Fault detection methods and devices for station screens, electronic equipment and storage media

Abstract

This invention discloses a fault detection method, device, electronic equipment, and storage medium for station display screens. The method includes: acquiring video frames from the perspective of the target station display screen; detecting a first graphic frame containing the target station display screen and a second graphic frame containing interfering objects within the video frames; performing occlusion detection on the target station display screen based on the first and second graphic frames; if the first and second graphic frames do not overlap, performing fault detection on the first graphic frame using a dilated convolution model to obtain the display status information of the target station display screen. This invention solves the technical problem of related technologies being unable to perform visual fault detection on station display screens, and has advantages such as high recognition accuracy, wide recognition range, and real-time reliable operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of safety supervision, in particular to a station screen fault detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] In the related art, in the field of video recognition, there is mainly the problem of poor universality. When it comes to the recognition of new scenes, it is necessary to adjust the parameters or design new algorithms. In the scenes of subway, airport and station, there is no related application at present. The platform layer of the subway station is quite different from other related scenes. The subway has a large passenger flow and a complex scene, which is easy to be blocked. If the conventional video recognition method is used, false positives are easy to occur.

[0003] The existing video recognition method in the related art does not have a detection method for passenger information screens and advertising light boxes. Most of them are used for platform door pinching or passenger flow statistics and prediction, and cannot be applied to complex scenes. The display screens in the station are automatically played. If hardware failure or software failure occurs, display problems will occur, and timely detection cannot be achieved.

[0004] For the above problems in the related art, no effective solution has been found. SUMMARY

[0005] The present application provides a station screen fault detection method and device, electronic equipment and storage medium.

[0006] According to one aspect of the present application, a station screen fault detection method is provided, which comprises: collecting a video frame of a target station screen; detecting a first graphic box where the target station screen is located and a second graphic box where an interference object is located in the video frame; performing occlusion detection on the target station screen based on the first graphic box and the second graphic box; if the first graphic box and the second graphic box do not overlap, performing fault detection on the first graphic box using a hollow convolution model to obtain display state information of the target station screen.

[0007] Further, detecting the first graphic box where the target station screen is located and the second graphic box where the interference object is located in the video frame comprises: performing sub-pixel corner point detection on the video frame to identify an interest domain and a non-interest domain in the video frame; performing pedestrian detection on the non-interest domain using a yolo algorithm to generate a pedestrian frame in the non-interest domain; determining the area where the interest domain is located as the first graphic box where the target station screen is located, and determining the area where the pedestrian frame is located as the second graphic box where the interference object is located.

[0008] Further, the sub-pixel corner point detection is performed on the video frame, and the interest domain and the non-interest domain in the video frame are identified, including: performing iterative detection on the sub-pixel corner points in the video frame to determine a set of sub-pixel points; for each sub-pixel point q in the set of sub-pixel points, the following steps are iteratively performed: in the video frame, the pixel points within a preset range from the q point are subjected to Gaussian filtering to obtain a set of smooth points; the derivative gradient G of each smooth point p in the set of smooth points is calculated respectively; the weight of the corresponding smooth point p is calculated according to the derivative gradient and the distance between the corresponding smooth point p and the q point; it is judged whether the q meets the preset accuracy requirement based on the weight; after the set of sub-pixel points is iterated, the region surrounded by all the smooth points meeting the preset accuracy requirement is determined as the interest domain, and the other regions in the video frame except the interest domain are determined as the non-interest domain.

[0009] Further, the weight of the corresponding smooth point p is calculated according to the derivative gradient and the distance between the corresponding smooth point p and the q point, including: the weight of the smooth point is calculated by using the following formula : ; wherein D i is the distance normalization value of the q point and the point, is the derivative gradient of the , and T is the transpose symbol.

[0010] Further, it is judged whether the q meets the preset accuracy requirement based on the weight, including: the accuracy value of the q is calculated by using the following formula: ; wherein is the weight of the smooth point , N is the total number of smooth points in the set of smooth points, is the two-dimensional coordinate value of the smooth point in the first graphic box.

[0011] Further, the display state information of the target station screen is obtained by performing fault detection on the first graphic box by using the empty convolution model, including: the first graphic box is used to configure an empty convolution layer to obtain an empty convolution model, wherein the empty convolution model includes an input layer, an empty convolution layer, an encoding layer, a self-attention layer, a decoding layer, and an output layer connected in sequence, and the empty convolution layer is used to down-sample the graphics output by the input layer; the first graphic box is input into the empty convolution model, and the display state information of the target station screen is output, wherein the state information is used to represent whether the target station screen has a display fault.

[0012] Further, the configuration of the first graphic box for the empty convolution layer comprises: analyzing the first graphic box, extracting the pixel amount contained in the first graphic box; selecting a matched kernel hole coefficient based on the pixel amount, wherein the kernel hole coefficient is used to indicate the distance between the pixel points processed by the kernel; and configuring the empty convolution layer by using the kernel hole coefficient.

[0013] According to another aspect of the embodiments of the present application, a fault detection device for a station screen is also provided, comprising: an acquisition module configured to acquire a video frame of a viewing angle in which a target station screen is located; a first detection module configured to detect a first graphic box in which the target station screen is located and a second graphic box in which an interference object is located in the video frame; a second detection module configured to perform occlusion detection on the target station screen based on the first graphic box and the second graphic box; and a third detection module configured to perform fault detection on the first graphic box by using an empty convolution model if the first graphic box and the second graphic box do not overlap, to obtain display state information of the target station screen.

[0014] Further, the first detection module comprises: an identification unit configured to perform sub-pixel corner point detection on the video frame, and identify an interest domain and a non-interest domain in the video frame; a generation unit configured to perform pedestrian detection on the non-interest domain by using a yolo algorithm, and generate a pedestrian box in the non-interest domain; and a determination unit configured to determine a region in which the interest domain is located as the first graphic box in which the target station screen is located, and determine a region in which the pedestrian box is located as the second graphic box in which the interference object is located.

[0015] Further, the identification unit comprises: a detection sub-unit configured to perform iterative detection on sub-pixel corner points in the video frame, to determine a set of sub-pixel points; a traversal sub-unit configured to perform the following steps by traversing each sub-pixel point q in the set of sub-pixel points: performing Gaussian filtering on pixel points within a preset range from the q point in the video frame to obtain a set of smoothed points; respectively calculating a derivative gradient G of each smoothed point p in the set of smoothed points; calculating a weight of the corresponding smoothed point p based on the derivative gradient and the distance between the corresponding smoothed point p and the q point; judging whether the q meets a preset accuracy requirement based on the weight; and a determination sub-unit configured to determine, after the set of sub-pixel points is traversed, a region surrounded by all the smoothed points meeting the preset accuracy requirement as the interest domain, and determine other regions in the video frame except the interest domain as the non-interest domain.

[0016] Further, the traversal sub-unit is further configured to calculate the weight of the smoothed point p by using the following formula: ; wherein D i is the distance between the q point and the p point. ​​distance normalized value of a point, is derivative gradient, T is a transpose symbol.

[0017] Further, the traversal subunit is further configured to calculate the accuracy value of the q by using the following formula: ; wherein, is a weight of a smoothing point , N is a total number of smoothing points in the smoothing point set, is a two-dimensional coordinate value of a smoothing point in the first graphic frame.

[0018] Further, the third detection module comprises: a configuration unit, configured to configure a dilated convolution layer by using the first graphic frame to obtain a dilated convolution model, wherein the dilated convolution model comprises an input layer, a dilated convolution layer, an encoding layer, a self-attention layer, a decoding layer and an output layer connected in sequence, and the dilated convolution layer is configured to down-sample a graphic output by the input layer; and an output unit, configured to input the first graphic frame into the dilated convolution model and output display state information of the target station screen, wherein the state information is configured to represent whether a display failure occurs in the target station screen.

[0019] Further, the configuration unit comprises: an analysis subunit, configured to analyze the first graphic frame and extract a pixel amount contained in the first graphic frame; a selection subunit, configured to select a matched kernel hole coefficient based on the pixel amount, wherein the kernel hole coefficient is configured to indicate a distance between pixel points processed by a kernel; and a configuration subunit, configured to configure the dilated convolution layer by using the kernel hole coefficient.

[0020] According to another aspect of the embodiments of the present application, a storage medium is also provided, which comprises a stored program, and the program performs the steps described above when running.

[0021] According to another aspect of the embodiments of the present application, an electronic device is also provided, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus; wherein: the memory is configured to store a computer program; and the processor is configured to execute the steps in the above method by running the program stored in the memory.

[0022] The embodiments of the present application also provide a computer program product comprising instructions which, when executed on a computer, cause the computer to perform the steps of the above method.

[0023] This invention acquires video frames from the perspective of the target station screen, detects a first graphic frame containing the target station screen and a second graphic frame containing interference objects within the video frames, and performs occlusion detection on the target station screen based on the first and second graphic frames. If the first and second graphic frames do not overlap, a dilated convolution model is used to detect faults in the first graphic frame, thereby obtaining the display status information of the target station screen. This invention can effectively address the location identification of station screens such as passenger information screens and advertising light boxes in complex scenarios. Occlusion detection can solve false alarms caused by pedestrian occlusion, and the dilated convolution model can quickly and accurately determine the display status information of the target station screen. This invention solves the technical problem that related technologies cannot perform visual fault detection on station displays, and has the advantages of high recognition accuracy, wide recognition range, and real-time reliable operation. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0025] Figure 1 This is a hardware structure block diagram of a computer according to an embodiment of the present invention;

[0026] Figure 2 This is a flowchart of a fault detection method for a station screen according to an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of the scene layout in which the present invention is applied according to an embodiment;

[0028] Figure 4 This is a flowchart of the sub-pixel corner point algorithm according to an embodiment of the present invention;

[0029] Figure 5 This is a schematic diagram of the detection results using the corner subpix algorithm in an embodiment of the present invention;

[0030] Figure 6 This is a schematic diagram of the structure of the dilated convolution model according to an embodiment of the present invention;

[0031] Figure 7 This is a flowchart illustrating the workflow of the dilated convolution model according to an embodiment of the present invention.

[0032] Figure 8 This is a schematic diagram of automatic fault detection according to an embodiment of the present invention;

[0033] Figure 9 This is a structural block diagram of a fault detection device for a station screen according to an embodiment of the present invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of them. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present application can be combined with each other.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] Example 1

[0037] The method embodiment provided in Embodiment 1 of this application can be executed in a controller, server, computer, tablet, or similar computing scheduling device. Taking running on a computer as an example, Figure 1 This is a hardware structure block diagram of a computer according to an embodiment of the present invention. For example... Figure 1 As shown, a computer may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. Optionally, the computer may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer described above. For example, the computer may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0038] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to a fault detection method for a station screen in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the aforementioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0039] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a computer's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0040] This embodiment provides a fault detection method for station screens. Figure 2 This is a flowchart of a fault detection method for a station screen according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0041] Step S202: Acquire video frames from the perspective of the target station screen;

[0042] In this embodiment, the station screen can be a passenger information screen, an advertising light box, etc.

[0043] Figure 3 This is a schematic diagram of the scene layout for the application of an embodiment of the present invention. The present invention takes an island platform as an example, including but not limited to side platforms and hybrid platforms, including cameras 1 to n, pedestrians PIS1 to PISn, and the target station screen is an advertising light box, including advertising light box 1 to advertising light box n, with the camera's shooting vision directed towards the advertising light box.

[0044] Step S204: Detect the first frame containing the target station screen and the second frame containing the interference object in the video frame;

[0045] Optionally, the distraction can be a dynamic object that visually obscures the target station screen, such as a pedestrian.

[0046] In one example, background subtraction can be used to further determine whether there are interfering objects in the video frame. Background subtraction is performed on the two acquired video frames to determine whether there are differences between the two video frames. If there are differences, the first graphic frame containing the target station screen and the second graphic frame containing the interfering object are detected in the video frame.

[0047] Step S206: Perform occlusion detection on the target station screen based on the first and second graphic frames;

[0048] This embodiment determines whether the first graphic frame and the second graphic frame overlap by judging whether the first graphic frame and the second graphic frame intersect, or by judging whether the area or number of pixels of the intersecting region is greater than a preset threshold.

[0049] Step S208: If the first graphic frame and the second graphic frame do not overlap, a dilated convolution model is used to perform fault detection on the first graphic frame to obtain the display status information of the target station screen.

[0050] The display status information includes whether the display is normal or abnormal.

[0051] Through the above steps, video frames from the perspective of the target station screen are acquired. The first bounding box containing the target station screen and the second bounding box containing interfering objects are detected within the video frames. Occlusion detection is performed on the target station screen based on the first and second bounding boxes. If the first and second bounding boxes do not overlap, a dilated convolution model is used to detect faults in the first bounding box, thus obtaining the display status information of the target station screen. This method can effectively address the location identification of station screens such as passenger information screens and advertising light boxes in complex scenarios. Occlusion detection can resolve false alarms caused by pedestrian obstruction. The dilated convolution model can quickly and accurately determine the display status information of the target station screen, solving the technical problem that related technologies cannot perform visual fault detection on station displays. It has advantages such as high recognition accuracy, wide recognition range, and real-time reliable operation.

[0052] In one embodiment of this example, detecting the first frame containing the target station screen and the second frame containing the interference object in the video frame includes:

[0053] S11, perform subpixel corner detection on the video frame to identify the domain of interest and non-domain of interest in the video frame;

[0054] In one example, subpixel corner detection of the video frame and identification of the region of interest (ROI) and non-ROI in the video frame include: iterative detection of subpixel corners in the video frame to determine a set of subpixel points; for each subpixel point q in the set of subpixel points, the following steps are performed: Gaussian filtering is applied to pixels within a preset range from point q in the video frame, centered on point q, to ​​obtain a set of smooth points; the gradient G of the derivative of each smooth point p in the set of smooth points is calculated; the weight of the corresponding smooth point p is calculated based on the gradient and the distance between the corresponding smooth point p and point q; based on the weight, it is determined whether q meets a preset accuracy requirement; after the subpixel point set is traversed, the region enclosed by all smooth points that meet the preset accuracy requirement is determined as the region of interest, and other regions in the video frame other than the region of interest are determined as non-ROIs.

[0055] Optionally, calculating the weight of the corresponding smoothing point p based on the derivative gradient and the distance between the corresponding smoothing point p and the point q includes: calculating the smoothing point using the following formula. weight : ; where D i For point q and Normalized distance value of the point for The gradient of the derivative, where T is the transpose.

[0056] S12, use the YOLO algorithm to perform pedestrian detection on the non-interest region and generate pedestrian bounding boxes in the non-interest region;

[0057] S13, the area where the region of interest is located is determined as the first graphic frame where the target station screen is located, and the area where the pedestrian frame is located is determined as the second graphic frame where the interference is located.

[0058] Based on the above example, determining whether q meets the preset precision requirement based on the weight includes: calculating the precision value of q using the following formula: ;in, For smooth points The weights are given by N, where N is the total number of smooth points in the set of smooth points. For smooth points Two-dimensional coordinate values ​​in the first graphic frame.

[0059] Figure 4 This is a flowchart of the sub-pixel corner point algorithm according to an embodiment of the present invention, including:

[0060] Obtain the iterated subpixel using the cornersubpix method. ;

[0061] Applying Gaussian filtering to the vicinity of point q can eliminate possible corner points with small differences in brightness.

[0062] Calculate the gradient G at N points p surrounding point q. G i = [ d x ⤢ d y ] , For q point to The coordinate increment of the point;

[0063] The weight w is calculated based on the gradient and distance. Where Di is the normalized distance between point p and point q. ;

[0064] Recalculate whether point q meets the accuracy requirements. ;

[0065] The region of interest is formed using all q points that meet the accuracy requirements.

[0066] The algorithm designed in this embodiment adds a gradient-based weight calculation method on the basis of considering distance, and obtains the corner points of areas with large light and dark contrast through Gaussian filtering to form a region of interest. It is suitable for passenger information screen and advertising light box detection scenarios and improves detection accuracy.

[0067] Figure 5 This is a schematic diagram of the detection results using the corner subpix algorithm in an embodiment of the present invention. The black dots are the detected corner points, and the diagonal lines represent the divided regions of interest.

[0068] In this embodiment, a dilated convolution model is used to perform fault detection on the first graphic frame, and the display status information of the target station screen is obtained, including:

[0069] S21, using the first graphic frame to configure a dilated convolutional layer to obtain a dilated convolutional model, wherein the dilated convolutional model includes an input layer, a dilated convolutional layer, an encoding layer, a self-attention layer, a decoding layer, and an output layer connected in sequence, and the dilated convolutional layer is used to downsample the graphic output by the input layer;

[0070] In one example, configuring a dilated convolutional layer using the first graphical frame includes: parsing the first graphical frame and extracting the number of pixels contained in the first graphical frame; selecting a matching convolution kernel dilation coefficient based on the number of pixels, wherein the convolution kernel dilation coefficient is used to indicate the spacing between pixels processed by the convolution kernel; and configuring the dilated convolutional layer using the convolution kernel dilation coefficient.

[0071] S22, input the first graphic frame into the dilated convolution model and output the display status information of the target station screen, wherein the status information is used to characterize whether the target station screen has a display failure.

[0072] Figure 6 This is a schematic diagram of the dilated convolution model according to an embodiment of the present invention. This model adds a dilated convolution layer, and the input to the model is an image within the region of interest. The convolution kernel used by the dilated convolution layer can be selected according to the number of pixels within the region of interest. In this example, the dilation rate of the convolution kernel used is 5.

[0073] In this embodiment, considering that advertising light boxes and PIS screens have high brightness and do not require content recognition, only the detection of whether they are emitting light normally is required, dilated convolution can effectively improve computational efficiency, while reconstruction method can improve recognition accuracy.

[0074] Figure 7 This is a flowchart of the dilated convolution model in an embodiment of the present invention. When the advertising sign or PIS screen is in a normal luminous state, a downsampled image is output after dilated convolution, and then the output result is normal after passing through the self-attention layer. When the advertising sign or PIS screen is in an abnormal state, a downsampled image is output after dilated convolution, and then the output result is abnormal after passing through the self-attention layer.

[0075] Figure 8 This is a schematic diagram of automatic fault detection according to an embodiment of the present invention. The method and apparatus for automatic fault detection of station passenger information screens and advertising light boxes based on video recognition are characterized by including the following steps:

[0076] Step 1: Capture one frame of the online video every 10 seconds;

[0077] Considering that the differences between adjacent frames are not significant, in order to obtain the positions of passenger information screens and advertising light boxes in the background, this invention obtains one frame image by long-interval sampling (default every 10 seconds).

[0078] Step 2: Perform background subtraction on the two acquired frames. If no difference is detected for the first time, proceed to Step 3 for position detection. Perform position detection only once. If no difference is detected again in subsequent steps, no further processing is performed. Otherwise, proceed to Step 4.

[0079] Background subtraction is performed on the two frames of images before and after the 10-second interval. When no difference is detected for the first time, the image is considered to be a background image without external interference and can be used for position localization. The image is then input into step 3 for position detection. Position detection is performed only once. If no difference is detected again, no processing is performed. Otherwise, proceed to step 4.

[0080] Step 3: Location detection. Here, an improved corner subpix algorithm is used to detect the location of the corner points of the passenger information screen and the advertising light box, and to divide the region of interest into other irrelevant areas. The region of interest is saved for subsequent calls, and the process returns to step 1 for repeated sampling.

[0081] Step 4: Pedestrian recognition. The YOLO algorithm is used to identify pedestrians and obtain their bounding boxes.

[0082] Step 5: Occlusion detection. Calculate the region of interest and pedestrian bounding box. If the pedestrian bounding box overlaps with the region of interest, occlusion is considered to have occurred, and subsequent state detection is not performed. If the pedestrian bounding box does not overlap with the region of interest, occlusion is considered not to have occurred, and step 6 state detection is performed.

[0083] Step 6: State detection. Input the image data of the passenger information screen and advertising light box within the domain of interest under unobstructed conditions into the reconstructed dilated convolution model and output the corresponding state.

[0084] This embodiment implements a method and apparatus for automatic fault detection of passenger information screens and advertising light boxes in subway stations based on video recognition. By designing a position detection method, an occlusion detection method, and a status detection method for complex subway platform scenarios, an automated fault detection method is provided for passenger information screens and advertising light boxes in subway stations. It can achieve position recognition, occlusion recognition, and status recognition of passenger information screens and advertising light boxes in complex subway platform scenarios.

[0085] Compared with existing technologies, the solution in this embodiment has the advantages of high recognition accuracy, wide recognition range, and real-time reliable operation.

[0086] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0087] Example 2

[0088] This embodiment also provides a fault detection device for a station screen, used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0089] Figure 9 This is a structural block diagram of a fault detection device for a station screen according to an embodiment of the present invention, such as... Figure 9 As shown, the device includes: a data acquisition module 90, a first detection module 92, a second detection module 94, and a third detection module 96, wherein...

[0090] Acquisition module 90 is used to acquire video frames from the perspective of the target station screen;

[0091] The first detection module 92 is used to detect the first graphic frame where the target station screen is located and the second graphic frame where the interference is located in the video frame;

[0092] The second detection module 94 is used to perform occlusion detection on the target station screen based on the first graphic frame and the second graphic frame.

[0093] The third detection module 96 is used to perform fault detection on the first graphic frame using a dilated convolution model if the first graphic frame and the second graphic frame do not overlap, so as to obtain the display status information of the target station screen.

[0094] Optionally, the first detection module includes: an identification unit, configured to perform sub-pixel corner detection on the video frame and identify the region of interest and non-region of interest in the video frame; a generation unit, configured to perform pedestrian detection on the non-region of interest using the YOLO algorithm and generate pedestrian bounding boxes in the non-region of interest; and a determination unit, configured to determine the area where the region of interest is located as the first graphic frame where the target station screen is located, and determine the area where the pedestrian bounding box is located as the second graphic frame where the interference is located.

[0095] Optionally, the recognition unit includes: a detection subunit, used to iteratively detect sub-pixel corner points in the video frame to determine a set of sub-pixel points; a traversal subunit, used to perform the following steps for each sub-pixel point q in the set of sub-pixel points: performing Gaussian filtering on pixels within a preset range from point q in the video frame, centered on point q, to ​​obtain a set of smooth points; calculating the derivative gradient G of each smooth point p in the set of smooth points; calculating the weight of the corresponding smooth point p based on the derivative gradient and the distance between the corresponding smooth point p and point q; determining whether q meets a preset accuracy requirement based on the weight; and a determination subunit, used to determine the region enclosed by all smooth points that meet the preset accuracy requirement as the region of interest after the traversal of the sub-pixel point set is completed, and determining other regions in the video frame other than the region of interest as non-regions of interest.

[0096] Optionally, the traversal subunit is further configured to: calculate the smoothing point using the following formula. weight : ; where D i For point q and Normalized distance value of the point for The gradient of the derivative, where T is the transpose.

[0097] Optionally, the traversal subunit is further configured to: calculate the precision value of q using the following formula: ;in, For smooth points The weights are given by N, where N is the total number of smooth points in the set of smooth points. For smooth points Two-dimensional coordinate values ​​in the first graphic frame.

[0098] Optionally, the third detection module includes: a configuration unit, configured to configure a dilated convolutional layer using the first graphic frame to obtain a dilated convolutional model, wherein the dilated convolutional model includes an input layer, a dilated convolutional layer, an encoding layer, a self-attention layer, a decoding layer, and an output layer connected in sequence, the dilated convolutional layer being used to downsample the graphic output by the input layer; and an output unit, configured to input the first graphic frame into the dilated convolutional model and output the display status information of the target station screen, wherein the status information is used to characterize whether the target station screen has experienced a display failure.

[0099] Optionally, the configuration unit includes: a parsing subunit for parsing the first graphic frame and extracting the number of pixels contained in the first graphic frame; a selection subunit for selecting a matching convolution kernel dilation coefficient based on the number of pixels, wherein the convolution kernel dilation coefficient is used to indicate the spacing between pixels processed by the convolution kernel; and a configuration subunit for configuring the dilated convolutional layer using the convolution kernel dilation coefficient.

[0100] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0101] Example 3

[0102] Embodiments of the present invention also provide a storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0103] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0104] S1, capture video frames from the perspective of the target station screen;

[0105] S2, detect the first graphic frame where the target station screen is located and the second graphic frame where the interference is located in the video frame;

[0106] S3, perform occlusion detection on the target station screen based on the first graphic frame and the second graphic frame;

[0107] S4. If the first graphic frame and the second graphic frame do not overlap, a dilated convolution model is used to perform fault detection on the first graphic frame to obtain the display status information of the target station screen.

[0108] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0109] Embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0110] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0111] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0112] S1, capture video frames from the perspective of the target station screen;

[0113] S2, detect the first graphic frame where the target station screen is located and the second graphic frame where the interference is located in the video frame;

[0114] S3, perform occlusion detection on the target station screen based on the first graphic frame and the second graphic frame;

[0115] S4. If the first graphic frame and the second graphic frame do not overlap, a dilated convolution model is used to perform fault detection on the first graphic frame to obtain the display status information of the target station screen.

[0116] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0117] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0118] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0119] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0120] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0121] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0122] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0123] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method of fault detection for a station screen, characterized by, The method comprises: collecting a video frame of a view angle in which a target station screen is located; detecting a first graphic box in which the target station screen is located and a second graphic box in which an interference object is located in the video frame; performing occlusion detection on the target station screen based on the first graphic box and the second graphic box; if the first graphic box and the second graphic box do not overlap, performing fault detection on the first graphic box by using a hollow convolution model to obtain display state information of the target station screen; detecting a first graphic box in which the target station screen is located and a second graphic box in which an interference object is located in the video frame comprises: performing sub-pixel corner point detection on the video frame to identify an interest domain and a non-interest domain in the video frame; performing pedestrian detection on the non-interest domain by using a YOLO algorithm to generate a pedestrian box in the non-interest domain; determining a region in which the interest domain is located as the first graphic box in which the target station screen is located, and determining a region in which the pedestrian box is located as the second graphic box in which the interference object is located; performing sub-pixel corner point detection on the video frame to identify an interest domain and a non-interest domain in the video frame comprises: iteratively detecting sub-pixel corner points in the video frame to determine a set of sub-pixel points; for each sub-pixel point q in the set of sub-pixel points, the following steps are iteratively performed: in the video frame, performing Gaussian filtering on pixel points within a preset range from the q point to obtain a set of smoothed points; calculating a derivative gradient G of each smoothed point p in the set of smoothed points; based on the derivative gradient and a distance between the corresponding smoothed point p and the q point, calculating a weight of the corresponding smoothed point p; judging whether the q meets a preset accuracy requirement based on the weight; after the set of sub-pixel points is iteratively completed, a region surrounded by all the smoothed points that meet the preset accuracy requirement is determined as the interest domain, and other regions in the video frame except the interest domain are determined as the non-interest domain.

2. The method of claim 1, wherein, based on the derivative gradient and a distance between the corresponding smoothed point p and the q point, calculating a weight of the corresponding smoothed point p comprises: The following formula is used to calculate the weight of a smoothed point :​ ； where D i is a distance normalized value of the q point and the p point, is a derivative gradient of T is a transpose symbol.

3. The method of claim 2, wherein, judging whether the q meets a preset accuracy requirement based on the weight comprises: calculating an accuracy value of the q by using the following formula: ； wherein, is a weight of the smoothing point, N is a total number of smoothing points in the smoothing point set, is a weight of the smoothing point, is a weight of the smoothing point, is a two-dimensional coordinate value in the first graphic frame.

4. The method of claim 1, wherein, performing fault detection on the first graphic box by using the hollow convolution model to obtain the display state information of the target station screen comprises: configuring a hollow convolution layer by using the first graphic box to obtain a hollow convolution model, wherein the hollow convolution model comprises an input layer, a hollow convolution layer, an encoding layer, a self-attention layer, a decoding layer, and an output layer connected in sequence, and the hollow convolution layer is used to down-sample a graph output by the input layer; inputting the first graphic box into the hollow convolution model to output the display state information of the target station screen, wherein the state information is used to represent whether the target station screen has a display fault.

5. The method of claim 4, wherein, configuring the hollow convolution layer by using the first graphic box comprises: analyzing the first graphic box to extract a pixel amount contained in the first graphic box; selecting a matched convolution kernel hollow coefficient based on the pixel amount, wherein the convolution kernel hollow coefficient is used to indicate a distance between pixel points processed by a convolution kernel; The convolution kernel is configured with the hole convolution layer.

6. A fault detection apparatus for a station screen, characterized by comprising: The method comprises the steps of: The acquisition module is configured to acquire a video frame of a target station screen in a perspective angle. The first detection module is configured to detect a first graphic frame in which the target station screen is located and a second graphic frame in which an interference object is located in the video frame, and comprises the following steps: Sub-pixel corner point detection is performed on the video frame to identify an interest domain and a non-interest domain in the video frame. A pedestrian detection is performed on the non-interest domain by using a YOLO algorithm to generate a pedestrian frame in the non-interest domain. An area in which the interest domain is located is determined as the first graphic frame in which the target station screen is located, and an area in which the pedestrian frame is located is determined as the second graphic frame in which the interference object is located. The sub-pixel corner point detection on the video frame to identify the interest domain and the non-interest domain comprises the following steps: An iterative detection is performed on the sub-pixel corner points in the video frame to determine a set of sub-pixel points. For each sub-pixel point q in the set of sub-pixel points, the following steps are iteratively performed: In the video frame, a set of smooth points is obtained by performing a Gaussian filtering on pixel points within a preset range from the q point and taking the q point as the center. A derivative gradient G of each smooth point p in the set of smooth points is calculated. According to the derivative gradient and a distance between the corresponding smooth point p and the q point, a weight of the corresponding smooth point p is calculated. Whether the q meets a preset accuracy requirement is determined based on the weight. After the set of sub-pixel points is iteratively completed, an area surrounded by all smooth points meeting the preset accuracy requirement is determined as the interest domain, and other areas in the video frame except the interest domain are determined as the non-interest domain. The second detection module is configured to perform an occlusion detection on the target station screen based on the first graphic frame and the second graphic frame. The third detection module is configured to, if the first graphic frame and the second graphic frame do not overlap, perform a fault detection on the first graphic frame by using a hole convolution model to obtain display state information of the target station screen.

7. An electronic device, comprising: The device comprises a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. The memory is configured to store a computer program. The processor is configured to execute the method steps of any one of claims 1 to 5 by running the program stored in the memory.

8. A storage medium, characterized by The storage medium comprises a stored program, wherein the program performs the method steps of any one of claims 1 to 5 when running. The storage medium comprises a stored program, wherein the program performs the method steps of any one of claims 1 to 5 when running.

Images