An outdoor digital advertising screen playing state monitoring and abnormality repairing method

Abstract

The application relates to the field of digital media playing monitoring, and discloses an outdoor digital advertising screen playing state monitoring and abnormality repairing method; the method generates a playing time sequence template according to advertisement scheduling and advertisement materials by a server; a broadcast control terminal collects broadcast control output evidence including a current advertisement segment identifier, a decoding frame timestamp, a rendering queue state, a video output interface state and a screen body control card feedback state when playing the advertisement materials; an image acquisition unit collects a screen display picture and forms screen display evidence; the server maps the playing time sequence template, the broadcast control output evidence and the screen display evidence to the same playing time axis to obtain scheduling consistency results, output consistency results and real display consistency results, and generates abnormality attribution results according to the conflict types among the three, and then corresponding repairing strategies are executed.

Description

Technical Field

[0001] This invention relates to the field of digital media playback monitoring, specifically to a method for monitoring and repairing the playback status of outdoor digital advertising screens. Background Technology

[0002] Outdoor digital advertising screens typically consist of a server, a broadcast control terminal, a video output link, a screen control card, and the screen itself. The server sends advertising materials to the broadcast control terminal according to the advertising schedule. The broadcast control terminal decodes and renders the advertising materials, then outputs a video signal to the screen control card via the video output interface. The screen control card then drives the screen to display the advertising image. Because outdoor digital advertising screens are mostly installed in open environments such as roads, squares, building facades, and bus stops, the equipment needs to operate continuously for extended periods. Furthermore, the advertising content usually has a specific playback time and sequence. Therefore, it is necessary to monitor the playback status of the advertising screen to promptly detect and address any playback abnormalities.

[0003] Existing methods for monitoring the playback status of outdoor digital advertising screens fall into two categories. One relies primarily on the operational status of the broadcast control terminal, such as monitoring player processes, playback progress, media playback logs, CPU usage, or local heartbeat information. The other method uses cameras or sensors to capture screen images to determine if there are black screens, distorted images, bright or dark areas, or other localized display anomalies. While the former reflects the internal operation of the broadcast control terminal, normal output from the terminal does not necessarily indicate a normal display on the screen itself. Abnormal video output interface handshakes, communication problems with the screen control card, receiver card disconnections, or localized display anomalies can all lead to discrepancies between the actual screen image and the scheduled advertisement. The latter reflects the surface display status, but outdoor environments present external factors such as strong light, reflections, pedestrian and vehicle obstructions, raindrops, and dirt. Relying solely on screen image capture can easily lead to misinterpreting environmental interference as playback malfunctions.

[0004] Therefore, existing technologies still have the following problems: During the monitoring of the playback status of outdoor digital advertising screens, relying solely on the status of the broadcast control terminal or solely on the screen capture image makes it difficult to accurately distinguish between abnormal material scheduling, abnormal material caching, abnormal decoding and rendering, abnormal video output links, abnormal screen receiving links, abnormal partial screen display, and outdoor environmental interference, resulting in inaccurate attribution of abnormalities; When repairing abnormalities, existing methods mostly adopt common processing methods such as restarting the player, restarting the broadcast control terminal, reloading materials, or simply reporting alarms, which makes it difficult to perform corresponding repairs for different abnormal links, easily causing normal playback interruption, advertising schedule deviation, and repeated failures. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method for monitoring and repairing the playback status of outdoor digital advertising screens, thereby resolving the technical problems existing in the prior art.

[0006] The above-mentioned technical objective of the present invention is achieved through the following technical solution: A method for monitoring and repairing the playback status of outdoor digital advertising screens includes the following steps: S1. The server generates a playback sequence template based on the advertising schedule and advertising materials. The playback sequence template includes advertising segment identifiers, the time period to be played, key frame time points, sampling area positions, and regional image features extracted from the original images of the advertising materials. S2. When the broadcast control terminal plays advertising materials, it collects broadcast control output evidence, which includes the current advertising segment identifier, decoding frame timestamp, rendering queue status, video output interface status, and screen control card feedback status. S3. The screen display image is captured by the image acquisition unit set at the corresponding position of the outdoor digital advertising screen, the effective display area of ​​the screen is extracted, and the screen display evidence is determined according to the ambient illuminance and the occluded area in the display image. S4. Map the playback timing template, broadcast control output evidence, and screen display evidence to the same playback timeline to obtain the scheduling consistency result, output consistency result, and actual display consistency result, respectively. S5. Generate anomaly attribution results based on the conflict types between scheduling consistency results, output consistency results, and actual displayed consistency results; S6. Execute the corresponding repair strategy according to the abnormal attribution results; S7. After the repair strategy is executed, re-collect the broadcast control output evidence and screen display evidence, and once again perform a consistency judgment with the playback timing template under the same playback time axis. Determine whether the repair is complete based on the re-obtained scheduling consistency result, output consistency result and actual display consistency result.

[0007] Preferably, in step S1, when the server generates the playback timing template, it performs frame-by-frame parsing of the advertising material and divides the advertising material into multiple advertising segments according to the advertising segment identifier; For each advertisement segment, select keyframes located at different playback times, and determine at least two sampling regions in each keyframe; For each sampling region, regional image features are extracted, including the region's average brightness value, region's color histogram, region's edge gradient value, and region's texture change value. The key frame time points, sampling area locations, and area image features are associated and stored with the corresponding advertising segment identifiers to form a playback timing template.

[0008] Preferably, the sampling area avoids the solid color background area, the scrolling subtitle area and the edge blank area in the advertising material, and each sampling area in the same keyframe is located in a different partition of the effective display area of ​​the screen; The server assigns sequential numbers to the regional image features of corresponding sampling areas in adjacent keyframes, and uses these sequential numbers as the inter-frame change order in the playback timing template.

[0009] Preferably, in step S2, when the broadcast control terminal plays advertising materials, it uses the advertising segment switching time, the decoding frame output time, and the rendering frame submission time as recording nodes to generate broadcast control output evidence. The timestamp of the decoded frame is the playback time of the material when the decoder outputs the video frame. The rendering queue status includes the number of frames to be rendered, the number of frames submitted for rendering, and the number of consecutive frames that have not been updated. The video output interface status includes the interface connection status, output resolution, output refresh rate, and interface handshake status. The screen control card feedback status includes the control card online status, the receiving card number, and the cabinet partition number.

[0010] Preferably, in step S3, after the image acquisition unit acquires the screen display image, it determines the effective display area of ​​the screen based on the position of the screen border or the preset screen corner coordinates. Perform perspective correction on the effective display area of ​​the screen, and extract the corresponding display sampling area according to the sampling area position in the playback timing template; The screen display evidence includes the acquisition timestamp, the corrected image of the effective display area of ​​the screen, the location of the display sampling area, the average brightness value of the display sampling area, the color histogram, the edge gradient value, the texture change value, the ambient illuminance value, and the location of the occluded area.

[0011] Preferably, the ambient illuminance is collected by an ambient light sensor located near the outdoor digital advertising screen, and the occlusion area is determined based on the brightness change boundary of consecutive pixels within the display sampling area, and the range of consecutive pixels located within the brightness change boundary and whose color difference is less than the color difference threshold configured by the server. When the overlap area between the display sampling area and the occlusion area reaches the overlap ratio threshold configured by the server, an occlusion mark is set for the display sampling area, and the display sampling area with the occlusion mark is excluded in the consistency judgment in step S4.

[0012] Preferably, in step S4, the advertising scheduling time of the server, the local playback time of the broadcast control terminal, and the acquisition time of the image acquisition unit are uniformly converted into time points on the same playback time axis. The scheduling consistency result is determined by the ad segment identifier corresponding to the current time point and the time period to be played. The consistency of the output is determined by whether the timestamp of the decoded frame increases with the playback time, whether the rendering queue status is updated, and whether the video output interface status remains connected. The consistency of the actual display is determined by whether the feature difference between the display sampling area features in the screen display evidence and the corresponding sampling area area features in the playback timing template falls within the feature difference range configured by the server.

[0013] Preferably, in step S5, the abnormal attribution results include abnormal material scheduling, abnormal material caching, abnormal decoding and rendering, abnormal video output link, abnormal screen receiving link, abnormal partial screen display, and outdoor environmental interference. Specifically, if the current ad segment identifier is inconsistent with the ad segment identifier at the corresponding time point in the playback timing template, the generated material scheduling is abnormal. When the current ad segment identifier is consistent and the local ad creative verification value is inconsistent with the ad creative verification value recorded on the server, the generated creative cache is abnormal; A decoding rendering error is generated when the timestamp of the decoded frame does not increase within N consecutive sampling periods, or when the number of consecutive unupdated frames in the rendering queue reaches the frame rate threshold configured by the server. N is a positive integer configured by the server based on the frame rate and sampling period of the advertising material.

[0014] Preferably, in step S5, when the scheduling consistency result and the output consistency result are both consistent, the actual display consistency result is inconsistent, and the video output interface status is abnormal, a video output link abnormality is generated; when the scheduling consistency result, the output consistency result, and the video output interface status are all consistent, the actual display consistency result is inconsistent, and the screen control card feedback status is abnormal, a screen receiving link abnormality is generated. When the scheduling consistency result, output consistency result, video output interface status, and screen control card feedback status are all consistent, and the actual display consistency result is inconsistent only in the display sampling area of ​​the effective display area of ​​the screen (less than the total display sampling area), a partial screen display anomaly is generated. Outdoor environmental interference is generated when the location of the display sampling area that shows an inconsistent display consistency result coincides with the location of the occluded area, or when the ambient illuminance value corresponding to the display sampling area exceeds the illuminance threshold configured by the server. The abnormal status of the video output interface includes: the interface connection status is disconnected, the interface handshake status is failed, the output resolution is inconsistent with the screen resolution recorded in the playback timing template, and the output refresh rate is inconsistent with the screen refresh rate recorded in the playback timing template. The abnormal feedback status of the screen control card includes control card offline, missing receiver card number, abnormal cabinet partition number, and receiver card feedback time exceeding the feedback time threshold configured by the server.

[0015] Preferably, in step S6, when the anomaly attribution result is an anomaly in the material scheduling or an anomaly in the material caching, the broadcast control terminal verifies or re-fetches the local advertising materials. When the anomaly attribution result is a decoding or rendering anomaly, the broadcast control terminal restarts the decoding or rendering process and locates the current time point when it should be played. When the anomaly attribution result is a video output link anomaly, the broadcast control terminal reinitializes the video output interface. When the anomaly attribution result is that the screen receiving link is abnormal, the broadcast control terminal re-issues the screen control card configuration parameters. When the anomaly attribution result is a partial display anomaly on the screen, a fault record containing the location of the anomaly area is generated. When the abnormal attribution result is outdoor environmental interference, the judgment weight of the screen display evidence in the current sampling period is reduced, and playback repair is not performed; When reinitializing the video output interface, the broadcast control terminal sequentially performs interface closing, interface re-enumeration, output resolution configuration, output refresh rate configuration, and interface handshake status confirmation. When re-issuing the configuration parameters of the screen control card, the broadcast control terminal determines the abnormal receiving card number or abnormal cabinet partition number based on the feedback status of the screen control card, and issues mapping parameters to the corresponding receiving card or corresponding cabinet partition. The mapping parameters include the receiving card address, cabinet partition coordinates, display start coordinates, and display size. In step S7, when the re-obtained schedule consistency result, output consistency result, and actual display consistency result are all consistent, the repair completion status is recorded; When the scheduling consistency result and the output consistency result are both consistent, but the actual display consistency result is still inconsistent, the anomaly attribution result is updated to video output link anomaly, screen receiving link anomaly, partial screen display anomaly, or outdoor environmental interference, and the corresponding repair strategy is executed or a fault record is generated according to the updated anomaly attribution result.

[0016] In summary, the present invention has the following main beneficial effects: By generating playback sequence templates based on advertising schedules and creative materials, and simultaneously collecting broadcast control output evidence and screen display evidence, the playback sequence templates, broadcast control output evidence, and screen display evidence are mapped to the same playback timeline for consistency judgment. This achieves the goal of simultaneously verifying the content that should be played, the actual content output by the broadcast control, and the actual content displayed on the screen. Therefore, it avoids misjudgments caused by relying solely on player progress, playback progress, or single-frame images to determine the playback status, enabling outdoor digital advertising screens to accurately identify the playback status even in complex environments such as strong light, obstructions, reflections, and localized display anomalies.

[0017] By generating anomaly attribution results based on the conflict types among scheduling consistency results, output consistency results, and actual display consistency results, and categorizing anomalies into material scheduling anomalies, material caching anomalies, decoding and rendering anomalies, video output link anomalies, screen receiving link anomalies, partial screen display anomalies, and outdoor environmental interference, the goal of link-specific playback anomaly localization is achieved. This allows for differentiation between different situations such as abnormal player output, abnormal video output interface, abnormal screen control card, partial screen malfunction, and outdoor environmental interference, avoiding the simplistic attribution of all playback anomalies to player or screen failures, and improving the specificity and traceability of anomaly diagnosis.

[0018] By implementing corresponding repair strategies based on the anomaly attribution results, and then re-collecting broadcast control output evidence and screen display evidence for verification after repair, the goal of closed-loop confirmation of anomaly handling is achieved. For material anomalies, decoding and rendering anomalies, video output link anomalies, and screen receiving link anomalies, respectively, the following actions are taken: re-fetching materials, restarting the decoding or rendering process, re-initializing the video output interface, and re-issuing the screen control card configuration parameters. However, for partial screen display anomalies and outdoor environmental interference, a full machine restart is not directly performed. This reduces unnecessary playback interruptions, minimizes scheduling deviations caused by repeated restarts, and improves the stability of continuous playback and remote maintenance of outdoor digital advertising screens. Attached Figure Description

[0019] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1 refer to Figure 1 A method for monitoring and repairing the playback status of outdoor digital advertising screens, comprising the following steps: S1. The server generates a playback sequence template based on the advertising schedule and advertising materials. The playback sequence template includes advertising segment identifiers, the time period to be played, key frame time points, sampling area positions, and regional image features extracted from the original images of the advertising materials. S2. When the broadcast control terminal plays advertising materials, it collects broadcast control output evidence, which includes the current advertising segment identifier, decoding frame timestamp, rendering queue status, video output interface status, and screen control card feedback status. S3. The screen display image is captured by the image acquisition unit set at the corresponding position of the outdoor digital advertising screen, the effective display area of ​​the screen is extracted, and the screen display evidence is determined according to the ambient illuminance and the occluded area in the display image. S4. Map the playback timing template, broadcast control output evidence, and screen display evidence to the same playback timeline to obtain the scheduling consistency result, output consistency result, and actual display consistency result, respectively. S5. Generate anomaly attribution results based on the conflict types between scheduling consistency results, output consistency results, and actual displayed consistency results; S6. Execute the corresponding repair strategy according to the abnormal attribution results; S7. After the repair strategy is executed, re-collect the broadcast control output evidence and screen display evidence, and once again perform a consistency judgment with the playback timing template under the same playback time axis. Determine whether the repair is complete based on the re-obtained scheduling consistency result, output consistency result and actual display consistency result.

[0022] This application applies to outdoor digital advertising screens installed on both sides of roads, in commercial plazas, bus stops, building facades, or park entrances. The outdoor digital advertising screen includes a screen body, a broadcast control terminal, a screen control card, an image acquisition unit, an ambient light sensor, and a server. The broadcast control terminal communicates with the server to receive advertising schedules and materials, and outputs video signals to the screen control card; the screen control card drives the screen body to display the corresponding video image; the image acquisition unit is positioned to capture the image displayed on the screen body; the ambient light sensor is located near the outdoor digital advertising screen to collect the illuminance value of the environment in which the screen body is located.

[0023] The server, broadcast control terminal, and image acquisition unit synchronize their times before operation. Time synchronization can be achieved via network time synchronization or by the server periodically sending time correction commands to the broadcast control terminal and image acquisition unit. Both the broadcast control terminal and image acquisition unit include their local acquisition timestamps when uploading data. The server then converts the data generated by different devices to the same playback timeline based on the time correction results. This allows the server to compare the content that should be played in the advertising schedule, the actual content output by the broadcast control terminal, and the content actually displayed on the screen at the same point in time.

[0024] In this embodiment, the brightness variance threshold, color difference threshold, overlap ratio threshold, feature difference range, illuminance threshold, frame rate threshold, and feedback time threshold configured by the server are all stored in the server's device configuration table and associated with the device number of the outdoor digital advertising screen. The device configuration table is generated during the installation and debugging phase of the outdoor digital advertising screen. During generation, the server collects the display screen under normal playback conditions, the environmental image under no obvious obstruction conditions, the image under pedestrian or vehicle obstruction conditions, the image under strong daylight conditions, and the communication records under normal feedback conditions of the screen control card. The server forms corresponding thresholds based on the above samples, so that the same device will perform judgments according to the device configuration table corresponding to the device during subsequent operation. The above thresholds are not limited to fixed values, but are determined based on the device installation location, the installation angle of the image acquisition unit, the screen brightness level, the frame rate of the advertising material, the processing capability of the broadcast control terminal, and the communication cycle of the screen control card.

[0025] After receiving the ad schedule and ad creatives, the server generates a playback sequence template based on the start and end times, ad segment identifiers, and creative file identifiers in the ad schedule. The ad creatives are either video files to be played or ad files generated from multiple images in a set order. The server parses the ad creatives and divides them into multiple ad segments according to the ad segment identifiers. Each ad segment corresponds to an ad segment identifier and a specified playback time period.

[0026] The server analyzes each ad segment frame by frame and selects keyframes from different playback moments. Keyframes can be selected at fixed time intervals or according to the changes in the ad's visuals. For ad segments with minor visual changes, the server increases the time interval between keyframes; for ad segments with rapid visual changes, the server shortens the time interval between keyframes. This ensures that the playback timing template covers the main visual changes during ad playback while avoiding excessive redundant sampling of static images.

[0027] In each keyframe, the server determines at least two sampling regions. These regions are rectangular or polygonal areas within the keyframe, their positions determined by the coordinates of their top-left corner, width, and height, or by the coordinates of multiple corner points. Sampling regions are preferentially selected from areas of the advertising material that exhibit stable visual changes. Solid-color background areas, scrolling text areas, and edge whitespace areas are not prioritized for sampling. Specifically, solid-color background areas are determined by ensuring the pixel brightness variance within the area is less than the brightness variance threshold configured by the server; scrolling text areas are determined by identifying high-contrast text edges with continuous horizontal movement in consecutive keyframes; and edge whitespace areas are determined based on continuous low-texture areas at the edges of the advertising material.

[0028] The server extracts regional image features for each sampling region. These features include the region's average brightness value, color histogram, edge gradient value, and texture variation value. The average brightness value characterizes the overall brightness of the sampling region; the color histogram characterizes the color distribution; the edge gradient value characterizes contour changes within the sampling region; and the texture variation value characterizes local changes in pixel grayscale distribution within the sampling region. The server associates and stores keyframe time points, sampling region locations, and corresponding regional image features with advertising segment identifiers to form a playback timing template. The server also sequentially numbers the regional image features of corresponding sampling regions in adjacent keyframes and uses this sequential numbering as the inter-frame change order in the playback timing template.

[0029] In this embodiment, the playback timing template is not formed by additionally overlaying visible markers, spectral fingerprints, or watermarks on the advertisement screen, but rather by extracting regional screen features from the original advertisement material. This processing method keeps the original advertisement material screen unchanged, while enabling the server to determine the actual display state of the screen itself based on the screen features of the advertisement material itself.

[0030] When the broadcast control terminal plays advertising materials, it uses the advertising segment switching time, decoding frame output time, and rendering frame submission time as recording nodes to generate broadcast control output evidence. The broadcast control output evidence includes the current advertising segment identifier, decoding frame timestamp, rendering queue status, video output interface status, and screen control card feedback status.

[0031] The current ad segment identifier indicates the ad segment currently being played on the broadcast control terminal. The decoded frame timestamp is the playback time of the material corresponding to the video frame output by the decoder. The rendering queue status includes the number of frames to be rendered, the number of submitted rendering frames, and the number of consecutive frames not updated. The number of frames to be rendered indicates the number of video frames that have been decoded but not yet submitted for display; the number of submitted rendering frames indicates the number of video frames that have been submitted to the video output link; and the number of consecutive frames not updated indicates the number of times the rendering queue has not added any new rendering frames within a consecutive sampling period.

[0032] The video output interface status includes interface connection status, output resolution, output refresh rate, and interface handshake status. Interface connection status indicates whether the video output interface between the broadcast control terminal and the screen control card is connected; output resolution indicates the current video resolution output by the broadcast control terminal; output refresh rate indicates the current video refresh rate output by the broadcast control terminal; and interface handshake status indicates whether the handshake between the video output interface and the subsequent display link is complete. The screen control card feedback status includes the control card's online status, receiver card number, and cabinet partition number. For outdoor digital advertising screens formed by splicing multiple cabinets, the cabinet partition number is used to determine the screen area corresponding to the anomaly.

[0033] The image acquisition unit captures screen displays according to the sampling period configured by the server. The sampling period is configured based on the frame rate of the advertising material, the frame rate of the image acquisition unit, and the processing capability of the broadcast control terminal. The sampling period is not required to be equal to the video frame period, but it should ensure that at least one keyframe of the same advertising segment can be captured during playback.

[0034] After the image acquisition unit captures the screen display, the server or edge processing unit determines the effective display area of ​​the screen based on the screen border position or preset screen corner coordinates. If there is a tilt angle between the image acquisition unit and the screen body, perspective correction is performed on the effective display area to establish a correspondence between the screen area in the acquired image and the screen coordinates in the playback timing template. After perspective correction is completed, the corresponding display sampling area is extracted according to the sampling area position in the playback timing template.

[0035] The evidence displayed on the screen includes the acquisition timestamp, the corrected image of the effective display area, the location of the sampled display area, the average brightness value of the sampled display area, the color histogram, the edge gradient value, the texture variation value, the ambient illuminance value, and the location of the occluded area. The ambient illuminance value is acquired by an ambient light sensor; the location of the occluded area is determined based on the brightness abrupt change boundaries and color differences in the displayed image.

[0036] Specifically, the edge processing unit searches for brightness abrupt change boundaries of consecutive pixels in the display sampling area, and detects a range of consecutive pixels within these boundaries whose color difference is less than the color difference threshold configured by the server. When this range of consecutive pixels covers a certain area, it is defined as an occlusion area. The color difference is determined using the distance between pixel color vectors in the same color space. The color difference threshold is configured during the on-site installation and debugging phase by collecting images of normally playing screens, images without obvious occlusion, and images with occlusion, and is stored in the server's device configuration table.

[0037] When the overlap area between the displayed sampling area and the occluded area reaches the overlap ratio threshold configured by the server, an occlusion marker is set for that displayed sampling area. In subsequent consistency checks of the actual display, the displayed sampling area with the occlusion marker is not considered a valid comparison area for the current sampling period, to reduce false positives caused by pedestrians, vehicles, tree shadows, raindrops, or stains. The overlap ratio threshold is determined during the installation and debugging phase based on the proportion of the displayed sampling area covered by occlusion objects in the occlusion samples and is written into the device configuration table.

[0038] The server uniformly converts the server's advertising scheduling time, the local playback time of the broadcast control terminal, and the acquisition time of the image acquisition unit into a single time point on the same playback timeline. For the data generated by the broadcast control terminal and the image acquisition unit, the server corrects it based on the time offset obtained during the time synchronization process.

[0039] The server determines the ad segment identifier and playback time period to be played in the ad schedule at the current time point on the same playback timeline, and compares it with the current ad segment identifier uploaded by the broadcast control terminal to obtain a schedule consistency result. If the current ad segment identifier is consistent with the ad segment identifier at the corresponding time point in the playback sequence template, and the current time point falls within the playback time period of the ad segment, then the schedule consistency result is consistent; otherwise, the schedule consistency result is inconsistent.

[0040] The server determines the output consistency result based on whether the decoded frame timestamp increases with playback time, whether the rendering queue status is updated, and whether the video output interface remains connected. If the decoded frame timestamp increases with the sampling period, there are submitted rendering frame updates in the rendering queue, and the video output interface remains connected, then the output consistency result is consistent; otherwise, the output consistency result is inconsistent.

[0041] The server determines the true display consistency result based on the feature difference between the display sampling area features in the screen display evidence and the regional image features of the corresponding sampling area in the playback timing template. Specifically, the server compares the average brightness value, color histogram, edge gradient value, and texture change value of the display sampling area with the regional image features corresponding to the same advertisement segment, the same keyframe time point, and the same sampling area position in the playback timing template. If the feature difference between the display sampling area features and the regional image features of the corresponding sampling area in the playback timing template falls within the feature difference range configured by the server, the true display consistency result of the display sampling area is consistent; otherwise, the true display consistency result of the display sampling area is inconsistent. For display sampling areas marked with occlusion, the server does not use this area in the true display consistency judgment during the current sampling period.

[0042] When determining the feature difference range, the server collects multiple sets of screen display evidence for the same advertising creative under normal playback conditions and calculates the feature difference between each set of screen display evidence and the playback sequence template. The server uses the feature difference distribution formed under normal playback conditions as a baseline range, and combines this baseline range with the installation angle of the image acquisition unit, screen brightness level, and ambient illuminance variation range to form the final feature difference range. If the advertising creative is mainly characterized by color changes, the server increases the proportion of the color histogram in the feature difference judgment; if the advertising creative is mainly characterized by text outlines or product edge changes, the server increases the proportion of edge gradient values ​​in the feature difference judgment; if the advertising creative is mainly characterized by texture changes, the server increases the proportion of texture change values ​​in the feature difference judgment. Related configurations are stored in the server's device configuration table or advertising creative configuration table.

[0043] The server generates anomaly attribution results based on the conflict types among scheduling consistency results, output consistency results, and actual display consistency results. Anomaly attribution results include material scheduling anomalies, material caching anomalies, decoding and rendering anomalies, video output link anomalies, screen receiving link anomalies, partial screen display anomalies, and outdoor environmental interference.

[0044] When the current ad segment identifier is inconsistent with the ad segment identifier at the corresponding time point in the playback timing template, it indicates that the ad segment currently being played by the broadcast control terminal is inconsistent with the ad scheduling requirements, and the server-generated material scheduling is abnormal.

[0045] When the current ad segment identifier is consistent, but the local ad creative verification value is inconsistent with the ad creative verification value recorded on the server, it indicates that the ad creative stored locally on the broadcast control terminal is inconsistent with the ad creative published by the server, and the server-generated creative cache is abnormal. The local ad creative verification value is obtained using a hash verification method, while the ad creative verification value recorded on the server is generated and stored when the ad creative is published.

[0046] The server generates a decoding rendering anomaly when the timestamp of the decoded frame does not increase within N consecutive sampling periods, or when the number of consecutive unupdated frames in the rendering queue reaches the frame count threshold configured by the server. Here, N is a positive integer configured by the server based on the ad creative frame rate and sampling period. N is determined by the allowed pause duration and sampling period configured by the server, which is based on the ad creative frame rate, the playback control terminal's decoding buffer length, and the video output link buffer duration. The frame count threshold corresponding to the number of consecutive unupdated frames in the rendering queue is determined based on the ad creative frame rate, the playback control terminal's rendering buffer length, and the allowed latency of the video output link. For the same playback control terminal, the server records the update interval of the rendering queue under normal playback conditions during the installation and debugging phase, and configures the upper limit of the allowed update interval above the normal update interval as the frame count threshold. This frame count threshold is used to distinguish between brief buffering and decoding rendering anomalies.

[0047] When both the scheduling consistency result and the output consistency result are consistent, but the actual display consistency result is inconsistent, and the video output interface status is abnormal, the server generates a video output link anomaly. An abnormal video output interface status includes an open interface connection, a failed interface handshake, an output resolution inconsistent with the screen resolution recorded in the playback timing template, and an output refresh rate inconsistent with the screen refresh rate recorded in the playback timing template. At this point, the broadcast control terminal has already played according to the schedule and completed decoding output, but the actual screen display is inconsistent with the expected playback image, and the video output interface is abnormal; therefore, the anomaly is attributed to the video output link.

[0048] When the scheduling consistency result, output consistency result, and video output interface status are all consistent, but the actual display consistency result is inconsistent, and the screen control card feedback status is abnormal, the server generates a screen receiving link anomaly. An abnormal screen control card feedback status includes control card offline, missing receiver card number, abnormal cabinet partition number, and receiver card feedback time exceeding the server-configured feedback time threshold. The feedback time threshold is determined based on the communication cycle between the broadcast control terminal and the screen control card. The server records the receiver card feedback time multiple times while the screen control card is normally online and configures the feedback time threshold based on the upper limit of the normal feedback time. When the receiver card feedback time exceeds this threshold, the server determines the screen control card feedback status is abnormal; when the communication link is restored and the receiver card feedback time falls back within the feedback time threshold, the server updates the corresponding receiver card status to online.

[0049] When the scheduling consistency result, output consistency result, video output interface status, and screen control card feedback status are all consistent, and the actual display consistency result is inconsistent only in display sampling areas within the effective display area of ​​the screen (less than the total display sampling area), the server generates a partial screen display anomaly. This anomaly corresponds to a situation where the display of a partial area of ​​the screen does not match the advertising material, such as partial cabinet display anomaly, partial image loss, or partial color anomaly.

[0050] When the displayed sampling area, which shows an inconsistent consistency result, coincides with an obstructed area, or when the ambient illuminance value corresponding to the displayed sampling area exceeds the illuminance threshold configured by the server, the server generates outdoor environmental interference. The illuminance threshold is configured during the equipment installation and commissioning phase based on the screen's installation orientation, the exposure capability of the image acquisition unit, and the sampling samples collected under strong daylight conditions. Outdoor environmental interference is not directly considered a playback link failure.

[0051] When determining whether a screen display anomaly is due to localized display issues or outdoor environmental interference, the server first checks if there are any obstruction or strong light interference markers in the display sampling areas where the actual display consistency result is inconsistent. If obstruction or strong light interference markers are present, outdoor environmental interference is prioritized. If no obstruction or strong light interference markers are present, and the inconsistent display sampling area is less than the total display sampling area, then a screen display anomaly is generated. This determination order prevents the same abnormal area from being classified as both a screen display anomaly and outdoor environmental interference simultaneously.

[0052] The server or broadcast control terminal executes the corresponding repair strategy based on the anomaly attribution results.

[0053] When the anomaly attribution result is an issue with the scheduling or caching of advertising materials, the broadcast control terminal verifies or re-fetches the local advertising materials. Specifically, the broadcast control terminal calculates the verification value of the current advertising segment and compares it with the verification value recorded on the server. If they do not match, the broadcast control terminal deletes the current local advertising segment and re-fetches the corresponding advertising material from the server based on the advertising segment identifier. After re-fetching, the broadcast control terminal determines the current playback time based on the same playback timeline and continues playing the corresponding advertising segment from that time.

[0054] When the anomaly is attributed to a decoding / rendering error, the broadcast control terminal restarts the decoding or rendering process and repositions itself to the current scheduled playback time. Specifically, the broadcast control terminal retains the current ad segment identifier, the current scheduled playback time, and the video output interface parameters. After the decoding or rendering process restarts, the broadcast control terminal loads the corresponding ad material according to the current ad segment identifier and sets the decoding start position according to the current scheduled playback time. This approach avoids playback schedule shifts caused by replaying from the ad segment's start position.

[0055] When the anomaly attribution result indicates a video output link failure, the broadcast control terminal reinitializes the video output interface. During reinitialization, the broadcast control terminal sequentially performs interface closure, interface re-enumeration, output resolution configuration, output refresh rate configuration, and interface handshake status confirmation. After the interface handshake status confirmation is complete, the broadcast control terminal resumes video output and uploads the new interface connection status, output resolution, output refresh rate, and interface handshake status to the server.

[0056] When the anomaly attribution result indicates a screen receiving link failure, the broadcast control terminal reissues the screen control card configuration parameters. Based on the screen control card's feedback status, the broadcast control terminal determines the abnormal receiving card number or the abnormal cabinet partition number and issues mapping parameters to the corresponding receiving card or cabinet partition. These mapping parameters include the receiving card address, cabinet partition coordinates, display start coordinates, and display size. After reissuance, the broadcast control terminal reads the screen control card's feedback status to confirm whether the corresponding receiving card or cabinet partition has returned to online status.

[0057] When the anomaly attribution result indicates a partial screen display anomaly, the broadcast control terminal or server generates a fault record containing the location of the abnormal area. The fault record includes the advertisement segment identifier, acquisition timestamp, location of the abnormal display sampling area, anomaly attribution result, video output interface status, and screen control card feedback status. For partial screen display anomalies, this embodiment does not directly perform a full system restart to avoid interrupting full-screen playback due to a localized anomaly.

[0058] When the anomaly attribution result is outdoor environmental interference, the server reduces the judgment weight of screen display evidence in the current sampling period and does not perform playback repair. Reducing the judgment weight means that display sampling areas marked with occlusion or strong light interference in the current sampling period are not included in the actual display consistency judgment, or their participation ratio is reduced when multiple display sampling areas are judged comprehensively. This processing method is used to distinguish between actual playback link failures and image acquisition deviations caused by external environment.

[0059] After the repair strategy is implemented, the server re-collects broadcast control output evidence and screen display evidence, and again performs a consistency check with the playback timing template under the same playback timeline. The server re-obtains scheduling consistency results, output consistency results, and actual display consistency results over multiple consecutive sampling periods.

[0060] When the re-obtained scheduling consistency result, output consistency result, and actual display consistency result are all consistent, the server records the repair completion status and writes the repair action, repair time point, anomaly attribution result, and review result into the fault log.

[0061] When both the scheduling consistency result and the output consistency result are consistent, but the actual display consistency result is still inconsistent, it indicates that the broadcast control terminal has resumed output according to the schedule, but the actual display on the screen has not yet recovered. At this time, the server updates the anomaly attribution result to video output link anomaly, screen receiving link anomaly, partial screen display anomaly, or outdoor environmental interference, and executes the corresponding repair strategy or generates a fault record again according to the updated anomaly attribution result.

[0062] If the scheduling consistency result is still inconsistent, the server prioritizes handling ad scheduling or material caching issues; if the output consistency result is still inconsistent, the server prioritizes handling decoding rendering anomalies or video output link anomalies. Through this review process, successful repair is not judged by process restart completion, but by the reconciliation of three types of evidence: ad scheduling, broadcast control output, and actual screen display.

[0063] In a specific application scenario, an outdoor digital advertising screen plays advertisement segment A, scheduled to play from 10:00:00 to 10:00:30. When the advertising material is released, the server selects multiple keyframes from advertisement segment A, and within each keyframe, selects multiple sampling regions, extracting the average brightness value, color histogram, edge gradient value, and texture change value of each region to form a playback timing template. The playback control terminal uploads playback control output evidence at 10:00:10, showing that the current advertisement segment is identified as A, the decoded frame timestamp is continuously increasing, the rendering queue status is normal, and the video output interface remains connected. After the image acquisition unit captures the screen display at the same time, the server discovers that the feature differences between multiple display sampling regions in the effective display area and the corresponding regions in the playback timing template exceed the feature difference range configured by the server, and the video output interface handshake status is failed. Based on this, the server generates a video output link anomaly and controls the playback control terminal to reinitialize the video output interface. After re-initialization, the server collects broadcast control output evidence and screen display evidence again. If the scheduling consistency result, output consistency result, and actual display consistency result are all restored to consistency, the repair completion status is recorded.

[0064] In another specific application scenario, the current advertisement segment identifier, decoding frame timestamp, and rendering queue status uploaded by the broadcast control terminal are all normal, as are the video output interface status and screen control card feedback status. However, only two display sampling areas in the lower right corner of the screen captured by the image acquisition unit are inconsistent with the playback timing template. If these two display sampling areas are not marked as obstructed areas, and the ambient illuminance value does not exceed the illuminance threshold configured by the server, the server generates a partial screen display anomaly and records the location of the abnormal display sampling area without performing a full system restart. If these two display sampling areas overlap with the obstructed area, the server generates outdoor environmental interference, reduces the judgment weight of the corresponding screen display evidence in this sampling period, and does not perform playback repair.

[0065] In this embodiment, anomaly detection is not based solely on player process status, CPU usage, or a single screen image. Instead, it maps three types of evidence—advertisement scheduling, broadcast control output, and actual screen display—to the same playback timeline and determines the anomaly attribution result based on the conflict relationships between these three types of evidence. Furthermore, repair is not performed by uniformly restarting the broadcast control terminal or the screen itself, but rather by repairing the corresponding links based on the anomaly attribution result. Through this process, anomalies in material scheduling, material caching, decoding and rendering, video output links, screen receiving links, partial screen display, and outdoor environmental interference can be identified and handled separately.

[0066] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for monitoring and repairing the playback status of outdoor digital advertising screens, characterized in that, Includes the following steps: S1. The server generates a playback sequence template based on the advertising schedule and advertising materials. The playback sequence template includes advertising segment identifiers, the time period to be played, key frame time points, sampling area positions, and regional image features extracted from the original images of the advertising materials. S2. When the broadcast control terminal plays advertising materials, it collects broadcast control output evidence, which includes the current advertising segment identifier, decoding frame timestamp, rendering queue status, video output interface status, and screen control card feedback status. S3. The screen display image is captured by the image acquisition unit set at the corresponding position of the outdoor digital advertising screen, the effective display area of ​​the screen is extracted, and the screen display evidence is determined according to the ambient illuminance and the occluded area in the display image. S4. Map the playback timing template, broadcast control output evidence, and screen display evidence to the same playback timeline to obtain the scheduling consistency result, output consistency result, and actual display consistency result, respectively. S5. Generate anomaly attribution results based on the conflict types between scheduling consistency results, output consistency results, and actual displayed consistency results; S6. Execute the corresponding repair strategy according to the abnormal attribution results; S7. After the repair strategy is executed, re-collect the broadcast control output evidence and screen display evidence, and once again perform a consistency judgment with the playback timing template under the same playback time axis. Determine whether the repair is complete based on the re-obtained scheduling consistency result, output consistency result and actual display consistency result.

2. The method for monitoring and repairing the playback status of an outdoor digital advertising screen according to claim 1, characterized in that, In step S1, when the server generates the playback timing template, it parses the advertising material frame by frame and divides the advertising material into multiple advertising segments according to the advertising segment identifier. For each advertisement segment, select keyframes located at different playback times, and determine at least two sampling regions in each keyframe; For each sampling region, regional image features are extracted, including the region's average brightness value, region's color histogram, region's edge gradient value, and region's texture change value. The key frame time points, sampling area locations, and area image features are associated and stored with the corresponding advertising segment identifiers to form a playback timing template.

3. The method for monitoring and repairing the playback status of an outdoor digital advertising screen according to claim 2, characterized in that, The sampling area avoids the solid color background area, the scrolling subtitle area and the edge white space area in the advertising material, and each sampling area in the same key frame is located in a different partition of the effective display area of ​​the screen. The server assigns sequential numbers to the regional image features of corresponding sampling areas in adjacent keyframes, and uses these sequential numbers as the inter-frame change order in the playback timing template.

4. The method for monitoring and repairing the playback status of an outdoor digital advertising screen according to claim 3, characterized in that, In step S2, when the broadcast control terminal plays advertising materials, it generates broadcast control output evidence by using the advertising segment switching time, the decoding frame output time, and the rendering frame submission time as recording nodes. The timestamp of the decoded frame is the playback time of the material when the decoder outputs the video frame. The rendering queue status includes the number of frames to be rendered, the number of frames submitted for rendering, and the number of consecutive frames that have not been updated. The video output interface status includes the interface connection status, output resolution, output refresh rate, and interface handshake status. The screen control card feedback status includes the control card online status, the receiving card number, and the cabinet partition number.

5. The method for monitoring and repairing the playback status of an outdoor digital advertising screen according to claim 4, characterized in that, In step S3, after the image acquisition unit acquires the screen display image, it determines the effective display area of ​​the screen based on the position of the screen border or the preset screen corner coordinates. Perform perspective correction on the effective display area of ​​the screen, and extract the corresponding display sampling area according to the sampling area position in the playback timing template; The screen display evidence includes the acquisition timestamp, the corrected image of the effective display area of ​​the screen, the location of the display sampling area, the average brightness value of the display sampling area, the color histogram, the edge gradient value, the texture change value, the ambient illuminance value, and the location of the occluded area.

6. The method for monitoring and repairing the playback status of an outdoor digital advertising screen according to claim 5, characterized in that, The ambient illuminance is collected by an ambient light sensor located near the outdoor digital advertising screen. The occlusion area is determined based on the brightness change boundary of consecutive pixels within the display sampling area, and the range of consecutive pixels located within the brightness change boundary and whose color difference is less than the color difference threshold configured by the server. When the overlap area between the display sampling area and the occlusion area reaches the overlap ratio threshold configured by the server, an occlusion mark is set for the display sampling area, and the display sampling area with the occlusion mark is excluded in the consistency judgment in step S4.

7. The method for monitoring and repairing the playback status of an outdoor digital advertising screen according to claim 6, characterized in that, In step S4, the advertising schedule time of the server, the local playback time of the broadcast control terminal, and the acquisition time of the image acquisition unit are uniformly converted into time points on the same playback time axis. The scheduling consistency result is determined by the ad segment identifier corresponding to the current time point and the time period to be played. The consistency of the output is determined by whether the timestamp of the decoded frame increases with the playback time, whether the rendering queue status is updated, and whether the video output interface status remains connected. The consistency of the actual display is determined by whether the feature difference between the display sampling area features in the screen display evidence and the corresponding sampling area area features in the playback timing template falls within the feature difference range configured by the server.

8. The method for monitoring and repairing the playback status of an outdoor digital advertising screen according to claim 7, characterized in that, In step S5, the abnormal attribution results include abnormal material scheduling, abnormal material caching, abnormal decoding and rendering, abnormal video output link, abnormal screen receiving link, abnormal partial screen display, and outdoor environmental interference. Specifically, if the current ad segment identifier is inconsistent with the ad segment identifier at the corresponding time point in the playback timing template, the generated material scheduling is abnormal. When the current ad segment identifier is consistent and the local ad creative verification value is inconsistent with the ad creative verification value recorded on the server, the generated creative cache is abnormal; A decoding rendering error is generated when the timestamp of the decoded frame does not increase within N consecutive sampling periods, or when the number of consecutive unupdated frames in the rendering queue reaches the frame rate threshold configured by the server. N is a positive integer configured by the server based on the frame rate and sampling period of the advertising material.

9. The method for monitoring and repairing the playback status of an outdoor digital advertising screen according to claim 8, characterized in that, In step S5, when the scheduling consistency result and the output consistency result are both consistent, the actual display consistency result is inconsistent, and the video output interface status is abnormal, a video output link abnormality is generated. When the scheduling consistency result, output consistency result, and video output interface status are all consistent, but the actual display consistency result is inconsistent, and the screen control card feedback status is abnormal, an abnormal screen receiving link is generated. When the scheduling consistency result, output consistency result, video output interface status, and screen control card feedback status are all consistent, and the actual display consistency result is inconsistent only in the display sampling area of ​​the effective display area of ​​the screen (less than the total display sampling area), a partial screen display anomaly is generated. Outdoor environmental interference is generated when the location of the display sampling area that shows an inconsistent display consistency result coincides with the location of the occluded area, or when the ambient illuminance value corresponding to the display sampling area exceeds the illuminance threshold configured by the server. The abnormal status of the video output interface includes: the interface connection status is disconnected, the interface handshake status is failed, the output resolution is inconsistent with the screen resolution recorded in the playback timing template, and the output refresh rate is inconsistent with the screen refresh rate recorded in the playback timing template. The abnormal feedback status of the screen control card includes control card offline, missing receiver card number, abnormal cabinet partition number, and receiver card feedback time exceeding the feedback time threshold configured by the server.

10. The method for monitoring and repairing the playback status of an outdoor digital advertising screen according to claim 9, characterized in that, In step S6, when the anomaly attribution result is an abnormal material scheduling or an abnormal material cache, the broadcast control terminal verifies the local advertising material or re-fetches it. When the anomaly attribution result is a decoding or rendering anomaly, the broadcast control terminal restarts the decoding or rendering process and locates the current time point when it should be played. When the anomaly attribution result is a video output link anomaly, the broadcast control terminal reinitializes the video output interface. When the anomaly attribution result is that the screen receiving link is abnormal, the broadcast control terminal re-issues the screen control card configuration parameters. When the anomaly attribution result is a partial display anomaly on the screen, a fault record containing the location of the anomaly area is generated. When the abnormal attribution result is outdoor environmental interference, the judgment weight of the screen display evidence in the current sampling period is reduced, and playback repair is not performed; When reinitializing the video output interface, the broadcast control terminal sequentially performs interface closing, interface re-enumeration, output resolution configuration, output refresh rate configuration, and interface handshake status confirmation. When re-issuing the configuration parameters of the screen control card, the broadcast control terminal determines the abnormal receiving card number or abnormal cabinet partition number based on the feedback status of the screen control card, and issues mapping parameters to the corresponding receiving card or corresponding cabinet partition. The mapping parameters include the receiving card address, cabinet partition coordinates, display start coordinates, and display size. In step S7, when the re-obtained schedule consistency result, output consistency result, and actual display consistency result are all consistent, the repair completion status is recorded; When the scheduling consistency result and the output consistency result are both consistent, but the actual display consistency result is still inconsistent, the anomaly attribution result is updated to video output link anomaly, screen receiving link anomaly, partial screen display anomaly, or outdoor environmental interference, and the corresponding repair strategy is executed or a fault record is generated according to the updated anomaly attribution result.

Images