Multi-screen based message display method, system and intelligent terminal

Abstract

The present invention discloses a message display method, system and intelligent terminal based on multiple screens, and relates to the technical field of multiple screen message display, including the following steps: for the monitoring screen of the repeated area, the time interval between two adjacent frames of monitoring video screen on one of the screens is obtained, the data of the interval length between two adjacent frames obtained by all screens in the repeated monitoring area are collected, a complete data set is established, and the frame interval length in the data set is analyzed for differences. The present invention preliminarily determines that the video is not synchronized by analyzing the time interval between adjacent frames, and then captures the monitoring screen at the same time to perform cosine similarity and peak signal-to-noise ratio analysis, accurately evaluates the screen similarity, identifies and screens out abnormal screens, and ensures the synchronization and consistency of the remaining screens. This method improves the reliability and real-time performance of the monitoring system, enabling staff to accurately piece together the timeline of events and better judge the cause, development and outcome of the event.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-screen message display, and in particular to a multi-screen-based message display method, system and intelligent terminal. Background Art

[0002] The multi-screen message display system is a technical architecture designed to present information simultaneously through multiple display screens to improve the effectiveness and efficiency of information transmission. This system is often used in public places, meeting rooms, control centers, retail environments and other places where real-time information display is required. It can distribute different types of information (such as text, pictures, videos, real-time data, etc.) to different screens to ensure that information can be conveyed to the target audience more widely and effectively. For example, in a control center, multiple screens can simultaneously display surveillance videos, system status, alarm information, and operation guides, helping staff to quickly obtain comprehensive information and improve the accuracy and response speed of decision-making.

[0003] The implementation of such a system usually involves the integration of hardware and software, including high-resolution display screens, image processors, control software, and network connections. Through the control software, users can flexibly configure and manage the display content on each screen to achieve synchronous or independent playback of content. In addition, the network connection enables the system to interact with external data sources in real time and dynamically update the display content. For example, in a retail store, a multi-screen system can display promotional information, product inventory, and customer feedback in real time, helping the store to adjust its marketing strategy in a timely manner and improve customer experience and sales performance. In short, the multi-screen-based message display system significantly improves the efficiency and effectiveness of information dissemination through a variety of information display methods, and is suitable for various application scenarios that require real-time information display and interaction.

[0004] When conducting video surveillance based on a multi-screen message display system, for surveillance images of repeated areas, if the surveillance videos displayed on multiple screens cannot be synchronized and the system itself cannot detect it, the monitoring system relies on continuous video streams to track and analyze the entire process of the event. If the video is not synchronized, it will be difficult for staff to piece together the complete timeline of the event and cannot accurately determine the cause, development and outcome of the event. This poses a great obstacle to work such as accident investigation and crime analysis, and may lead to wrong conclusions and ineffective disposal plans.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] The purpose of the present invention is to provide a message display method, system and intelligent terminal based on multiple screens. By obtaining the time interval between two adjacent frames of monitoring video images and performing difference analysis on the data of all screens in the repeated monitoring area, the potential abnormality of video asynchrony is preliminarily judged. Subsequently, by capturing the monitoring images at the same time, pairwise similarity analysis is performed, and similarity indicators such as cosine similarity and peak signal-to-noise ratio are used to accurately evaluate the image similarity, identify and screen out abnormal images, and ensure the synchronization and consistency of the remaining images. This improves the reliability and real-time performance of the monitoring system, enables staff to accurately piece together the event timeline, and better judge the cause, development and outcome of the event, so as to solve the problems in the above-mentioned background technology.

[0007] In order to achieve the above object, the present invention provides the following technical solution: a message display method based on multiple screens, comprising the following steps:

[0008] For the surveillance images in the repeated area, the time interval between two adjacent frames of surveillance video images on one of the screens is obtained, and the data on the interval between two adjacent frames obtained from all screens in the repeated surveillance area are collected to establish a complete data set. The frame interval duration in the data set is analyzed for differences, and a preliminary judgment is made as to whether there is a potential anomaly of asynchrony in the surveillance video images displayed on multiple screens.

[0009] After initially determining the potential anomaly of video asynchrony, obtain several surveillance images of the same time in the repeated surveillance area, and ensure that the comparative analysis is based on the image content at the same time by capturing frames at the same time point;

[0010] Perform pairwise similarity analysis on several surveillance images and calculate the similarity between each pair of surveillance images;

[0011] The similarity values ​​after the pairwise similarity analysis are used to establish an analysis set, and the similarity of each monitoring screen with other monitoring screens at the same time is summed up to obtain the similarity sum value;

[0012] The calculated similarity sum values ​​are sorted in descending order, and the monitoring picture corresponding to the minimum similarity sum value is marked and the monitoring picture is screened out, thereby removing the monitoring picture with potential abnormalities of video asynchrony.

[0013] Preferably, the interval time data of two adjacent frames obtained by all screens in the repeated monitoring area are collected to establish a complete data set, which is marked as C and the interval time is marked as Interval v , then: C = {Interval v}, where v represents the number of each monitoring screen in the repeated monitoring area, v = 1, 2, 3, 4, ..., V, and V is a positive integer.

[0014] Perform difference analysis on the frame interval duration in the data set to generate the interval duration variation coefficient. Use the interval duration variation coefficient to preliminarily determine whether there is a potential anomaly of asynchrony in the surveillance video images displayed on multiple screens. The specific steps are as follows:

[0015] The mean and standard deviation of the interval duration are calculated by the interval duration in the data set, and the mean and standard deviation of the interval duration are calibrated as μ Interval and σ Interval ;

[0016] By the mean interval length μ Interval and the standard deviation of the interval duration σ Interval Calculate the coefficient of variation of the interval duration. The expression is: In the formula, Interval 变异 represents the coefficient of variation of interval duration.

[0017] Preferably, the interval duration variation coefficient generated after the difference analysis of the frame interval duration in the data set is compared and analyzed with the preset interval duration variation coefficient reference threshold, and a preliminary judgment is made as to whether there is a potential abnormality of asynchrony in the monitoring video images displayed on multiple screens. The specific judgment is as follows:

[0018] If the interval duration variation coefficient is greater than or equal to the interval duration variation coefficient reference threshold, a potential abnormality signal is generated, indicating that the surveillance images of the repeated area have a potential abnormality of video asynchrony;

[0019] If the interval duration variation coefficient is less than the interval duration variation coefficient reference threshold, an efficient monitoring signal is generated, indicating that the monitoring screen of the repeated area can achieve efficient video synchronization.

[0020] Preferably, a pairwise similarity analysis is performed on several monitoring screens, and the similarity between each pair of monitoring screens is calculated. The similarity indicators include cosine similarity and peak signal-to-noise ratio. After obtaining the cosine similarity and peak signal-to-noise ratio generated by pairwise similarity analysis of several monitoring screens, the cosine similarity and peak signal-to-noise ratio are formulated and analyzed to generate a similarity index, and the similarity of two monitoring screens is evaluated by the similarity index.

[0021] Preferably, a pairwise similarity analysis is performed on a plurality of monitoring images, and the cosine similarity between each pair of monitoring images is calculated. The specific steps are as follows:

[0022] Convert each monitoring screen into a vector. Suppose the monitoring screen is a grayscale image of size m×n, flatten it into a vector of length m×n, and suppose the vectors corresponding to the two monitoring screens are: A=(a i )=(a 1 , a2 , a 3 , …, a mn ), B=(b i )=(b 1 , b 2 , b 3 , …, b mn ), where vector A and vector B are vectors of two monitoring images, respectively. i and b i are the grayscale values ​​of the i-th pixel in the picture;

[0023] Calculate the dot product of vector A and vector B. The dot product represents the sum of the products of two vectors in corresponding dimensions. The calculation expression is:

[0024] Calculate the magnitude of vector A and vector B. The magnitude of a vector is the square root of the sum of the squares of all its components. The calculation expression is:

[0025] Calculate the cosine similarity between vector A and vector B. Cosine similarity is the dot product divided by the product of the two vector moduli. The value range is between [-1, 1] and represents the cosine value of the angle between the two vectors. The calculation expression is: Cosine Similarity(A, B) represents the cosine similarity between vector A and vector B.

[0026] Preferably, a pairwise similarity analysis is performed on a plurality of monitoring images, and the peak signal-to-noise ratio between each pair of monitoring images is calculated. The specific steps are as follows:

[0027] Calculate the mean square error between the two images. The mean square error is used to measure the difference in corresponding pixel values ​​between the two images. The calculation expression is: x(i', j') and y(i', j') are the pixel values ​​of image x and image y at the i'th row and j'th column respectively, M and N are the width and height of the image, that is, the resolution of the image, MN is the total number of pixels in the image, and mse(x, y) is the mean square error between image x and image y;

[0028] Calculate the peak signal-to-noise ratio. The expression is: Where PSNR(x, y) represents the peak signal-to-noise ratio, max I is the maximum pixel value of the image.

[0029] Preferably, the cosine similarity and peak signal-to-noise ratio are analyzed in a formula to generate a similarity index: Similarityτ = r 1 *Cosine SImilarity(A,B)+r 2*PSNR(x, y), where r 1 、r 2 are the preset proportional coefficients of cosine similarity CosineSimilarity(A, B) and peak signal-to-noise ratio PSNR(x, y), and r 1 、r 2 Both are greater than 0.

[0030] Preferably, the similarity index after the pairwise similarity analysis is used to establish an analysis set, and the similarity between each monitoring screen and other monitoring screens at the same time is summed up and calculated. The calculation process is:

[0031] For a given time t, there are w monitoring images for the repeated area. Let the similarity index between each pair of monitoring images p and q be S pq , then for monitoring screen p, the similarity sum calculation formula between it and all other monitoring screens is: In the formula, Sum p It represents the sum of similarities between monitoring picture p and all other monitoring pictures.

[0032] The message display system based on multiple screens includes a data collection and preliminary analysis module, a synchronous data capture module, a similarity analysis module, a similarity sum value calculation module and an abnormal screening module;

[0033] The data collection and preliminary analysis module obtains the time interval between two adjacent frames of surveillance video on one of the repeated surveillance images, collects the data on the interval between two adjacent frames obtained on all screens in the repeated surveillance area, establishes a complete data set, performs difference analysis on the frame intervals in the data set, and preliminarily determines whether there are potential anomalies of asynchrony in the surveillance video images displayed on multiple screens;

[0034] The synchronous data capture module initially determines the potential anomaly of video asynchrony and obtains several surveillance images of the same time in the repeated monitoring area. By capturing frames at the same time point, it ensures that the comparative analysis is based on the image content at the same time.

[0035] The similarity analysis module performs pairwise similarity analysis on a number of monitoring images and calculates the similarity between each pair of monitoring images;

[0036] The similarity sum value calculation module establishes an analysis set of similarity values ​​after pairwise similarity analysis, and performs sum calculation on the similarity of each monitoring screen with other monitoring screens at the same time to obtain a similarity sum value;

[0037] The abnormal screening module sorts the calculated similarity sum values ​​in descending order, marks the monitoring picture corresponding to the minimum similarity sum value, and screens out the monitoring picture, thereby removing the monitoring picture with potential abnormalities of video asynchrony.

[0038] A multi-screen message display intelligent terminal includes a multi-screen message display system.

[0039] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0040] The present invention obtains the time interval between two adjacent frames of surveillance video and performs difference analysis on the data of all screens in the repeated monitoring area. It can preliminarily determine whether there is a potential abnormality of video asynchrony, ensure that the system can promptly identify and respond to potential problems, and avoid monitoring blind spots and information omissions caused by asynchrony.

[0041] After identifying the potential anomaly of video asynchrony, the present invention captures the monitoring images at the same time to perform pairwise similarity analysis, and uses similarity indicators such as cosine similarity and peak signal-to-noise ratio to accurately evaluate the similarity between each pair of monitoring images. It can identify those abnormal images that are greatly different from other images, thereby achieving accurate screening, ensuring the high consistency of the remaining images, ensuring that the system can provide continuous and synchronized images, and effectively reflecting the entire process of the event, greatly improving the reliability and real-time performance of the monitoring system, and ensuring that the monitoring images ultimately used for analysis and decision-making are highly synchronized and consistent in time and content, so that staff can accurately piece together the complete timeline of the event, thereby better judging the cause, development and outcome of the event. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0043] Figure 1 The present invention is a method flow chart of a method for displaying messages on multiple screens.

[0044] Figure 2 It is a module schematic diagram of the message display system based on multiple screens of the present invention. DETAILED DESCRIPTION

[0045] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of the present disclosure will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those skilled in the art.

[0046] The present invention provides Figure 1 The multi-screen message display method shown includes the following steps:

[0047] For the surveillance images in the repeated area, the time interval between two adjacent frames of surveillance video images on one of the screens is obtained, and the data on the interval between two adjacent frames obtained from all screens in the repeated surveillance area are collected to establish a complete data set. The frame interval duration in the data set is analyzed for differences, and a preliminary judgment is made as to whether there is a potential anomaly of asynchrony in the surveillance video images displayed on multiple screens.

[0048] Collect the interval time data of two adjacent frames obtained from all screens in the repeated monitoring area, establish a complete data set, mark the data set as C, and mark the interval time as Interval v , then: C = {Interval v}, where v represents the number of each monitoring screen in the repeated monitoring area, v = 1, 2, 3, 4, ..., V, and V is a positive integer.

[0049] Perform difference analysis on the frame interval duration in the data set to generate the interval duration variation coefficient. Use the interval duration variation coefficient to preliminarily determine whether there is a potential anomaly of asynchrony in the surveillance video images displayed on multiple screens. The specific steps are as follows:

[0050] The mean and standard deviation of the interval duration are calculated by the interval duration in the data set, and the mean and standard deviation of the interval duration are calibrated as μ Interval and σ Interval ;

[0051] By the mean interval length μ Inervar and the standard deviation of the interval duration σ Inervar Calculate the coefficient of variation of the interval duration. The expression is: In the formula, Interval 变异 represents the coefficient of variation of interval duration;

[0052] A large interval duration variation coefficient indicates that the frame interval duration between each screen has large fluctuations and inconsistencies, indicating that the video synchronization is poor and there may be serious asynchrony. On the contrary, a small interval duration variation coefficient indicates that the frame interval duration between each screen is relatively stable and consistent, indicating that the video synchronization is good, and the video content displayed by the system on multiple screens is highly synchronized, and asynchrony is not likely to occur.

[0053] The interval duration variation coefficient generated after the difference analysis of the frame interval duration in the data set is compared and analyzed with the preset interval duration variation coefficient reference threshold to preliminarily determine whether there is a potential abnormality of asynchrony in the surveillance video images displayed on multiple screens. The specific judgment is as follows:

[0054] If the interval duration variation coefficient is greater than or equal to the interval duration variation coefficient reference threshold, it indicates that the frame interval durations between the screens have large fluctuations and inconsistencies, thereby generating a potential abnormality signal, indicating that the surveillance screen video synchronization of the repeated area is poor, and there is a potential abnormality of video asynchrony;

[0055] If the interval duration variation coefficient is less than the interval duration variation coefficient reference threshold, it indicates that the frame interval durations between the screens are relatively stable and consistent, thereby generating an efficient monitoring signal, indicating that the monitoring screens in the repeated areas can achieve efficient video synchronization.

[0056] After initially determining the potential anomaly of video asynchrony, obtain several surveillance images of the same time in the repeated surveillance area, and ensure that the comparative analysis is based on the image content at the same time by capturing frames at the same time point;

[0057] After initially determining the potential anomaly of video asynchrony, that is, when a potential abnormal signal is generated during video monitoring, it is necessary to obtain several surveillance images of the same time in the repeated monitoring area. This can be achieved by grabbing frames at the same time point to ensure that the comparative analysis is based on the content of the images at the same time. The purpose of this step is to provide sufficient data samples for subsequent similarity analysis, so as to identify and confirm the specific situation of video asynchrony in detail.

[0058] Perform pairwise similarity analysis on several surveillance images and calculate the similarity between each pair of surveillance images;

[0059] Perform pairwise similarity analysis on several monitoring screens, calculate the similarity between each pair of monitoring screens, and the similarity indicators include cosine similarity and peak signal-to-noise ratio. After obtaining the cosine similarity and peak signal-to-noise ratio generated by pairwise similarity analysis of several monitoring screens, the cosine similarity and peak signal-to-noise ratio are formulated and analyzed to generate a similarity index, and the similarity of two monitoring screens is evaluated by the similarity index.

[0060] Perform pairwise similarity analysis on several monitoring images and calculate the cosine similarity between each pair of monitoring images. The specific steps are as follows:

[0061] Convert each monitoring screen into a vector. Suppose the monitoring screen is a grayscale image of size m×n, flatten it into a vector of length m×n, and suppose the vectors corresponding to the two monitoring screens are: A=(a i )=(a 1 , a 2 , a 3 , …, a mn ), B=(b i )=(b 1 , b 2 , b 3 , …, b mn ), where vector A and vector B are vectors of two monitoring images, respectively. i and b i are the grayscale values ​​of the i-th pixel in the picture;

[0062] Calculate the dot product of vector A and vector B. The dot product represents the sum of the products of two vectors in corresponding dimensions. The calculation expression is:

[0063] Calculate the magnitude (length) of vector A and vector B. The magnitude of a vector is the square root of the sum of the squares of all its components. The calculation expression is:

[0064] Calculate the cosine similarity between vector A and vector B. Cosine similarity is the dot product divided by the product of the two vector moduli. The value range is between [-1, 1] and represents the cosine value of the angle between the two vectors. The calculation expression is: Cosine Similarity(A, B) represents the cosine similarity between vector A and vector B.

[0065] Perform pairwise similarity analysis on several surveillance images. When calculating the cosine similarity between each pair of surveillance images, the larger the cosine similarity value, the higher the similarity between the corresponding two surveillance images and the more similar the image contents are; conversely, the smaller the cosine similarity value, the lower the similarity between the corresponding two surveillance images and the greater the difference in image contents. The value range of cosine similarity is between -1 and 1, where 1 means exactly the same, 0 means irrelevant, and -1 means completely opposite. Therefore, by comparing the size of cosine similarity, the similarity between surveillance images can be effectively evaluated and judged.

[0066] Perform pairwise similarity analysis on several surveillance images and calculate the peak signal-to-noise ratio between each pair of surveillance images. The specific steps are as follows:

[0067] Calculate the mean square error between the two images. The mean square error is used to measure the difference in corresponding pixel values ​​between the two images. The calculation expression is: x(i', j') and y(i', j') are the pixel values ​​of image x and image y at the i'th row and j'th column respectively, M and N are the width and height of the image, that is, the resolution of the image, MN is the total number of pixels in the image, and mse(x, y) is the mean square error between image x and image y;

[0068] Calculate the peak signal-to-noise ratio. The expression is: Where PSNR(x, y) represents the peak signal-to-noise ratio, max I is the maximum pixel value of the image. For example, for an 8-bit image, the pixel value range is 0 to 255, then max I =255;

[0069] When performing pairwise similarity analysis on several surveillance images, the peak signal-to-noise ratio between each pair of surveillance images is calculated. The larger the peak signal-to-noise ratio, the higher the similarity between the corresponding two surveillance images, which means that the difference between the images is small and the quality is relatively consistent; conversely, the smaller the peak signal-to-noise ratio, the lower the similarity between the corresponding two surveillance images, the larger the difference between the images, and the inconsistent quality. Therefore, the peak signal-to-noise ratio can effectively evaluate the similarity between surveillance images and identify images that may have synchronization problems.

[0070] The cosine similarity and peak signal-to-noise ratio are analyzed in a formula to generate the similarity index: τ =r 1 *Cosine Similarity(A,B)+r 2 *PSNR(x, y), where r 1 、r 2 are the preset proportional coefficients of cosine similarity (A, B) and peak signal-to-noise ratio (PSNR) (x, y), and r 1 、r 2 Both are greater than 0.

[0071] It can be seen from the similarity index that when performing pairwise similarity analysis on several monitoring images, the greater the cosine similarity and peak signal-to-noise ratio performance values ​​generated between each pair of monitoring images, that is, the greater the performance value of the similarity index, the higher the similarity between the corresponding two monitoring images, and the more similar the image contents are. Conversely, the lower the similarity between the corresponding two monitoring images, the less similar the image contents are.

[0072] The similarity values ​​after the pairwise similarity analysis are used to establish an analysis set, and the similarity of each monitoring screen with other monitoring screens at the same time is summed up to obtain the similarity sum value;

[0073] The similarity index after pairwise similarity analysis is used to establish an analysis set, and the similarity of each monitoring screen with other monitoring screens at the same time is summed up and calculated. The calculation process is:

[0074] For a given time t, there are w monitoring images for the repeated area. Let the similarity index between each pair of monitoring images p and q be S pq , then for monitoring screen p, the similarity sum calculation formula between it and all other monitoring screens is: In the formula, Sum p It represents the sum of similarities between monitoring picture p and all other monitoring pictures.

[0075] The calculated similarity sum values ​​are sorted in descending order, and the surveillance picture corresponding to the minimum similarity sum value is marked and the surveillance picture is screened out, thereby removing the surveillance picture with potential abnormalities of video asynchrony;

[0076] The calculated similarity sum values ​​are sorted in descending order, and the monitoring screen corresponding to the minimum similarity sum value is marked, and then the monitoring screen is screened out because the monitoring screen with the minimum similarity sum value shows the lowest similarity in comparison with all other screens. This means that the content of this screen is the most different from the rest of the monitoring screens at the same time, and it is most likely to be out of sync. Low similarity indicates that the content of this screen is inconsistent with that of other screens, which may be caused by misalignment, delay, frame loss or other technical failures on the timeline. By screening out the monitoring screen with the minimum similarity sum value, those abnormal out-of-sync screens can be effectively removed, thereby improving the synchronization and consistency of the remaining monitoring screens. This method ensures that the screens provided by the monitoring system are coherent at the same time, accurately reflect the real situation of the event, avoid misjudgments and decision-making errors caused by asynchrony, and improve the reliability and effectiveness of the overall monitoring system.

[0077] The present invention obtains the time interval between two adjacent frames of surveillance video and performs difference analysis on the data of all screens in the repeated monitoring area. It can preliminarily determine whether there is a potential abnormality of video asynchrony, ensure that the system can promptly identify and respond to potential problems, and avoid monitoring blind spots and information omissions caused by asynchrony.

[0078] After identifying the potential abnormality of video asynchrony, the present invention captures the monitoring images at the same time to perform pairwise similarity analysis, and uses similarity indicators such as cosine similarity and peak signal-to-noise ratio to accurately evaluate the similarity between each pair of monitoring images, so as to identify those abnormal images that are greatly different from other images, thereby achieving accurate screening, ensuring the high consistency of the remaining images, ensuring that the system can provide continuous and synchronized images, effectively reflecting the whole process of the event, greatly improving the reliability and real-time performance of the monitoring system, ensuring that the monitoring images ultimately used for analysis and decision-making are highly synchronized and consistent in time and content, so that the staff can accurately piece together the complete timeline of the event, thereby better judging the cause, development and outcome of the event, especially in fields such as accident investigation and crime analysis that require highly accurate information, the high-quality synchronized images provided by this solution greatly reduce the misjudgment and erroneous conclusions caused by video asynchrony, and improve the effectiveness and scientificity of the disposal plan, which not only improves the efficiency of emergency response, but also provides a more solid foundation for the analysis and utilization of long-term monitoring data, and promotes the overall progress of monitoring technology.

[0079] The present invention provides Figure 2 The multi-screen based message display system shown includes a data collection and preliminary analysis module, a synchronous data capture module, a similarity analysis module, a similarity sum value calculation module and an abnormal screening module;

[0080] The data collection and preliminary analysis module obtains the time interval between two adjacent frames of surveillance video on one of the repeated surveillance images, collects the data on the interval between two adjacent frames obtained on all screens in the repeated surveillance area, establishes a complete data set, performs difference analysis on the frame intervals in the data set, and preliminarily determines whether there are potential anomalies of asynchrony in the surveillance video images displayed on multiple screens;

[0081] The synchronous data capture module initially determines the potential anomaly of video asynchrony and obtains several surveillance images of the same time in the repeated monitoring area. By capturing frames at the same time point, it ensures that the comparative analysis is based on the image content at the same time.

[0082] The similarity analysis module performs pairwise similarity analysis on a number of monitoring images and calculates the similarity between each pair of monitoring images;

[0083] The similarity sum value calculation module establishes an analysis set of similarity values ​​after pairwise similarity analysis, and performs sum calculation on the similarity of each monitoring screen with other monitoring screens at the same time to obtain a similarity sum value;

[0084] The abnormal screening module sorts the calculated similarity sum values ​​in descending order, marks the monitoring picture corresponding to the minimum similarity sum value, and screens out the monitoring picture, thereby removing the monitoring picture with potential abnormalities of video asynchrony;

[0085] The multi-screen message display method provided in an embodiment of the present invention is implemented by the multi-screen message display system mentioned above. The specific method and process of the multi-screen message display system are detailed in the embodiment of the multi-screen message display method mentioned above, which will not be repeated here.

[0086] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0087] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center containing one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0088] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0089] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0090] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0091] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0092] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0093] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A message display method based on multiple screens, characterized in that: The following steps are involved: For the surveillance images in the repeated area, the time interval between two adjacent frames of surveillance video images on one of the screens is obtained, and the data on the interval between two adjacent frames obtained from all screens in the repeated surveillance area are collected to establish a complete data set. The frame interval duration in the data set is analyzed for differences, and a preliminary judgment is made as to whether there is a potential anomaly of asynchrony in the surveillance video images displayed on multiple screens. After initially determining the potential anomaly of video asynchrony, obtain several surveillance images of the same time in the repeated surveillance area, and ensure that the comparative analysis is based on the image content at the same time by capturing frames at the same time point; Perform pairwise similarity analysis on several surveillance images and calculate the similarity between each pair of surveillance images; The similarity values ​​after the pairwise similarity analysis are used to establish an analysis set, and the similarity of each monitoring screen with other monitoring screens at the same time is summed up to obtain the similarity sum value; The calculated similarity sum values ​​are sorted in descending order, and the monitoring picture corresponding to the minimum similarity sum value is marked and the monitoring picture is screened out, thereby removing the monitoring picture with potential abnormalities of video asynchrony.

2. The message display method based on multiple screens according to claim 1, characterized in that: Collect the interval time data of two adjacent frames obtained from all screens in the repeated monitoring area, establish a complete data set, mark the data set as C, and mark the interval time as Interval v , then: C = {Interval v }, where v represents the number of each monitoring screen in the repeated monitoring area, v = 1, 2, 3, 4, ..., V, V is a positive integer; Perform difference analysis on the frame interval duration in the data set to generate the interval duration variation coefficient. Use the interval duration variation coefficient to preliminarily determine whether there is a potential anomaly of asynchrony in the surveillance video images displayed on multiple screens. The specific steps are as follows: The mean and standard deviation of the interval duration are calculated by the interval duration in the data set, and the mean and standard deviation of the interval duration are calibrated as μ Interval and σ Interval ; By the mean interval length μ Interval and the standard deviation of the interval duration σ Interval Calculate the coefficient of variation of the interval duration. The expression is: In the formula, Interval 变异 represents the coefficient of variation of interval duration.

3. The message display method based on multiple screens according to claim 2, characterized in that: The interval duration variation coefficient generated after the difference analysis of the frame interval duration in the data set is compared and analyzed with the preset interval duration variation coefficient reference threshold to preliminarily determine whether there is a potential abnormality of asynchrony in the surveillance video images displayed on multiple screens. The specific judgment is as follows: If the interval duration variation coefficient is greater than or equal to the interval duration variation coefficient reference threshold, a potential abnormality signal is generated, indicating that the surveillance images of the repeated area have a potential abnormality of video asynchrony; If the interval duration variation coefficient is less than the interval duration variation coefficient reference threshold, an efficient monitoring signal is generated, indicating that the monitoring screen of the repeated area can achieve efficient video synchronization.

4. The message display method based on multiple screens according to claim 1, characterized in that: Perform pairwise similarity analysis on several monitoring screens, calculate the similarity between each pair of monitoring screens, and the similarity indicators include cosine similarity and peak signal-to-noise ratio. After obtaining the cosine similarity and peak signal-to-noise ratio generated by pairwise similarity analysis of several monitoring screens, the cosine similarity and peak signal-to-noise ratio are formulated and analyzed to generate a similarity index, and the similarity of two monitoring screens is evaluated by the similarity index.

5. The message display method based on multiple screens according to claim 4, characterized in that: Perform pairwise similarity analysis on several monitoring images and calculate the cosine similarity between each pair of monitoring images. The specific steps are as follows: Convert each monitoring screen into a vector. Suppose the monitoring screen is a grayscale image of size m×n, flatten it into a vector of length m×n, and suppose the vectors corresponding to the two monitoring screens are: A=(a i )=(a1,a2,a3,…,a mn ), B=(b i )=(b1,b2,b3,…,b mn ), where vector A and vector B are vectors of two monitoring images, respectively. i and b i They are the grayscale values ​​of the i-th pixel in the picture: Calculate the dot product of vector A and vector B. The dot product represents the sum of the products of two vectors in corresponding dimensions. The calculation expression is: Calculate the magnitude of vector A and vector B. The magnitude of a vector is the square root of the sum of the squares of all its components. The calculation expression is: Calculate the cosine similarity between vector A and vector B. Cosine similarity is the dot product divided by the product of the two vector moduli. The value range is between [-1, 1] and represents the cosine value of the angle between the two vectors. The calculation expression is: Cosine Similarity(A, B) represents the cosine similarity between vector A and vector B.

6. The message display method based on multiple screens according to claim 5, characterized in that: Perform pairwise similarity analysis on several surveillance images and calculate the peak signal-to-noise ratio between each pair of surveillance images. The specific steps are as follows: Calculate the mean square error between the two images. The mean square error is used to measure the difference in corresponding pixel values ​​between the two images. The calculation expression is: x(i′, j′) and y(i′, j′) are the pixel values ​​of image x and image y at the i′th row and j′th column respectively, M and N are the width and height of the image, that is, the resolution of the image, MN is the total number of pixels in the image, and mse(x, y) is the mean square error between image x and image y; Calculate the peak signal-to-noise ratio. The expression is: Where PSNR(x, y) represents the peak signal-to-noise ratio, max I is the maximum pixel value of the image.

7. The message display method based on multiple screens according to claim 6, characterized in that: The cosine similarity and peak signal-to-noise ratio are analyzed in a formula to generate the similarity index: τ =r1*CosineSimilarity(A, B)+r2*PSNR(x, y), where r1 and r2 are preset proportional coefficients of cosine similarity Cosine Similarity(A, B) and peak signal-to-noise ratio PSNR(x, y), respectively, and r1 and r2 are both greater than 0.

8. The message display method based on multiple screens according to claim 7, characterized in that: The similarity index after pairwise similarity analysis is used to establish an analysis set, and the similarity of each monitoring screen with other monitoring screens at the same time is summed up and calculated. The calculation process is: For a given time t, there are w monitoring screens for the repeated area. Let the similarity index between each pair of monitoring screens p and q be Spq. Then the similarity sum calculation formula between monitoring screen p and all other monitoring screens is: In the formula, Sum p It represents the sum of similarities between monitoring picture p and all other monitoring pictures.

9. A message display system based on multiple screens, used to implement the message display method based on multiple screens as described in any one of claims 1 to 8, characterized in that: It includes data collection and preliminary analysis module, synchronous data capture module, similarity analysis module, similarity sum value calculation module and abnormal screening module; The data collection and preliminary analysis module obtains the time interval between two adjacent frames of surveillance video on one of the repeated surveillance images, collects the data on the interval between two adjacent frames obtained on all screens in the repeated surveillance area, establishes a complete data set, performs difference analysis on the frame intervals in the data set, and preliminarily determines whether there are potential anomalies of asynchrony in the surveillance video images displayed on multiple screens; The synchronous data capture module initially determines the potential anomaly of video asynchrony and obtains several surveillance images of the same time in the repeated monitoring area. By capturing frames at the same time point, it ensures that the comparative analysis is based on the image content at the same time. The similarity analysis module performs pairwise similarity analysis on a number of monitoring images and calculates the similarity between each pair of monitoring images; The similarity sum value calculation module establishes an analysis set of similarity values ​​after pairwise similarity analysis, and performs sum calculation on the similarity of each monitoring screen with other monitoring screens at the same time to obtain a similarity sum value; The abnormal screening module sorts the calculated similarity sum values ​​in descending order, marks the monitoring picture corresponding to the minimum similarity sum value, and screens out the monitoring picture, thereby removing the monitoring picture with potential abnormalities of video asynchrony.

10. A multi-screen message display intelligent terminal, comprising the multi-screen message display system according to claim 9.

Images