Multimedia monitoring method, system and electronic device

Abstract

The present disclosure provides a multimedia monitoring method, system and electronic device, belonging to the technical field of media dissemination, which includes generating a first verification image of a playback plan according to a pre-generated first identification image and configuration information of a display terminal through an image generation algorithm; the first identification image is used to represent first identity authentication information; the first identity authentication information is used to represent the playback plan; generating data to be displayed according to the received playback plan, the multimedia file corresponding to the playback plan and the first verification image, so that the display terminal displays in response to the data to be displayed; receiving the display image sent by the display terminal, parsing the display image, and after parsing out the second identification image in the display image and determining that the second identification image is consistent with the first identification image, obtaining second identity authentication information; comparing the second identity authentication information with the first identity authentication information, and judging whether the display image of the display terminal is accurate according to the comparison result.

Description

Technical Field

[0001] The present disclosure belongs to the technical field of media transmission, and particularly relates to a multimedia monitoring method, system and electronic device. Background Art

[0002] For the multimedia content that has been published, if it is necessary to monitor the playback of the multimedia content finally displayed on the display screen of the monitoring terminal, such as whether the multimedia content is displayed on time or whether the displayed content is correct, etc., in a traditional solution, remote monitoring can be carried out by means of manual patrol, but this method has high labor costs, low detection efficiency, low detection accuracy, and low execution efficiency of the countermeasure in the case of detecting incorrect displayed content. Another solution is to perform optical monitoring through an external camera of the display screen and use an algorithm to analyze and detect the multimedia content. However, this method requires a monitoring camera to be equipped for each display screen, and the camera needs to be installed according to specific height, angle and other standards, and the engineering of the camera-related structure, wiring, etc. is often limited by the application scenario during the installation process, making it difficult to implement in engineering. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art, and provides a multimedia monitoring method, system and electronic device.

[0004] In a first aspect, the technical solution adopted to solve the technical problems of the present disclosure is a multimedia monitoring method, including:

[0005] Generating a first verification image of the playback plan according to a pre-generated first identification image and the configuration information of the display terminal, and through an image generation algorithm; the first identification image is used to represent first identity authentication information; the first identity authentication information is used to represent the playback plan in the playback schedule;

[0006] Generating data to be displayed according to the received playback plan, the multimedia file corresponding to the playback plan, and the first verification image, so that the display terminal displays in response to the data to be displayed;

[0007] Receiving the display image sent by the display terminal, parsing the display image, and after parsing out the second identification image in the display image and determining that the second identification image is consistent with the first identification image, obtaining the second identity authentication information corresponding to the second identification image;

[0008] Comparing the second identity authentication information with the first identity authentication information, and judging whether the display image of the display terminal is accurate according to the comparison result.

[0009] In some examples, generating a first verification image of a playback plan according to the pre-generated first identification image and the configuration information of the display terminal through an image generation algorithm includes:

[0010] Generating a first identification image according to the first identity information;

[0011] Processing the first identification image according to the number of pixel points corresponding to a specific area of the pre-generated transparency image to obtain a first target identification image;

[0012] Encoding the display information of each pixel point of the first target identification image to obtain first encoded data;

[0013] Replacing the display data of the pixel points corresponding to the specific area of the transparency image with the first encoded data to generate a first verification image.

[0014] In some examples, after generating the first identification image according to the first identity information, it further includes:

[0015] Generating the transparency image according to the resolution information of the display terminal; the resolution of the transparency image is the same as that of the display terminal.

[0016] In some examples, the transparency image has a middle area and an edge area surrounding the middle area; the specific area is the edge area.

[0017] In some examples, when the number of pixel points in the specific area is greater than the number of pixel points of the first target identification image, the step of replacing the display data of the pixel points corresponding to the specific area of the transparency image with the first encoded data to generate a first verification image includes:

[0018] While replacing the display data of the pixel points corresponding to the specific area of the transparency image with the first encoded data, replacing the display data of the pixel points in the specific area of the transparency image except those corresponding to the first target identification image with second encoded data to obtain the first verification image.

[0019] In some examples, the specific steps of parsing the display image to determine whether the display image of the display terminal is accurate include:

[0020] Parsing the display image to obtain the identification image actually displayed by the display terminal;

[0021] Processing the actually displayed identification image and obtaining a second identification image according to the resolution information of the first identification image;

[0022] Compare the second identification image with the first identification image. If they are inconsistent, it indicates that the displayed image is incorrect.

[0023] When the second identification image is consistent with the first identification image, obtain second identity authentication information based on the second identification image.

[0024] Compare the second identity authentication information with the first identity authentication information to determine whether the displayed image is accurate.

[0025] In some examples, the processing of the actually displayed identification image and obtaining the second identification image according to the resolution information of the first identification image includes:

[0026] Process the actually displayed identification image according to the number of pixel points corresponding to the specific area of the transparency image to obtain a second target identification image.

[0027] Determine whether there is target encoded data obtained by encoding the display information of each pixel point in the second target identification image.

[0028] When it is determined that there is target encoded data obtained by encoding the display information of each pixel point in the second target identification image, decode the target encoded data to obtain decoded data.

[0029] Generate a second identification image according to the resolution information of the first identification image and the decoded data.

[0030] In some examples, the encoding of the display information of each pixel point in the first target identification image to obtain first encoded data includes:

[0031] Encode the display information of each pixel point in the first target identification image in sequence according to a first preset order to obtain first encoded data.

[0032] The processing of the actually displayed identification image according to the number of pixel points corresponding to the specific area of the transparency image to obtain a second target identification image includes:

[0033] Assemble the display data of each pixel point in the actually displayed identification image in sequence according to the first preset order according to the number of pixel points corresponding to the specific area of the transparency image to obtain a second target identification image.

[0034] In some examples, the replacing of the display data of the pixel points corresponding to the specific area of the transparency image with the first encoded data to generate a first verification image includes:

[0035] Replacing the display data of the corresponding pixel points in the specific area of the transparency image with the first encoded data in a preset order to generate the first verification image;

[0036] Parsing the display image to obtain the identification image actually displayed by the display terminal, including:

[0037] Sequentially extracting the display data of the corresponding pixel points in the specific area of the display image according to the preset order, and obtaining the identification image actually displayed by the display terminal according to the extracted display data.

[0038] In some examples, the playback plan includes the playback window information and playback order information of each multimedia file in the display terminal;

[0039] Generating the data to be displayed according to the received playback plan, the multimedia file corresponding to the playback plan, and the first verification image, including:

[0040] Generating the data to be displayed according to the display area information and display order information of each multimedia file in the display terminal, the multimedia file corresponding to the playback plan, and the first verification image, so that the display terminal responds to the data to be displayed and displays according to the playback window information and playback order information of each multimedia file in the display terminal.

[0041] In some examples, when analyzing the display image, it further includes:

[0042] When it is determined that the display image is inaccurate, sending a control instruction to cancel the display to the display terminal, so that the display terminal cancels the display of the display image and generates an alarm message.

[0043] In some examples, the multimedia monitoring method further includes:

[0044] Encoding the parameter information in the playback plan to generate the first identity authentication information.

[0045] In a second aspect, an embodiment of the present disclosure further provides a multimedia monitoring system, which includes a verification image generation module, a playback control module, an analysis module, and a display terminal; wherein,

[0046] The verification image generation module is configured to generate a first verification image of the playback plan according to a pre-generated first identification image and the configuration information of the display terminal, and through an image generation algorithm; the first identification image is used to represent the first identity authentication information; the first identity authentication information is used to represent the playback plan in the playback plan table;

[0047] The playback control module is configured to generate data to be displayed according to the received playback plan, the multimedia file corresponding to the playback plan, and the first verification image;

[0048] The display terminal is configured to display in response to the data to be displayed, and intercept the currently displayed image at a preset frequency as the display image;

[0049] The analysis module is configured to parse the display image, and after parsing out the second identification image in the display image and determining that the second identification image is consistent with the first identification image, obtain the second identity authentication information corresponding to the second identification image, compare the second identity authentication information with the first identity authentication information, and determine whether the display image of the display terminal is accurate according to the comparison result.

[0050] In a third aspect, an embodiment of the present disclosure further provides a computer device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible multimedia monitoring method in the first aspect are executed.

[0051] In a fourth aspect, an embodiment of the present disclosure further provides a computer non-transitory readable storage medium. A computer program is stored on the computer non-transitory readable storage medium. When the computer program is run by a processor, the steps of the above-mentioned first aspect, or any possible multimedia monitoring method in the first aspect are executed.

[0052] In a fifth aspect, an embodiment of the present disclosure further provides an electronic device, including the multimedia monitoring system and the display terminal described in the second aspect. Description of the Drawings

[0053] Figure 1 It is a schematic structural diagram of a multimedia monitoring system provided by an embodiment of the present disclosure;

[0054] Figure 2 It is a schematic processing flow diagram of a verification image generation module provided by an embodiment of the present disclosure;

[0055] Figure 3 It is a schematic diagram of the encoding order of the first target identification image provided by an embodiment of the present disclosure;

[0056] Figure 4 It is a schematic replacement order diagram of the first encoded data replacing the transparent image provided by an embodiment of the present disclosure;

[0057] Figure 5Schematic diagram of the processing flow of the second identification image analysis unit provided by the embodiments of the present disclosure;

[0058] Figure 6 Schematic diagram of the display image provided by the embodiments of the present disclosure;

[0059] Figure 7 Flowchart of a multimedia monitoring method provided by the embodiments of the present disclosure;

[0060] Figure 8 Schematic diagram of the structure of a computer device provided by the embodiments of the present application. Detailed implementation manners

[0061] To enable those skilled in the art to better understand the technical solutions of the present invention / utility model, the present invention / utility model will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0062] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure shall have the ordinary meanings understood by those of ordinary skill in the art belonging to the field of the present disclosure. The "first", "second" and similar terms used in the present disclosure do not denote any order, quantity or importance, but are only used to distinguish different components. Similarly, the terms such as "a", "an" or "the" do not denote a quantity limitation, but mean that there is at least one. The terms such as "comprising" or "including" mean that the elements or objects appearing before this term cover the elements or objects listed after this term and their equivalents, without excluding other elements or objects. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left" and "right" are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0063] In a first aspect, Figure 1 Schematic diagram of the structure of a multimedia monitoring system provided by the embodiments of the present disclosure. As Figure 1 shown, the multimedia monitoring system 100 is configured to monitor the display content of the display terminal 101 to prompt the user of the current display situation of the display terminal 101. The multimedia monitoring system 100 includes a verification image generation module 11, a playback control module 12 and an analysis module 13.

[0064] The verification image generation module 11 is configured to generate a first verification image of the play schedule according to a pre-generated first identification image and the configuration information of the display terminal 101 through an image generation algorithm. The first identification image is generated according to the first authentication information and is used to represent the first authentication information; the first authentication information is generated by the information management module 14 according to the play schedule in the play schedule table and is used to represent the play schedule in the play schedule table. The play schedule table is generated by the information management module 14 using the parameter information of at least one play schedule.

[0065] It should be noted that one play schedule corresponds to one first authentication information, and one first authentication information corresponds to one first identification image. In the case where the play schedule table includes multiple play schedules, the first verification image corresponding to each play schedule can be generated respectively through the image generation algorithm.

[0066] In some examples, the first authentication information and the configuration information of the display terminal 101 are used as the input data of the image generation algorithm, and through the processing of the image generation algorithm, the first verification image of the play schedule can be generated.

[0067] In some examples, the verification image generation module 11 is configured to send the first verification image of the play schedule to the distribution control module, so that the distribution control module, according to the play schedule table, sends the currently pending play schedule, the first verification image of this play schedule, and the multimedia file corresponding to this play schedule to the play control module 12 at the same time.

[0068] The play control module 12 is configured to generate the data to be displayed according to the received play schedule, the multimedia file corresponding to the play schedule, and the first verification image for the display terminal 101 to display. The data to be displayed specifically includes the play schedule, the multimedia file corresponding to this play schedule, and the first verification image.

[0069] The play control module 12 can send the data to be displayed to the display terminal 101 for display through a High Definition Multimedia Interface (HDMI).

[0070] The analysis module 13 is configured to parse the display images intercepted by the display terminal 101 at a preset frequency. After parsing out the second identification image and determining that the second identification image is consistent with the first identification image, obtain the second authentication information corresponding to the second identification image, compare the second authentication information with the first authentication information, and determine whether the display of the display terminal 101 is accurate according to the comparison result.

[0071] In the embodiments of the present disclosure, without affecting the playback content of the multimedia file, the playback content of the display terminal 101 is monitored in a timely manner. Specifically, by taking a screenshot of the image currently displayed on the display terminal 101 and further analyzing the displayed image, it is possible to quickly monitor whether the displayed image of the display terminal 101 is accurate, greatly improving the security of multimedia monitoring and broadcasting. Compared with the traditional method of manual patrol points, the embodiments of the present disclosure can reduce the cost of multimedia monitoring and broadcasting and have high real-time performance; compared with the optical monitoring and broadcasting solution using an external camera for the display screen, the multimedia monitoring and broadcasting system 100 provided by the embodiments of the present disclosure belongs to a system-built multimedia monitoring and broadcasting algorithm, which uses background data processing and can efficiently complete multimedia monitoring and broadcasting without external detection devices.

[0072] The display terminal 101 displays in response to the data to be displayed, intercepts the current displayed image at a preset frequency, and sends the displayed image to the analysis module 13. The display terminal 101 may include a display screen for displaying the picture corresponding to the multimedia file. Specifically, the display terminal 101 responds to the data to be displayed, controls the playback of the multimedia file using the playback card in the display terminal 101, and intercepts the image currently displayed on the display screen at a preset frequency, and uses this image as the displayed image for timely monitoring of the accuracy of the multimedia content currently played by the display terminal 101.

[0073] Figure 2 It is a schematic diagram of the processing flow of the verification image generation module 11 provided by the embodiments of the present disclosure, as Figure 2 shown, the verification image generation module 11 includes a first identification image generation unit, an image processing unit, an encoding unit, and a first verification image generation unit.

[0074] The first identification image generation unit is configured to generate a first identification image according to the first identity authentication information.

[0075] Specifically, the first identification image generation unit is configured to arrange the identification of the first identity authentication information according to a specific rule using an identification generation function and generate a first identification image according to a specific geometric figure.

[0076] Exemplarily, the identification generation function may be a two-dimensional code generation function, and the first identification image generation unit is configured to call the two-dimensional code generation function to generate an image of an identity information two-dimensional code with a resolution of L1*L1, and this image of the identity information two-dimensional code is the first identification image.

[0077] In the embodiments of the present disclosure, using the identity information two-dimensional code to represent the first identity authentication information can realize the encoding protection of the first identity authentication information and enhance the robustness of the first identity authentication information, that is, prevent the first identity authentication information from being contaminated by noise during storage and transmission, resulting in information extraction failure.

[0078] The image processing unit is configured to process the first identification image according to the number of pixel points corresponding to a specific area of a pre-generated transparency map, so as to obtain a first target identification image.

[0079] The transparency map can be pre-generated by the transparency map generation unit in the verification image generation module 11. The specific area of the transparency map can be a pre-set area, and the area range can be defined according to the actual application scenario and experience. The transparency map has a middle area and an edge area surrounding the middle area. The specific area can include the edge area of the transparency map, or the top corner area of the transparency map, or a set area with an irregular shape of the transparency map, etc.

[0080] Specifically, the image processing unit is configured to process the first identification image according to the number of pixel points corresponding to a specific area of a pre-generated transparency map, so that the number of pixel points in the obtained first target identification image is less than or equal to the number of pixel points corresponding to the specific area of the transparency map.

[0081] Taking the image corresponding to the identity information two-dimensional code as an example of the first identification image (hereinafter simply referred to as the identity information two-dimensional code), the identity information two-dimensional code is scaled according to the number of pixel points corresponding to a specific area of a pre-generated transparency map, so as to obtain a first target identification image with the scaled identity information two-dimensional code.

[0082] For example, the resolution information of the identity information two-dimensional code is L1*L1 = N1 pixel points. If the number of pixel points N1 of the identity information two-dimensional code is greater than or equal to the number of pixel points N' corresponding to the specific area of the transparency map, then the resolution of the identity information two-dimensional code is scaled down proportionally to L2*L2, where L2*L2 = N2 ≤ N', and L2 is the largest positive integer less than or equal to inside, so as to obtain a first target identification image with a resolution of L2*L2. If the number of pixel points N1 of the identity information two-dimensional code is less than or equal to the number of pixel points corresponding to the specific area of the transparency map, then the resolution of the identity information two-dimensional code is scaled up proportionally to L2*L2, where L2*L2 = N2 ≤ N', and L2 is the largest positive integer less than or equal to inside, so as to obtain a first target identification image with a resolution of L2*L2.

[0083] The encoding unit is configured to encode the display information of each pixel point of the first target identification image to obtain first encoded data.

[0084] The display information includes information about the colors shown by the first target identification image, which may include information about different colors, such as but not limited to black and white, etc. The encoding unit is configured to encode the display information of each pixel point of the first target identification image, and different display information can encode different first encoded data. For example, black pixel points can be encoded as first encoded sub-data (e.g., 0), and white pixel points can be encoded as second encoded sub-data (e.g., 1), etc.

[0085] In some examples, the encoding unit is configured to sequentially encode the display information of each pixel point of the first target identification image in a first preset order to obtain first encoded data arranged in a preset order. Figure 3 It is a schematic diagram of the encoding order of the first target identification image provided by the embodiments of the present disclosure. As Figure 3 shown, the first preset order can be Figure 3 as shown, traversing each pixel point of the first target identification image in the order from left to right and from top to bottom, encoding each pixel point, and recording the first encoded data of each pixel point in a preset order.

[0086] The first verification image generation unit is configured to replace the display data of the corresponding pixel points in a specific area of the transparent image with the first encoded data to generate a first verification image.

[0087] The transparent image is a single-channel image, that is, an image with an alpha channel. The display data of the corresponding pixel points in a specific area of the transparent image can be the data of the alpha channel of the transparent image. For example, the first verification image generation unit is configured to replace the data of the alpha channel of the corresponding pixel points in a specific area of the transparent image with the first encoded data to generate a transparent image with a scaled identity information two-dimensional code (that is, the first verification image).

[0088] In some examples, the first verification image generation subunit is configured to sequentially replace the display data of the corresponding pixel points in a specific area of the transparent image with the first encoded data in a second preset order to generate a first verification image.

[0089] Taking the specific area of the transparent image as the edge area as an example, the first encoded data of each pixel point recorded in the first target identification image in a preset order can be sequentially replaced with the display data of each pixel point in the edge area of the transparent image in the order from left to right, from top to bottom, from right to left, and from bottom to top to generate a transparent image with a scaled identity information two-dimensional code (that is, the first verification image). Figure 4 It is a schematic diagram of the replacement order of the first encoded data replacing the transparent image provided by the embodiments of the present disclosure. As Figure 4As shown, in the order from top to bottom, from right to left, and from bottom to top of the transparency image, the first encoded data from left to right and from top to bottom in the first target identification image is sequentially replaced with the display data of each pixel point from top to bottom, from right to left, from bottom to top, and from bottom to top (arrow direction) of the transparency image.

[0090] It should be noted that in this example, the second preset order may not be limited to the order of "from left to right, from top to bottom, from right to left, from bottom to top", and can be defined according to experience and actual application scenarios. The non-disclosed embodiments will not be listed one by one.

[0091] In order to be able to display a relatively regular image on the display terminal 101 finally, when the number of pixel points in a specific area is greater than the number of pixel points of the first target identification image, the first verification image generation unit is configured to, while replacing the display data of the corresponding pixel points in the specific area of the transparency image with the first encoded data, replace the display data of the pixel points in the specific area of the transparency image, except for the pixel points corresponding to the first target identification image, with the second encoded data to obtain the first verification image. In some examples, the first verification image generation unit is configured to, while sequentially replacing the display data of the corresponding pixel points in the specific area of the transparency image with the first encoded data according to the second preset order, continue to sequentially replace the display data of the pixel points in the specific area of the transparency image, except for the pixel points corresponding to the first target identification image, with the second encoded data according to the second preset order to obtain the first verification image.

[0092] The pixel points other than the pixel points corresponding to the first target identification image, that is, among the N' pixel points in the specific area, the pixel points except for the N2 pixel points in the first target identification image.

[0093] Since this second encoded data does not participate in the judgment of the accuracy of the image displayed on the display terminal 101 subsequently, therefore, the second encoded data can be randomly set encoded data, or can also be the same data as the first encoded data or the second encoded data. The embodiments of the present disclosure do not limit this.

[0094] In some examples, in the process of generating the transparency image, specifically, the verification image generation module 11 further includes a transparency image generation unit; the transparency image generation unit is configured to generate a transparency image according to the resolution information of the display terminal 101; the resolution of the transparency image is consistent with the resolution of the display terminal 101.

[0095] It should be noted that the resolution of the transparency image may not be consistent with the resolution of the display terminal 101. For example, the positions of the pixel points of the transparency image are located at the positions of some pixel points in the display terminal 101.

[0096] Such as Figure 2As shown in the figure, the analysis module 13 includes an image parsing unit, a second identification image parsing unit, a first comparison unit, a second identity authentication information parsing unit, and a second comparison unit.

[0097] The image parsing unit is configured to parse the display image to obtain the identification image actually displayed by the display terminal 101.

[0098] Specifically, the image parsing unit is configured to extract the display data of each pixel corresponding to the position range of the pixel points corresponding to the specific area of the pre-generated transparent image from the display image, so as to obtain the identification image actually displayed by the display terminal 101.

[0099] In some examples, when the first verification image generation unit is configured to sequentially replace the display data of the pixel points corresponding to the specific area of the transparent image with the first encoded data to generate the first verification image, the image parsing unit is configured to sequentially extract the display data of the pixel points corresponding to the specific area of the display image (that is, the area corresponding to the position range of the pixel points of the specific area of the transparent image) in the second preset order, and obtain the identification image actually displayed by the display terminal 101 according to the extracted display data.

[0100] When the first verification image generation subunit generates the first verification image, taking the specific area of the transparent image as the edge area and the second preset order as the order from left to right, from top to bottom, from right to left, and from bottom to top as an example, in this example, the display data of the pixel points corresponding to the edge area of the display image can be sequentially extracted in the order from left to right, from top to bottom, from right to left, and from bottom to top, and the identification image actually displayed by the display terminal 101 can be obtained according to the extracted display data.

[0101] It should be noted that the replacement rule of the first verification image generation unit using the first encoded data to replace the transparent image (that is, the method of sequentially replacing in the second preset order) is the same as the extraction rule of the image parsing unit for extracting the pixel points of the display image (that is, the method of extracting in the second preset order).

[0102] The second identification image parsing unit is configured to process the actually displayed identification image and obtain the second identification image according to the resolution information of the first identification image.

[0103] Figure 5 The figure is a schematic diagram of the processing flow of the second identification image parsing unit provided by the embodiments of the present disclosure. The second identification image parsing unit includes an image processing subunit, a first judgment subunit, a decoding subunit, and a second identification image generation subunit. The actually displayed identification image is processed by each subunit of the second identification image parsing unit, and the specific process is as follows:

[0104] The image processing subunit is configured to process the actually displayed identification image according to the number of pixel points corresponding to a specific area of the transparency image, so as to obtain a second target identification image. Specifically, the image processing subunit is configured to assemble the display data of each pixel point in the actually displayed identification image into a second target identification image according to the number of pixel points corresponding to the specific area of the transparency image.

[0105] For example, the number of display data of each pixel point in the actually displayed identification image is N3, the number of pixel points corresponding to the specific area of the transparency image is N', and N3 = N'. During the process of assembling the display data, it is satisfied that the resolution L3*L3 of the second target identification image is N4 ≤ N', and L3 is the largest positive integer less than or equal to the following value.

[0106] In some examples, the encoding unit is configured to encode the display information of each pixel point of the first target identification image in sequence according to a first preset order, so as to obtain first encoded data arranged in a preset order. During the data assembly process of the above-mentioned image processing subunit, the display data of each pixel point in the actually displayed identification image can be assembled in sequence according to the first preset order. If the resolution L3*L3 of the second target identification image is N4 < N', the display data of other pixel points in the actually displayed identification image that have not been assembled are deleted to obtain the second target identification image.

[0107] Since the display image intercepted from the display terminal 101 may have unencoded images, the reason for this situation may be that the currently played multimedia file does not belong to the multimedia file in the play plan, that is, the play content is inaccurate. Or, it may also be that the multimedia file played by the display terminal 101 is the multimedia file in the play plan. Due to hardware problems of the display terminal 101, the intercepted display image is an incomplete image, and thus the intercepted display image may not have the encoded part of the image. Therefore, the first judgment subunit in the second identification image parsing unit is used to preliminarily verify the second target identification image. Specifically, the first judgment subunit is configured to judge whether there is target encoded data obtained by encoding the display information of each pixel point in the second target identification image. If it is judged that there is no target encoded data obtained by encoding the display information of each pixel point in the second target identification image, the display image is inaccurate.

[0108] When the number of pixel points in the specific area is equal to the number of pixel points of the first target identification image, the target encoded data is the first encoded data; when the number of pixel points in the specific area is equal to the number of pixel points of the first target identification image, the target encoded data includes the first encoded data and the second encoded data.

[0109] The decoding subunit is configured to decode the target encoded data to obtain decoded data when it is determined that there is target encoded data obtained by encoding the display information of each pixel point in the second target identification image.

[0110] Different target encoded data can be decoded into different display information (i.e., decoded data). For example, the first encoded sub-data is decoded into black, the second encoded sub-data is decoded into white (the first decoded sub-data), the second encoded data is decoded into black (the second decoded sub-data), etc.

[0111] The second identification image generation subunit is configured to generate a second identification image according to the resolution information of the first identification image and the decoded data.

[0112] Specifically, the second identification image generation subunit is specifically configured to scale the second target identification image with decoded data after decoding according to the resolution information of the first identification image, so that the resolution L4*L4 of the scaled second identification image is the same as the resolution L1*L1 of the first identification image.

[0113] When the first target identification image has a scaled identity information two-dimensional code and the playback plan is normal, the second target identification image has a two-dimensional code corresponding to the decoded data. The second identification image generation subunit is configured to scale the two-dimensional code corresponding to the decoded data according to the resolution information of the first identification image to obtain the second identification image.

[0114] The scaling method here is the same as the scaling method of "scaling the identity information two-dimensional code to obtain the first target identification image with the scaled identity information two-dimensional code". If the resolution L3*L3 of the second target identification image with decoded data after decoding is less than L1*L1, the second target identification image is enlarged according to the resolution L1*L1 of the first identification image. For example, pixels are filled by interpolation until the resolution of the generated second identification image is the same as the resolution of the first identification image. If the resolution L3*L3 of the second target identification image with decoded data after decoding is greater than L1*L1, the second target identification image is reduced according to the resolution L1*L1 of the first identification image. For example, some pixel points are removed until the resolution of the generated second identification image is the same as the resolution of the first identification image.

[0115] The first comparison unit is configured to compare the type of the second identification image with the type of the first identification image. If the types of the two are inconsistent, it indicates that the displayed image is incorrect.

[0116] Specifically, the type of the first identification image is a QR code. If the type of the second identification image is also a QR code, it is determined that the type of the second identification image is consistent with that of the first identification image; if the type of the second identification image is not an identification image in the form of a QR code, it is determined that the type of the second identification image is inconsistent with that of the first identification image. Specifically, the first comparison unit is configured to use a QR code reading function to read the QR code information of the second identification image. If the reading is successful, it indicates that the second identification image is an image of the QR code type. If the reading fails, it indicates that the type of the second identification image is not of the QR code type, and thus it is determined that the displayed image is incorrect.

[0117] The second identity authentication information parsing unit is configured to obtain the second identity authentication information according to the second identification image when the second identification image is consistent with the first identification image.

[0118] Specifically, the second identity authentication information parsing unit is configured to, when the second identification image is consistent with the first identification image, use an inverse identification generation function to inversely parse the second identification image according to a specific rule to obtain the second identity authentication information.

[0119] The inverse identification generation function is the inverse operation function of the above-mentioned identification generation function. Taking the second identification image as a QR code as an example, the identification generation function can be a QR code generation function, and the inverse identification generation function is the inverse operation process of the QR code generation function. The QR code is inversely parsed into the second identity authentication information by using the inverse operation process of the QR code generation function.

[0120] Since the displayed image currently played by the display terminal 101 may not be the multimedia content corresponding to the preset current play plan, it is possible that the scheduled play time of the current play plan is disrupted, and the displayed image currently played by the display terminal 101 is the multimedia content corresponding to another play plan in the play schedule. Therefore, a second comparison unit is set in the analysis module 13. The second comparison unit is configured to compare the second identity authentication information with the first identity authentication information to determine whether the displayed image is accurate.

[0121] Specifically, the second comparison unit is configured to compare the second identity authentication information with the first identity authentication information to determine the similarity of the two identity information. When the similarity of the identity information is greater than or equal to a preset similarity threshold, it is determined whether the displayed image is accurate. When the similarity of the identity information is less than the preset threshold, it is determined that the displayed image is incorrect. Among them, the preset similarity threshold can be determined according to the actual application scenario and experience, and the embodiments of the present disclosure do not make specific limitations.

[0122] In some examples, the analysis module 13 is further configured to, when it is determined that the displayed image is inaccurate, send a control instruction to cancel the display to the display terminal 101, so that the display terminal 101 cancels the display of the displayed image and generates an alarm message.

[0123] When the display terminal 101 receives the control instruction to cancel the display sent by the analysis module 13, it can use the playback card to turn off the screen and generate an alarm message. The alarm message can be displayed on the screen in text form; alternatively, the alarm message can also be played as an alarm prompt tone.

[0124] Taking the specific area of the transparent image as the edge area and the first identification image as the identity information QR code as an example, the process of monitoring each module in the multimedia monitoring system 100 will be described in detail below.

[0125] The first identification image generation unit is configured to generate an identity information QR code according to the first identity authentication information.

[0126] The transparent image generation unit is configured to generate a transparent image according to the resolution information of the display terminal 101; the resolution of the transparent image is the same as that of the display terminal 101.

[0127] The image processing unit is configured to process the identity information QR code according to the number of pixel points corresponding to the edge area of the pre-generated transparent image, and obtain a first target identification image with the scaled identity information QR code.

[0128] The encoding unit is configured to encode the display information of each pixel point of the first target identification image in the order from left to right, from top to bottom (i.e., the first preset order). For example, the first color pixel (e.g., black) is encoded as the first encoded sub-data, and the second color pixel (e.g., white) is encoded to obtain the second encoded sub-data. The first encoded data includes the first encoded sub-data and the second encoded sub-data.

[0129] The first verification image generation unit is configured to sequentially replace the display data of the corresponding pixel points in the edge area of the transparent image with the first encoded data in the order of "from left to right, from top to bottom, from right to left, and from bottom to top". When the number of pixel points in the edge area is greater than the number of pixel points of the first target identification image, the display data of the pixel points in the edge area of the transparent image, except for the pixel points corresponding to the first target identification image, is continued to be replaced with the second encoded data in the order of "from left to right, from top to bottom, from right to left, and from bottom to top" to obtain the first verification image.

[0130] The playback control module 12 is configured to generate the data to be displayed according to the received playback plan, the multimedia file corresponding to the playback plan, and the first verification image.

[0131] Specifically, the display terminal 101 responds to the data to be displayed for display and intercepts the currently displayed image at a preset frequency as the display image. Figure 6Schematic diagram of the displayed image provided by the embodiments of the present disclosure. As Figure 6 shown, it is the displayed image of the display terminal 101 under normal playback according to the playback plan, including the structural border of the display terminal 101, the screen black edge of the display terminal 101, the picture area of the display terminal 101, each pixel point and the edge area in the display terminal 101.

[0132] Taking the specific area of the transparent image as the edge area as an example, the image parsing unit is specifically configured to sequentially extract the display data of the pixel points corresponding in the edge area of the displayed image in the order of "from left to right, from top to bottom, from right to left, and from bottom to top", and obtain the identification image actually displayed by the display terminal 101 according to the extracted display data.

[0133] The image processing sub-unit is configured to assemble the display data of each pixel point in the actually displayed identification image in the order from left to right and from top to bottom (i.e., the first preset order) according to the number of pixel points corresponding to the edge area of the transparent image, and obtain the second target identification image.

[0134] The first determination sub-unit is configured to determine whether there is target encoded data (including the first encoded data and the second encoded data) after encoding the color information of each pixel point in the second target identification image.

[0135] The decoding sub-unit is configured to decode the target encoded data when it is determined that there is target encoded data after encoding the color information of each pixel point in the second target identification image. For example, decode the first encoded sub-data into the first color information, decode the second encoded sub-data into the second color information, and decode the second encoded data into the third color information (the third color information may be the first color information or the second color information). The decoded data includes the first color information, the second color information, and the third color information.

[0136] The second identification image generation sub-unit is configured to scale the second target identification image with decoded data after decoding to generate a second identification image. The resolution information of the second identification image is the same as the resolution information of the first identification image.

[0137] The first comparison unit is configured to call the two-dimensional code reading function to determine whether the second identification image can be read. If the reading fails, it indicates that the displayed image is incorrect.

[0138] The second identity authentication information parsing unit is configured to obtain the second identity authentication information according to the second identification image when the second identification image is successfully read.

[0139] The second comparison unit is configured to compare the second identity authentication information with the first identity authentication information to determine whether the displayed image is accurate.

[0140] Taking the specific area of the transparent image as the edge area and the first identification image as the QR code for identity information as an example, since the specific area of the transparent image is the edge area, it does not affect the user's viewing of the playback content of the multimedia file displayed on the display terminal 101, and can monitor the playback content of the display terminal 101 in a timely manner. The specific monitoring process: By taking a screenshot of the image currently displayed on the display terminal 101 and further analyzing the displayed image, it is possible to timely discover whether the playback content is played in the playback order and whether the playback content is accurate, greatly improving the security of multimedia monitoring and broadcasting. Compared with the manual patrol method in the traditional solution, the embodiment of the present disclosure can reduce the cost of multimedia monitoring and broadcasting and has high timeliness; compared with the optical monitoring and broadcasting solution using an external camera for the display screen, the multimedia monitoring and broadcasting system 100 provided by the embodiment of the present disclosure belongs to a system-built multimedia monitoring and broadcasting algorithm, which uses background data processing and can efficiently complete multimedia monitoring and broadcasting without external detection equipment.

[0141] In some examples, the information management module 14 is configured to encode the parameter information in the playback plan to generate the first identity authentication information.

[0142] The information management module 14 can be an Internet web terminal.

[0143] Referring to Table 1, the parameter information in the playback plan includes a playback plan identifier, the number of multimedia files, the type of playback window in the display terminal 101, the number of playback windows, the identifier of each multimedia file, the type of each multimedia file, the identifier of the playback window used to play each multimedia file, the playback order of each multimedia file, and the number of times each multimedia file is looped for playback.

[0144] Table 1

[0145]

[0146] The information management module 14 is configured to encode the parameter information in the play schedule. Referring to Table 1, the first 0 to 3 bytes represent the play schedule identifier, the 4th byte represents the number of multimedia files, the 5th byte represents the type of the play window in the display terminal 101, the 6th byte represents the number of play windows, the 7th to 10th bytes represent the identifier of the multimedia file, the 11th byte represents the type of the multimedia file, the 12th byte represents the identifier of the play window for playing the multimedia file, the 13th represents the play order of the multimedia file, and the 14th represents the number of times the multimedia file is looped. If a play schedule includes multiple multimedia files to be played, the parameter information of the second multimedia file is encoded using the next 8 bytes, and so on. A play schedule containing M multimedia files requires a total of 7 + 8M bytes. Among them, the first 7 bytes represent the attributes of the play schedule, and the next 8M bytes represent the play information of a multimedia file, where M is a positive integer.

[0147] The type of the play window can be represented by numbers. Among them, 1 represents one window; 2 represents two windows and are horizontally distributed; 3 represents two windows and are vertically distributed; 4 represents three windows, in one row and horizontally distributed; 5 represents three windows, in two rows, with two windows in the first row and one window in the second row; 6 represents three windows, in two rows, with one window in the first row and two windows in the second row.

[0148] The types of multimedia files include at least one of the following: video type, image type, text type, and audio type. The types of multimedia files can be represented by numbers. Among them, 1 represents the video type, that is, the.mp4 format; 2 represents the image type, that is, the.jpg format and.png format; 3 represents the file type, such as the.pptx format and.txt format; 4 represents the audio format, that is,.mp3.

[0149] For the above-mentioned play control module 12, in some examples, the play schedule includes the play window information and play order information of each multimedia file in the display terminal 101. Among them, the play window information includes the identifier of the play window, the number of play windows, and the type of the play window. The play order information includes the play order of each multimedia file in a play schedule.

[0150] The display terminal 101 may include at least one play window, and each play window can independently execute the play schedule, or multiple play windows play the multimedia file indicated by a play schedule together.

[0151] The playback control module 12 is specifically configured to generate data to be displayed based on the display area information and display sequence information of each multimedia file in the display terminal 101, as well as the multimedia file and the first verification image corresponding to the playback plan. The display terminal 101 is specifically configured to respond to the data to be displayed and display it according to the playback window information and display sequence information of each multimedia file in the display terminal 101.

[0152] The multimedia monitoring system 100 further includes a file distribution module 15. The distribution control module is configured to send the playback plan and the multimedia file corresponding to the playback plan to the playback control module 12. In some examples, the distribution control module is configured to receive the playback plan table sent by the information management module 14, and according to the playback plan table, send the current playback plan to be played and the multimedia file corresponding to the playback plan to the playback control module 12 simultaneously in the playback sequence of each multimedia file in the playback plan.

[0153] After the playback control module 12 receives the playback plan table and the multimedia file corresponding to the playback plan, the playback control module 12 is configured to match the multimedia file corresponding to the playback plan and the first verification image according to the parameter information of the current playback plan in the playback plan table, and generate the data to be displayed.

[0154] Taking the playback plan 2 in Table 1 as an example, the data to be displayed generated by the playback control module 12 includes the multimedia file x1, the multimedia file x2, the first verification image, the playback window of the multimedia file x1, the playback window of the multimedia file x2, the loop times of the multimedia file x1, and the loop times of the multimedia file x2. The display terminal 101 is specifically configured to respond to the data to be displayed, and play the multimedia file x1 (audio) on the first-line playback window and play the multimedia file x2 (video) on the second-line playback window.

[0155] Based on the same inventive concept, an embodiment of the present disclosure also provides a multimedia monitoring method corresponding to the multimedia monitoring system 100. Since the principle of solving problems by the multimedia monitoring method in the embodiment of the present disclosure is similar to that of the above multimedia monitoring system 100 in the embodiment of the present disclosure, the implementation of the multimedia monitoring method can refer to the implementation of the multimedia monitoring system 100, and the repeated parts will not be described again.

[0156] The following takes the platform where the multimedia monitoring system 100 is located as the execution subject to illustrate the multimedia monitoring method provided by the embodiment of the present disclosure.

[0157] In a second aspect, Figure 7 is a flowchart of a multimedia monitoring method provided by an embodiment of the present disclosure. As Figure 7 shown, the method includes steps S1 to S4:

[0158] S1. Generate a first verification image of the play schedule according to a pre-generated first identification image and the configuration information of the display terminal 101, and through an image generation algorithm; the first identification image is used to represent first identity authentication information; the first identity authentication information is used to represent the play schedule in the play schedule table.

[0159] In some examples, S1 can be implemented by the verification image generation module 11 in the multimedia monitoring system 100 in the above embodiment.

[0160] S2. Generate data to be displayed according to the received play schedule, the multimedia file corresponding to the play schedule, and the first verification image, so that the display terminal 101 performs display in response to the data to be displayed.

[0161] In some examples, S2 can be implemented by the play control module 12 in the multimedia monitoring system 100 in the above embodiment.

[0162] S3. Receive the display image sent by the display terminal 101, parse the display image, and after parsing out the second identification image in the display image and determining that the second identification image is consistent with the first identification image, obtain the second identity authentication information corresponding to the second identification image.

[0163] S4. Compare the second identity authentication information with the first identity authentication information, and judge whether the display image of the display terminal 101 is accurate according to the comparison result.

[0164] In some examples, S3 and S4 can be implemented by the analysis module 13 in the multimedia monitoring system 100 in the above embodiment.

[0165] In the embodiment of the present disclosure, without affecting the playback content of the multimedia file, the playback content of the display terminal 101 is monitored in a timely manner. Specifically, by taking a screenshot of the image currently displayed on the display terminal 101 and further parsing the display image, it is possible to quickly monitor whether the display image of the display terminal 101 is accurate, greatly improving the security of multimedia monitoring. Compared with the traditional method of manual patrol points, the embodiment of the present disclosure can reduce the cost of multimedia monitoring and has high timeliness; compared with the optical monitoring solution using an external camera for the display screen, the multimedia monitoring system 100 provided by the embodiment of the present disclosure belongs to a system-built multimedia monitoring algorithm, which uses background data processing and can efficiently complete multimedia monitoring without external detection equipment.

[0166] To clarify the multimedia monitoring method of the embodiments of the present disclosure, an exemplary monitoring method is given below. Among them, when the verification image generation module 11 in the above monitoring system includes a first identification image generation unit, an image processing unit, a transparent image generation unit, an encoding unit, and a first verification image generation unit; and the analysis module 13 includes an image parsing unit, a second identification image parsing unit, a first comparison unit, a second identity authentication information parsing unit, and a second comparison unit, this method can be implemented using the multimedia monitoring system 100. Below, taking the specific area of the transparent image as the edge area and the first identification image as the identity information QR code as an example for illustration. The method specifically includes the following steps:

[0167] S101. Generate a first identification image according to the first identity authentication information.

[0168] In some examples, S101 can be implemented by the first identification image generation unit. Specifically, use the identification generation function to arrange the identifiers of the first identity authentication information according to a specific rule, and generate a first identification image according to a specific geometric figure.

[0169] Exemplarily, the identification generation function can be a QR code generation function. Specifically, the QR code generation function can be called to generate an image of the identity information QR code with a resolution of L1*L1, and this image of the identity information QR code is the first identification image.

[0170] In the embodiments of the present disclosure, using the identity information QR code to represent the first identity authentication information can realize the encoding protection of the first identity authentication information, enhance the robustness of the first identity authentication information, that is, prevent the first identity authentication information from being contaminated by noise during storage and transmission, resulting in information extraction failure.

[0171] The multimedia monitoring system 100 further includes an information management module 14. The first identity authentication information can be generated by the information management module 14. Specifically, the parameter information in the play plan is encoded to generate the first identity authentication information.

[0172] S102. Generate a transparent image according to the resolution information of the display terminal 101; the resolution of the transparent image is the same as that of the display terminal 101.

[0173] In some examples, S102 can be implemented by the transparent image generation unit.

[0174] The transparent image has a middle area and an edge area surrounding the middle area. The specific area can include the edge area of the transparent image, or the top corner area of the transparent image, or a set area with an irregular shape of the transparent image, etc.

[0175] S103. Process the first identification image according to the number of pixel points corresponding to a specific area of the pre-generated transparent image to obtain a first target identification image.

[0176] In some examples, this S103 can be implemented by an image processing unit. Specifically, process the identity information QR code according to the number of pixel points corresponding to the edge area of the pre-generated transparent image to obtain a first target identification image with a scaled identity information QR code.

[0177] Here, the scaling method can refer to the scaling process of the above image processing unit. For example, the resolution information of the identity information QR code is L1*L1 = N1 pixel points. If the number of pixel points N1 of the identity information QR code is greater than or equal to the number of pixel points N' corresponding to the specific area of the transparent image, then scale the resolution of the identity information QR code proportionally to L2*L2, where L2*L2 = N2 ≤ N', and L2 is the largest positive integer less than or equal to √N', to obtain a first target identification image with a resolution of L2*L2. If the number of pixel points N1 of the identity information QR code is less than or equal to the number of pixel points corresponding to the specific area of the transparent image, then scale the resolution of the identity information QR code proportionally to L2*L2, where L2*L2 = N2 ≤ N', and L2 is the largest positive integer less than or equal to the largest positive integer within, to obtain a first target identification image with a resolution of L2*L2.

[0178] S104. Encode the display information of each pixel point of the first target identification image to obtain first encoded data.

[0179] The display information includes the information of the colors shown by the first target identification image, which can include information of different colors, such as including but not limited to black and white, etc.

[0180] In some examples, this S104 can be implemented by an encoding unit. Specifically, encode the display information of each pixel point of the first target identification image. Different display information can be encoded into different first encoded data. For example, black pixel points can be encoded into a first encoded sub-data (for example, 0), and white pixel points can be encoded into a second encoded sub-data (for example, 1), etc.

[0181] In some examples, in the order from left to right and from top to bottom (i.e., the first preset order), encode the display information of each pixel point of the first target identification image in sequence. For example, encode the first color pixel (for example, black) into a first encoded sub-data, and encode the second color pixel (for example, white) into a second encoded sub-data. The first encoded data includes the first encoded sub-data and the second encoded sub-data.

[0182] S105. Replace the display data of the pixel points corresponding to the specific area of the transparency image with the first encoded data to generate a first verification image.

[0183] The transparency image is a single-channel image, that is, an image with an alpha channel. The display data of the pixel points corresponding to the specific area of the transparency image can be the data of the alpha channel of the transparency image. For example, replace the data of the alpha channel of the pixel points corresponding to the specific area of the transparency image with the first encoded data to generate a transparency image with a QR code of scaled identity information (that is, the first verification image).

[0184] In some examples, this S105 can be implemented by a first verification image generation unit.

[0185] Specifically, replace the display data of the pixel points corresponding to the specific area of the transparency image with the first encoded data in sequence according to the second preset order to generate a first verification image. In order to be able to finally display a relatively regular picture on the display terminal 101, when the number of pixel points in the specific area is greater than the number of pixel points of the first target identification image, while replacing the display data of the pixel points corresponding to the specific area of the transparency image with the first encoded data, replace the display data of the pixel points in the specific area of the transparency image, except for the pixel points corresponding to the first target identification image, with the second encoded data to obtain the first verification image.

[0186] The pixel points other than the pixel points corresponding to the first target identification image, that is, among the N' pixel points in the specific area, except for the N2 pixel points in the first target identification image.

[0187] Taking the edge area of the transparency image as an example, the first encoded data of each pixel point recorded in the first target identification image in the preset order can be replaced with the display data of the corresponding pixel points in the edge area of the transparency image in the order of "from left to right, from top to bottom, from right to left, and from bottom to top" in sequence. When the number of pixel points in the edge area is greater than the number of pixel points of the first target identification image, the display data of the pixel points in the edge area of the transparency image, except for the pixel points corresponding to the first target identification image, are continuously replaced with the second encoded data in the order of "from left to right, from top to bottom, from right to left, and from bottom to top" in sequence to obtain the first verification image.

[0188] It should be noted that in this example, the second preset order is not limited to the order of "from left to right, from top to bottom, from right to left, from bottom to top", and can be defined according to experience and actual application scenarios. The non-disclosed embodiments will not be listed one by one.

[0189] Since the second encoded data is not involved in the subsequent judgment of the accuracy of the displayed screen of the display terminal 101, the second encoded data may be randomly set encoded data, or may also be the same as the first encoded data or the second encoded data. The embodiments of the present disclosure do not limit this.

[0190] The multimedia monitoring and broadcasting system 100 further includes a file distribution module 15. The step of simultaneously sending the current playback plan to be played and the multimedia file corresponding to the playback plan to the playback control module 12 can be implemented by the file distribution module 15.

[0191] S106. Generate the data to be displayed according to the received playback plan, the multimedia file corresponding to the playback plan, and the first verification image.

[0192] In some examples, S106 can be implemented by the playback control module 12. Specifically, according to the display area information and display order information of each multimedia file in the display terminal 101, as well as the multimedia file corresponding to the playback plan and the first verification image, generate the data to be displayed, so that the display terminal 101 displays according to the playback window information and playback order information of each multimedia file in the display terminal 101.

[0193] The display terminal 101 displays in response to the data to be displayed, intercepts the current display image at a preset frequency, and sends the display image to the analysis module 13. The display terminal 101 may include a display screen for displaying the screen corresponding to the multimedia file. Specifically, the display terminal 101 responds to the data to be displayed, controls the playback of the multimedia file by using the playback card in the display terminal 101, intercepts the image displayed on the current display screen at a preset frequency, and uses this image as the display image for subsequent timely monitoring of the accuracy of the multimedia content currently played by the display terminal 101.

[0194] S107. Analyze the display image to obtain the identification image actually displayed by the display terminal 101.

[0195] In some examples, S107 can be implemented by an image parsing unit. Taking the specific area of the transparent image as the edge area as an example, in the order of "from left to right, from top to bottom, from right to left, and from bottom to top", sequentially extract the display data of the corresponding pixel points in the edge area of the display image, and obtain the identification image actually displayed by the display terminal 101 according to the extracted display data.

[0196] S108. Process the actually displayed identification image, and obtain a second identification image according to the resolution information of the first identification image.

[0197] In some examples, the S108 may be implemented by a second identification image parsing unit. The second identification image parsing unit includes an image processing sub-unit, a first determination sub-unit, a decoding sub-unit, and a second identification image generation sub-unit. S108 specifically includes the following steps (S108-1 to S108-4):

[0198] S108-1. Process the actually displayed identification image according to the number of pixel points corresponding to the specific area of the transparent image to obtain a second target identification image.

[0199] In some examples, the S108-1 may be implemented by the image processing sub-unit. Specifically, the display data of each pixel point in the actually displayed identification image is assembled into a second target identification image according to the number of pixel points corresponding to the specific area of the transparent image.

[0200] Taking the edge area of the transparent image as an example of the specific area, according to the number of pixel points corresponding to the edge area of the transparent image, the display data of each pixel point in the actually displayed identification image is assembled in sequence from left to right and top to bottom (i.e., the first preset order) to obtain a second target identification image.

[0201] The number of display data of each pixel point in the actually displayed identification image is N3, and the number of pixel points corresponding to the specific area of the transparent image is N', and N3 = N'. During the process of assembling the display data, it is satisfied that the resolution L3*L3 of the second target identification image is N4 ≤ N', and L3 is the largest positive integer less than or equal to .

[0202] In some examples, the display information of each pixel point of the first target identification image is encoded in sequence according to the first preset order to obtain first encoded data arranged in a preset order. During the data assembly process of the above image processing sub-unit, the display data of each pixel point in the actually displayed identification image can be assembled in sequence according to the first preset order. If the resolution L3*L3 of the second target identification image is N4 < N', the display data of other pixel points in the actually displayed identification image that have not been assembled is deleted to obtain a second target identification image.

[0203] S108-2. Determine whether there is target encoded data obtained by encoding the display information of each pixel point in the second target identification image.

[0204] In some examples, the S108-2 can be implemented by a first determination subunit. Specifically, it is determined whether there is target encoded data (including first encoded data and second encoded data) obtained by encoding the color information of each pixel point in the second target identification image. If it is determined that there is no target encoded data obtained by encoding the display information of each pixel point in the second target identification image, the displayed image is inaccurate.

[0205] S108-3. When it is determined that there is target encoded data obtained by encoding the display information of each pixel point in the second target identification image, decode the target encoded data to obtain decoded data.

[0206] In some examples, the S108-3 can be implemented by a decoding subunit.

[0207] When the number of pixel points in a specific area is equal to the number of pixel points in the first target identification image, the target encoded data is the first encoded data; when the number of pixel points in a specific area is equal to the number of pixel points in the first target identification image, the target encoded data includes the first encoded data and the second encoded data.

[0208] Specifically, when it is determined that there is target encoded data obtained by encoding the color information of each pixel point in the second target identification image, decode the target encoded data. For example, decode the first encoded sub-data into first color information, decode the second encoded sub-data into second color information, and decode the second encoded data into third color information (the third color information can be the first color information or the second color information). The decoded data includes the first color information, the second color information, and the third color information.

[0209] S108-4. Generate a second identification image according to the resolution information of the first identification image and the decoded data.

[0210] In some examples, the S108-4 can be implemented by a second identification image generation subunit. Specifically, according to the resolution information of the first identification image, scale the second target identification image with decoded data after decoding so that the resolution L4*L4 of the scaled second identification image is the same as the resolution L1*L1 of the first identification image.

[0211] When the first target identification image has a scaled identity information QR code and the playback plan is normal, the second target identification image has a QR code corresponding to the decoded data. Specifically, according to the resolution information of the first identification image, scale the QR code corresponding to the decoded data to obtain the second identification image.

[0212] The scaling method here is the same as the scaling method in "scaling the identity information QR code to obtain the first target identification image with the scaled identity information QR code". If the resolution L3*L3 of the second target identification image with decoded data is less than L1*L1, the second target identification image is enlarged according to the resolution L1*L1 of the first identification image. For example, pixels are filled using the interpolation method until the resolution of the generated second identification image is the same as that of the first identification image. If the resolution L3*L3 of the second target identification image with decoded data is greater than L1*L1, the second target identification image is reduced according to the resolution L1*L1 of the first identification image. For example, some pixel points are removed until the resolution of the generated second identification image is the same as that of the first identification image.

[0213] S109. Compare the type of the second identification image with the type of the first identification image. If the two types are inconsistent, it indicates that the image is incorrect.

[0214] In some examples, S109 can be implemented using a first comparison unit. Specifically, the type of the first identification image is a QR code. If the type of the second identification image is also a QR code, it is determined that the type of the second identification image is the same as that of the first identification image. If the type of the second identification image is not an identification image in the form of a QR code, it is determined that the type of the second identification image is different from that of the first identification image. Specifically, using the QR code reading function, the QR code information of the second identification image is read. If the reading is successful, it indicates that the second identification image is an image of the QR code type. If the reading fails, it indicates that the type of the second identification image is not of the QR code type, and thus it is determined that the displayed image is incorrect.

[0215] S110. When the type of the second identification image is the same as that of the first identification image, obtain the second identity authentication information according to the second identification image.

[0216] In some examples, S110 can be implemented using a second identity authentication information parsing unit.

[0217] Specifically, when the second identification image is the same as the first identification image, the second identification image is inversely parsed according to a specific rule using the inverse function of the identification generation to obtain the second identity authentication information.

[0218] The inverse function of the identification generation is the inverse operation function of the above-mentioned identification generation function. Taking the second identification image as a QR code as an example, the identification generation function can be a QR code generation function, and the inverse function of the identification generation is the inverse operation process of the QR code generation function. The QR code is inversely parsed into the second identity authentication information using the inverse operation process of the QR code generation function.

[0219] S111. Compare the second identity authentication information with the first identity authentication information to determine whether the displayed image is accurate. If it is determined that the displayed image is accurate, the process ends; if it is determined that the displayed image is incorrect, execute S112.

[0220] In some examples, this S111 can be implemented by a second comparison unit.

[0221] Specifically, compare the second identity authentication information with the first identity authentication information to determine the similarity of the two identity information. When the similarity of the identity information is greater than or equal to a preset similarity threshold, determine whether the displayed image is accurate. When the similarity of the identity information is less than the preset threshold, determine that the displayed image is incorrect. Among them, the preset similarity threshold can be determined according to the actual application scenario and experience, and the embodiments of the present disclosure do not make specific limitations.

[0222] S112. When it is determined that the displayed image is inaccurate, send a control instruction to cancel the display to the display terminal 101, so that the display terminal 101 cancels the display of the displayed image and generates an alarm message.

[0223] In some examples, this S112 can be implemented by a second comparison unit.

[0224] When the display terminal 101 receives the control instruction to cancel the display sent by the analysis module 13, it can turn off the screen using the playback card and generate an alarm message. The alarm message can be displayed on the screen in text form; or, the alarm message can also be played as an alarm prompt tone.

[0225] Based on the same inventive concept, the embodiments of the present application also provide a computer device. Refer to Figure 8 As shown, it is a schematic structural diagram of the computer device provided by the embodiments of the present application, including:

[0226] A processor 81, a memory 82, and a bus 83. Among them, the memory 82 stores machine-readable instructions executable by the processor 81. The processor 81 is used to execute the machine-readable instructions stored in the memory 82. When the machine-readable instructions are executed by the processor 81, the processor 81 performs the following steps: S1. Generate a first verification image of the playback plan according to a pre-generated first identification image and the configuration information of the display terminal 101, and through an image generation algorithm; the first identification image is used to represent the first identity authentication information; the first identity authentication information is used to represent the playback plan in the playback schedule. S2. Generate data to be displayed according to the received playback plan, the multimedia file corresponding to the playback plan, and the first verification image, so that the display terminal 101 displays in response to the data to be displayed. S3. Receive the display image sent by the display terminal 101, parse the display image, and after parsing out the second identification image in the display image and determining that the second identification image is consistent with the first identification image, obtain the second identity authentication information corresponding to the second identification image. S4. Compare the second identity authentication information with the first identity authentication information, and determine whether the display image of the display terminal 101 is accurate according to the comparison result.

[0227] The above-mentioned memory 82 includes a memory 821 and an external memory 822; here, the memory 821 is also called an internal memory, which is used to temporarily store the operation data in the processor 81 and the data exchanged with the external memory 822 such as a hard disk. The processor 81 exchanges data with the external memory 822 through the memory 821. When the computer device is running, the processor 81 communicates with the memory 82 through the bus 83, so that the processor 81 executes the execution instructions mentioned in the above method embodiments.

[0228] The embodiments of the present disclosure also provide a computer non-transitory readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the multimedia monitoring method described in the above method embodiments. Among them, the storage medium can be a volatile or non-volatile computer non-transitory readable storage medium.

[0229] The embodiments of the present disclosure also provide an electronic device, including the multimedia monitoring system 100 and the display terminal 101 in the above-mentioned disclosed embodiments.

[0230] It can be understood that the above embodiments are merely exemplary embodiments adopted to illustrate the principles of the present invention / utility model. However, the present invention / utility model is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention / utility model, and these modifications and improvements are also regarded as the protection scope of the present invention / utility model.

Claims

1. A multimedia monitoring method, wherein, including: generating a first verification image of a play schedule according to a pre-generated first identification image and configuration information of a display terminal, and through an image generation algorithm; the first identification image is used to represent first identity authentication information; the first identity authentication information is used to represent the play schedule in the play schedule table; generating display data to be displayed according to the received play schedule, a multimedia file corresponding to the play schedule, and the first verification image, so that the display terminal performs display in response to the display data to be displayed; receiving a display image sent by the display terminal, parsing the display image, and after parsing out a second identification image in the display image and determining that the second identification image is consistent with the first identification image, obtaining second identity authentication information corresponding to the second identification image; comparing the second identity authentication information with the first identity authentication information, and judging whether the display image of the display terminal is accurate according to the comparison result; the generating a first verification image of a play schedule according to a pre-generated first identification image and configuration information of the display terminal, and through an image generation algorithm, includes: generating a first identification image according to the first identity authentication information; processing the first identification image according to the number of pixel points corresponding to a specific area of a pre-generated transparent image, to obtain a first target identification image; the transparent image has a middle area and an edge area surrounding the middle area, and the specific area includes the edge area of the transparent image, or the top corner area of the transparent image, or a set area with an irregular shape of the transparent image; encoding display information of each pixel point of the first target identification image, to obtain first encoded data; replacing display data of pixel points corresponding to the specific area of the transparent image with the first encoded data, to generate a first verification image.

2. The multimedia monitoring method according to claim 1, wherein, after generating the first identification image according to the first identity authentication information, further including: generating the transparent image according to resolution information of the display terminal; the resolution of the transparent image is consistent with the resolution of the display terminal.

3. The multimedia monitoring method according to claim 1, wherein, the transparent image has a middle area and an edge area surrounding the middle area; the specific area is the edge area.

4. The multimedia monitoring method according to claim 1, wherein, when the number of pixel points in the specific area is greater than the number of pixel points of the first target identification image, the replacing display data of pixel points corresponding to the specific area of the transparent image with the first encoded data, to generate a first verification image, includes: while replacing display data of pixel points corresponding to the specific area of the transparent image with the first encoded data, replacing display data of pixel points in the specific area of the transparent image, except for pixel points corresponding to the first target identification image, with second encoded data, to obtain the first verification image.

5. The multimedia monitoring method according to claim 1, wherein, specific steps of parsing the display image to judge whether the display image of the display terminal is accurate include: parsing the display image to obtain an identification image actually displayed by the display terminal; processing the actually displayed identification image, and according to resolution information of the first identification image, obtaining the second identification image; Compare the second identification image with the first identification image. If they are inconsistent, it indicates that the displayed image is incorrect; When the second identification image is consistent with the first identification image, obtain second identity authentication information based on the second identification image; Compare the second identity authentication information with the first identity authentication information to determine whether the displayed image is accurate.

6. The multimedia monitoring method according to claim 5, wherein, The processing of the actually displayed identification image and obtaining the second identification image according to the resolution information of the first identification image includes: Process the actually displayed identification image according to the number of pixel points corresponding to the specific area of the transparent image to obtain a second target identification image; Determine whether there is target encoded data obtained by encoding the display information of each pixel point in the second target identification image; When it is determined that there is target encoded data obtained by encoding the display information of each pixel point in the second target identification image, decode the target encoded data to obtain decoded data; Generate a second identification image according to the resolution information of the first identification image and the decoded data.

7. The multimedia monitoring method according to claim 6, wherein The encoding of the display information of each pixel point of the first target identification image to obtain first encoded data includes: Encode the display information of each pixel point of the first target identification image in sequence according to a first preset order to obtain first encoded data; The processing of the actually displayed identification image according to the number of pixel points corresponding to the specific area of the transparent image to obtain a second target identification image includes: Assemble the display data of each pixel point in the actually displayed identification image in sequence according to the first preset order according to the number of pixel points corresponding to the specific area of the transparent image to obtain a second target identification image.

8. The multimedia monitoring method according to claim 5, wherein, The replacement of the display data of the pixel points corresponding to the specific area of the transparent image with the first encoded data to generate a first verification image includes: Replace the display data of the pixel points corresponding to the specific area of the transparent image with the first encoded data in sequence according to a preset order to generate the first verification image; The parsing of the displayed image to obtain the identification image actually displayed by the display terminal includes: Extract the display data of the pixel points corresponding to the specific area of the displayed image in sequence according to the preset order, and obtain the identification image actually displayed by the display terminal based on the extracted display data.

9. The multimedia monitoring method according to claim 1, wherein, The playback plan includes the playback window information and playback order information of each multimedia file in the display terminal; The generation of the data to be displayed according to the received playback plan, the multimedia file corresponding to the playback plan, and the first verification image includes: Generate the data to be displayed according to the display area information and display order information of each multimedia file in the display terminal, the multimedia file corresponding to the playback plan, and the first verification image, so that the display terminal responds to the data to be displayed and displays according to the playback window information and playback order information of each multimedia file in the display terminal.

10. The multimedia monitoring method according to claim 1, wherein, When analyzing the displayed image, it further includes: When it is determined that the displayed image is inaccurate, a control instruction to cancel the display is sent to the display terminal, so that the display terminal cancels the display of the displayed image and generates an alarm message.

11. The multimedia monitoring method according to claim 1, wherein, It further includes: Encoding the parameter information in the play plan to generate the first identity authentication information.

12. A multimedia monitoring and broadcasting system, which includes a verification image generation module, a play control module, and an analysis module. Among them, The verification image generation module is configured to generate a first verification image of the play plan according to a pre-generated first identification image and the configuration information of the display terminal, and through an image generation algorithm; The first identification image is used to represent the first identity authentication information; The first identity authentication information is used to represent the play plan in the play schedule; The play control module is configured to generate data to be displayed according to the received play plan, the multimedia file corresponding to the play plan, and the first verification image, for the display terminal to display; The analysis module is configured to parse the display image intercepted by the display terminal at a preset frequency, and after parsing out the second identification image, and determining that the second identification image is consistent with the first identification image, obtain the second identity authentication information corresponding to the second identification image, compare the second identity authentication information with the first identity authentication information, and determine whether the display of the display terminal is accurate according to the comparison result; Among them, the verification image generation module includes a first identification image generation unit, an image processing unit, an encoding unit, and a first verification image generation unit; The first identification image generation unit is configured to generate a first identification image according to the first identity authentication information; The image processing unit is configured to process the first identification image according to the number of pixel points corresponding to a specific area of a pre-generated transparent image. The transparent image has a middle area and an edge area surrounding the middle area. The specific area includes the edge area of the transparent image, or the top corner area of the transparent image, or a set area with an irregular shape of the transparent image; The encoding unit is configured to encode the display information of each pixel point of the first target identification image to obtain first encoded data; The first verification image generation unit is configured to replace the display data of the pixel points corresponding to the specific area of the transparent image with the first encoded data to generate a first verification image.

13. A computer device, characterized in that, It includes: A processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the multimedia monitoring and broadcasting method according to any one of claims 1 to 11 are executed.

14. A computer non-transitory readable storage medium, characterized in that, A computer program is stored on this non-transitory computer-readable storage medium. When the computer program is run by the processor, the steps of the multimedia monitoring and broadcasting method according to any one of claims 1 to 11 are executed.

15. An electronic device, wherein, Including the multimedia monitoring system and display terminal described in claim 12.

Images