Information processing method and device of display control system

Abstract

The invention provides an information processing method and device of a display control system. The display control system comprises a multimedia information processing unit, a client and a display screen; the information processing method comprises the following steps: the multimedia information processing unit receives multi-channel multimedia information sent by a multimedia signal source; the multimedia information processing unit is provided with an image processing module for detecting target information; image information in the multi-path multimedia information is processed through an image processing module, and a target information detection result is determined; based on the target information detection result, determining target indication information and sending the target indication information to a client; target multimedia information for the display screen is determined based on the display content control rule corresponding to the target information detection result, and the target multimedia information is sent to the display screen to be displayed, so that the display content of the display screen in the display control system is controlled to meet the requirement, the detection efficiency is high, and the error rate is low.

Description

Technical Field

[0001] The present application relates to the field of multimedia data transmission, and in particular to an information processing method and device for a display control system. Background Art

[0002] With the progress of society, the application of display and control systems can be seen everywhere in production and life.

[0003] For example, various video information displayed on large screens in public places, or live broadcasts of various large command centers, all use spliced ​​large screens. Most of the content displayed on large screens comes from network video sources, live video sources, local video sources, etc. In actual applications, large screen display content is frequently misplayed, which can easily cause adverse effects. For example, the wrong video is played, sensitive images appear on the large screen during a meeting, etc. In the prior art, the video source displayed on the screen often needs to be manually screened, and manual screening will inevitably lead to mistakes, such as mistakes caused by not noticing sensitive information, mistakes caused by not checking each frame, etc. At the same time, this work can be very boring.

[0004] For example, the display and control system of a factory production line, in the existing technology, determines whether the monitored content is compliant based on the video images, but it still requires manual judgment, which is prone to errors and inefficient. Summary of the invention

[0005] In view of this, the purpose of the present application is to provide an information processing method and device for a display control system, which can monitor the pictures in the multimedia information in the display control system, so as to control the display content of the display screen in the display control system to meet the requirements, with high detection efficiency and low error rate.

[0006] An information processing method of a display control system provided in an embodiment of the present application is applied to a display control system, wherein the display control system includes a multimedia information processing unit, a client, and a display screen; the information processing method includes:

[0007] The multimedia information processing unit receives multiple channels of multimedia information sent by a multimedia signal source; the multimedia information processing unit is configured with an image processing module for detecting target information;

[0008] The multimedia information processing unit processes the image information in the multiple multimedia information channels through the image processing module, and determines the target information detection result of each multimedia information channel; the multimedia information processing unit is configured with display content control rules corresponding to different types of target information detection results; different display control systems correspond to different display content control rules; the display content control rules are used to control the display screen to display multimedia information that meets the preset detection requirements;

[0009] The multimedia information processing unit determines target indication information based on the target information detection result, and sends the target indication information to the client;

[0010] The multimedia information processing unit determines the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, and sends the target multimedia information to the display screen for display.

[0011] In some embodiments, in the information processing method of the display control system, when the multiple channels of multimedia information are multiple channels of video, the multimedia information processing unit further includes a control module and a video encoding module;

[0012] The multimedia information processing unit processes the image information in the multiple multimedia information by the image processing module to determine the target information detection result of each multimedia information, including:

[0013] The multimedia information processing unit processes each image frame of each video through the image processing module, detects whether there is target information in each image frame, and determines the target information detection result of each image frame.

[0014] In some embodiments, in the information processing method of the display control system, the multimedia information processing unit determines the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, and sends the target multimedia information to the display screen, including:

[0015] When the target information detection results of each image frame of each video are normal, the multimedia information processing unit re-encodes the image frame into multiple videos through the video encoding module, and sends the multiple videos to the display screen for display;

[0016] When an abnormal image frame whose target information detection result is abnormal is detected, first abnormal target information in the abnormal image frame is determined, and an abnormal video in which the abnormal image frame is detected is determined;

[0017] The control module of the multimedia information processing unit controls to close the abnormal video, obtains the target normal video to replace the abnormal video, and sends the target normal video and other videos to the display screen for display.

[0018] In some embodiments, in the information processing method of the display control system, the step of acquiring a target normal video to replace the abnormal video includes:

[0019] Directly retrieve the normal video to replace the abnormal video;

[0020] or,

[0021] The control module processes the abnormal image frame based on the target image frame processing strategy to obtain a normal image frame, and re-encodes the normal image frame into a target normal video; different types of the first abnormal target information in the multimedia target image frame correspond to different image frame processing strategies.

[0022] In some embodiments, in the information processing method of the display control system, the control module processes the abnormal image frame based on the target image frame processing strategy to obtain a normal image frame, and re-encodes the normal image frame into a target normal video, including:

[0023] Based on the attribute of the first abnormal target information in the target image frame, determining whether a preset blur condition is met;

[0024] If it is in compliance, the control module performs fuzzy processing on the first abnormal target information in the abnormal image frame to obtain a normal image frame;

[0025] If not, the control module obtains the number of abnormal image frames and calls the target backup video; the frame number difference between the target backup video and the abnormal image frame is within a preset difference range;

[0026] The abnormal image frame is replaced by the image frame of the target spare video to obtain a normal image frame.

[0027] In some embodiments, in the information processing method of the display control system, the multimedia information processing unit determines the target indication information based on the target information detection result, and sends the target indication information to the client, including:

[0028] When an abnormal image frame whose target information detection result is abnormal is detected, the image processing module of the multimedia information processing unit sends the abnormal target information detection result to the control module, and the control module generates target indication information based on the abnormal target information detection result, and sends the target indication information to the client;

[0029] The client generates scheduling information for the target indication information in response to a scheduling operation for a specific type of target indication information, and sends the scheduling information to a control module; the scheduling information is used to determine the content of the display content control rule.

[0030] In some embodiments, in the information processing method of the display control system, before the multimedia information processing unit receives the multi-channel multimedia information sent by the multimedia signal source, the information processing method further includes:

[0031] A plurality of devices to be monitored in the production line are connected to a host computer, and the host computer captures target data of the devices in the production line as image data based on a pre-configured device control tool; the devices include production equipment and / or testing equipment;

[0032] The host computer saves the image data of different devices in groups to obtain multiple channels of image data;

[0033] The host computer sends the multiple channels of image data to the multimedia information processing unit.

[0034] In some embodiments, in the information processing method of the display control system, the multimedia information processing unit processes the image information in the multiple multimedia information channels through the image processing module to determine the target information detection result of each multimedia information channel, including:

[0035] The multimedia information processing unit processes the image data of the different devices through the image processing module to determine whether there is second abnormal target information in the image data of each device;

[0036] If so, determining that the target information detection result of the device image data is image data abnormality;

[0037] If not, determining that the target information detection result of the device image data is that the image data is normal;

[0038] Correspondingly, the multimedia information processing unit determines the target indication information based on the target information detection result, and sends the target indication information to the client, including:

[0039] The target information detection result of the image data anomaly and the corresponding image data are sent to the client.

[0040] In some embodiments, in the information processing method of the display control system, the multimedia information processing unit determines the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, and sends the target multimedia information to the display screen for display, including:

[0041] The target multimedia information for the display screen is determined to be normal image data, and image data of the normal image data is sent to the display screen for display.

[0042] In some embodiments, an information processing device of a display and control system is further provided, which is applied to the display and control system. The display and control system includes a multimedia information processing unit, a client, and a display screen; the information processing device includes:

[0043] A receiving module, used to enable the multimedia information processing unit to receive multiple multimedia information sent by a multimedia signal source; the multimedia information processing unit is equipped with an image processing module for detecting target information;

[0044] A first determination module is used to enable the multimedia information processing unit to process the image information in the multiple multimedia information channels through the image processing module, and determine the target information detection result of each multimedia information channel; the multimedia information processing unit is configured with display content control rules corresponding to different types of target information detection results; different display control systems correspond to different display content control rules; the display content control rules are used to control the display screen to display multimedia information that meets the preset detection requirements;

[0045] A second determination module, configured to enable the multimedia information processing unit to determine target indication information based on the target information detection result, and send the target indication information to the client;

[0046] The third determination module is used to enable the multimedia information processing unit to determine the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, and send the target multimedia information to the display screen for display.

[0047] In some embodiments, a method and device for processing information of a display and control system are provided, which are applied to the display and control system, wherein the display and control system includes a multimedia information processing unit, a client and a display screen; the information processing method includes: the multimedia information processing unit receives multiple multimedia information sent by a multimedia signal source; the multimedia information processing unit is configured with an image processing module for detecting target information; the multimedia information processing unit processes the image information in the multiple multimedia information through the image processing module, and determines the target information detection result of each multimedia information; the multimedia information processing unit is configured with display content control rules corresponding to different types of target information detection results; different display and control systems correspond to different display content control rules; the display content control rules are used to control the display screen to display multimedia information that meets preset detection requirements; the multimedia information processing unit determines target indication information based on the target information detection result, and sends the target indication information to the client; the multimedia information processing unit determines the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, so that comprehensive supervision and control of multiple multimedia information can be achieved through one client, and the detection efficiency is high and the error rate is low, so as to ensure that the content displayed on the display screen in the display and control system meets the requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 A flowchart showing an information processing method of a display control system according to an embodiment of the present application is shown;

[0050] Figure 2 A schematic diagram of the structure of a display and control system according to an embodiment of the present application is shown;

[0051] Figure 3 A flow chart of a method for determining target multimedia information for the display screen according to an embodiment of the present application is shown;

[0052] Figure 4 A flow chart of a method for the control module in an embodiment of the present application to process the abnormal image frame based on the target image frame processing strategy is shown;

[0053] Figure 5 A flowchart showing an information processing method of another display control system according to an embodiment of the present application is shown;

[0054] Figure 6 A schematic diagram of the structure of another display and control system according to an embodiment of the present application is shown;

[0055] Figure 7 A schematic diagram showing the structure of an information processing device of a display control system according to an embodiment of the present application is shown;

[0056] Figure 8 A schematic structural diagram of an electronic device described in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0057] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0058] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0059] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0060] With the progress of society, the application of display and control systems can be seen everywhere in production and life.

[0061] For example, various video information displayed on large screens in public places, or live broadcasts of various large command centers, all use spliced ​​large screens. Most of the content displayed on large screens comes from network video sources, live video sources, local video sources, etc. In actual applications, large screen display content is frequently misplayed, which can easily cause adverse effects. For example, the wrong video is played, sensitive images appear on the large screen during a meeting, etc. In the prior art, the video source displayed on the screen often needs to be manually screened, and manual screening will inevitably lead to mistakes, such as mistakes caused by not noticing sensitive information, mistakes caused by not checking each frame, etc. At the same time, this work can be very boring.

[0062] For example, the display and control system of a factory production line, in the existing technology, determines whether the monitored content is compliant based on the video images, but it still requires manual judgment, which is prone to errors and inefficient.

[0063] Based on this, in some embodiments, a method and device for processing information of a display and control system are provided, which are applied to a display and control system, wherein the display and control system includes a multimedia information processing unit, a client and a display screen; the information processing method includes: the multimedia information processing unit receives multiple multimedia information sent by a multimedia signal source; the multimedia information processing unit is configured with an image processing module for detecting target information; the multimedia information processing unit processes the image information in the multiple multimedia information through the image processing module, and determines the target information detection result of each multimedia information; the multimedia information processing unit is configured with display content control rules corresponding to different types of target information detection results; different display and control systems correspond to different display content control rules; the display content control rules are used to control the display screen to display multimedia information that meets preset detection requirements; the multimedia information processing unit determines target indication information based on the target information detection result, and sends the target indication information to the client; the multimedia information processing unit determines the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, so that comprehensive supervision and control of multiple multimedia information can be achieved through one client, and at the same time, the detection efficiency is high and the error rate is low, so as to ensure that the content displayed on the display screen of the display and control system meets the requirements.

[0064] Based on this, please refer to Figure 1 , Figure 1 A flowchart of the information processing method of the display control system according to the embodiment of the present application is shown; Figure 1 As shown, the information processing method of the display control system is applied to the display control system, and the display control system includes a multimedia information processing unit, a client and a display screen; the information processing method includes the following steps S101-S104:

[0065] S101, the multimedia information processing unit receives multiple channels of multimedia information sent by a multimedia signal source; the multimedia information processing unit is configured with an image processing module for detecting target information;

[0066] S102, the multimedia information processing unit processes the image information in the multiple multimedia information through the image processing module, and determines the target information detection result of each multimedia information; the multimedia information processing unit is configured with display content control rules corresponding to different types of target information detection results; different display control systems correspond to different display content control rules; the display content control rules are used to control the display screen to display multimedia information that meets the preset detection requirements;

[0067] S103, the multimedia information processing unit determines target indication information based on the target information detection result, and sends the target indication information to the client;

[0068] S104: The multimedia information processing unit determines the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, and sends the target multimedia information to the display screen for display.

[0069] The display control system usually includes a client and a display screen. The display screen is the part of the display control system that directly faces the user and is responsible for presenting various information in a visual way. This information may include real-time monitoring data, process flow, equipment status, navigation information, various videos, etc., depending on the application field of the display control system; the display screen is used to display content, and the client is the interface for users to interact with the display control system. Users can input instructions, adjust parameters, view data, etc. through the client.

[0070] The display and control system can be applied to various fields, such as a large screen for playing videos, or a display and control system for a factory production line; different display and control systems have different specific structures.

[0071] In the step S101, the multimedia information processing unit receives multiple channels of multimedia information sent by a multimedia signal source; the multimedia information processing unit is configured with an image processing module for detecting target information.

[0072] The multimedia information may be multiple videos, multiple image data files, etc.

[0073] The target detection information is information that the display and control system needs to recognize in a target scene in a target field, such as specific text, logo, or image category on an image, or more complex objects, scenes, or behaviors, etc.

[0074] For example, specific text information is detected and recognized in images or videos, such as license plate numbers, product labels, slogans, etc. This is of great significance for scenarios such as traffic monitoring, product identification, and advertising monitoring.

[0075] Based on the target information to be identified, the image processing module may adopt an algorithm and / or an image processing model.

[0076] That is to say, the multimedia information processing unit is used to detect whether multiple channels of multimedia information include target information.

[0077] As an example only, identify specific logos in an image, such as trademarks, QR codes, etc.

[0078] Image classification, that is, classifying objects in images into predefined categories, such as faces, vehicles, animals, buildings, etc.

[0079] Complex Objects and Scenes: Identify and understand multiple objects and their relationships in images in more complex scenarios, such as crowd behavior analysis.

[0080] Behavior recognition: Detecting and identifying specific behavior patterns in images or videos, such as human movements.

[0081] Therefore, the target information in different display and control systems is different, and the image processing module is to be identified based on the configuration of the target information.

[0082] When it is necessary to detect multiple target information of the display and control system, multiple algorithms and / or multiple image processing models can be used.

[0083] In step S102, the multimedia information processing unit processes the image information in the multiple multimedia information channels through the image processing module, and determines the target information detection result of each multimedia information channel; the multimedia information processing unit is configured with display content control rules corresponding to different types of target information detection results; different display control systems correspond to different display content control rules; the display content control rules are used to control the display screen to display multimedia information that meets preset detection requirements.

[0084] The different types of target information detection results include normal, abnormal, recognition failure, image type, etc.

[0085] In different display control systems, different contents are to be displayed on the display screen based on the target information detection results. Therefore, it is necessary to configure display content control rules corresponding to different types of target information detection results to control the display screen to display multimedia information that meets the preset detection requirements.

[0086] The display content control rules corresponding to the different types of target information detection results may be pre-configured, or may be determined based on the interaction between the user and the client terminal after the target information detection result is detected.

[0087] Pre-configured display content control rules are usually set before the system is deployed based on the experience of the system designer and the needs of the target scenario. These rules define how the content on the display should be adjusted when the system detects different types of target information detection results, and are automatically executed.

[0088] In addition to pre-configured control rules, some display control systems can also support dynamic display content control rules based on the interaction between users and client terminals. This method allows users to adjust the display content on the display screen in real time according to their needs and preferences after detecting the target information. For example, one of the multiple alternative display contents can be selected to replace the original display content; this method provides higher flexibility and customizability, allowing the display control system to better adapt to different scenarios and user needs.

[0089] Based on this, in the step S103, the multimedia information processing unit determines the target indication information based on the target information detection result, and sends the target indication information to the client.

[0090] The target indication information includes the target information detection result, and may also be the detected target information itself, generated reminder information, and the like.

[0091] In some embodiments, the target indication information is sent to a client, and a user views the target indication information to determine specific content of a display content control rule corresponding to the target information detection result based on the target prompt information and the client interaction.

[0092] In the step S104, the multimedia information processing unit determines the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, and sends the target multimedia information to the display screen for display.

[0093] The target multimedia information for the display screen may be original multimedia information processing, or multimedia information after processing, or a target information detection result.

[0094] The following specifically describes the information processing method of the display control system when the multi-channel multimedia information is multi-channel video. In daily life, public large screens can be seen everywhere. In the application of large screens, multiple signal sources are often required to be displayed on the spliced ​​large screen. However, in the current control system, the screening of video source quality is generally manually confirmed through management software, which may lead to human errors.

[0095] Based on this, intelligent screening is performed in the embodiments of the present application to reduce the error frequency and save labor costs.

[0096] When the display control system is used to display multiple channels of video, such as a large screen spliced ​​with multiple channels of video in public places, please refer to Figure 2The display and control system specifically includes: an audio and video signal source 200, a multimedia information processing unit, a network switching module 203, a video decoding module 204, a client 206, and a display screen 205; the multimedia information processing unit includes an image processing module, a control module 202 and a video encoding module.

[0097] Control module 202: used for controlling the whole system and accepting the scheduling of the client 206 software;

[0098] The control module 202 is a module of the original display control system. In the embodiment of the present application, an image processing module and a video encoding module are added to the original display control system. The image processing module is used to identify target information based on an image processing model, so it can also be called an AI recognition module.

[0099] The image processing module and the video encoding module are arranged in the AI ​​box 201. By pretending that the AI ​​box 201 is in the original display and control system, the information processing method of the display and control system described in the embodiment of the present application is implemented.

[0100] In some display and control systems, a network switching module 203 needs to be installed.

[0101] Specifically, the audio and video signal source 200 and the AI ​​box 201 are connected via a video cable, and the AI ​​box 201 is connected to the network switching module 203 via the network; the network switching module 203 and the control module 202 and the video decoding module 204 are respectively connected via the network; and control signals are exchanged between the control module 202 and the AI ​​box 201 and the video decoding module 204.

[0102] The audio and video signal source 200 can be a network video or a local video, and can be multi-channel or single-channel. The signal source is connected to the AI ​​box 201 via a video cable.

[0103] The image processing module loads the image processing model obtained through training. Different algorithm models can be loaded in different application scenarios. The image processing module identifies the received video frames and determines whether the video frames meet the requirements; encodes the normal video frames that meet the requirements, intercepts or alarms the abnormal video frames that do not meet the requirements and feeds back the results to the client 206 software.

[0104] The network switching module 203 is connected to the AI ​​box 201 via a network and is used to exchange encoded multi-channel video images.

[0105] The video decoding module 204 decodes the received video and transmits it to the display via a video cable.

[0106] The client 206 is installed with the client 206 software and accepts manual operations. The specific operations may be to cut off the video that does not meet the requirements, retrieve the plan to supplement, directly process the video frame, etc.

[0107] The display screen 205 is used to display video images, and may be a spliced ​​screen or a single display screen.

[0108] In some embodiments, the multimedia information processing unit processes the image information in the multiple multimedia information channels through the image processing module to determine the target information detection result of each multimedia information channel, including:

[0109] The multimedia information processing unit processes each image frame of each video through the image processing module, detects whether there is target information in each image frame, and determines the target information detection result of each image frame.

[0110] A video is composed of a series of continuous image frames, each of which is a static image. For each video, the image processing module processes it frame by frame.

[0111] In each frame of the image, the image processing module identifies whether there is a preset "target information". The "target information" here can be varied, such as faces, license plate numbers, specific behaviors, business logos, etc., depending on the application requirements and the design of the detection algorithm.

[0112] For multiple connected video sources, the multimedia information processing unit will process each video source in parallel or sequentially to ensure that each channel of information is analyzed and detected.

[0113] In the information processing method of the display control system described in some embodiments, please refer to Figure 3 The multimedia information processing unit determines the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, and sends the target multimedia information to the display screen, including the following steps S301-S303:

[0114] S301, when the target information detection results of each image frame of each video are normal, the multimedia information processing unit re-encodes the image frame into multiple videos through the video encoding module, and sends the multiple videos to the display screen for display;

[0115] S302, when an abnormal image frame whose target information detection result is abnormal is detected, determining first abnormal target information in the abnormal image frame, and determining an abnormal video in which the abnormal image frame is detected;

[0116] S303: The control module of the multimedia information processing unit controls to close the abnormal video, obtains a target normal video to replace the abnormal video, and sends the target normal video and other videos to a display screen for display.

[0117] Determine the target information detection result: For each frame, there will be a corresponding target information detection result, which can be "target information detected" or "target information not detected".

[0118] Here, the normality or abnormality of the target information detection result is determined based on specific requirements. For example, for a promotional video carrying business information, it is normal when the trademark LOGO is detected, and it is abnormal when the trademark LOGO cannot be detected. For abnormal behavior, when abnormal behavior is detected in the video, the target information detection result is abnormal, and when no abnormal behavior is detected, the target information detection result is normal.

[0119] Determine the first abnormal target information in the abnormal image frame. Specifically, when the image processing module detects target information in a certain frame, it will record some attributes of the target information, such as position, size, content, etc. The first abnormal target information is the position, size, content, etc. of the target information.

[0120] An abnormal video in which an abnormal image frame is detected is determined, that is, a video that does not meet the conditions is determined, for example, a signal source identifier of the abnormal video is determined.

[0121] As an example, the signal source identifier is the source information of the abnormal video, such as camera ID, location, timestamp, etc.

[0122] The control module of the multimedia information processing unit controls to close the abnormal video, obtains the target normal video to replace the abnormal video, and sends the target normal video and other videos to the display screen for display.

[0123] That is to say, this abnormal video cannot be played, and the control module obtains the target normal video to replace the abnormal video. At this time, the control module serves as a new signal source.

[0124] Specifically, obtaining a target normal video to replace the abnormal video, and sending the target normal video and other channels of video to a display screen for display is achieved based on the interaction between a control module and a network switching module.

[0125] Specifically, under the control of the control module, the abnormal video is closed, and the abnormal video is sent to the control module through the network switching module, and / or the control module sends the acquired target normal video to the decoding module through the network switching module. In this way, the video transmission and processing logic of the display and control system itself is maintained, and the modification process of the original display and control system is more convenient.

[0126] In the embodiment of the present application, the step of obtaining a target normal video to replace the abnormal video includes:

[0127] Directly retrieve the normal video to replace the abnormal video;

[0128] or,

[0129] The control module processes the abnormal image frame based on the target image frame processing strategy to obtain a normal image frame, and re-encodes the normal image frame into a target normal video; different types of the first abnormal target information in the multimedia target image frame correspond to different image frame processing strategies.

[0130] The normal video directly retrieved may come from other cameras, historical records, backup storage, live videos from other live broadcast rooms, network videos, etc. of the same display and control system.

[0131] In case of live broadcast, video source errors, or abnormal videos with a large number of abnormal behaviors detected, directly switch to the abnormal video and play the retrieved normal video.

[0132] When the number of abnormal image frames is small, the overall content of the video is less affected, the abnormal video can be played later (for example, some local videos played repeatedly in public places), or there are some small-area error marks, the abnormal image frames can be directly processed to obtain normal image frames, so as to continue playing the video. Because the embodiment of the present application has processed the video into image frames and has detected the position and size of the abnormal information, the efficiency of simply processing the abnormal image frames is very high, and normal videos can be generated in time based on the processed normal image frames.

[0133] For details, please refer to Figure 4 In the information processing method of the display control system, the control module processes the abnormal image frame based on the target image frame processing strategy to obtain a normal image frame, and re-encodes the normal image frame into a target normal video, including the following steps S401-S404:

[0134] S401, judging whether the preset blur condition is met based on the attribute of the first abnormal target information in the target image frame;

[0135] S402: If the information is in compliance with the requirements, the control module performs fuzzy processing on the first abnormal target information in the abnormal image frame to obtain a normal image frame;

[0136] S403, if not, the control module obtains the number of abnormal image frames and calls the target backup video; the frame number difference between the target backup video and the abnormal image frame is within a preset difference range;

[0137] S404: Replace the abnormal image frame with the image frame of the target backup video to obtain a normal image frame.

[0138] Based on the attributes of the first abnormal target information in the target image frame, determine whether it meets the preset blur conditions, that is, whether the type, size and position of the first abnormal target information meet the preset blur conditions, for example, the type is trademark information (logo or slogan), the area is smaller than the preset area threshold, and the position is within a preset position range (generally below or in a corner of the video).

[0139] When the preset blur condition is met, it indicates that the influence of the first abnormal target information on the video content is within an acceptable range. Therefore, the first abnormal target information in the abnormal image frame is blurred to obtain a normal image frame.

[0140] If it does not match, the target backup video with a frame number that is not much different from the abnormal image frame can be spliced ​​into a normal video.

[0141] The target backup video may be a promotional video, etc.

[0142] The abnormal image frame is replaced by the image frame of the target spare video to obtain a normal image frame, and the normal image frame is inserted into the position of the original abnormal image frame to achieve the effect of inserting other content into the video content.

[0143] If the number of frames of the target backup video does not completely match the number of frames of the abnormal image, the control module may need to crop or fill the target backup video (such as using repeated frames, static frames or transition effects) to match the target length.

[0144] That is to say, when the video data of the display and control system described in the embodiment of the present application is transmitted, multiple video sources are transmitted to the installed AI box through video cables; the AI ​​box loads the image processing model, and the image processing model recognizes the image frames in the received video frame by frame; the image processing model recognizes that it contains target information (such as abnormal behavior, abnormal content, specific identification, etc.); when the target information is recognized, the video source channel of this channel is closed, and the target information is reported to the control module, and the control module obtains the normal video of the plan and projects it to the large screen; if no sensitive screen is recognized: the image frame is encoded, and the encoded video is transmitted to the video decoding module through the network switching module, and the video is output to the spliced ​​large screen after decoding, so that the video source is intelligently detected frame by frame, with high detection accuracy and low error rate, saving human resources, and facilitating the transformation and upgrading of the existing display and control system.

[0145] The following specifically describes the application of the information processing method of the display and control system in the smart factory scenario.

[0146] In a test workshop, each test device usually requires manual monitoring of production data and / or test data, which will incur labor costs; the present application applies the information processing method of the display and control system described in the embodiment of the present application to the workshop, so that one person can supervise multiple production equipment and test equipment, and the generated data and test data can be intuitively presented on the large screen without the need for continuous attention, and an alarm will be automatically issued when an error occurs.

[0147] Based on this, please refer to Figure 5 Before the multimedia information processing unit receives the multi-channel multimedia information sent by the multimedia signal source, the information processing method further includes the following steps S501-S503:

[0148] S501, multiple devices to be monitored in the production line are connected to a host computer, and the host computer captures target data of the devices in the production line as image data based on a pre-configured device control tool; the devices include production equipment and / or testing equipment;

[0149] S502, the host computer saves the image data of different devices in groups to obtain multiple channels of image data;

[0150] S503: The host computer sends the multiple channels of image data to a multimedia information processing unit.

[0151] The device control tool is software used to control the device.

[0152] The target data of the equipment in the production line is captured as image data. Specifically, when the production equipment or the test equipment is executed, the execution result of the equipment control tool is the corresponding data. For example, when the test fails, "error" or "error" will be output; the image data captured by the host PC is the print information on the equipment control tool.

[0153] Here, the multimedia information processing unit is an AI box equipped with an image processing module. By installing the AI ​​box into the original display and control system of the workshop, intelligent detection can be realized.

[0154] Based on this, please refer to Figure 6 The display and control system of the workshop includes: equipment 600, a host computer 601, an AI box 602, and a display screen 603; the equipment 600 is connected to the host computer 601 through a network port or a serial port, the host computer 601 is connected to the AI ​​box 602 through a video cable, and the AI ​​box 602 is connected to the display screen 603 through a video cable.

[0155] The host computer can be a computer or the like.

[0156] In the embodiment of the present application, the multimedia information processing unit processes the image information in the multiple multimedia information channels through the image processing module to determine the target information detection result of each multimedia information channel, including:

[0157] The multimedia information processing unit processes the image data of the different devices through the image processing module to determine whether there is second abnormal target information in the image data of each device;

[0158] If so, determining that the target information detection result of the device image data is image data abnormality;

[0159] If not, determining that the target information detection result of the device image data is that the image data is normal;

[0160] Correspondingly, the multimedia information processing unit determines the target indication information based on the target information detection result, and sends the target indication information to the client, including:

[0161] The target information detection result of the image data anomaly and the corresponding image data are sent to the client.

[0162] The image processing module loads a specific image recognition algorithm, and the image processing module recognizes the multiple channels of image data generated by different devices, thereby detecting whether the image data of each device contains the second abnormal target information. Specifically, the image processing module recognizes the multiple channels of image data generated by different devices, and can recognize text images in the image data, for example, the image data output by the host computer contains text such as "error" or "error".

[0163] If second abnormal target information is detected in the image data of the device, the target information detection result of the device image data is determined to be image data abnormality, that is, when second abnormal target information of the type "error" or "error" is detected, the target information detection result of the device image data is determined to be image data abnormality, and the target information detection result of the image data abnormality and the corresponding image data are sent to the client to remind the operator to handle it.

[0164] Alternatively, when it is determined that the target information detection result of the device image data is that the image data is abnormal, the prompt module of the AI ​​box directly prompts based on a preset prompt module.

[0165] The preset prompt module of the AI ​​box may be a sound playing module, an audio and video prompter, and the like.

[0166] In some embodiments, in the information processing method of the display control system, the multimedia information processing unit determines the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, and sends the target multimedia information to the display screen for display, including:

[0167] The target multimedia information for the display screen is determined to be normal image data, and image data of the normal image data is sent to the display screen for display.

[0168] That is to say, if the image processing module does not recognize the target information in the image data, the image content is directly displayed on the display screen.

[0169] In the application scenario of a smart factory, the information processing method of the display and control system described in the embodiment of the present application can alarm for abnormal operations and abnormal results of the generating equipment or the testing equipment to save labor costs, and enable one person to supervise and control multiple devices on multiple production lines.

[0170] Based on the same inventive concept, an information device of a display and control system corresponding to the information processing method of the display and control system is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the information processing method of the display and control system in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0171] Please refer to Figure 7 In some embodiments, the information processing device of the display and control system is applied to the display and control system, and the display and control system includes a multimedia information processing unit, a client and a display screen; the information processing device includes:

[0172] The receiving module 701 is used to enable the multimedia information processing unit to receive multiple multimedia information sent by the multimedia signal source; the multimedia information processing unit is equipped with an image processing module for detecting target information;

[0173] The first determination module 702 is used to enable the multimedia information processing unit to process the image information in the multiple multimedia information through the image processing module, and determine the target information detection result of each multimedia information; the multimedia information processing unit is configured with display content control rules corresponding to different types of target information detection results; different display control systems correspond to different display content control rules; the display content control rules are used to control the display screen to display multimedia information that meets the preset detection requirements;

[0174] A second determination module 703 is configured to enable the multimedia information processing unit to determine target indication information based on the target information detection result, and send the target indication information to the client;

[0175] The third determination module 704 is used to enable the multimedia information processing unit to determine the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, and send the target multimedia information to the display screen for display.

[0176] In some embodiments, in the information processing device of the display and control system, when the multiple channels of multimedia information are multiple channels of video, the multimedia information processing unit further includes a control module and a video encoding module;

[0177] The first determination module, when causing the multimedia information processing unit to process the image information in the multiple multimedia information channels through the image processing module to determine the target information detection result of each multimedia information channel, is specifically used to:

[0178] The multimedia information processing unit processes each image frame of each video through the image processing module, detects whether there is target information in each image frame, and determines the target information detection result of each image frame.

[0179] In some embodiments, in the information processing device of the display control system, the third determination module, when enabling the multimedia information processing unit to determine the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, and sending the target multimedia information to the display screen, is specifically used to:

[0180] When the target information detection results of each image frame of each video are normal, the multimedia information processing unit re-encodes the image frame into multiple videos through the video encoding module, and sends the multiple videos to the display screen for display;

[0181] When an abnormal image frame whose target information detection result is abnormal is detected, first abnormal target information in the abnormal image frame is determined, and an abnormal video in which the abnormal image frame is detected is determined;

[0182] The control module of the multimedia information processing unit controls to close the abnormal video, obtains the target normal video to replace the abnormal video, and sends the target normal video and other videos to the display screen for display.

[0183] In some embodiments, in the information processing device of the display control system, the third determination module, when acquiring a target normal video to replace the abnormal video, is specifically used to:

[0184] Directly retrieve the normal video to replace the abnormal video;

[0185] or,

[0186] The control module processes the abnormal image frame based on the target image frame processing strategy to obtain a normal image frame, and re-encodes the normal image frame into a target normal video; different types of the first abnormal target information in the multimedia target image frame correspond to different image frame processing strategies.

[0187] In some embodiments, in the information processing device of the display control system, the third determination module, when causing the control module to process the abnormal image frame based on the target image frame processing strategy to obtain a normal image frame, and re-encode the normal image frame into a target normal video, is specifically used to:

[0188] Based on the attribute of the first abnormal target information in the target image frame, determining whether a preset blur condition is met;

[0189] If it is in compliance, the control module performs fuzzy processing on the first abnormal target information in the abnormal image frame to obtain a normal image frame;

[0190] If not, the control module obtains the number of abnormal image frames and calls the target backup video; the frame number difference between the target backup video and the abnormal image frame is within a preset difference range;

[0191] The abnormal image frame is replaced by the image frame of the target spare video to obtain a normal image frame.

[0192] In some embodiments, in the information processing device of the display control system, the second determination module, when enabling the multimedia information processing unit to determine the target indication information based on the target information detection result and sending the target indication information to the client, is specifically used to:

[0193] When an abnormal image frame whose target information detection result is abnormal is detected, the image processing module of the multimedia information processing unit sends the abnormal target information detection result to the control module, and the control module generates target indication information based on the abnormal target information detection result, and sends the target indication information to the client;

[0194] The client generates scheduling information for the target indication information in response to a scheduling operation for a specific type of target indication information, and sends the scheduling information to a control module; the scheduling information is used to determine the content of the display content control rule.

[0195] In some embodiments, the information processing device of the display and control system further includes:

[0196] A capture module is used for connecting a plurality of devices to be monitored in the production line to a host computer before the multimedia information processing unit receives the multi-channel multimedia information sent by the multimedia signal source, and the host computer captures the target data of the devices in the production line as image data based on a pre-configured device control tool; the devices include production equipment and / or testing equipment;

[0197] The host computer saves the image data of different devices in groups to obtain multiple channels of image data;

[0198] The host computer sends the multiple channels of image data to the multimedia information processing unit.

[0199] In some embodiments, the first determination module in the information processing device of the display control system is specifically used to:

[0200] The multimedia information processing unit processes the image data of the different devices through the image processing module to determine whether there is second abnormal target information in the image data of each device;

[0201] If so, determining that the target information detection result of the device image data is image data abnormality;

[0202] If not, determining that the target information detection result of the device image data is that the image data is normal;

[0203] Correspondingly, the second determination module in the information processing device of the display control system, when enabling the multimedia information processing unit to determine the target indication information based on the target information detection result and send the target indication information to the client, is specifically used to:

[0204] The target information detection result of the image data anomaly and the corresponding image data are sent to the client.

[0205] In some embodiments, the third determination module in the information processing device of the display control system, when enabling the multimedia information processing unit to determine the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, and sending the target multimedia information to the display screen for display, is specifically used to:

[0206] The target multimedia information for the display screen is determined to be normal image data, and image data of the normal image data is sent to the display screen for display.

[0207] Based on the same inventive concept, an electronic device corresponding to the information processing method of the display and control system is also provided in the embodiment of the present application. Since the principle of solving the problem by the electronic device in the embodiment of the present application is similar to the information processing method of the display and control system in the embodiment of the present application, the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0208] Please refer to Figure 8 , Figure 8 A schematic diagram of the structure of the electronic device described in an embodiment of the present application is shown; the electronic device 800 includes: a processor 801, a memory 802 and a bus, the memory 802 stores machine-readable instructions executable by the processor 801, and when the electronic device 800 is running, the processor 801 communicates with the memory 802 through the bus, and when the machine-readable instructions are executed by the processor 801, the steps of the information processing method of the display and control system are executed.

[0209] Based on the same inventive concept, an electronic device corresponding to the information processing method of the display and control system is also provided in the embodiment of the present application. Since the principle of solving the problem by the electronic device in the embodiment of the present application is similar to the information processing method of the display and control system in the embodiment of the present application, the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0210] A computer-readable storage medium stores a computer program, which executes the steps of the information processing method of the display and control system when the computer program is run by a processor.

[0211] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0212] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0213] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0214] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a platform server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0215] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An information processing method for a display control system, characterized in that: Applied to a display control system, the display control system includes a multimedia information processing unit, a client and a display screen; the information processing method includes: The multimedia information processing unit receives multiple channels of multimedia information sent by a multimedia signal source; the multimedia information processing unit is configured with an image processing module for detecting target information; The multimedia information processing unit processes the image information in the multiple multimedia information channels through the image processing module, and determines the target information detection result of each multimedia information channel; the multimedia information processing unit is configured with display content control rules corresponding to different types of target information detection results; different display control systems correspond to different display content control rules; the display content control rules are used to control the display screen to display multimedia information that meets the preset detection requirements; The multimedia information processing unit determines target indication information based on the target information detection result, and sends the target indication information to the client; The multimedia information processing unit determines the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, and sends the target multimedia information to the display screen for display.

2. The information processing method of the display control system according to claim 1, characterized in that: When the multi-channel multimedia information is multi-channel video, the multimedia information processing unit further includes a control module and a video encoding module; The multimedia information processing unit processes the image information in the multiple multimedia information by the image processing module to determine the target information detection result of each multimedia information, including: The multimedia information processing unit processes each image frame of each video channel through the image processing module, detects whether there is target information in each image frame, and determines the target information detection result of each image frame.

3. The information processing method of the display control system according to claim 2, characterized in that: The multimedia information processing unit determines the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, and sends the target multimedia information to the display screen, including: When the target information detection results of each image frame of each video are normal, the multimedia information processing unit re-encodes the image frame into multiple videos through the video encoding module, and sends the multiple videos to the display screen for display; When an abnormal image frame whose target information detection result is abnormal is detected, first abnormal target information in the abnormal image frame is determined, and an abnormal video in which the abnormal image frame is detected is determined; The control module of the multimedia information processing unit controls to close the abnormal video, obtains the target normal video to replace the abnormal video, and sends the target normal video and other videos to the display screen for display.

4. The information processing method of the display control system according to claim 3, characterized in that: The step of obtaining a target normal video to replace the abnormal video comprises: Directly retrieve the normal video to replace the abnormal video; or, The control module processes the abnormal image frame based on the target image frame processing strategy to obtain a normal image frame, and re-encodes the normal image frame into a target normal video; different types of the first abnormal target information in the multimedia target image frame correspond to different image frame processing strategies.

5. The information processing method of the display control system according to claim 4, characterized in that: The control module processes the abnormal image frame based on the target image frame processing strategy to obtain a normal image frame, and re-encodes the normal image frame into a target normal video, including: Based on the attribute of the first abnormal target information in the target image frame, determining whether it meets the preset blur condition; If it is in compliance, the control module performs fuzzy processing on the first abnormal target information in the abnormal image frame to obtain a normal image frame; If not, the control module obtains the number of abnormal image frames and calls the target backup video; the frame number difference between the target backup video and the abnormal image frame is within a preset difference range; The abnormal image frame is replaced by the image frame of the target spare video to obtain a normal image frame.

6. The information processing method of the display control system according to claim 2, characterized in that: The multimedia information processing unit determines target indication information based on the target information detection result, and sends the target indication information to the client, including: When an abnormal image frame whose target information detection result is abnormal is detected, the image processing module of the multimedia information processing unit sends the abnormal target information detection result to the control module, and the control module generates target indication information based on the abnormal target information detection result, and sends the target indication information to the client; The client generates scheduling information for the target indication information in response to a scheduling operation for a specific type of target indication information, and sends the scheduling information to a control module; the scheduling information is used to determine the content of the display content control rule.

7. The information processing method of the display control system according to claim 1, characterized in that: Before the multimedia information processing unit receives the multi-channel multimedia information sent by the multimedia signal source, the information processing method further includes: A plurality of devices to be monitored in the production line are connected to a host computer, and the host computer captures target data of the devices in the production line as image data based on a pre-configured device control tool; the devices include production equipment and / or testing equipment; The host computer saves the image data of different devices in groups to obtain multiple channels of image data; The host computer sends the multiple channels of image data to the multimedia information processing unit.

8. The information processing method of the display control system according to claim 7, characterized in that: The multimedia information processing unit processes the image information in the multiple multimedia information by the image processing module to determine the target information detection result of each multimedia information, including: The multimedia information processing unit processes the image data of the different devices through the image processing module to determine whether there is second abnormal target information in the image data of each device; If so, determining that the target information detection result of the device image data is image data abnormality; If not, determining that the target information detection result of the device image data is that the image data is normal; Correspondingly, the multimedia information processing unit determines the target indication information based on the target information detection result, and sends the target indication information to the client, including: The target information detection result of the image data anomaly and the corresponding image data are sent to the client.

9. The information processing method of the display control system according to claim 8, characterized in that: The multimedia information processing unit determines the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, and sends the target multimedia information to the display screen for display, including: The target multimedia information for the display screen is determined to be normal image data, and image data of the normal image data is sent to the display screen for display.

10. An information processing device for a display and control system, characterized in that: Applied to a display and control system, the display and control system includes a multimedia information processing unit, a client and a display screen; the information processing device includes: A receiving module, used to enable the multimedia information processing unit to receive multiple multimedia information sent by a multimedia signal source; the multimedia information processing unit is equipped with an image processing module for detecting target information; A first determination module is used to enable the multimedia information processing unit to process the image information in the multiple multimedia information channels through the image processing module, and determine the target information detection result of each multimedia information channel; the multimedia information processing unit is configured with display content control rules corresponding to different types of target information detection results; different display control systems correspond to different display content control rules; the display content control rules are used to control the display screen to display multimedia information that meets the preset detection requirements; A second determination module, configured to enable the multimedia information processing unit to determine target indication information based on the target information detection result, and send the target indication information to the client; The third determination module is used to enable the multimedia information processing unit to determine the target multimedia information for the display screen based on the display content control rule corresponding to the target information detection result, and send the target multimedia information to the display screen for display.

Images