Multi-terminal display control system based on MVC and C / S mode

Abstract

The invention provides a multi-terminal display control system based on MVC and C / S modes. The multi-terminal display control system comprises a single main control terminal and a plurality of clients. The main control end comprises a communication module, a data analysis module, an image processing module, a rendering module, a JSON analysis module and a cache module; the client comprises a communication module, a data analysis module, an image processing module and a rendering module; the communication module is composed of a TCP component, a UDP component and an OpenSSL security component; according to the image processing module, a bottom layer frame is built by Gstreamer and is further packaged into an RTSP assembly and a JPEG assembly. The JSON analysis module is composed of a JSON Parser component and a Bbox component; the rendering module is composed of an OpenGL component and a QPainter component; and the cache module is composed of a Cache component and a QHash component. According to the invention, the quality, maintainability, expandability and reusability of the multi-terminal display control system can be improved, and the session information security of multi-terminal communication is ensured.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a multi-terminal display control system based on MVC and C / S modes. Background Art

[0002] The multi-terminal display control system connects multiple display terminals through a digital network to achieve centralized management, unified control and content distribution. It supports multiple signal source inputs, including video, audio, pictures, text, etc., and can display these signal sources on multiple display terminals according to preset rules and layouts. It is widely used in application scenarios such as monitoring and command, exhibition display, etc.

[0003] The current multi-terminal display control system lacks a systematic architecture design, and has problems such as chaotic structure, poor scalability and maintainability, and security risks. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention proposes a multi-terminal display control system based on MVC and C / S mode. By separating the model, view, and controller and combining the C / S mode, components with high scalability, strong reusability, and strong maintainability are designed that can be used on the main control end and the client, so as to realize multi-terminal system module reuse, multi-terminal device safe and efficient communication, and fast rendering of high-resolution images.

[0005] The technical solution of the present invention is:

[0006] A multi-terminal display control system based on MVC and C / S mode, including a single master terminal and multiple clients;

[0007] The main control end includes a communication module, a data parsing module, an image processing module, a rendering module, a JSON parsing module, and a cache module;

[0008] The client includes a communication module, a data parsing module, an image processing module and a rendering module;

[0009] The communication module consists of a TCP component, a UDP component, and an OpenSSL security component;

[0010] The image processing module is built with Gstreamer as the underlying framework and is further encapsulated into RTSP components and JPEG components;

[0011] The JSON parsing module consists of a JSON Parser component and a Bbox component;

[0012] The rendering module is composed of OpenGL components and QPainter components;

[0013] The cache module is composed of Cache component and QHash component;

[0014] In the master control terminal:

[0015] The TCP component of the communication module parses the JSON format control data received from the client, and sends the parsed data to the OpenSSL component for interpretation, and the interpreted data is sent to the data parsing module;

[0016] The UDP component of the communication module parses the received image data and video data, and sends the parsed data to the data parsing module;

[0017] The data parsing module performs format analysis on the input data, decomposes it into JPEG image information, RTSP video information, and JSON format control information, and sends them to the corresponding processing components respectively, wherein the JPEG image information is sent to the JPEG component of the image processing module, the RTSP video information is sent to the RTSP component of the image processing module, and the JSON format control information is sent to the JSON Parser component of the JSON parsing module;

[0018] The JPEG component uses the NvJPEG decoder to decode the input JPEG image information and convert the decoded data into QImage information; the RTSP component decodes the input RTSP video information according to the rtsp protocol and converts the decoded data into QImage information;

[0019] The Cache component caches QImage information;

[0020] The JSON Parser component converts the JSON format control information into the target recognition box information in the Bounding Box format, and sends it to the Bbox component for sorting and merging to obtain the Bbox information;

[0021] The QHash component matches the QImage information cached by the Cache component with the Bbox information one by one to obtain the corresponding mapping relationship QHash<QImage,Bbox> ;

[0022] The QPainter component performs rectangular rendering on the QImage information and the Bbox information according to the mapping relationship to obtain a QImage image with the target frame information, and renders the generated QImage image through the OpenGL component and draws it on the screen;

[0023] In the client:

[0024] The UDP component of the communication module parses the received image data and video data, and after passing through the data parsing module, decomposes them into JPEG image information and RTSP video information; and sends the JPEG image information to the JPEG component of the image processing module, and sends the RTSP video information to the RTSP component of the image processing module;

[0025] The JPEG component uses the NvJPEG decoder to decode the input JPEG image information and convert the decoded data into QImage information; the RTSP component decodes the input RTSP video information according to the rtsp protocol and converts the decoded data into QImage information;

[0026] OpenGL renders the QImage information and draws it on the screen.

[0027] Furthermore, in the main control end, the communication module, data parsing module, image processing module and the JSON Parser component in the JSON parsing module constitute the model part of the MVC architecture; the cache module, rendering module and the Bbox component in the JSON parsing module constitute the controller part of the MVC architecture; the external screen display constitutes the view part of the MVC architecture.

[0028] Beneficial Effects

[0029] The present invention divides the multi-terminal display control system into module components that realize different functions. The master terminal and the client terminal respectively realize the arrangement and combination of specific modules according to their own needs to build their own software system. The present invention can improve the quality, maintainability, scalability and reusability of the multi-terminal display control system, while ensuring the security of multi-terminal communication session information.

[0030] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0032] Figure 1 : Multi-terminal display control system connection diagram;

[0033] Figure 2 : Typical module information flow of the master control end;

[0034] Figure 3 : Typical client module information flow. DETAILED DESCRIPTION

[0035] Embodiments of the present invention are described in detail below. The embodiments are exemplary and intended to be used to explain the present invention, but should not be construed as limiting the present invention.

[0036] This embodiment proposes a multi-terminal display control system based on MVC and C / S mode. Based on the MVC design pattern of view (View), model (Model), and controller (Controller), the data, interface and logic of the multi-terminal display control system are separated, and a large number of reusable modules are separated, thereby making it more maintainable and scalable.

[0037] In the MVC mode, the model is the core of the multi-terminal display control system, responsible for processing data and business logic, and the model does not interact directly with the view. The view is only responsible for displaying data for the user and does not contain any business logic. The view can receive user input and notify the controller, which processes the user request. The controller is the coordinator between the model and the view. It receives input information through the view, that is, the input information obtained by the JSON data parsing module, and updates the model and view based on the input. The controller can ensure the overall data flow of the master and client, the view display logic, etc. The MVC mode adopts a QT-based signal / slot communication mechanism. When the state of the model changes, a signal is sent to the view and controller to ensure that the view and controller respond to the state change of the model in a timely manner; at the same time, when the user interacts with the view, the controller receives the interaction logic to update the model.

[0038] The modular design is combined with the MVC mode and the C / S mode, including a single master terminal and multiple clients. The system separates the responsibilities of the MVC, realizes the decoupling of the multi-terminal display control system, and improves the reusability and maintainability of the system.

[0039] (I) System composition:

[0040] like Figure 1 As shown, the main control end includes a communication module, a data analysis module, an image processing module, a rendering module, a JSON analysis module, and a cache module; the client function is relatively simple, including a communication module, a data analysis module, an image processing module, and a rendering module.

[0041] Each of the above modules is composed of smaller components, including:

[0042] The communication module is composed of TCP component, UDP component and OpenSSL security component; the data parsing module is independently grouped; the image processing module is built with Gstreamer as the underlying framework and is further encapsulated into RTSP component and JPEG component; the JSON parsing module is composed of JSON Parser component and Bbox component; the rendering module is composed of OpenGL component and QPainter component; the cache module is composed of QHash component and Cache component.

[0043] (II) Data flow:

[0044] Determine the data flow of the multi-terminal display control system according to the C / S concept:

[0045] like Figure 2 As shown, in the master control terminal:

[0046] The communication module is responsible for internal module connections and external information reception and transmission; the TCP component of the communication module parses the JSON format control data sent by the client, and sends the parsed data to the OpenSSL component for decryption, and the decrypted data is sent to the data parsing module; the UDP component of the communication module parses the received image data and video data, and sends the parsed data to the data parsing module.

[0047] The data parsing module performs format analysis on the input data, decomposes it into JPEG image information, RTSP video information, and JSON format control information, and sends them to the corresponding processing components respectively. The JPEG image information is sent to the JPEG component of the image processing module, the RTSP video information is sent to the RTSP component of the image processing module, and the JSON format control information is sent to the JSON Parser component of the JSON parsing module.

[0048] The JPEG component uses the NvJPEG decoder to decode the input JPEG image information and converts the decoded data into QImage information; the RTSP component decodes the input RTSP video information according to the rtsp protocol and converts the decoded data into QImage information; the Cache component caches the QImage information.

[0049] The JSON Parser component converts the control information in JSON format into target recognition box information in Bounding Box format, and sends it to the Bbox component for sorting and merging to obtain Bbox information.

[0050] The QHash component matches the QImage information cached by the Cache component with the Bbox information one by one to obtain the mapping relationship QHash<QImage,BBox> The QPainter component performs rectangular rendering on the QImage information and the Bbox information according to the mapping relationship, generates a QImage image with a target identification information box, and OpenGL renders the generated QImage image and draws it on the screen.

[0051] like Figure 3 As shown, in the client:

[0052] The UDP component of the communication module parses the received image data and video data, which are decomposed into JPEG image information and RTSP video information by the data parsing module; and the JPEG image information is sent to the JPEG component of the image processing module, and the RTSP video information is sent to the RTSP component of the image processing module;

[0053] The JPEG component uses the NvJPEG decoder to decode the input JPEG image information and convert the decoded data into QImage information; the RTSP component decodes the input RTSP video information according to the rtsp protocol and converts the decoded data into QImage information;

[0054] OpenGL renders the QImage information and draws it on the screen.

[0055] (3) Secure communication:

[0056] The communication between the master and the client is based on the JSON protocol communication encrypted by OpenSSL, which can avoid eavesdropping and confirm the identity of the other end of the connection, ensuring the confidentiality and reliability of the communication between the two applications. It can be supported on both the master and the client. The main steps are as follows:

[0057] Step 1: Initialize the OpenSSL library on the master.

[0058] Step 2: Create an encryption / decryption context object, which will store settings related to encryption and authentication.

[0059] Step 3: Generate a pair of public and private keys on the master end for encrypting and decrypting data.

[0060] Step 4: Send the public key of the master to the client so that the client can use it to encrypt data.

[0061] Step 5: Use the public key of the master terminal to encrypt the data on the client.

[0062] Step 6: Send the encrypted data to the master based on TCP.

[0063] Step 7: Use the private key to decrypt the data on the master.

[0064] Step 8: Perform multiple encrypted and decrypted communications between the master and the client until the required data transmission is completed.

[0065] Step 9: After the communication is completed, close the communication connection between the master and the client. Delete the generated key pair and encrypted data file to enhance security.

[0066] (IV) High-resolution image caching, target recognition frame drawing and real-time refreshing; the main steps are as follows:

[0067] Step 1: Receive high-resolution image JPEG encoded data according to the communication protocol. In the JPEG component, use NVIDIA CUDA acceleration to implement hardware decoding of JPEG images, convert them into QImage information, and convert Bbox information and QImage information into QHash<QImage,Bbox> The forms are organized and stored in a hash table.

[0068] Step 2: Receive the target recognition box Bbox information according to the communication protocol, take out the corresponding image in the hash table container in the previous step, use the QPainter component to accelerate, draw the target recognition box on the high-resolution image, and generate a high-resolution image with the target recognition box rectangle information.

[0069] Step 3: Use OpenGL to render and display the high-resolution image generated in the third step.

[0070] (V) Hash table operation:

[0071] The high-resolution image cache and target recognition frame drawing use hash table cache, and the average time complexity of search, insertion and deletion operations is O(1), which makes it show excellent performance in large-scale data processing scenarios. By using hash functions to map keys to values, hash tables can quickly locate stored and retrieved image data. The main steps include:

[0072] Step 1: Decode the QImage information of the JPEG component according to QHash<QImage,Bbox> Format for mapping.

[0073] Step 2: Take out the corresponding frame of the target recognition frame according to the JSON format, and read the QImage information of the Value value in the Hash table according to the frame number.

[0074] Step 3: Use the QPainter component to draw the target recognition box Bbox information on the QImage information.

[0075] Step 4: After the target recognition frame is drawn, the signal / slot mechanism is used to notify the display interface view element to use the OpenGL component to refresh the QImage image.

[0076] Step 5: After the display view element refreshes the image, the signal slot mechanism is used to notify the hash table to clear the displayed image frame cache and release memory.

[0077] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention without departing from the principles and intent of the present invention.

Claims

1. A multi-terminal display control system based on MVC and C / S mode, characterized in that: Includes a single master and multiple clients; The main control end includes a communication module, a data parsing module, an image processing module, a rendering module, a JSON parsing module, and a cache module; The client includes a communication module, a data parsing module, an image processing module and a rendering module; The communication module consists of a TCP component, a UDP component, and an OpenSSL security component; The image processing module is built with Gstreamer as the underlying framework and is further encapsulated into RTSP components and JPEG components; The JSON parsing module consists of a JSON Parser component and a Bbox component; The rendering module is composed of OpenGL components and QPainter components; The cache module is composed of a Cache component and a QHash component.

2. According to claim 1, a multi-terminal display control system based on MVC and C / S mode, characterized in that: In the master control terminal: The TCP component of the communication module parses the JSON format control data received from the client, and sends the parsed data to the OpenSSL component for interpretation, and the interpreted data is sent to the data parsing module; The UDP component of the communication module parses the received image data and video data, and sends the parsed data to the data parsing module; The data parsing module performs format analysis on the input data, decomposes it into JPEG image information, RTSP video information, and JSON format control information, and sends them to the corresponding processing components respectively, wherein the JPEG image information is sent to the JPEG component of the image processing module, the RTSP video information is sent to the RTSP component of the image processing module, and the JSON format control information is sent to the JSON Parser component of the JSON parsing module; The JPEG component uses the NvJPEG decoder to decode the input JPEG image information and convert the decoded data into QImage information; the RTSP component decodes the input RTSP video information according to the rtsp protocol and converts the decoded data into QImage information; The Cache component caches QImage information; The JSON Parser component converts the JSON format control information into the target recognition box information in the Bounding Box format, and sends it to the Bbox component for sorting and merging to obtain the Bbox information; The QHash component matches the QImage information cached by the Cache component with the Bbox information one by one to obtain the corresponding mapping relationship QHash<QImage,Bbox> ; The QPainter component performs rectangular rendering on the QImage information and the Bbox information according to the mapping relationship to obtain a QImage image with the target frame information, and renders the generated QImage image through the OpenGL component and draws it on the screen; In the client: The UDP component of the communication module parses the received image data and video data, and after passing through the data parsing module, decomposes them into JPEG image information and RTSP video information; And send the JPEG image information to the JPEG component of the image processing module, and send the RTSP video information to the RTSP component of the image processing module; The JPEG component uses the NvJPEG decoder to decode the input JPEG image information and convert the decoded data into QImage information; the RTSP component decodes the input RTSP video information according to the rtsp protocol and converts the decoded data into QImage information; OpenGL renders the QImage information and draws it on the screen.

3. According to claim 1, a multi-terminal display control system based on MVC and C / S mode, characterized in that: In the main control end, the communication module, data parsing module, image processing module and the JSONParser component in the JSON parsing module constitute the model part of the MVC architecture; the cache module, rendering module and the Bbox component in the JSON parsing module constitute the controller part of the MVC architecture; the external screen display constitutes the view part of the MVC architecture.

4. According to claim 1, a multi-terminal display control system based on MVC and C / S mode, characterized in that: The communication between the master and the client is based on the JSON protocol encrypted by OpenSSL.

Images