A picture real-time preview method applied to a KVM system

Abstract

The application discloses a picture real-time preview method applied to a KVM system, the KVM system comprising a TX box, a system server, a streaming server, an RTSP server and a client, and comprising the following steps: the system server acquires TX box information of the TX box; a TX box data object is generated according to the TX box information and is stored in a global array TXs; original TX box information of a database is updated; the streaming server creates a UDP thread and an RTP thread, acquires pictures of the TX box through the UDP thread, is responsible for encapsulating the acquired pictures into format pictures meeting RTSP requirements through the RTP thread, and pushes the format pictures to the RTSP server; and the client obtains the pushed Rtsp stream, so that the client obtains a video stream formed based on target pictures, and the target picture is previewed on the client. The application realizes preview, is high in efficiency, and can stably obtain a preview effect.

Description

Technical Field

[0001] This invention relates to the field of image preview technology in KVM systems, specifically a real-time image preview method applied to KVM systems. Background Technology

[0002] In existing KVM systems, such as KVM console systems used for command and dispatch, it is often difficult to achieve image preview. This is because the TX signal acquisition box (TX box for short) of the KVM system does not generate a real-time preview screen, so the required images cannot be displayed on the client, that is, it is impossible to preview the images acquired by the TX box on the client. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the present invention aims to provide a real-time preview method for KVM systems, which can solve the problems described in the background art.

[0004] The technical solution to achieve the objective of this invention is as follows: a real-time preview method for a KVM system, wherein the KVM system includes a TX box, a system server, a streaming server, an RTSP server, and a client, comprising the following steps:

[0005] Step 1: The system server obtains the TX box information, which includes the TX box's IP address, MAC address, RTPS stream address, box name, and online status.

[0006] Step 2: The streaming server obtains TX box information from the system server and generates TX box data objects based on the TX box information. The TX box data objects include the TX box IP address, TX box name, TX box MAC address, TX box stream address, current screen image of the TX box, and TX box image ready flag. All TX box data objects are stored in a global array TXs.

[0007] Step 3: The streaming server sends the global array TXs back to the system server. The system server updates the original TX box information in the database, including updating the Rtsp stream address in the original TX box record to the RtspAddress field in the global array TXs.

[0008] Step 4: The streaming server creates a corresponding UDP thread and RTP thread for each TX box based on the TX box data objects in the TXs array.

[0009] The streaming server uses a UDP thread to obtain images from the TX box and an RTP thread to encapsulate the obtained images into RTP data packets that meet RTSP requirements and push them to the RTSP server.

[0010] Step 5: The client obtains the Rtsp address from the system server, sends a first request to the RTSP server through the Rtsp address, and the RTSP server notifies the push server to start the UDP thread and the RTP thread after receiving the first request,

[0011] On the UDP thread, a UDP request is sent every preset time to obtain the target picture of the TX box, and the obtained target picture is cached in the TX box data object,

[0012] On the RTP thread, the target image is obtained from the TX box data object and encapsulated into an RTP data packet, and then the push server pushes the RTP data packet to the RTSP server, the RTSP server responds to the first request of the client and pushes the Rtsp stream to the client, so that the client continuously obtains the video stream formed based on the target picture, and realizes previewing the target picture on the client.

[0013] Further, after step 5 is executed, the following steps are further included:

[0014] The client sends a close or stop Rtsp stream notification to the RTSP server, the RTSP server receives the close or stop Rtsp stream notification, and notifies the push server to destroy the current thread, and the push server notifies the thread pool to stop the UDP thread and the RTP thread.

[0015] Further, in step 5, when the client pulls the Rtsp stream, the client creates a TCP connection with the RTSP server, and saves this TCP connection in a temporary hashMap, the hashMap stores key-value pair data, the key is the UUID value generated by the RTSP server for the connection, and the value is the TX box IP address parsed from the TX box stream address, the IP address is used to uniquely identify the TX box information as the communication between the RTSP server and the push server,

[0016] When the client sends an OPTION request to the RTSP server, the UUID value of the current request connection and the parsed TX box IP address are stored in the hashMap, the RTSP server notifies the push server to create a corresponding thread, and the push server creates a UDP thread and an RTP thread if the corresponding TX box does not create a UDP thread and an RTP thread,

[0017] When the client disconnects from the RTSP server or receives a TEARDOWN request, the RTSP server finds the TX box IP address in the hashMap according to the UUID value of the TCP connection, notifies the push server to close or stop the corresponding thread of the TX box, and if there is no multi-client connection, stops the UDP thread and the RTP thread, and deletes the corresponding key-value pair of the hashMap.

[0018] Further, in step 1, the acquired TX box information is written into the database on the system server,

[0019] In step 2, the push server establishes a connection with the database, queries the database through a SQL query instruction, and obtains the TX box information.

[0020] Further, in step 5, the specific process of obtaining the target picture through the UDP thread includes the following steps:

[0021] Step S1: The push server sends a target picture push request to the TX box;

[0022] Step S2: The TX box responds to the request of the push server and sends the related information of the target picture to the push server, including the picture size;

[0023] Step S3: After receiving the related information of the target picture, the push server returns a confirmation of receiving the related information of the picture to the TX box;

[0024] Step S4: The multiple receiving sub-threads under the UDP thread created by the push server receive the UDP data packets of the target picture continuously sent by the TX box, and the target picture is written into the UDP data packet in binary format, thereby completing the push picture;

[0025] Step S5: The client reads the UDP data packet, and after reading all the UDP data packets to form a complete file, sends a complete push target picture instruction to the push server, and if the confirmation information of the push server responding to the complete push target picture instruction is received, then one round of picture push is completed.

[0026] Further, the push server receives all the UDP data packets and stores them in UDPData, and after receiving the complete push target picture instruction sent by the above client, parses and splices the data in UDPData into a complete BMP picture, and stores the binary data of the converted JPEG picture in the JPEGData of the TX box data object, and sets JPEGReady to true.

[0027] Further, a counter is arranged in the push server, the counter is used to count the number of clients currently connected to pull the stream, the counter is increased by 1 each time a first request for obtaining the Rtsp stream is obtained, and the counter is decreased by 1 when the Rtsp connection is disconnected,

[0028] It is judged whether the current count of the counter is 0, wherein, if the first request is received, if the current count is 0, the UDP thread and the RTP thread are created, the first request is responded, and then the count of the counter is increased by 1; if the first request is stopped or closed, the counter is decreased by 1 each time the stop or close request is received, and then it is judged whether the current count of the counter is 0, if the current count is 0, the current UDP thread and the RTP thread are closed, and the Rtsp stream is stopped.

[0029] The application has the advantages that the target picture of the TX box can be previewed on the client, the efficiency is high, and the preview effect can be stably obtained. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 is a flowchart of the application;

[0031] Figure 2 is a flowchart of the process of sending or stopping the RTSP request by multiple clients;

[0032] Figure 3 is a flowchart of the process of sending the target picture of the TX box by the receiving sub-thread under the UDP thread. DETAILED DESCRIPTION

[0033] The application will be further described below in combination with the drawings and specific implementation schemes:

[0034] As shown in the drawings, a picture real-time preview method applied to a KVM system, the KVM system includes a TX box, a system server, a push server, an RTSP server and a client, and includes the following steps: Figures 1-3

[0035] Step 1: The system server obtains TX box information of the TX box, and the TX box information includes the ip address, the mac address, the Rtps stream address, the box name and the online state of the TX box and other basic information.

[0036] In this step, after the KVM system is started, a communication connection with the TX box is established by sending a heartbeat packet and other related protocols through TCP connection. The obtained TX box information can be written into a database on the system server.

[0037] ​Step 2: The push server establishes a connection with the database, queries the database through a SQL query instruction, thereby obtaining TX box information, and generates a TX box data object according to the TX box information, the TX box data object including a TX box IP address, a TX box name, a TX box MAC address, a TX box stream address, a TX box current picture, and a TX box picture ready identifier, and stores all TX box data objects of the TX boxes in a global array TXs. The TX box information in the global array TXs is shown in the following table:

[0038] Attribute Type Explanation Ip String TX box IP address Name String TX box name Mac String TX box MAC address RtspAddress String TX box stream address JPEGData Byte array TX box current picture picture JPEGReady Boolean type TX box picture ready identifier

[0039] The TX box stream address is obtained by splicing a configuration file of the push server and TX box information. The TX box picture ready identifier JPEGReady is an initial empty byte array.

[0040] Step 3: The push server establishes a communication connection with the system server, for example, a TCP connection, and returns the global array TXs to the system server, and the system server writes the global array TXs back to the database to update the original TX box information in the database, including updating the Rtsp stream address in the original TX box record to the RtspAddress field in the global array TXs, so that in subsequent processing, the client can obtain the Rtsp stream address of the TX box by establishing a communication connection with the system server, thereby performing a pull stream picture operation.

[0041] Step 4: The push server creates a corresponding UDP thread and RTP thread for each TX box according to the TX box data object in the TXs array, all UDP threads are placed in a UDP thread pool, and all RTP threads are placed in an RTP thread pool, and the thread pool is responsible for storing and managing threads.

[0042] The push server obtains the picture of the TX box through the UDP thread, and is responsible for encapsulating the obtained picture into RTP data satisfying the RTSP requirement, for example, encapsulating into an M-JPEG format RTP data frame (packet), and pushing to the RTSP server through the RTP thread.

[0043] Step 5: The client obtains TX box information by sending a related protocol through a TCP connection with the system server, and the system server returns related data to the client after querying the database, the returned related data including a Rtsp address.

[0044] After the client obtains the Rtsp address, the client sends a first request to the RTSP server. After the RTSP server receives the first request, the RTSP server notifies the push server to start a UDP thread and an RTP thread, and allocates one UDP thread and one RTP thread from a UDP thread pool and an RTP thread pool respectively. On the UDP thread, a UDP request is sent every preset time (for example, 300 ms) to obtain a target picture of the TX box, and the obtained target picture is cached to a TX box data object. On the RTP thread, the target picture is obtained from the TX box data object and encapsulated into an RTP data packet. Then, the push server pushes the RTP data packet to the RTSP server. The RTSP server responds to the first request of the client and pushes an Rtsp stream to the client, so that the client continuously obtains a video stream formed based on the target picture, and the client previews the target picture. That is, new target pictures are continuously obtained through the UDP thread, the target pictures form a video stream, and the video stream is finally played on the client to preview the target picture. Then, the client sends a close or stop Rtsp stream notification to the RTSP server. After the RTSP server receives the close or stop Rtsp stream notification, the RTSP server notifies the push server to destroy the current thread, and the push server notifies the thread pool to stop the UDP thread and the RTP thread.

[0045] The object structure on the UDP thread is shown in the following table:

[0046]

[0047]

[0048] The object structure on the RTP thread is shown in the following table:

[0049] Attribute Type Explanation TX TX box object TX box object information (above) SUQ_NUM Int UDP connection port Tx TX box object TX box object information (above) Tx_ip Byte array TX box IP address byte array Status Boolean type Whether the UDP thread sends the get picture request Reading Boolean type Whether the UDP thread reads data UDPDatas Two-dimensional byte array Store picture data sent by the TX box SignalTag Signal connection identifier Used to identify whether the current TX box is signal input

[0050] In this step, when the client pulls the Rtsp stream, the client creates a TCP connection with the RTSP server, and at this time, the TCP connection is saved to a temporary hashMap. The hashMap stores key-value pair data, the key is the UUID value generated by the RTSP server for the connection, and the value is the TX box IP address parsed from the TX box stream address, which is used to uniquely identify the TX box information as the communication between the RTSP server and the push server. When the client sends an OPTION request to the RTSP server, the UUID value of the current request connection and the parsed TX box IP address are stored in the hashMap. The RTSP server notifies the push server to create a corresponding thread, and if the corresponding TX box of the push server has not created a UDP and RTP thread, it creates a UDP thread and an RTP thread. When the client disconnects the connection with the RTSP server or receives a TEARDOWN request, the RTSP server finds the TX box IP address in the hashMap according to the UUID value of the TCP connection. The push server is notified to close or stop the corresponding thread of the corresponding TX box, and if there is no multi-client connection, the UDP thread and the RTP thread are stopped, and the corresponding key-value pair in the hashMap is deleted.

[0051] Reference Figure 3 , Figure 3 is a flowchart of the TX box sending the target picture through the receiving sub-thread under the UDP thread. The UDP thread is only responsible for requesting the TX box to obtain the target picture for preview. After the UDP thread is created, a picture request sub-thread and multiple picture receiving sub-threads are started. After the UDP thread is closed, all sub-threads under the UDP thread are closed. The specific process of requesting and obtaining the target picture through the UDP thread includes the following steps:

[0052] Step S1: The push server sends a target picture push request to the TX box.

[0053] Step S2: The TX box responds to the request of the push server and sends the related information of the target picture to the push server, including the picture size.

[0054] Step S3: After receiving the related information of the target picture, the push server returns a confirmation to the TX box that the picture information has been received.

[0055] Step S4: The multiple receiving sub-threads under the UDP thread created by the push server receive the UDP data packets of the target picture continuously sent by the TX box. The target picture is written in binary format into the UDP data packet, thereby completing the push picture.

[0056] Step S5: The client reads the UDP data packet, waits until all the UDP data packets are read to constitute a complete file (i.e. the target picture), and sends a complete pushing target picture instruction to the streaming server. If the streaming server receives the confirmation information of the complete pushing target picture instruction, then one round of picture pushing is completed, i.e. one round of picture preview is realized.

[0057] Wherein, the streaming server receives all the UDP data packets and stores them in UDPDatas. After receiving the complete pushing target picture instruction sent by the device (i.e. the client), the data in UDPDatas is parsed and spliced into a complete BMP picture, which is stored in the binary data of the JPEG picture of the TX box data object by conversion, and JPEGReady is set to true.

[0058] Reference Figure 2 , Figure 2 is a flowchart of a plurality of clients sending or stopping the RTSP request (i.e. the first request). In an optional embodiment, a counter is set in the streaming server, which is used to count the number of clients currently connected to pull the stream. Whenever a first request for obtaining the Rtsp stream is obtained, the count of the counter is increased by 1. When the Rtsp connection is disconnected, the count of the counter is decreased by 1. It is judged whether the current count of the counter is 0. If yes, in the process of receiving the request, if the current count is 0, the UDP thread and the RTP thread are created, the first request is responded, and then the count of the counter is increased by 1. If yes, in the process of stopping or closing the request, whenever the first request is received, the counter is decreased by 1, and then it is judged whether the current count of the counter is 0. If yes, the current UDP thread and the RTP thread are closed, and the Rtsp stream is stopped, so that the resources can be saved and the server carrying capacity can be improved under the condition of meeting the Rtsp stream pulled by a plurality of clients.

[0059] The embodiments disclosed in the specification are only an example of the one-sided features of the present application, and the protection scope of the present application is not limited to this embodiment. Any other functionally equivalent embodiments fall within the protection scope of the present application. For those skilled in the art, other various corresponding changes and modifications can be made according to the above described technical solutions and concepts, and all these changes and modifications should belong to the protection scope of the claims of the present application.

Claims

1. A picture real-time preview method applied to a KVM system, characterized in that, The KVM system comprises a TX box, a system server, a push stream server, an RTSP server and a client, and comprises the following steps: Step 1: The system server acquires TX box information of the TX box, and the TX box information comprises an IP address, a MAC address, an Rtps stream address, a box name and an online state of the TX box; Step 2: The push stream server acquires the TX box information from the system server, generates a TX box data object according to the TX box information, the TX box data object comprises a TX box IP address, a TX box name, a TX box MAC address, a TX box stream address, a TX box current picture and a TX box picture ready identifier, and stores all TX box data objects of the TX box in a global array TXs; Step 3: The push stream server returns the global array TXs to the system server, and the system server updates original TX box information in the database, comprising updating the Rtsp stream address in the original TX box record to the RtspAddress field in the global array TXs; Step 4: The push stream server creates a corresponding UDP thread and RTP thread for each TX box according to the TX box data object in the TXs array, The push stream server acquires the picture of the TX box through the UDP thread, and is responsible for encapsulating the acquired picture into an RTP data packet meeting the RTSP requirement through the RTP thread, and pushing to the RTSP server; Step 5: The client obtains the Rtsp address from the system server, sends a first request to the RTSP server through the Rtsp address, the RTSP server receives the first request, and notifies the push stream server to start the UDP thread and the RTP thread, On the UDP thread, a UDP request is sent every preset time to acquire the target picture of the TX box, and the acquired target picture is cached in the TX box data object, On the RTP thread, the target image is acquired from the TX box data object and encapsulated into an RTP data packet, and then the push stream server pushes the RTP data packet to the RTSP server, the RTSP server responds to the first request of the client and pushes the Rtsp stream to the client, so that the client continuously obtains the video stream formed based on the target picture, and realizes previewing the target picture on the client, After step 5 is executed, the following steps are further included: The client sends a close or stop Rtsp stream notification to the RTSP server, the RTSP server receives the close or stop Rtsp stream notification, and notifies the push stream server to destroy the current thread, and the push stream server notifies the thread pool to stop the UDP thread and the RTP thread, In step 5, when the client pulls the Rtsp stream, the client creates a TCP connection with the RTSP server, and saves this TCP connection to a temporary hashMap, the hashMap stores key-value pair data, the key is the UUID value generated by the RTSP server for this connection, and the value is the TX box IP address parsed from the TX box stream address, which is used to uniquely identify the TX box information as the communication between the RTSP server and the push server, When the client sends an OPTION request to the RTSP server, the UUID value of the current request connection and the parsed TX box IP address are stored in the hashMap, and the RTSP server notifies the push server to create a corresponding thread, and if the corresponding TX box of the push server has not created a UDP, RTP thread, then create a UDP thread, RTP thread, When the client disconnects the connection with the RTSP server or receives a TEARDOWN request, the RTSP server finds the TX box IP address in the hashMap according to the UUID value of the TCP connection, notifies the push server to close or stop the corresponding thread of the corresponding TX box, and if there is no multi-client connection, stop the UDP thread and the RTP thread, and delete the corresponding key-value pair in the hashMap.

2. The picture real-time preview method applied to a KVM system according to claim 1, wherein, In step 1, the TX box information obtained is written into the database on the system server, In step 2, the push server establishes a connection with the database, and queries the database through a SQL query instruction to obtain the TX box information.

3. The picture real-time preview method applied to a KVM system according to claim 2, characterized in that, In step 5, the specific process of obtaining the target picture through the UDP thread includes the following steps: Step S1: The push server sends a target picture push request to the TX box; Step S2: The TX box responds to the request of the push server and sends the related information of the target picture to the push server, including the picture size; Step S3: After receiving the related information of the target picture, the push server returns a confirmation to the TX box that the picture information has been received; Step S4: The multiple receiving sub-threads under the UDP thread created by the push server receive the UDP data packets of the target picture continuously sent by the TX box, and the target picture is written in binary format into the UDP data packet, thereby completing the push picture; Step S5: The client reads the UDP data packet, and after reading all the UDP data packets to form a complete file, sends a complete push target picture instruction to the push server, and if the push server responds to the confirmation information of the complete push target picture instruction, then a round of picture push is completed.

4. The picture real-time preview method applied to a KVM system according to claim 3, characterized in that, The push server receives all the UDP data packets and stores them in UDPData, and after receiving the above-mentioned complete push target picture instruction sent by the client, parses and splices the data in UDPData into a complete BMP picture, and stores the binary data of the converted JPEG picture in the TX box data object JPEGData, and sets JPEGReady to true.

5. The picture real-time preview method applied to a KVM system according to claim 4, characterized in that, A counter is arranged in the push server, the counter is used to count the number of clients currently connected to pull stream, the count of the counter is increased by 1 whenever a first request for obtaining Rtsp stream is obtained, and the count of the counter is decreased by 1 whenever the Rtsp connection is disconnected, It is judged whether the current count of the counter is 0, wherein, if the first request is received, if the current count is 0, the UDP thread and the RTP thread are created, the first request is responded, and then the count of the counter is increased by 1; if the first request is stopped or closed, the count of the counter is decreased by 1 whenever the first request is received, and then it is judged whether the current count of the counter is 0, if the current count is 0, the current UDP thread and the RTP thread are closed, and the Rtsp stream is stopped.

Images