Data processing method and device, equipment and medium

Abstract

The invention provides a data processing method, device and equipment and a medium, the method is applied to a server, and the method comprises the following steps: receiving an audio and video stream and task information of the audio and video stream sent by a conference end; the task information is processed according to a preset algorithm, a task score is determined, and the task score is used for evaluating the capacity of an effectiveness calculation container; establishing an efficiency calculation container according to the task score and the task information; and processing the audio and video stream by using the efficiency calculation container to obtain the processed audio and video stream, thereby effectively reducing the occupation of data bandwidth and network congestion at the articulated naturality web side, and improving the transmission efficiency of the articulated naturality web.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data processing method, device, equipment and medium. Background Art

[0002] The visual network adopts VisionVera real-time high-definition video exchange technology, realizing real-time transmission of high-definition video across the entire network, which is currently impossible with the Internet. It integrates dozens of video, voice, image, text, communication, data and other services such as high-definition video conferencing, video monitoring, remote training, intelligent monitoring and analysis, emergency command, video calls, live broadcasts, TV mails, information releases, etc. into one system platform, and realizes real-time interconnection of high-definition quality video communications through a variety of terminal devices.

[0003] In the visual network, a visual network gateway is deployed, such as the visual network conference gateway XMCU, which is responsible for connecting and integrating the visual network with the Internet, ensuring that the visual network and the Internet can be interconnected and conduct video conferencing and other services.

[0004] In the related technology, XMCU sends all the data on the Internet side directly to the devices of the visual network for processing, and the devices of the visual network process and integrate the data, thereby occupying the data bandwidth of the visual network and affecting the transmission efficiency of the visual network. Summary of the invention

[0005] The present invention provides a data processing method, device, equipment and medium to solve the problem in the prior art that data integration occupies too much data bandwidth, thereby affecting the transmission efficiency of the visual network.

[0006] In order to solve the above problems, the present invention discloses a data processing method, which is applied to a server and includes:

[0007] Receive audio and video streams and task information of audio and video streams sent by the conference terminal;

[0008] Processing the task information according to a preset algorithm to determine a task score, wherein the task score is used to evaluate the capacity of the performance calculation container;

[0009] Establishing a performance calculation container according to the task score and the task information;

[0010] The audio and video stream is processed using the performance computing container to obtain a processed audio and video stream.

[0011] Optionally, the task information includes: the number of channels of the audio and video streams, the video enhancement ratio and the video blur coefficient;

[0012] The processing of the task information according to a preset algorithm to obtain a task score includes:

[0013] The number of channels, video enhancement ratio and video fuzzy coefficient of the audio and video streams are processed according to a preset algorithm to obtain a task score.

[0014] Optionally, the task information includes: the number of channels of the audio and video streams, the video enhancement ratio, the video fuzzy coefficient and the video integration method;

[0015] The processing of the task information according to a preset algorithm to obtain a task score includes:

[0016] Obtaining the fraction of the CPU occupied by the audio and video stream from the video integration conversion table according to the video integration mode;

[0017] The number of channels of the audio and video streams, the video enhancement ratio, the video fuzzy coefficient and the fraction of the CPU occupied by the audio and video streams are processed according to a set algorithm to obtain a task score.

[0018] Optionally, establishing a performance calculation container according to the task score and the task information includes:

[0019] A performance calculation container is established according to the task score and the task amount in the task information.

[0020] Optionally, the using the performance computing container to process the audio and video stream to obtain a processed video stream includes:

[0021] The performance computing container is used to process the audio and video streams respectively, wherein the processing includes any one or more of the following: visual enhancement, frame interpolation processing, screen merging processing, and background blurring processing.

[0022] In order to solve the above problem, the present invention further discloses a data processing device, which is applied to a server and includes:

[0023] A receiving module, used to receive the audio and video streams and task information of the audio and video streams sent by the conference terminal;

[0024] A task score module, used to process the task information according to a preset algorithm to obtain a task score, wherein the task score is used to evaluate the capacity of the performance calculation container;

[0025] A building module, used for building a performance calculation container according to the task score and the task information;

[0026] The processing module is used to process the audio and video stream using the performance computing container to obtain a processed audio and video stream.

[0027] Optionally, the task information includes: the number of channels of the audio and video streams, the video enhancement ratio and the video blur coefficient;

[0028] The task score module includes: a first processing unit;

[0029] The first processing unit is used to process the number of channels, video enhancement ratio and video fuzzy coefficient of the audio and video stream according to a preset algorithm to obtain a task score.

[0030] Optionally, the task information includes: the number of channels of the audio and video streams, the video enhancement ratio, the video fuzzy coefficient and the video integration method;

[0031] The task score module includes:

[0032] A search unit, used for obtaining the fraction of the CPU occupied by the audio and video stream from the video integration conversion table according to the video integration mode;

[0033] The second processing unit is used to process the number of channels of the audio and video streams, the video enhancement ratio, the video fuzzy coefficient and the fraction of the CPU occupied by the audio and video streams according to a set algorithm to obtain a task score.

[0034] In order to solve the above problems, the present invention also discloses an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the data processing method when executing the computer program.

[0035] In order to solve the above problem, the present invention further discloses a computer-readable storage medium, which stores a computer program for implementing the data processing method when executed by a processor.

[0036] Compared with the prior art, the present invention has the following advantages:

[0037] The present invention processes the audio and video streams and the task information of the audio and video streams sent by the XMCU on the Internet side according to a preset algorithm, determines the task score, and establishes a performance computing container according to the task score and the task information; uses the performance computing container to process the audio and video streams to obtain the processed audio and video streams; sends the processed audio and video streams to a visual networking device. Compared with the prior art, multiple audio and video streams are integrated into one audio and video stream on the Internet side, and then the processed audio and video streams are sent to the device on the visual networking side, which effectively reduces the data bandwidth occupancy and network congestion on the visual networking side, thereby improving the transmission efficiency of the visual networking.

[0038] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1It is a schematic diagram of the overall concept of a data processing method described in an embodiment of the present invention;

[0040] Figure 2 is a flow chart of a data processing method according to an embodiment of the present invention;

[0041] Figure 3 is a flow chart of a data processing method according to an embodiment of the present invention;

[0042] Figure 4 is a flow chart of a data processing method according to an embodiment of the present invention;

[0043] Figure 5 It is a structural diagram of a data processing device described in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] See also Figure 1 , which shows a schematic diagram of the overall concept of a data processing method according to an embodiment of the present invention, specifically comprising:

[0046] SIP terminal / MCU1, H323 terminal / MCU2, Liangyan Cloud Conference 3, conference terminal 4, server 5 and visual Internet device 6.

[0047] Among them, the conference terminal 4 can be an XMCU or a terminal, and the server 5 can be an audio and video computing server or other servers, and the present invention does not make specific limitations on this.

[0048] SIP terminal / MCU1, H323 terminal / MCU2, Liangyan Cloud Conference 3 are connected to XMCU4 via the Internet, XMCU4 is connected to server 5 via the Internet, and XMCU4 is connected to visual network device 6 via visual network.

[0049] SIP terminal / MCU1, H323 terminal / MCU2, and Liangyan Cloud Conference 3 send audio and video streams to XMCU4 via the Internet. XMCU4 sends multiple audio and video streams to the server via the Internet. The server processes the audio and video streams and audio and video task information to obtain the processed audio and video streams. The server sends the processed audio and video streams to XMCU4. XMCU4 sends the processed audio and video streams to the visual network device 6 via the same channel of the visual network.

[0050] Based on the above concept, Figure 2 , which shows a flow chart of a data processing method according to an embodiment of the present invention, the method is applied to a server, and specifically includes:

[0051] Step 201: Receive the audio and video stream and task information of the audio and video stream sent by XMCU.

[0052] In actual applications, before receiving the multi-channel audio and video streams and the task information of the audio and video streams sent by the XMCU, the server also includes:

[0053] The SIP terminal, H323 terminal, and Liangyan Cloud Conference send a signaling notification to XMCU, which is used to establish a communication connection between the SIP terminal, H323 terminal, and Liangyan Cloud Conference and XMCU. After the communication connection is successfully established, the SIP terminal, H323 terminal, and Liangyan Cloud Conference send audio and video streams and task information of the audio and video streams to XMCU, and XMCU sends the audio and video streams and the task information of the audio and video streams to the server. Among them, the task information of the audio and video stream includes but is not limited to any one or more of the following: the number of channels of the audio and video stream, the video enhancement ratio, the video fuzziness coefficient, and the video integration method.

[0054] Step 202: Process the task information according to a preset algorithm to determine a task score.

[0055] The server processes the task information according to a preset algorithm to determine a task score, wherein the task score is used to evaluate the capacity of the performance computing container, that is, different task scores correspond to different performance computing containers, that is, the higher the task score, the larger the capacity of the performance computing container.

[0056] Step 203: Establish a performance calculation container according to the task score and the task information.

[0057] Different performance computing containers are established according to different task scores and task information. That is, different performance computing containers can be established simultaneously according to the task scores and task information. Different performance computing containers can work simultaneously, thereby improving the processing efficiency of audio and video streams.

[0058] Step 204: Use the performance computing container to process the audio and video stream to obtain a processed audio and video stream.

[0059] The server sends the processed audio and video stream to XMCU, and XMCU sends it to the visual networking device through the same channel.

[0060] The main function of the performance computing container is to process audio and video images, for example, converting low-definition images to high-definition images, or compensating for missing frames.

[0061] For example, the resolution of the 4-channel audio and video streams on the IP side is 1080, the encoding method is: h264, and the estimated bit rate is: 1.5Mb / s*4.

[0062] After the optimization and screen integration of the above steps 201 to 204, a video stream with a resolution of 1080, an encoding method of h264, and an estimated bit rate of 1.5 Mb / s*1 can be obtained.

[0063] In this embodiment, the audio and video streams and the task information of the audio and video streams sent by the XMCU on the Internet side are processed according to a preset algorithm to determine the task score, and a performance computing container is established according to the task score and the task information; the audio and video streams are processed using the performance computing container to obtain a processed audio and video stream; the processed audio and video stream is sent to a visual networking device. Compared with the prior art, multiple audio and video streams are integrated into one audio and video stream on the Internet side, and then the processed audio and video stream is sent to the device on the visual networking side, which effectively reduces the data bandwidth occupancy and network congestion on the visual networking side, thereby improving the transmission efficiency of the visual networking side.

[0064] Based on the above concept, Figure 3 , which shows a flow chart of a data processing method according to an embodiment of the present invention, the method is applied to a server, and specifically includes:

[0065] Step 301: Receive the audio and video streams and task information of the audio and video streams sent by XMCU.

[0066] Receive multiple audio and video streams sent by XMCU, wherein the task information includes: the number of audio and video streams, video enhancement ratio and video fuzzy coefficient.

[0067] Step 302: Process the number of channels, video enhancement ratio and video fuzzy coefficient of the audio and video streams according to a preset algorithm to obtain a task score.

[0068] In a specific application, the number of channels, video enhancement ratio and video fuzziness coefficient of the audio and video streams are multiplied to obtain a task score, and a performance computing container with corresponding capabilities is allocated according to the task score to process the audio and video streams.

[0069] In practical applications, the task score can be obtained by the following formula:

[0070] C=N*I*V

[0071] Among them, C represents the task score, N represents the number of audio and video streams, I represents the video enhancement ratio, and V represents the video blur coefficient, which is estimated according to the video scene.

[0072] Step 303: Establish a performance calculation container according to the task score and the task information.

[0073] In a specific application, the server directly establishes a performance computing container according to the task score. The higher the task score, the larger the performance computing container established, and the smaller the task score, the smaller the performance computing container established.

[0074] The server can also establish a performance computing container based on the task score and the task amount in the task information. The higher the task score and the larger the task amount, the larger the performance computing container is established; the smaller the task score and the smaller the task amount, the smaller the performance computing container is established.

[0075] Step 304: Use the performance computing container to process the audio and video stream to obtain a processed audio and video stream.

[0076] In practical applications, the performance computing container is used to process the audio and video streams respectively, wherein the processing includes any one or more of the following: visual enhancement, frame interpolation, screen merging, and background blurring.

[0077] Step 305: Send the processed audio and video stream to the visual networking device.

[0078] The server sends the processed audio and video streams to the visual networking device through the same channel of XMCU.

[0079] In this embodiment, the audio and video streams and the task information of the audio and video streams sent by the XMCU on the Internet side are processed according to a preset algorithm to obtain a task score, and a performance computing container is established according to the task score and the task information; the audio and video streams are processed using the performance computing container to obtain a processed audio and video stream; the processed audio and video stream is sent to a visual network device. Compared with the prior art, multiple audio and video streams are integrated into one audio and video stream on the Internet side, and then the processed audio and video stream is sent to the device on the visual network side, which effectively reduces the data bandwidth occupancy and network congestion on the visual network side, thereby improving the transmission efficiency of the visual network.

[0080] Based on the above concept, Figure 4 , which shows a flow chart of a data processing method according to an embodiment of the present invention, the method is applied to a server, and specifically includes:

[0081] Step 401: Receive the audio and video stream and task information of the audio and video stream sent by XMCU.

[0082] The task information includes: the number of audio and video streams, video enhancement ratio, video fuzzy coefficient and video integration method.

[0083] Step 402: Process the task information according to a preset algorithm to obtain a task score.

[0084] Step 402 includes the following sub-steps:

[0085] Sub-step 4021: Obtain the fraction of the CPU occupied by the audio and video stream from the video integration conversion table according to the video integration method.

[0086] Among them, the video integration conversion table stores the number of audio and video streams, video enhancement ratio, video blur coefficient, video integration method and the fraction of CPU occupied by audio and video streams. Among them, the video integration method includes: screen combination, which includes: left and right split screen and top and bottom split screen. For example, a 480P resolution video from SIP is enhanced, and then combined with a 720P video from H323 (left and right split screen).

[0087] Sub-step 4022: Process the number of channels of the audio and video streams, the video enhancement ratio, the video fuzzy coefficient and the fraction of the CPU occupied by the audio and video streams according to a preset algorithm to obtain a task score.

[0088] In practical applications, the task score can be obtained by the following formula:

[0089] C=F*N*I*V

[0090] Among them, F represents the fraction of CPU occupied by audio and video streams, C represents the task fraction, N represents the number of audio and video streams, I represents the video enhancement ratio, and V represents the video blur coefficient, which is estimated according to the video scene.

[0091] Step 403: Establish a performance calculation container according to the task score and the task amount in the task information.

[0092] Step 404: Use the performance computing container to process the audio and video stream to obtain a processed audio and video stream.

[0093] Step 405: Send the processed audio and video stream to the visual networking device.

[0094] In this embodiment, the audio and video streams and the task information of the audio and video streams sent by the XMCU on the Internet side are processed according to a preset algorithm to determine the task score, and a performance computing container is established according to the task score and the task information; the audio and video streams are processed using the performance computing container to obtain a processed audio and video stream; the processed audio and video stream is sent to a visual networking device. Compared with the prior art, multiple audio and video streams are integrated into one audio and video stream on the Internet side, and then the processed audio and video stream is sent to the device on the visual networking side, which effectively reduces the data bandwidth occupancy and network congestion on the visual networking side, thereby improving the transmission efficiency of the visual networking side.

[0095] It should be noted that, for the above method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the order of the actions described, because according to the present invention, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0096] Based on the description of the above method embodiment, the present invention also provides a corresponding screenshot device embodiment to implement the contents described in the above method embodiment.

[0097] Reference Figure 5 , which shows a structural diagram of a data processing device according to an embodiment of the present invention, wherein the device is applied to a server and specifically comprises:

[0098] The receiving module 501 is used to receive the audio and video streams and the task information of the audio and video streams sent by the conference terminal.

[0099] The task score module 502 is used to process the task information according to a preset algorithm to determine a task score, and the task score is used to evaluate the capacity of the performance calculation container.

[0100] The establishing module 503 is used to establish a performance calculation container according to the task score and the task information.

[0101] The processing module 504 is used to process the audio and video stream using the performance computing container to obtain a processed audio and video stream.

[0102] Optionally, the task information includes: the number of channels of the audio and video streams, the video enhancement ratio and the video fuzzy coefficient;

[0103] The task score module includes: a first processing unit;

[0104] The first processing unit is used to process the number of channels, video enhancement ratio and video fuzzy coefficient of the audio and video stream according to a preset algorithm to obtain a task score.

[0105] Optionally, the task information includes: the number of channels of the audio and video streams, the video enhancement ratio, the video fuzzy coefficient and the video integration method;

[0106] The task score module includes:

[0107] A search unit, used for obtaining the fraction of the CPU occupied by the audio and video stream from the video integration conversion table according to the video integration mode;

[0108] The second processing unit is used to process the number of channels of the audio and video streams, the video enhancement ratio, the video fuzzy coefficient and the fraction of the CPU occupied by the audio and video streams according to a set algorithm to obtain a task score.

[0109] Optionally, the establishing module is specifically configured to establish a performance calculation container according to the task score and the task amount in the task information.

[0110] Optionally, the processing module is specifically used to use the performance computing container to process the audio and video streams respectively, wherein the processing includes any one or more of the following: visual enhancement, frame interpolation processing, screen merging processing, and background blurring processing.

[0111] In this embodiment, the audio and video streams and the task information of the audio and video streams sent by the XMCU on the Internet side are processed according to a preset algorithm to determine the task score, and a performance computing container is established according to the task score and the task information; the audio and video streams are processed using the performance computing container to obtain a processed audio and video stream; the processed audio and video stream is sent to a visual networking device. Compared with the prior art, multiple audio and video streams are integrated into one audio and video stream on the Internet side, and then the processed audio and video stream is sent to the device on the visual networking side, which effectively reduces the data bandwidth occupancy and network congestion on the visual networking side, thereby improving the transmission efficiency of the visual networking side.

[0112] As for the above-mentioned device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts may refer to the partial description of the method embodiment.

[0113] An embodiment of the present invention further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data processing method when executing the computer program.

[0114] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program that enables a processor to execute the data processing method.

[0115] Alternatively, the computer-readable storage medium may be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0116] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0117] It is easy for a person skilled in the art to think that any combination of the above embodiments is feasible, so any combination of the above embodiments is an implementation scheme of the present invention, but due to space limitations, this specification will not be described in detail here. Although the preferred embodiments of the present invention have been described, once a person skilled in the art knows the basic creative concept, he can make other changes and modifications to these embodiments. Therefore, the attached claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0118] The data processing method, device, equipment and medium provided by the present invention are introduced in detail above. The principles and implementation methods of the present invention are explained in this article using specific examples. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A data processing method, characterized in that: The method is applied to a server and includes: Receive audio and video streams and task information of audio and video streams sent by the conference terminal; Processing the task information according to a preset algorithm to determine a task score, wherein the task score is used to evaluate the capacity of the performance calculation container; Establishing a performance calculation container according to the task score and the task information; The audio and video stream is processed using the performance computing container to obtain a processed audio and video stream.

2. The method according to claim 1, characterized in that The task information includes: the number of channels of the audio and video streams, the video enhancement ratio and the video fuzziness coefficient; The processing of the task information according to a preset algorithm to determine the task score includes: The number of channels, video enhancement ratio and video fuzzy coefficient of the audio and video streams are processed according to a preset algorithm to obtain a task score.

3. The method according to claim 1, characterized in that The task information includes: the number of channels of the audio and video streams, the video enhancement ratio, the video fuzziness coefficient and the video integration method; The processing of the task information according to a preset algorithm to determine the task score includes: Obtaining the fraction of the CPU occupied by the audio and video stream from the video integration conversion table according to the video integration mode; The number of channels of the audio and video streams, the video enhancement ratio, the video fuzzy coefficient and the fraction of the CPU occupied by the audio and video streams are processed according to a preset algorithm to obtain a task score.

4. The method according to claim 1, characterized in that: The establishing of the performance calculation container according to the task score and the task information comprises: A performance calculation container is established according to the task score and the task amount in the task information.

5. The method according to claim 1, characterized in that: The using the performance computing container to process the audio and video stream to obtain a processed video stream includes: The performance computing container is used to process the audio and video streams respectively, wherein the processing includes any one or more of the following: visual enhancement, frame interpolation processing, screen merging processing, and background blurring processing.

6. A data processing device, characterized in that: The device is applied to a server and includes: A receiving module, used to receive the audio and video streams and task information of the audio and video streams sent by the conference terminal; A task score module, used to process the task information according to a preset algorithm to determine a task score, wherein the task score is used to evaluate the capacity of the performance calculation container; A building module, used for building a performance calculation container according to the task score and the task information; The processing module is used to process the audio and video stream using the performance computing container to obtain a processed audio and video stream.

7. The device according to claim 6, characterized in that The task information includes: the number of channels of the audio and video streams, the video enhancement ratio and the video fuzziness coefficient; The task score module includes: a first processing unit; The first processing unit is used to process the number of channels, video enhancement ratio and video fuzzy coefficient of the audio and video stream according to a preset algorithm to obtain a task score.

8. The device according to claim 6, characterized in that The task information includes: the number of channels of the audio and video streams, the video enhancement ratio, the video fuzziness coefficient and the video integration method; The task score module includes: A search unit, used for obtaining the fraction of the CPU occupied by the audio and video stream from the video integration conversion table according to the video integration mode; The second processing unit is used to process the number of channels of the audio and video streams, the video enhancement ratio, the video fuzzy coefficient and the fraction of the CPU occupied by the audio and video streams according to a preset algorithm to obtain a task score.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the data processing method according to any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that implements the data processing method according to any one of claims 1 to 5 when executed by a processor.

Images