Video conference communication method, device and system

Abstract

The embodiment of the invention provides a video conference communication method, device and system. The method comprises the following steps: collecting video stream data; calling a pre-integrated video stream coding algorithm to carry out coding processing on the video stream data to obtain video coding data; and sending the video coding data to a conference sending device through a video stream transmission channel, so that the conference sending device performs slicing and packaging processing on the video coding data, and sends a packaged video data packet to a conference receiving device. According to the embodiment of the invention, the purpose that CPU resources of a host are not used at all can be achieved, and the problems of hardware inconsistency and the like generated when the MediaCodec calls the GPU are avoided.

Description

Technical Field

[0001] The present application relates to the technical field of video conference communication, and in particular to a video conference communication method, device and system. Background Art

[0002] Existing video communication encoding is generally divided into two categories: one is based on the CPU (Central Processing Processor) software encoding. The other is based on the MediaCodec (audio and video codec) method to call the hardware GPU (Graphics Processing Unit) method for encoding. The CPU-based software encoding method transplants the ffmpeg-like software codec library, and the application layer calls to implement image encoding and decoding. The MediaCodec-based method relies on the hardware GPU, which is usually a hardware-accelerated encoding library provided by hardware manufacturers and relies on the resources and architecture of special hardware.

[0003] The CPU-based software encoding method is usually very limited in the computing power of the hardware CPU on low-performance embedded devices. This type of CPU generally does not support real-time encoding / decoding of large resolutions. At the same time, in the scenario of video communication, audio / video acquisition and rendering are also required, so it is easy to cause the CPU to reach its peak, resulting in a low encoding frame rate and a large delay. This is especially true when the video image has a high resolution and a high bit rate.

[0004] The encoding method based on MediaCodec relies on the hardware-related GPU encoding library, hardware resources and architecture. Due to the great differences in embedded device hardware and systems, inconsistencies are easily caused. Some hardware does not even have good GPU resources, so the underlying layer still uses CPU encoding. Another more serious situation is that the version of the video format encoded by MediaCodec is relatively old. For example, the H.264 format only supports the Baseline profile, which has many disadvantages. Summary of the invention

[0005] The technical problem to be solved by the embodiments of the present application is to provide a video conferencing communication method, device and system to achieve the purpose of completely not using the CPU resources of the host and avoid problems such as hardware inconsistency caused by MediaCodec calling GPU.

[0006] In a first aspect, an embodiment of the present application provides a video conference communication method, which is applied to a camera device, and the method includes:

[0007] Collect video stream data;

[0008] Calling a pre-integrated video stream encoding algorithm to encode the video stream data to obtain video encoding data;

[0009] The video encoding data is sent to the conference sending device through the video stream transmission channel, so that the conference sending device slices and packages the video encoding data and sends the packaged video data packets to the conference receiving device.

[0010] Optionally, before collecting the video stream data, the method further includes:

[0011] After detecting the power-on operation, performing an initialization operation, the initialization operation includes: a hardware detection operation, a software configuration operation and a parameter setting operation;

[0012] Initializing the UVC library, the initial UVC library comprising: loading a UVC driver, establishing a video stream transmission channel with the conference sending device through a USB interface, and configuring video stream information according to configuration parameters corresponding to the parameter setting operation;

[0013] The video stream information includes: video stream format, video stream resolution and video stream frame rate.

[0014] Optionally, the calling of a pre-integrated video stream encoding algorithm to encode the video stream data to obtain the video encoding data includes:

[0015] Get the video stream bitrate configured during initialization;

[0016] Determine the number of encoded frames according to the video stream bit rate;

[0017] The video stream encoding algorithm is called to encode the collected video stream data according to the encoding frame number to obtain the video encoding data.

[0018] Optionally, after collecting the video stream data, the method further includes:

[0019] The video stream data is sent to a video encoding device, so that the video encoding device encodes the video stream data to obtain video stream encoded data, and the video stream encoded data is sent to the conference sending device.

[0020] In a second aspect, an embodiment of the present application provides a video conference communication device, which is applied to a camera device, and the device includes:

[0021] Video data acquisition module, used for acquiring video stream data;

[0022] A video encoding data acquisition module is used to call a pre-integrated video stream encoding algorithm to encode the video stream data to obtain video encoding data;

[0023] The video encoding data sending module is used to send the video encoding data to the conference sending device through the video stream transmission channel, so that the conference sending device slices and packages the video encoding data and sends the packaged video data packets to the conference receiving device.

[0024] Optionally, the device further comprises:

[0025] An initialization module, used to perform an initialization operation after detecting a power-on operation, wherein the initialization operation includes: a hardware detection operation, a software configuration operation, and a parameter setting operation;

[0026] A UVC library initialization module is used to initialize the UVC library, and the initial UVC library includes: loading a UVC driver, establishing a video stream transmission channel with the conference sending device through a USB interface, and configuring video stream information according to configuration parameters corresponding to the parameter setting operation;

[0027] The video stream information includes: video stream format, video stream resolution and video stream frame rate.

[0028] Optionally, the video encoding data acquisition module includes:

[0029] A video stream bit rate acquisition unit, used to acquire the video stream bit rate configured during the initialization process;

[0030] A coding frame number determination unit, used to determine the coding frame number according to the video stream bit rate;

[0031] The video encoding data acquisition unit is used to call the video stream encoding algorithm to encode the collected video stream data according to the encoding frame number to obtain the video encoding data.

[0032] Optionally, the device further comprises:

[0033] The video stream data sending module is used to send the video stream data to the video encoding device so that the video encoding device encodes the video stream data to obtain video stream encoded data, and send the video stream encoded data to the conference sending device.

[0034] In a third aspect, an embodiment of the present application provides a video conference communication system, the system comprising: a camera device, a conference sending device and a conference receiving device, wherein:

[0035] The camera device is used to collect video stream data; call a pre-integrated video stream encoding algorithm to encode the video stream data to obtain video encoding data; and send the video encoding data to a conference sending device through a video stream transmission channel;

[0036] The conference sending device is used to slice and package the video encoding data to obtain a video data packet, and send the video data packet to the conference receiving device;

[0037] The conference receiving device is used to decode the video data packet to obtain a decoded video stream, and render and display the decoded video stream.

[0038] In a fourth aspect, an embodiment of the present application provides a video conference communication system, the system comprising: a camera device, a video encoding device, a conference sending device and a conference receiving device, wherein:

[0039] The camera device is used to collect video stream data and send the video stream data to the video encoding device;

[0040] The video encoding device is used to call a pre-integrated video stream encoding algorithm to encode the video stream data to obtain video encoding data; and send the video encoding data to a conference sending device through a video stream transmission channel;

[0041] The conference sending device is used to slice and package the video encoding data to obtain a video data packet, and send the video data packet to the conference receiving device;

[0042] The conference receiving device is used to decode the video data packet to obtain a decoded video stream, and render and display the decoded video stream.

[0043] Compared with the prior art, the embodiments of the present application have the following advantages:

[0044] In an embodiment of the present application, video stream data is collected. A pre-integrated video stream encoding algorithm is called to encode the video stream data to obtain video encoding data. The video encoding data is sent to a conference sending device through a video stream transmission channel, so that the conference sending device slices and packages the video encoding data, and sends the packaged video data packets to a conference receiving device. In an embodiment of the present application, a video stream encoding algorithm is integrated with a camera device, and video encoding is performed through an external hardware camera, so that the video conference sending device obtains the encoded H.264 / H.265 video stream, and then slices and packages it for transmission. The host's CPU resources can be completely eliminated, and there are no disadvantages such as hardware inconsistencies caused by MediaCodec calling the GPU.

[0045] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flowchart of the steps of a video conference communication method provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of a video stream data processing flow provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of another video stream data processing flow provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of the structure of a video conferencing communication device provided in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of the structure of a video conferencing communication system provided in an embodiment of the present application;

[0051] Figure 6 A schematic diagram of the structure of another video conferencing communication system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0053] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0054] Reference Figure 1 , shows a flow chart of the steps of a video conference communication method provided by an embodiment of the present application, and the video conference communication method can be applied to a camera device. Figure 1 As shown, the video conference communication method may include the following steps:

[0055] Step 101: Collect video stream data.

[0056] The embodiment of the present application can be applied to a camera device, that is, the execution subject is a camera device. In a specific implementation, when two electronic devices conduct a video conference, the camera device can be a camera device on the two electronic devices side, and the camera device can be a camera device independent of the two electronic devices, which can be a hardware device such as a camera.

[0057] Video conferencing is the real-time transmission of video and audio data over the Internet, enabling participants in different locations to communicate and discuss as if they were in the same place.

[0058] When two electronic devices are conducting a video conference, the video stream data can be collected through the camera devices on the two electronic devices. That is, the camera device is responsible for collecting video stream data to capture the images of the conference participants. In a video conference, each participant's electronic device (such as a computer, mobile phone, or professional video conferencing terminal, etc.) is equipped with a camera device to collect local video images.

[0059] In this embodiment, when the camera device is started, an initialization process may be performed first. The specific initialization process may be described in detail in conjunction with the following specific implementation method.

[0060] In a specific implementation of the present application, before the above step 101, the following may also be included:

[0061] Step A1: After detecting the power-on operation, perform an initialization operation, which includes: a hardware detection operation, a software configuration operation, and a parameter setting operation.

[0062] In this embodiment, when the camera device detects a power-on operation, it will start to perform a series of initialization operations, including hardware detection (checking whether the camera module, sensor, lens and other hardware are working properly), software configuration (loading necessary firmware, drivers and applications) and parameter settings (such as exposure, white balance, focus, video stream bit rate, etc.).

[0063] Step A2: Initializing the UVC library, the initial UVC library includes: loading a UVC driver, establishing a video stream transmission channel with the conference sending device through a USB interface, and configuring video stream information according to configuration parameters corresponding to the parameter setting operation.

[0064] UVC (USB Video Class) is a USB device class specification that defines the standard for transmitting video data through the USB interface.

[0065] Initializing the UVC library is a key step in establishing a video stream transmission channel between a camera device and a video conferencing sending device (such as a computer, video conferencing terminal, etc.). After the camera device completes the initialization operation, the UVC library can be initialized in the camera device. The process of initializing the UVC library may include:

[0066] 1. Load the UVC driver: On the camera device side, you need to load the UVC driver to support USB transmission of video data. The driver is responsible for processing the USB communication protocol to ensure that the video data can be correctly transmitted from the camera device to the video conferencing sending device.

[0067] 2. Establish a video stream transmission channel: Through the USB interface, a video stream transmission channel is established between the camera device and the video conference sending device. This channel is used to transmit video data in real time, ensuring that both parties in the video conference can see each other's video images in real time.

[0068] 3. According to the configuration parameters corresponding to the parameter setting operation, the camera device will configure the video stream information. The video stream information may include: video stream format (such as YUV, RGB, MJPEG, H.246, H.265, etc.), video stream resolution (such as 720p, 1080p, 2K, 4K, etc.) and video stream frame rate (such as 30fps, 60fps, etc.).

[0069] Configuring these parameters can ensure the quality, fluency, and compatibility of video data and meet the requirements of video conferencing.

[0070] After the video stream data is collected, step 102 is executed.

[0071] Step 102: calling a pre-integrated video stream encoding algorithm to encode the video stream data to obtain video encoding data.

[0072] A video stream encoding algorithm is pre-integrated inside the camera device. The video stream encoding algorithm is usually based on industry-standard video compression technology, such as H.264, H.265 (HEVC) or AV1.

[0073] In a specific implementation, a dedicated video encoding chip or module can be integrated into the camera device. These chips or modules usually contain hardware accelerators for performing complex calculations in the video encoding algorithm. Alternatively, a processor or SoC (system-on-chip) with built-in video encoding function is used, which usually integrates multiple functions such as image processing, encoding and transmission.

[0074] When integrating the video stream encoding algorithm, you can choose a suitable video encoding library, such as FFmpeg, x264 (for H.264 encoding), etc., and integrate it into the camera's software system. These libraries provide rich video encoding functions and support multiple encoding formats and parameter settings.

[0075] When the video stream data is collected, the camera device will call the pre-integrated video stream encoding algorithm to encode the video stream data, thereby obtaining video encoding data.

[0076] In this embodiment, when the camera device encodes the video stream data, the number of encoded frames can be determined according to the pre-configured video stream bit rate, and encoding is performed once when a video stream with a specified number of frames is received. The implementation process can be described in detail in conjunction with the following specific implementation methods.

[0077] In a specific implementation of the present application, the above step 102 may include:

[0078] Sub-step B1: Obtain the video stream bit rate configured during the initialization process.

[0079] In this embodiment, during the initialization process of the camera device, a video stream bit rate can be preset according to application requirements and network conditions. This bit rate determines the bit rate of the video data after encoding, that is, the number of bits transmitted per second. The video stream bit rate directly affects the quality of the video and the required network bandwidth.

[0080] During initialization, this bitrate may be stored in the device's configuration file or set through the user interface. Before the encoding process begins, the bitrate value needs to be read from these storage locations.

[0081] When encoding a video stream, you can obtain the video stream bit rate configured during the initialization process.

[0082] After obtaining the video stream bit rate configured during the initialization process, sub-step B2 is performed.

[0083] Sub-step B2: Determine the number of encoded frames according to the video stream bit rate.

[0084] After obtaining the video stream bit rate configured during the initialization process, the number of encoded frames can be determined according to the video stream bit rate.

[0085] In specific implementations, there is a certain relationship between the video stream bitrate and the number of encoded frames. At a fixed bitrate, in order to maintain the quality and smoothness of the video, the number of encoded frames needs to be adjusted according to the bitrate. A higher bitrate generally allows more frames to be encoded, thereby providing higher video quality and details. Conversely, a lower bitrate may require a reduction in the number of encoded frames to reduce the amount of data and the required network bandwidth.

[0086] After determining the number of encoded frames according to the video stream bit rate, perform sub-step B3.

[0087] Sub-step B3: calling the video stream encoding algorithm to encode the collected video stream data according to the encoding frame number to obtain the video encoding data.

[0088] After determining the number of encoded frames according to the video stream bit rate, the video stream encoding algorithm can be called to encode the collected video stream data according to the number of encoded frames to obtain video encoded data. That is, when the camera device receives the video stream data of the encoded frame number, it will encode the video stream data once.

[0089] After the pre-integrated video stream encoding algorithm is called to encode the video stream data to obtain the video encoding data, step 103 is executed.

[0090] Step 103: Send the video encoding data to the conference sending device through the video stream transmission channel, so that the conference sending device slices and packages the video encoding data and sends the packaged video data packets to the conference receiving device.

[0091] After calling the pre-integrated video stream encoding algorithm to encode the video stream data to obtain video encoding data, the video encoding data can be sent to the conference sending device through the video stream transmission channel, so that the conference sending device can slice and package the video encoding data and send the packaged video data packets to the conference receiving device.

[0092] In the above process, the video stream encoding algorithm is integrated into the camera device, and the video is encoded through the external hardware camera, so that the video conference sending device obtains the encoded H.264 / H.265 video stream, and then slices and packages it for transmission. It can completely avoid using the host's CPU resources, and there is no disadvantage such as hardware inconsistency caused by MediaCodec calling GPU.

[0093] Next, combine Figure 2 The video encoding and transmission process is described in detail. Figure 2 As shown, the process may include:

[0094] 1. External camera initialization, that is, the initialization operation of the camera.

[0095] 2. Initialize the UVC library, that is, initialize the UVC library in the camera to establish a video stream transmission channel between the camera and the video conferencing sending device.

[0096] 3. Obtain H.264 / H.265 data, that is, the video stream encoding algorithm integrated in the camera encodes the collected video stream data to obtain video encoding data.

[0097] 4. Video coding data acquisition, that is, the camera sends the video coding data to the video conference sending device, and the video conference sending device can acquire the video coding data.

[0098] 5. Video frame slicing and packaging. That is, the video conferencing sending device slices and packages the video encoding data to obtain video data packets.

[0099] 6. Data packet transmission: The video conferencing sending device transmits the video data packet to the video conferencing receiving device.

[0100] 7. Data packet reception and decoding: The video conferencing receiving device receives the video data packet and decodes the video data packet to obtain decoded video stream data.

[0101] 8. Image rendering: The video conference receiving device renders the decoded video stream data to display the video conference image.

[0102] Next, combine Figure 3 The data encoding and processing flow of the RTP format is described in detail.

[0103] like Figure 3 As shown, the process may include:

[0104] 1. External camera initialization. Before starting video capture, the external camera needs to be initialized. This usually includes setting the camera parameters (such as resolution, frame rate, exposure, etc.) and establishing a communication connection with the camera. The external camera can collect video stream data and call the pre-integrated video stream encoding algorithm to encode the video stream data to obtain encoded video frames.

[0105] 2. Get video frames via UVC. UVC (USB Video Class) is used to connect to the video conferencing sending device via the USB interface. In this step, the UVC protocol can be used to get encoded video frames from the camera. These frames will be provided in encoded formats such as H.264 or H.265.

[0106] 3. Image frame format switching: Since the image frames have been pre-programmed, there is no need to convert the image data format, which improves the overall efficiency.

[0107] 4. RTP format packaging. RTP (Real-time Transport Protocol) is a protocol for transmitting real-time data on the Internet. In this step, the image frame data is packaged into RTP packets. This includes adding RTP header information, and possible data fragmentation (if the frame data is too large to fit into a single RTP packet).

[0108] 5. RTP format transmission. Use network protocols (such as UDP) to transmit RTP data packets from the sender to the receiver. Since UDP is a connectionless protocol, the transmission speed is faster, but the order or integrity of the data packets may not be guaranteed.

[0109] 6. RTP format reception. On the receiving end, receive and process RTP packets. This may include reordering packets (if they arrive in a different order than when they were sent) and handling lost or duplicate packets.

[0110] 7. RTP format depacketization. Extract the original image frame data from the RTP data packet. This includes removing the RTP header information and possible data reassembly (if the frame data is divided into multiple RTP data packets).

[0111] 8. Merge into video frames. If the image frame data is fragmented or scattered in multiple RTP packets, they need to be merged back into complete video frames in this step.

[0112] 9. Video rendering. Finally, the decoded video frame is displayed to the user. This usually involves passing the video frame data to a display device (such as a monitor or screen) and may include some post-processing steps (such as color correction, scaling, etc.) to improve the display effect.

[0113] The embodiment of the present application combines the two steps of external camera acquisition and encoding in the application of video communication scenarios, and can reduce the performance requirements of the host hardware through the parallel operation of internal and external hardware. It has fast speed, high performance, supports various types of different embedded hardware, and can be deployed to multiple platforms according to this method.

[0114] In another specific implementation of the present application, a video encoding device can be set up, and the camera device can send the collected video stream data to the video encoding device, so that the video encoding device encodes the video stream data to obtain video stream encoded data, and sends the video stream encoded data to the conference sending device.

[0115] The embodiment of the present application uses an external video encoding device to integrate a video stream encoding algorithm for video encoding, and the conference sending device obtains the encoded H.264 / H.265 video stream, and then slices and packages it for transmission. It can completely avoid using the host's CPU resources, and there is no hardware inconsistency caused by MediaCodec calling the GPU.

[0116] The video conferencing communication method provided in the embodiment of the present application collects video stream data. A pre-integrated video stream encoding algorithm is called to encode the video stream data to obtain video encoding data. The video encoding data is sent to the conference sending device through the video stream transmission channel, so that the conference sending device slices and packages the video encoding data, and sends the packaged video data packets to the conference receiving device. The embodiment of the present application integrates a video stream encoding algorithm through a camera device, performs video encoding through an external hardware camera, and obtains the encoded H.264 / H.265 video stream by the video conferencing sending device, and then slices and packages it for transmission. The host's CPU resources can be completely eliminated, and there are no disadvantages such as hardware inconsistencies caused by MediaCodec calling the GPU.

[0117] Reference Figure 4 , shows a schematic diagram of the structure of a video conference communication device provided by an embodiment of the present application, and the video conference communication device can be applied to a camera device. Figure 4 As shown, the video conference communication device 400 may include the following modules:

[0118] The video data acquisition module 410 is used to acquire video stream data;

[0119] The video encoding data acquisition module 420 is used to call a pre-integrated video stream encoding algorithm to encode the video stream data to obtain video encoding data;

[0120] The video encoding data sending module 430 is used to send the video encoding data to the conference sending device through the video stream transmission channel, so that the conference sending device slices and packages the video encoding data and sends the packaged video data packets to the conference receiving device.

[0121] Optionally, the device further comprises:

[0122] An initialization module, used to perform an initialization operation after detecting a power-on operation, wherein the initialization operation includes: a hardware detection operation, a software configuration operation, and a parameter setting operation;

[0123] A UVC library initialization module is used to initialize the UVC library, and the initial UVC library includes: loading a UVC driver, establishing a video stream transmission channel with the conference sending device through a USB interface, and configuring video stream information according to configuration parameters corresponding to the parameter setting operation;

[0124] The video stream information includes: video stream format, video stream resolution and video stream frame rate.

[0125] Optionally, the video encoding data acquisition module includes:

[0126] A video stream bit rate acquisition unit, used to acquire the video stream bit rate configured during the initialization process;

[0127] A coding frame number determination unit, used to determine the coding frame number according to the video stream bit rate;

[0128] The video encoding data acquisition unit is used to call the video stream encoding algorithm to encode the collected video stream data according to the encoding frame number to obtain the video encoding data.

[0129] Optionally, the device further comprises:

[0130] The video stream data sending module is used to send the video stream data to the video encoding device so that the video encoding device encodes the video stream data to obtain video stream encoded data, and send the video stream encoded data to the conference sending device.

[0131] The video conferencing communication device provided in the embodiment of the present application collects video stream data. A pre-integrated video stream encoding algorithm is called to encode the video stream data to obtain video encoding data. The video encoding data is sent to the conference sending device through the video stream transmission channel, so that the conference sending device slices and packages the video encoding data, and sends the packaged video data packets to the conference receiving device. The embodiment of the present application integrates a video stream encoding algorithm through a camera device, and performs video encoding through an external hardware camera, so that the video conferencing sending device obtains the encoded H.264 / H.265 video stream, and then slices and packages it for transmission. The host's CPU resources can be completely eliminated, and there are no disadvantages such as hardware inconsistencies caused by MediaCodec calling the GPU.

[0132] Reference Figure 5 , shows a schematic diagram of the structure of a video conference communication system provided by an embodiment of the present application. Figure 5 As shown, the video conference communication system 500 may include: a camera device 510, a conference sending device 520 and a conference receiving device 530, wherein:

[0133] The camera device is used to collect video stream data; call a pre-integrated video stream encoding algorithm to encode the video stream data to obtain video encoding data; and send the video encoding data to a conference sending device through a video stream transmission channel;

[0134] The conference sending device is used to slice and package the video encoding data to obtain a video data packet, and send the video data packet to the conference receiving device;

[0135] The conference receiving device is used to decode the video data packet to obtain a decoded video stream, and render and display the decoded video stream.

[0136] In this embodiment, the video stream encoding algorithm pre-integrated in the camera device can encode the collected video stream data, so that video encoding can be performed through an external hardware camera, and the video conferencing sending device obtains the encoded H.264 / H.265 video stream, and then slices and packages it for transmission.

[0137] The video conferencing communication system provided in the embodiment of the present application collects video stream data through a camera device. The pre-integrated video stream encoding algorithm is called to encode the video stream data to obtain video encoding data. The video encoding data is sent to the conference sending device through the video stream transmission channel, so that the conference sending device slices and packages the video encoding data, and sends the packaged video data packets to the conference receiving device. The conference receiving device decodes the video data packet to obtain a decoded video stream, and renders and displays the decoded video stream. The embodiment of the present application integrates a video stream encoding algorithm through a camera device, performs video encoding through an external hardware camera, and the video conferencing sending device obtains the encoded H.264 / H.265 video stream, which is then sliced ​​and packaged for transmission. The host's CPU resources can be completely eliminated, and there are no disadvantages such as hardware inconsistencies caused by MediaCodec calling the GPU.

[0138] Reference Figure 6 , shows a schematic diagram of the structure of another video conference communication system provided by an embodiment of the present application. Figure 6 As shown, the video conference communication system 600 may include: a camera device 610, a video encoding device 620, a conference sending device 630 and a conference receiving device 640, wherein:

[0139] The camera device is used to collect video stream data and send the video stream data to the video encoding device;

[0140] The video encoding device is used to call a pre-integrated video stream encoding algorithm to encode the video stream data to obtain video encoding data; and send the video encoding data to a conference sending device through a video stream transmission channel;

[0141] The conference sending device is used to slice and package the video encoding data to obtain a video data packet, and send the video data packet to the conference receiving device;

[0142] The conference receiving device is used to decode the video data packet to obtain a decoded video stream, and render and display the decoded video stream.

[0143] In this embodiment, by pre-integrating the video stream encoding algorithm in the video encoding device, the camera device is responsible for collecting video stream data and transmitting it to the video encoding device. The video encoding device calls the pre-integrated video stream encoding algorithm to encode the video stream data, and the video conferencing sending device obtains the encoded H.264 / H.265 video stream, and then slices and packages it for transmission.

[0144] In the video conferencing communication system provided in the embodiment of the present application, the camera device is used to collect video stream data and send the video stream data to the video encoding device. The video encoding device is used to call the pre-integrated video stream encoding algorithm to encode the video stream data, obtain the video encoding data, and send the video encoding data to the conference sending device through the video stream transmission channel. The conference sending device is used to slice and package the video encoding data to obtain a video data packet, and send the video data packet to the conference receiving device. The conference receiving device is used to decode the video data packet to obtain a decoded video stream, and render and display the decoded video stream. In the embodiment of the present application, the video encoding is performed by integrating the video stream encoding algorithm through an external video encoding device, and the conference sending device obtains the encoded H.264 / H.265 video stream, and then slices and packages it for transmission. It is possible to completely avoid using the host's CPU resources, and there are no disadvantages such as hardware inconsistency caused by MediaCodec calling the GPU.

[0145] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0146] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0147] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminals (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0148] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal so that a series of operating steps are executed on the computer or other programmable terminal to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0150] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0151] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or terminal including the elements.

[0152] The video conferencing communication method, device and system provided by the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A video conference communication method, characterized in that: Applied to a camera device, the method comprises: Collect video stream data; Calling a pre-integrated video stream encoding algorithm to encode the video stream data to obtain video encoding data; The video encoding data is sent to the conference sending device through the video stream transmission channel, so that the conference sending device slices and packages the video encoding data and sends the packaged video data packets to the conference receiving device.

2. The method according to claim 1, characterized in that Before collecting the video stream data, the method further includes: After detecting the power-on operation, performing an initialization operation, the initialization operation includes: a hardware detection operation, a software configuration operation, and a parameter setting operation; Initializing the UVC library, the initial UVC library comprising: loading a UVC driver, establishing a video stream transmission channel with the conference sending device through a USB interface, and configuring video stream information according to configuration parameters corresponding to the parameter setting operation; The video stream information includes: video stream format, video stream resolution and video stream frame rate.

3. The method according to claim 1, characterized in that The calling of a pre-integrated video stream encoding algorithm to encode the video stream data to obtain video encoding data includes: Get the video stream bitrate configured during initialization; Determine the number of encoded frames according to the video stream bit rate; The video stream encoding algorithm is called to encode the collected video stream data according to the encoding frame number to obtain the video encoding data.

4. The method according to claim 1, characterized in that: After collecting the video stream data, the method further includes: The video stream data is sent to a video encoding device, so that the video encoding device encodes the video stream data to obtain video stream encoded data, and the video stream encoded data is sent to the conference sending device.

5. A video conference communication device, characterized in that: Applied to a camera device, the device comprises: Video data acquisition module, used for acquiring video stream data; A video encoding data acquisition module is used to call a pre-integrated video stream encoding algorithm to encode the video stream data to obtain video encoding data; The video encoding data sending module is used to send the video encoding data to the conference sending device through the video stream transmission channel, so that the conference sending device slices and packages the video encoding data and sends the packaged video data packets to the conference receiving device.

6. The device according to claim 5, characterized in that The device also includes: An initialization module, used to perform an initialization operation after detecting a power-on operation, wherein the initialization operation includes: a hardware detection operation, a software configuration operation, and a parameter setting operation; A UVC library initialization module is used to initialize the UVC library, and the initial UVC library includes: loading a UVC driver, establishing a video stream transmission channel with the conference sending device through a USB interface, and configuring video stream information according to configuration parameters corresponding to the parameter setting operation; The video stream information includes: video stream format, video stream resolution and video stream frame rate.

7. The device according to claim 5, characterized in that The video encoding data acquisition module comprises: A video stream bit rate acquisition unit, used to acquire the video stream bit rate configured during the initialization process; A coding frame number determination unit, used to determine the coding frame number according to the video stream bit rate; The video encoding data acquisition unit is used to call the video stream encoding algorithm to encode the collected video stream data according to the encoding frame number to obtain the video encoding data.

8. The device according to claim 5, characterized in that The device also includes: The video stream data sending module is used to send the video stream data to the video encoding device so that the video encoding device encodes the video stream data to obtain video stream encoded data, and send the video stream encoded data to the conference sending device.

9. A video conference communication system, characterized in that: The system includes: a camera device, a conference sending device and a conference receiving device, wherein: The camera device is used to collect video stream data; call a pre-integrated video stream encoding algorithm to encode the video stream data to obtain video encoding data; and send the video encoding data to a conference sending device through a video stream transmission channel; The conference sending device is used to slice and package the video encoding data to obtain a video data packet, and send the video data packet to the conference receiving device; The conference receiving device is used to decode the video data packet to obtain a decoded video stream, and render and display the decoded video stream.

10. A video conference communication system, characterized in that: The system includes: a camera device, a video encoding device, a conference sending device and a conference receiving device, wherein: The camera device is used to collect video stream data and send the video stream data to the video encoding device; The video encoding device is used to call a pre-integrated video stream encoding algorithm to encode the video stream data to obtain video encoding data; and send the video encoding data to a conference sending device through a video stream transmission channel; The conference sending device is used to slice and package the video encoding data to obtain a video data packet, and send the video data packet to the conference receiving device; The conference receiving device is used to decode the video data packet to obtain a decoded video stream, and render and display the decoded video stream.

Images