Wireless remote video conferencing device and method
By using data transmission processing units in wireless remote video conferencing equipment to detect terminal resources and network environment, determine the priority of conference data and adjust the transmission process, the lag problem caused by equipment and network instability in remote video conferencing is solved, and the problem of lag caused by equipment and network instability is improved. Stability and fluency of the meeting.
Patent Information
- Application Number
- CN202311511821.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-14
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2043-11-14
AI Technical Summary
In the remote video conference, the performance of participants' terminal equipment and network environments is uneven, resulting in frequent problems such as lag and delay in video conferences, affecting the meeting effect.
Design a wireless remote video conferencing device and method, adopting a data transmission processing unit, including a participating terminal resource detection module, a priority order determination module, a traffic resource comparison module and a data determination module to be transmitted, by detecting terminal resources and network environment, Determine the priority of the conference data type, and pause low-priority data transmission when network resources are insufficient to optimize the data transmission process.
It improves the stability and fluency of remote video conferencing, ensures smooth transmission of audio data, and reduces the risk of lag caused by insufficient network.
Smart Images

Figure CN117615093B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of remote conferencing, and in particular to a wireless remote video conferencing device and method. Background Art
[0002] Remote video conferencing technology can be applied to online meetings, online education, online consultation, online business negotiations and other scenarios of large enterprises. Participants do not need to go to a specific location for face-to-face conversations, saving expenses on commuting, food and accommodation, and greatly reducing the communication costs between participants; however, in many remote video conferencing scenarios, participants usually conduct the conference at their own homes, and participants usually use personal computers or smart mobile communication devices, resulting in uneven performance of the participant's terminal and network environment, causing problems such as freezes and delays in video conferencing, thus affecting the effectiveness of remote video conferencing. Summary of the invention
[0003] In order to improve the stability of remote video conferencing, the present application provides a wireless remote video conferencing device and method.
[0004] The first invention objective of this application is achieved by adopting the following technical solution:
[0005] A wireless remote video conferencing device includes a camera component, an audio recording and broadcasting component, a display component, a command input component, a wireless communication component and a local processor, wherein the wireless communication component is communicatively connected to a remote conferencing platform with a cloud server as a carrier; the camera component, the audio recording and broadcasting component, the display component, the command input component and the wireless communication component are all communicatively connected to the local processor, and the local processor has a built-in data transmission processing unit, and the data transmission processing unit includes:
[0006] A participant terminal resource detection module is used to obtain the available device resources and available network resources of each participant terminal based on a preset detection period;
[0007] A priority order determination module, used to obtain conference attribute information and determine the priority order of each conference data type based on the conference attribute information;
[0008] The traffic resource comparison module is used to receive various conference data from the participating terminals in real time, count the type data traffic and comprehensive data traffic corresponding to each conference data type, and compare the comprehensive data traffic with the available network resources;
[0009] The module for determining data to be transmitted is used to set the restriction condition that the sum of type data flow is less than or equal to the available network resources if the available network resources are less than the comprehensive data flow, and determine several conference data types as data to be transmitted based on the priority of each conference data type from high to low.
[0010] By adopting the above technical solution, the camera component is used to shoot the video data of the participants, the audio recording and broadcasting component is used to record and play the audio data of the participants, the display component is used to display the video data, text data and display file data of the participants, etc., the instruction input component is used to control the wireless remote video conferencing equipment and input text data, etc., the wireless communication component is used to realize wireless data transmission and communication functions, the local processor is used to store and run computer programs, and perform data processing, and the data transmission processing unit is used to run the wireless remote video conferencing method, which specifically includes a participant terminal resource detection module, a priority order determination module, a traffic resource comparison module and a data to be transmitted determination module, thereby improving the stability of the remote video conferencing.
[0011] The second invention objective of this application is achieved by the following technical solution:
[0012] A wireless remote video conferencing method, comprising:
[0013] Obtaining available device resources and available network resources of each participating terminal based on a preset detection cycle;
[0014] Acquire conference attribute information, and determine the priority order of each conference data type based on the conference attribute information;
[0015] Receive various conference data from participating terminals in real time, count the type data flow and comprehensive data flow corresponding to each conference data type, and compare the comprehensive data flow with available network resources;
[0016] If the available network resources are less than the comprehensive data flow, a restriction condition is set that the total data flow of each type is less than or equal to the available network resources, and a number of conference data types are determined as data to be transmitted based on the priority of each conference data type from high to low;
[0017] The data to be transmitted is conference data to be transmitted to the remote conference platform.
[0018] By adopting the above technical solution, each participating terminal is regularly detected according to a preset detection cycle so as to know the computer performance and network environment of each participating terminal; because in a remote video conference, participants may need to send audio data, video data, text data, presentation files and other types of conference data to the remote conference platform, therefore, the conference attribute information is obtained to determine the needs of the required remote conference, analyze the urgency of the participants' needs for different types of conference data, and then determine the priority order of different conference data types; after the meeting starts, all types of conference data from the participating terminals are received in real time, and the type data traffic corresponding to each conference data type is counted , and count the sum of the data traffic of all conference data types as the comprehensive data traffic, and compare the comprehensive data traffic with the available network resources to determine whether the network resources corresponding to the participating terminal are sufficient to transmit all types of conference data in real time; if the available network resources are not sufficient to transmit all types of conference data in real time, then according to the priority order of each conference data type, the conference data type with a higher priority will be determined as the data to be transmitted. For example, if the focus of the meeting is on voice communication, the transmission of video data can be stopped when the user network is poor, reducing the demand for network resources connected to the participating terminal to ensure smooth transmission of audio data, thereby facilitating the improvement of the stability of remote video conferencing.
[0019] In a preferred example of the present application, the obtaining of conference attribute information and determining the priority order of each conference data type based on the conference attribute information include:
[0020] Receiving a conference attribute form sent by a conference initiator, and identifying corresponding conference attribute information based on the conference attribute form;
[0021] After the meeting starts, the meeting data is collected regularly based on the preset attribute evaluation cycle, the current meeting attributes are evaluated based on the real-time meeting data, and the meeting attribute information is updated;
[0022] The conference attribute information includes voice discussion type, video display type and file display type;
[0023] The priority order of the voice discussion conference data types is: text data > participant audio data > presentation file data > participant video data;
[0024] The priority order of the video display conference data types is: text data > participant video data > participant audio data > display file data;
[0025] The priority order of the file display type conference data types is: text data>display file data>participant audio data>participant video data.
[0026] By adopting the above technical solution, before the meeting starts, the meeting attribute form filled in by the meeting initiator is received so that the meeting attribute information can be determined from the meeting attribute form, which is convenient for subsequently determining the priority order of the corresponding meeting data types according to different meeting attribute information; because the meeting attribute form filled in by the meeting initiator may contain errors, or the topics and contents communicated by the participants may change with the progress of the meeting, therefore, real-time meeting data is regularly collected to evaluate the current meeting attributes, and then the meeting attribute information is updated, thereby improving the accuracy of the meeting attribute information.
[0027] In a preferred example of the present application: the pre-processing attributes of the conference data include local pre-processing type and cloud pre-processing type;
[0028] After determining a number of conference data types as data to be transmitted based on the priorities of the conference data types from high to low, the method further includes:
[0029] Setting the preprocessing attributes of several conference data types that are not determined as data to be transmitted to local preprocessing type;
[0030] The locally pre-processed conference data is input to the local processor, and after being processed according to the preset corresponding pre-processing program and the corresponding compression processing program, the pre-processed compressed data is generated and sent to the remote conference platform.
[0031] By adopting the above technical solution, the preprocessing attributes of several conference data types that have not been determined as data to be transmitted are set to local preprocessing type, so that these conference data can be processed locally later; the locally preprocessed conference data are input into the local processor, and the conference data are preprocessed and compressed accordingly according to the preset corresponding preprocessing program and corresponding compression program, and the data after corresponding preprocessing and compression are sent to the remote conference platform. Compared with directly sending the conference data to the remote conference platform, the computer resources of the local processor are utilized, the data flow required for transmission is reduced, and the fluency of the remote video conference is improved.
[0032] In a preferred example of the present application: after setting the preprocessing attributes of several conference data types that are not determined as data to be transmitted to the local preprocessing type, the following steps are included:
[0033] local processing time data required for preprocessing all conference data of the local preprocessing type based on the available device resources evaluation;
[0034] If the local processing time data is greater than a preset delay threshold, a data screening instruction is generated to determine a number of locally pre-processed conference data types as locally retained based on the priorities of each conference data type from low to high, and store the locally retained conference data locally;
[0035] When the available network resources are greater than the comprehensive data flow, a retained data upload instruction is generated to transmit the locally retained conference data to the remote conference platform.
[0036] By adopting the above technical solution, based on the known available device resources, the time required for preprocessing all conference data of the local preprocessing type is evaluated to obtain local processing time data; if the local processing time data is greater than the preset delay threshold, it means that the computer performance of the corresponding participating device is weak and it is impossible to preprocess all the conference data of the local preprocessing type in time, so several types of conference data of the local preprocessing type are selected from low to high based on the priority order and defined as local retention type, and the conference data of the local retention type is stored in the local database, so that when the computer performance and network environment of the participating device are poor, the sending of the participating data with a lower priority order is suspended to ensure the smooth conduct of the remote video conference as much as possible; when the available network resources of the participating terminal are greater than the comprehensive data traffic, that is, after the network environment is optimized or restored, a retained data upload instruction is generated to upload the locally stored outdated conference data to the remote conference platform, so as to form a playback file of the remote video conference, so that each participant can review the content that was not displayed in the remote video conference due to the network abnormality of a certain participant when they are free after the meeting.
[0037] In a preferred example of the present application, after determining that a number of local pre-processed conference data types are locally retained based on the priorities of each conference data type from low to high, the method further includes:
[0038] When the participant video data is locally stored, images are captured from the participant video data according to preset video capture rules and transmitted to the remote conference platform.
[0039] By adopting the above technical solution, when the video data of a participant is determined to be locally retained, images are captured regularly or randomly according to preset video capture rules and transmitted to the remote conference platform so that the conference host or other managers can monitor the participant's attendance status.
[0040] In a preferred example of the present application: after comparing the comprehensive data traffic with the available network resources, the method further includes:
[0041] If the available network resources are greater than the comprehensive data flow, local processing time data required for preprocessing all conference data of the local preprocessing type is evaluated based on the available device resources;
[0042] If the local processing time data is greater than a preset delay threshold, a pre-processing transfer instruction is generated to change the attribute of the conference data to a cloud pre-processing type.
[0043] By adopting the above technical solution, if the available network resources are greater than the comprehensive data traffic, the network environment to which the current participating terminal is connected is sufficient to meet the transmission requirements of all conference data, and the time required for preprocessing all locally preprocessed conference data is further evaluated based on the known available device resources; if the local processing time data is greater than the preset delay threshold, a preprocessing transfer instruction is generated to change the attributes of these conference data to cloud-preprocessed type, so that when the available network resources are sufficient and the user's conference terminal computer performance is poor, the amount of local preprocessed data can be reduced, and part of the conference data can be transmitted to the cloud for preprocessing, thereby reducing the possibility of remote video conferencing being stuck due to poor computer performance of the conference terminal.
[0044] In a preferred example, the present application also includes:
[0045] When receiving a participant's joining signal, a network detection instruction is sent to the corresponding participant terminal to obtain its currently available network resources;
[0046] The latest available network resources of each participating terminal are compared with the preset network warning threshold. If the available network resources of any participating terminal are less than the network warning threshold, a display file upload prompt message is generated and sent to all participating terminals.
[0047] By adopting the above technical solution, when a new participant joins a remote video conference, the available network resources corresponding to the participant terminal of the new participant are immediately detected; the latest available network resources of each participant terminal are compared with the preset network warning threshold value to find out whether the current participant terminal has a poor connected network environment; when the available network resources of any participant terminal are less than the network warning threshold value, a presentation file prompt message is generated and sent to all participant terminals to prompt each participant that there are participants with poor network in the current meeting, so that each participant can upload the presentation files needed for the meeting to the cloud in advance, so that the participants with poor network can use idle network resources to download or upload presentation files in advance.
[0048] The third invention objective of this application is achieved by the following technical solution:
[0049] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned wireless remote video conferencing method when executing the computer program.
[0050] The fourth invention objective of this application is achieved by the following technical solution:
[0051] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the wireless remote video conferencing method are implemented.
[0052] In summary, the present application includes at least one of the following beneficial technical effects:
[0053] 1. The camera component is used to shoot the video data of the participants, the audio recording and broadcasting component is used to record and play the audio data of the participants, the display component is used to display the video data, text data and display file data of the participants, etc., the command input component is used to control the wireless remote video conferencing equipment and input text data, etc., the wireless communication component is used to realize wireless data transmission and communication functions, the local processor is used to store and run computer programs, and perform data processing, and the data transmission processing unit is used to run the wireless remote video conferencing method, which specifically includes a participant terminal resource detection module, a priority order determination module, a traffic resource comparison module and a data to be transmitted determination module, so as to improve the stability of the remote video conferencing.
[0054] 2. Each participant terminal is regularly detected according to a preset detection cycle so as to know the computer performance and network environment of each participant terminal; since in a remote video conference, participants may need to send audio data, video data, text data, presentation files and other types of conference data to the remote conference platform, therefore, the conference attribute information is obtained to determine the needs of the remote conference to be conducted, analyze the urgency of the participants' needs for different types of conference data, and then determine the priority order of different conference data types; after the meeting starts, all types of conference data from the participant terminals are received in real time, the type data flow corresponding to each conference data type is counted, and the sum of the data flow of all conference data types is counted as the comprehensive data flow, and the comprehensive data flow is compared with the available network resources to determine whether the network resources corresponding to the participant terminal are sufficient to transmit all types of conference data in real time; if the available network resources are insufficient to transmit all types of conference data in real time, then according to the priority order of each conference data type, the conference data type with a higher priority is determined as the data to be transmitted. For example, if the focus of the meeting is on voice communication, the transmission of video data can be stopped when the user network is poor, reducing the demand for network resources connected to the participant terminal to ensure smooth transmission of audio data, so as to improve the stability of the remote video conference.
[0055] 3. Set the preprocessing properties of several conference data types that have not been determined as data to be transmitted as local preprocessing type, so that these conference data can be processed locally later; input the locally preprocessed conference data into the local processor, and perform corresponding preprocessing and compression on the conference data according to the preset corresponding preprocessing program and corresponding compression program, and send the corresponding preprocessed and compressed data to the remote conference platform. Compared with directly sending the conference data to the remote conference platform, the computer resources of the local processor are utilized, the data flow required for transmission is reduced, and the fluency of the remote video conference is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] Figure 1 It is a principle block diagram of the data transmission processing unit in Example 1 of the present application.
[0057] Figure 2 It is a flow chart of the wireless remote video conferencing method in the second embodiment of the present application.
[0058] Figure 3 It is a schematic diagram of the equipment in Example 3 of the present application. DETAILED DESCRIPTION
[0059] The following is combined with Figures 1 to 3 This application is described in further detail. Embodiment 1
[0060] The present application discloses a wireless remote video conferencing device, which specifically includes a camera component, an audio recording and broadcasting component, a display component, a command input component, a wireless communication component and a local processor, wherein the camera component, the audio recording and broadcasting component, the display component, the command input component and the wireless communication component are all communicatively connected to the local processor; the camera component is used to shoot the video data of the participants, the audio recording and broadcasting component is used to record and play the audio data of the participants, the display component is used to display the video data, text data and display file data of the participants, etc., the command input component is used to control the wireless remote video conferencing device and input text data, etc., the wireless communication component is used to realize wireless data transmission and communication functions, and the local processor is used to store and run computer programs, and perform data processing.
[0061] Each wireless remote video conferencing device can be used as a conference terminal. The wireless communication component is connected to the remote conferencing platform based on the cloud server of the operator of the remote video conferencing program. The remote conferencing platform is used to receive conference data sent by each conference terminal, and can process the conference data according to actual needs and send the processed data to each conference terminal.
[0062] Among them, the wireless remote video conferencing equipment can be a smart phone, tablet computer, laptop or desktop computer, etc., as long as it meets the above hardware requirements.
[0063] The local processor has a built-in data transfer processing unit, such as Figure 1 As shown, the data transmission processing unit includes a participating terminal resource detection module, a priority order determination module, a traffic resource comparison module and a to-be-transmitted data determination module. The detailed description of each functional module is as follows:
[0064] A participant terminal resource detection module is used to obtain the available device resources and available network resources of each participant terminal based on a preset detection period;
[0065] A priority order determination module, used to obtain conference attribute information and determine the priority order of each conference data type based on the conference attribute information;
[0066] The traffic resource comparison module is used to receive various conference data from the participating terminals in real time, count the type data traffic and comprehensive data traffic corresponding to each conference data type, and compare the comprehensive data traffic with the available network resources;
[0067] The module for determining data to be transmitted is used to set the restriction condition that the sum of type data flow is less than or equal to the available network resources if the available network resources are less than the comprehensive data flow, and determine several conference data types as data to be transmitted based on the priority of each conference data type from high to low.
[0068] For the specific definition of the data transmission processing unit, please refer to the definition of the wireless remote video conferencing method below, which will not be repeated here; the various modules in the above-mentioned data transmission processing unit can be implemented in whole or in part through software, hardware and their combination; the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of the above-mentioned modules. Embodiment 2
[0069] Reference Figure 2 The present application discloses a wireless remote video conferencing method, which can be applied to the wireless remote video conferencing device in the above-mentioned embodiment 1, and specifically comprises the following steps:
[0070] S10: Obtain available device resources and available network resources of each participating terminal based on a preset detection period.
[0071] In this embodiment, the detection period refers to the period for detecting the computer performance of each participating terminal and the performance indicators of the connected network. Preferably, each detection period can be set to 20S, or it can be set to other time according to actual needs; available device resources refer to the data of computer resources that the participating terminal can use to run the remote video conferencing program, including processor resources and memory resources; available network resources refer to the data of network resources that the participating terminal can use to run the remote video conferencing program, including upload speed and download speed.
[0072] Specifically, when a participant is detected to join a remote video conference, a device detection instruction is sent to the corresponding participating terminal to regularly detect the available device resources and available network resources of each participating terminal according to a preset detection cycle, so as to obtain the computer performance and network environment of each participating terminal.
[0073] S20: Acquire conference attribute information, and determine the priority order of each conference data type based on the conference attribute information.
[0074] In this embodiment, conference attribute information refers to information used to determine the importance of various information carriers in communication in a remote video conference; conference data types include but are not limited to text data, participant audio data, presentation file data, participant video data, etc.
[0075] Specifically, since in a remote video conference, participants may need to send various types of conference data such as text data, participant audio data, presentation file data, participant video data, etc. to the remote conference platform, it is necessary to obtain conference attribute information to determine the needs of the required remote conference, analyze the urgency of the participants' needs for different types of conference data, and then determine the priority order of different conference data types, so as to determine the transmission and processing priority of different types of conference data when the available device resources and available network resources of the participating terminals are insufficient.
[0076] Before step S20, the following steps are included:
[0077] S21: receiving a conference attribute form sent by a conference initiator, and identifying corresponding conference attribute information based on the conference attribute form.
[0078] In this embodiment, the conference attribute form refers to a form regarding conference attribute information that is filled in by the conference initiator when the conference initiator creates a conference.
[0079] Specifically, when the conference initiator creates a conference, a conference attribute form is sent to the conference participant terminal corresponding to the conference initiator, and the conference attribute form filled out by the conference initiator is received so as to determine the conference attribute information from the conference attribute form, so as to facilitate the subsequent determination of the priority order of corresponding conference data types according to different conference attribute information.
[0080] The conference attribute information is determined based on the most important content carrier in the online video conference, and the conference attribute information includes voice discussion type, video display type and file display type;
[0081] The priority order of the conference data types for voice discussion is: text data > participant audio data > presentation file data > participant video data;
[0082] The priority order of video presentation conference data types is: text data > participant video data > participant audio data > presentation file data;
[0083] The priority order of conference data types of file display type is: text data > display file data > participant audio data > participant video data.
[0084] S22: After the meeting starts, the meeting data is collected regularly based on a preset attribute evaluation cycle, the current meeting attributes are evaluated according to the real-time meeting data, and the meeting attribute information is updated.
[0085] In this embodiment, the attribute evaluation period refers to a period for re-evaluating the conference attributes of the remote video conference. Preferably, the attribute evaluation period can be set to 5 minutes.
[0086] Specifically, since the meeting attribute form filled out by the meeting initiator may contain errors, or the topics and content communicated by the participants may change as the meeting progresses, real-time meeting data is regularly collected to evaluate the current meeting attributes, and then the meeting attribute information is updated to improve the accuracy of the meeting attribute information.
[0087] Specifically, the participant video data, participant audio data and display file data are obtained. If only the head portrait or upper body image of each participant appears in the participant video data, and the movement changes slightly, the video display type of conference attribute information is excluded. If the image changes significantly in the speaker's participant video data, the conference attribute information is determined to be a video display type. The speaker refers to the participant who currently occupies the main image display window in the remote video conference or is inputting real-time participant audio data. If the current main image display window displays display file data, the conference attribute information is determined to be a file display type. If the current main image display window displays neither participant video data nor display file data, and a participant is inputting real-time participant audio data, the conference attribute information is determined to be a voice discussion type.
[0088] Furthermore, since whether the remote video conference can run smoothly depends on the participating terminal with the worst available device resources and / or available network resources, in other embodiments of the present application, the participant corresponding to the participating terminal with the worst available device resources and / or available network resources can also manually select the conference attribute information when he or she is the speaker, so as to facilitate the targeted transmission of conference data of the type that the speaker considers to be most important.
[0089] Furthermore, the priority order determination module also includes:
[0090] A conference attribute identification submodule, used to receive a conference attribute form sent by a conference initiator, and identify corresponding conference attribute information based on the conference attribute form;
[0091] The conference attribute information updating submodule is used to collect conference data regularly based on a preset attribute evaluation cycle after the conference starts, evaluate the current conference attributes according to the real-time conference data, and update the conference attribute information.
[0092] S30: receiving various conference data from participating terminals in real time, counting the type data flow and comprehensive data flow corresponding to each conference data type, and comparing the comprehensive data flow with available network resources.
[0093] In this embodiment, the type data flow refers to the data flow of a specific type of conference data; the comprehensive data flow refers to the total data flow of all types of conference data from the participating terminals.
[0094] Specifically, after the meeting starts, all types of conference data are received from the participating terminals in real time, the type data flow corresponding to each conference data type is counted, and the sum of the data flow of all conference data types is counted as the comprehensive data flow, and the comprehensive data flow is compared with the available network resources to determine whether the network resources corresponding to the participating terminal are sufficient to transmit all types of conference data in real time; among which, the criterion for determining whether the network resources corresponding to the participating terminal are sufficient to transmit all types of conference data in real time is to complete the data transmission within the network delay time that does not affect the smoothness of the meeting.
[0095] S40: If the available network resources are less than the comprehensive data flow, a restriction condition is set that the total type of data flow is less than or equal to the available network resources, and a number of conference data types are determined as data to be transmitted based on the priority of each conference data type from high to low.
[0096] In this embodiment, the data to be transmitted is conference data to be transmitted to the remote conference platform.
[0097] Specifically, if the available network resources are insufficient to transmit all types of conference data in real time, several conference data types with higher priorities will be determined as data to be transmitted based on the priority order of each conference data type. When determining the data to be transmitted, a certain amount of available network resources should be reserved, for example, 20% of the available network resources should be reserved to reduce the possibility of remote video conferencing being stuck due to network fluctuations. For example, if the focus of the meeting is on voice communication, the transmission of video data can be stopped when the user network is poor, reducing the demand for network resources connected to the participating terminals to ensure smooth transmission of audio data and facilitate improving the stability of remote video conferencing.
[0098] After step S40, the following steps are included:
[0099] S41: setting the preprocessing attributes of several conference data types that are not determined as data to be transmitted to a local preprocessing type.
[0100] In this embodiment, the preprocessing attributes of conference data include local preprocessing and cloud preprocessing, wherein the local preprocessing type conference data is preprocessed by the local processor by default, and the cloud preprocessing type conference data is preprocessed by the remote conference platform by default; wherein, data preprocessing specifically includes background cropping / replacement of person images in participant video data, speech-to-text conversion, audio subtitle generation, speech-to-text translation, video compression, audio compression, video beautification, presentation file editing and processing, etc.
[0101] Specifically, the preprocessing attributes of several conference data types that are not determined as data to be transmitted are set to local preprocessing type, so that these conference data can be processed locally later.
[0102] S42: The locally pre-processed conference data is input into the local processor, and after the conference data is processed according to the preset corresponding pre-processing program and the corresponding compression processing program, the pre-processed compressed data is generated and sent to the remote conference platform.
[0103] In this embodiment, the corresponding preprocessing program refers to a program for preprocessing a specific type of conference data; the corresponding compression processing program refers to a program for compressing a specific type of conference data.
[0104] Specifically, the locally preprocessed conference data is input into the local processor, and the conference data is preprocessed and compressed accordingly according to the preset corresponding preprocessing program and the corresponding compression program, wherein the order of preprocessing and compression can be adjusted according to actual needs, and the data after corresponding preprocessing and compression is sent to the remote conference platform. Compared with directly sending the conference data to the remote conference platform, the computer resources of the local processor are utilized, the data flow required for transmission is reduced, and the fluency of the remote video conference is improved.
[0105] Furthermore, the module for determining data to be transmitted further includes:
[0106] A preprocessing attribute setting submodule, used to set the preprocessing attributes of several conference data types that are not determined as data to be transmitted to a local preprocessing type;
[0107] The data preprocessing and compression submodule is used to input the locally preprocessed conference data into the local processor, process the conference data according to the preset corresponding preprocessing program and the corresponding compression processing program, generate preprocessed compressed data and send it to the remote conference platform.
[0108] After step S41, the following steps are included:
[0109] S43: local processing time data required for preprocessing all conference data of the local preprocessing type based on the available device resources evaluation.
[0110] Specifically, based on known available device resources, the time required for preprocessing all conference data of the local preprocessing type is evaluated to obtain local processing time data.
[0111] S44: If the local processing time data is greater than a preset delay threshold, a data screening instruction is generated, and a number of locally pre-processed conference data types are determined as locally retained based on the priority of each conference data type from low to high, and the locally retained conference data is stored locally.
[0112] In this embodiment, the delay threshold refers to the value of the maximum acceptable data processing delay time to ensure the smooth progress of the remote video conference. Preferably, the delay threshold can be set to 50MS; local retention type refers to conference data whose processing and transmission are temporarily suspended due to insufficient available device resources and available network resources of the current participating devices.
[0113] Specifically, if the local processing time data is greater than the preset delay threshold, it means that the computer performance of the corresponding participating device is weak and cannot pre-process all locally pre-processed conference data in time. Therefore, several types of locally pre-processed conference data are selected from low to high based on priority and defined as locally retained types. The locally retained conference data is stored in the local database so that when the computer performance and network environment of the participating device are poor, the sending of participating data with lower priority can be suspended to ensure the smoothness of remote video conferencing as much as possible.
[0114] S45: When the available network resources are greater than the comprehensive data flow, a retained data upload instruction is generated to transmit the locally retained conference data to the remote conference platform.
[0115] In this embodiment, the stored data upload instruction refers to an instruction for controlling the transmission of locally stored conference data to the remote conference platform.
[0116] Specifically, when the available network resources of the participating terminals are greater than the comprehensive data traffic, that is, after the network environment is optimized or restored, a retained data upload instruction is generated to upload the locally stored outdated conference data to the remote conference platform, so as to form a playback file of the remote video conference, so that each participant can review the content that was not displayed in the remote video conference due to the network abnormality of a participant when they are free after the meeting.
[0117] The module for determining data to be transmitted further includes:
[0118] A local preprocessing time evaluation submodule, used for evaluating the local processing time data required for preprocessing all conference data of the local preprocessing type based on the available device resources;
[0119] A local data storage submodule is used to generate a data screening instruction if the local processing time data is greater than a preset delay threshold, determine a number of local pre-processed conference data types as local retention types based on the priority of each conference data type from low to high, and store the locally retained conference data locally;
[0120] The retained data uploading submodule is used to generate a retained data uploading instruction to transmit the locally retained conference data to the remote conference platform when the available network resources are greater than the comprehensive data flow.
[0121] Wherein, after step S44, the following steps are included:
[0122] S46: When the participant video data is locally stored, images are captured from the participant video data according to a preset video capture rule and transmitted to the remote conference platform.
[0123] Specifically, when the participant video data is determined to be locally retained, images are captured regularly or randomly according to preset video capture rules and transmitted to the remote conference platform, so that the meeting host or other managers can monitor the participant's attendance status without obtaining the participant's corresponding video data.
[0124] The module for determining data to be transmitted further includes:
[0125] The video image capture submodule is used to capture images from the participant's video data according to preset video capture rules and transmit them to the remote conference platform when the participant's video data is locally stored.
[0126] Wherein, after step S30, the wireless remote video conferencing method further includes:
[0127] S50: If the available network resources are greater than the comprehensive data flow, local processing time data required for preprocessing all conference data of the local preprocessing type is evaluated based on the available device resources.
[0128] Specifically, if the available network resources are greater than the comprehensive data traffic, the network environment to which the current participating terminal is connected is sufficient to meet the transmission requirements of all conference data, and the time required for preprocessing all locally preprocessed conference data is further evaluated based on known available device resources.
[0129] S60: If the local processing time data is greater than a preset delay threshold, a pre-processing transfer instruction is generated to change the attribute of the conference data to a cloud pre-processing type.
[0130] Specifically, if the local processing time data is greater than a preset delay threshold, a preprocessing transfer instruction is generated to change the attributes of these conference data to cloud preprocessing type, so that when the available network resources are sufficient and the user's conference terminal computer performance is poor, the amount of local preprocessing data can be reduced, and part of the conference data can be transferred to the cloud for preprocessing, thereby reducing the possibility of remote video conferencing being stuck due to poor computer performance of the conference terminal; among them, changing the attributes of the conference data to cloud preprocessing type needs to be fully or partially changed according to the actual available device resources of the participating terminals.
[0131] Furthermore, the wireless remote video conferencing system also includes:
[0132] A local processing time data generating module, configured to evaluate the local processing time data required for preprocessing all conference data of the local preprocessing type based on the available device resources if the available network resources are greater than the comprehensive data flow;
[0133] The conference data attribute changing module is used to generate a pre-processing transfer instruction if the local processing time data is greater than a preset delay threshold, so as to change the attribute of the conference data to a cloud pre-processing type.
[0134] The wireless remote video conferencing method further includes:
[0135] S70: When a participant joining signal is received, a network detection instruction is sent to the corresponding participant terminal to obtain its currently available network resources.
[0136] In this embodiment, the network detection instruction refers to an instruction for enabling an available network resource detection program.
[0137] Specifically, when a new participant joins the remote video conference, the available network resources corresponding to the participating terminal of the new participant are detected in real time.
[0138] S80: Compare the latest available network resources of each participating terminal with a preset network warning threshold. If the available network resources of any participating terminal are less than the network warning threshold, generate a presentation file upload prompt message and send it to all participating terminals.
[0139] In this embodiment, the network warning threshold refers to a threshold used to determine whether the participating terminal is connected to a poor network environment.
[0140] Specifically, the latest available network resources of each participating terminal are compared with a preset network warning threshold to determine whether the current participating terminal is connected to a poor network environment. When the available network resources of any participating terminal are less than the network warning threshold, a presentation file prompt message is generated and sent to all participating terminals to remind each participant that there are participants with poor network in the current meeting, so that each participant can upload the presentation files needed for the meeting to the cloud in advance, and the participants with poor network can use idle network resources to download or upload presentation files in advance.
[0141] Furthermore, the wireless remote video conferencing system also includes:
[0142] The network detection module is used to send a network detection instruction to the corresponding participating terminal to obtain its currently available network resources when receiving a participant's joining signal;
[0143] The display file upload prompt module is used to compare the latest available network resources of each participating terminal with the preset network warning threshold. If the available network resources of any participating terminal are less than the network warning threshold, a display file upload prompt message is generated and sent to all participating terminals.
[0144] It should be understood that the serial numbers of the steps in the above embodiments do not imply a sequence of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. Embodiment 3
[0145] A computer device, which may be a server, may have an internal structure as shown in Figure 3 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as detection cycles, available device resources, available network resources, conference attribute information, conference data and corresponding priority order, type data flow, comprehensive data flow and data to be transmitted. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a wireless remote video conferencing method is implemented.
[0146] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:
[0147] S10: Obtaining available device resources and available network resources of each participating terminal based on a preset detection period;
[0148] S20: Acquire conference attribute information, and determine the priority order of each conference data type based on the conference attribute information;
[0149] S30: receiving various conference data from participating terminals in real time, counting the type data flow and comprehensive data flow corresponding to each conference data type, and comparing the comprehensive data flow with available network resources;
[0150] S40: If the available network resources are less than the comprehensive data flow, a restriction condition is set that the total type of data flow is less than or equal to the available network resources, and a number of conference data types are determined as data to be transmitted based on the priority of each conference data type from high to low.
[0151] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
[0152] S10: Obtaining available device resources and available network resources of each participating terminal based on a preset detection period;
[0153] S20: Acquire conference attribute information, and determine the priority order of each conference data type based on the conference attribute information;
[0154] S30: receiving various conference data from participating terminals in real time, counting the type data flow and comprehensive data flow corresponding to each conference data type, and comparing the comprehensive data flow with available network resources;
[0155] S40: If the available network resources are less than the comprehensive data flow, a restriction condition is set that the total type of data flow is less than or equal to the available network resources, and a number of conference data types are determined as data to be transmitted based on the priority of each conference data type from high to low.
[0156] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink), DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0157] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0158] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the features thereof may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A wireless remote video conferencing device, characterized in that: It includes a camera component, an audio recording and broadcasting component, a display component, a command input component, a wireless communication component and a local processor, wherein the wireless communication component is communicatively connected to a remote conference platform based on a cloud server; the camera component, the audio recording and broadcasting component, the display component, the command input component and the wireless communication component are all communicatively connected to the local processor, and the local processor has a built-in data transmission processing unit, and the data transmission processing unit includes: A participant terminal resource detection module is used to obtain the available device resources and available network resources of each participant terminal based on a preset detection period; A priority order determination module, used to obtain conference attribute information and determine the priority order of each conference data type based on the conference attribute information; The traffic resource comparison module is used to receive various conference data from the participating terminals in real time, count the type data traffic and comprehensive data traffic corresponding to each conference data type, and compare the comprehensive data traffic with the available network resources; A module for determining data to be transmitted, for setting a restriction condition that the sum of the type data flow is less than or equal to the available network resources if the available network resources are less than the comprehensive data flow, and determining a number of conference data types as data to be transmitted based on the priority of each conference data type from high to low; The priority determination module also includes: A conference attribute identification submodule, used to receive a conference attribute form sent by a conference initiator, and identify corresponding conference attribute information based on the conference attribute form; The conference attribute information updating submodule is used to collect conference data regularly based on a preset attribute evaluation cycle after the conference starts, evaluate the current conference attributes based on the real-time conference data, and update the conference attribute information; The conference attribute information includes voice discussion type, video display type and file display type; The priority order of the voice discussion conference data types is: text data > participant audio data > presentation file data > participant video data; The priority order of the video display conference data types is: text data > participant video data > participant audio data > display file data; The priority order of the conference data types of the file display type is: text data > display file data > participant audio data > participant video data; The module for determining data to be transmitted also includes: A preprocessing attribute setting submodule, used to set the preprocessing attributes of several conference data types that are not determined as data to be transmitted to a local preprocessing type; The data preprocessing and compression submodule is used to input the locally preprocessed conference data into the local processor, process the conference data according to the preset corresponding preprocessing program and the corresponding compression processing program, generate preprocessed compressed data and send it to the remote conference platform; A local preprocessing time evaluation submodule, used for evaluating the local processing time data required for preprocessing all conference data of the local preprocessing type based on the available device resources; A local data storage submodule is used to generate a data screening instruction if the local processing time data is greater than a preset delay threshold, determine a number of local pre-processed conference data types as local retention types based on the priority of each conference data type from low to high, and store the locally retained conference data locally; The retained data uploading submodule is used to generate a retained data uploading instruction to transmit the locally retained conference data to the remote conference platform when the available network resources are greater than the comprehensive data flow.
2. A wireless remote video conferencing method, characterized in that: The wireless remote video conferencing device as claimed in claim 1 comprises: Obtaining available device resources and available network resources of each participating terminal based on a preset detection cycle; Acquire conference attribute information, and determine the priority order of each conference data type based on the conference attribute information; Receive various conference data from participating terminals in real time, count the type data flow and comprehensive data flow corresponding to each conference data type, and compare the comprehensive data flow with available network resources; If the available network resources are less than the comprehensive data flow, a restriction condition is set that the total data flow of each type is less than or equal to the available network resources, and a number of conference data types are determined as data to be transmitted based on the priority of each conference data type from high to low; The data to be transmitted is conference data to be transmitted to the remote conference platform; The obtaining of conference attribute information and determining the priority order of each conference data type based on the conference attribute information include: Receive a conference attribute form sent by a conference initiator, and identify corresponding conference attribute information based on the conference attribute form; after the meeting starts, regularly collect conference data based on a preset attribute evaluation cycle, evaluate the current conference attributes based on real-time conference data, and update the conference attribute information; The conference attribute information includes voice discussion type, video display type and file display type; The priority order of the voice discussion conference data types is: text data > participant audio data > presentation file data > participant video data; The priority order of the video display conference data types is: text data > participant video data > participant audio data > display file data; The priority order of the conference data types of the file display type is: text data > display file data > participant audio data > participant video data; After determining a number of conference data types as data to be transmitted based on the priorities of the conference data types from high to low, the method further includes: Setting the preprocessing attributes of several conference data types that are not determined as data to be transmitted to local preprocessing type; The locally pre-processed conference data is input into the local processor, and after the conference data is processed according to the preset corresponding pre-processing program and the corresponding compression processing program, the pre-processed compressed data is generated and sent to the remote conference platform; The preprocessing attributes of the conference data include local preprocessing type and cloud preprocessing type; After the preprocessing attributes of the plurality of conference data types that are not determined as data to be transmitted are set to the local preprocessing type, the following steps are included: local processing time data required for preprocessing all conference data of the local preprocessing type based on the available device resources evaluation; If the local processing time data is greater than a preset delay threshold, a data screening instruction is generated to determine a number of locally pre-processed conference data types as locally retained based on the priorities of each conference data type from low to high, and store the locally retained conference data locally; When the available network resources are greater than the comprehensive data flow, a retained data upload instruction is generated to transmit the locally retained conference data to the remote conference platform.
3. A wireless remote video conferencing method according to claim 2, characterized in that: After determining that several locally pre-processed conference data types are locally retained based on the priorities of each conference data type from low to high, the method also includes: when the participant video data is locally retained, capturing images from the participant video data according to preset video capture rules and transmitting them to the remote conference platform.
4. A wireless remote video conferencing method according to claim 2, characterized in that: After comparing the comprehensive data traffic with the available network resources, the method further includes: If the available network resources are greater than the comprehensive data flow, local processing time data required for preprocessing all conference data of the local preprocessing type is evaluated based on the available device resources; If the local processing time data is greater than a preset delay threshold, a pre-processing transfer instruction is generated to change the attribute of the conference data to a cloud pre-processing type.
5. A wireless remote video conferencing method according to claim 2, characterized in that: Also includes: When receiving a participant's joining signal, a network detection instruction is sent to the corresponding participant terminal to obtain its currently available network resources; The latest available network resources of each participating terminal are compared with the preset network warning threshold. If the available network resources of any participating terminal are less than the network warning threshold, a display file upload prompt message is generated and sent to all participating terminals.
6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the wireless remote video conferencing method according to any one of claims 2 to 5 are implemented.
7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the wireless remote video conferencing method according to any one of claims 2 to 5 are implemented.
