Video conference processing method and device, equipment and medium

Abstract

The embodiment of the invention provides a video conference processing method and device, equipment and a medium, and the method comprises the steps: receiving a conference audio and video stream sent by a first conference client in a video conference process; generating interactive multimedia data according to the conference audio and video stream; and sending the conference audio and video stream and the interactive multimedia data to a second conference client so as to present the conference audio and video stream and the interactive multimedia data at the second conference client. According to the embodiment of the invention, the interaction content is presented in the video conference, and the interactivity of the video conference is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video conferencing, and in particular to a method, device, equipment and medium for video conferencing processing. Background Art

[0002] With the development of science and technology, video conferencing technology has been widely used in fields such as online education, telemedicine, emergency command, etc., bringing more intelligent and efficient solutions to all walks of life.

[0003] However, existing video conferencing technologies have poor expansion capabilities and can usually only present the audio and video streams collected by the conference client in the video conference. Video conferencing lacks interactivity and is difficult to meet demand. Summary of the invention

[0004] In view of the above problems, a method, apparatus, device and medium for video conference processing are proposed to overcome the above problems or at least partially solve the above problems, including:

[0005] A method for video conferencing processing, the method comprising:

[0006] During the video conference, receiving the conference audio and video stream sent by the first conference client;

[0007] Generate interactive multimedia data according to the conference audio and video streams;

[0008] The conference audio and video stream and the interactive multimedia data are sent to the second conference client so that the conference audio and video stream and the interactive multimedia data are presented on the second conference client.

[0009] Optionally, generating interactive multimedia data according to the conference audio and video streams includes:

[0010] Analyze the conference audio and video streams to obtain analysis results;

[0011] Interactive multimedia data is generated according to the analysis result.

[0012] Optionally, the analyzing the conference audio and video stream to obtain an analysis result includes:

[0013] Performing speech recognition on the conference audio and video stream to obtain a speech recognition result;

[0014] The speech recognition result is analyzed to obtain an analysis result.

[0015] Optionally, analyzing the speech recognition result to obtain the analysis result includes:

[0016] The speech recognition results are analyzed using a preset big data model to obtain analysis results including target words; wherein the target words include any one or more of the following: high-frequency words, keywords.

[0017] Optionally, generating interactive multimedia data according to the analysis result includes:

[0018] Receive interactive control instructions;

[0019] Parsing the interactive control instructions to obtain interactive data generation rules,

[0020] According to the interactive data generation rule, interactive multimedia data for the analysis result is generated.

[0021] Optionally, the interactive data generation rule includes candidate words and multimedia data types corresponding to the candidate words, and generating interactive multimedia data for the analysis result according to the interactive data generation rule includes:

[0022] When the target word in the analysis result matches the candidate word in the interactive data generation rule, interactive multimedia data for the target word is generated according to the type of multimedia data corresponding to the candidate word in the interactive data generation rule.

[0023] Optionally, the multimedia data type includes any one or more of the following: video, picture, emoticon.

[0024] A device for video conference processing, the device comprising:

[0025] A data receiving module, used to receive the conference audio and video stream sent by the first conference client during the video conference;

[0026] A data generation module, used to generate interactive multimedia data according to the conference audio and video streams;

[0027] The data sending module is used to send the conference audio and video stream and the interactive multimedia data to the second conference client, so that the conference audio and video stream and the interactive multimedia data are presented on the second conference client.

[0028] An electronic device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method described above when executed by the processor.

[0029] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0030] The embodiments of the present invention have the following advantages:

[0031] In an embodiment of the present invention, during a video conference, a conference audio and video stream sent by a first conference client is received, interactive multimedia data is generated based on the conference audio and video stream, and the conference audio and video stream and the interactive multimedia data are sent to a second conference client so that the conference audio and video stream and the interactive multimedia data are presented on the second conference client. This enables the presentation of interactive content in the video conference and improves the interactivity of the video conference. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0033] Figure 1 is a flowchart of the steps of a method for video conference processing provided by some embodiments of the present invention;

[0034] Figure 2 is a flowchart of another method for video conference processing provided by some embodiments of the present invention;

[0035] Figure 3 is a flowchart of another method for video conference processing provided by some embodiments of the present invention;

[0036] Figure 4 is a flowchart of another method for video conference processing provided by some embodiments of the present invention;

[0037] Figure 5 is a flowchart of another method for video conference processing provided by some embodiments of the present invention;

[0038] Figure 6 It is a structural block diagram of a video conference processing device provided by some embodiments of the present invention. DETAILED DESCRIPTION

[0039] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Reference Figure 1, shows a flow chart of steps of video conference processing provided by some embodiments of the present invention, which can be applied to the server side, such as Figure 2 Medium computing power server, computing power server is a server specially designed to provide computing power, which is usually equipped with high-performance processors, large amounts of memory and high-speed storage devices. Such servers are widely used in data centers, cloud computing platforms, high-performance computing environments, and occasions that require large amounts of data processing and analysis. Computing power servers can support complex scientific computing, big data analysis, machine learning, and training and reasoning of artificial intelligence algorithms.

[0041] Specifically, the following steps may be included:

[0042] Step 101: During a video conference, a conference audio and video stream sent by a first conference client is received.

[0043] In some examples, the video conference can be a video conference based on a visual network, or a video conference based on the Internet or other networks. The conference clients participating in the video conference can be visual network clients, or the conference clients can also be Internet clients, or the conference clients can also be partly visual network clients and partly Internet clients.

[0044] After starting the video conference, the first conference client can collect the conference audio and video streams through devices such as cameras and microphones, and then send the collected conference audio and video streams to the server side.

[0045] In some examples, such as Figure 2 The first conference client can be a visual network client, and the server side can be a computing power server. The visual network client can collect the conference audio and video streams of the host or participants, and then send the conference audio and video streams to the core server. The core server can then send the conference audio and video streams to the computing power server for subsequent processing on the computing power server.

[0046] In some examples, after a conference client joins a video conference, the identity role of the conference client can be determined based on the account information logged into the conference client, such as the identity role being a host, a participant, etc.

[0047] Among them, a video conferencing terminal is a device specifically used for video conferencing. It can provide high-quality video and audio communications and support interaction between remote conference participants. This terminal usually includes a high-definition camera, microphone, speaker, and video conferencing software, and can be connected to the network to achieve remote video communication. Video conferencing terminals are widely used in enterprises, educational institutions, government agencies, and other occasions where remote collaboration and communication are required.

[0048] The visual networking core server is the core part of the entire visual networking system. It is responsible for processing and managing all video streams and data in the network. The core server usually has high-performance processing capabilities, large-capacity storage space, and highly reliable network connections to ensure the smooth operation of applications such as video conferencing, distance education, and video surveillance. When designing and deploying the visual networking core server, its scalability, security, and stability need to be considered to adapt to the growing user needs and data traffic.

[0049] Step 102: Generate interactive multimedia data based on the conference audio and video streams.

[0050] After obtaining the conference audio and video stream, the server side can generate interactive multimedia data based on the received conference audio and video stream, such as generating interactive multimedia data based on the audio data in the conference audio and video stream, or generating interactive multimedia data based on the video data in the conference audio and video stream, or generating interactive multimedia data based on the audio data and video data in the conference audio and video stream.

[0051] In some examples, the interactive multimedia data may be data that can interact with the participants, such as interactive multimedia data such as videos, pictures, emoticons, etc.

[0052] In some embodiments of the present invention, generating interactive multimedia data based on the conference audio and video stream includes: analyzing the conference audio and video stream to obtain an analysis result; and generating interactive multimedia data based on the analysis result.

[0053] In actual applications, the server side can analyze the conference audio and video streams to obtain analysis results, and then generate corresponding interactive multimedia data based on the analysis results. The interactive multimedia data corresponds to the analysis results. In some examples, the interactive multimedia data can include the analysis results. For example, if the analysis results include target words, the interactive multimedia data can carry the target words in the form of multimedia.

[0054] In some embodiments of the present invention, analyzing the conference audio and video stream to obtain an analysis result includes: performing speech recognition on the conference audio and video stream to obtain a speech recognition result; and analyzing the speech recognition result to obtain an analysis result.

[0055] In a specific implementation, the server side may first pre-process the conference audio and video streams. The pre-processing process is to perform real-time speech recognition on the audio data in the conference audio and video streams to obtain a speech recognition result, such as the speech recognition result being the text content appearing in the audio data. After obtaining the speech recognition result, the server side may analyze the speech recognition result to obtain an analysis result.

[0056] In some embodiments of the present invention, analyzing the speech recognition results to obtain analysis results includes: using a preset big data model to analyze the speech recognition results to obtain analysis results including target words; wherein the target words include any one or more of the following: high-frequency words, keywords.

[0057] In actual applications, a big data model can be pre-set or a third-party big data model can be called. After obtaining the speech recognition result, the big data model can be used to analyze the speech recognition result to obtain an analysis result including the target word. In some examples, the speech recognition result is the text content appearing in the audio data, and the big data model can be used to extract the target word from the text content.

[0058] In some examples, the target words may be high-frequency words or keywords. For example, the big data model may determine high-frequency words based on the frequency of the words appearing in the text content. Alternatively, the big data model may determine keywords in the text content based on the contextual semantics of the text content.

[0059] In some embodiments of the present invention, generating interactive multimedia data according to the analysis results includes: receiving interactive control instructions; parsing the interactive control instructions to obtain interactive data generation rules, and generating interactive multimedia data for the analysis results according to the interactive data generation rules.

[0060] In actual applications, the server side can obtain interactive control instructions for audio and video streams, and then obtain interactive data generation rules by parsing the interactive control instructions, and then convert the analysis results into interactive multimedia data according to the instructions of the interactive data generation rules.

[0061] In some examples, such as Figure 2 , the interactive control instructions can be sent to the computing power server by the core server while forwarding the audio and video streams to the computing power server.

[0062] In some embodiments of the present invention, the interactive data generation rule includes candidate words and multimedia data types corresponding to the candidate words, and generating interactive multimedia data for the analysis result according to the interactive data generation rule includes:

[0063] When the target word in the analysis result matches the candidate word in the interactive data generation rule, interactive multimedia data for the target word is generated according to the type of multimedia data corresponding to the candidate word in the interactive data generation rule.

[0064] In practical applications, the interactive data generation rules include multiple candidate words and multimedia data types corresponding to each candidate word, such as multimedia data types including any one or more of the following: video, picture, emoticon. After obtaining the analysis results, the analysis results carry the target word, and the target word can be matched in the candidate words. When the candidate word matching the target word is found, the multimedia data type corresponding to the candidate word matching the target word can be determined.

[0065] After obtaining the multimedia data type, the target words in the analysis results can be converted into interactive multimedia data corresponding to the multimedia data type, such as videos, pictures, and emoticons.

[0066] Step 103: Send the conference audio and video stream and the interactive multimedia data to the second conference client, so that the conference audio and video stream and the interactive multimedia data are presented on the second conference client.

[0067] In some examples, the second conference client may be another conference client participating in the conference.

[0068] In actual applications, the conference audio and video stream and interactive multimedia data can be sent to the second conference client, and then the conference audio and video stream and interactive multimedia data can be superimposed and played in the second conference client. In some examples, the superimposed playback can be to synchronize the interactive multimedia data and the conference audio and video stream according to the timestamp and play them in the same interface, such as playing the interaction and multimedia data corresponding to the target word when the target word is mentioned.

[0069] In an embodiment of the present invention, by superimposing and playing the conference audio and video streams and interactive multimedia data, it is achieved that during the video conference, by monitoring specific audio words or sentences, some pre-made logo images or animation videos or real-time synthesized video clips will appear on the screen of the conference client, thereby solving the problem of poor expansion function of video conferencing and inability to interact in real time according to keyword rules.

[0070] In some examples, the conference audio and video streams and interactive multimedia data can be sent to the second conference client at the same time, or the conference audio and video streams and interactive multimedia data can be packaged into a data set and sent to the second conference client, or the conference audio and video streams and interactive multimedia data can be sent separately.

[0071] In some examples, the analysis result may also be sent to the second conference client, so that the second client may be informed whether there is interactive multimedia data to be presented.

[0072] In some examples, after receiving the conference audio and video stream, the second client can monitor whether the corresponding interactive multimedia data is received. If the interactive multimedia data corresponding to the conference audio and video stream is received within a specified time range, the conference audio and video stream and the interactive multimedia data can be superimposed and played. If the interactive multimedia data corresponding to the conference audio and video stream is not received within the specified time range, a regeneration instruction can be sent to the server side, thereby prompting the server side to resend the interactive multimedia data. In some examples, when resending the interactive multimedia data, in order to allow the conference audio and video stream and the interactive multimedia data to be superimposed and played, the timestamp corresponding to the conference audio and video stream can be carried for synchronization.

[0073] In an embodiment of the present invention, during a video conference, a conference audio and video stream sent by a first conference client is received, interactive multimedia data is generated based on the conference audio and video stream, and the conference audio and video stream and the interactive multimedia data are sent to a second conference client so that the conference audio and video stream and the interactive multimedia data are presented on the second conference client. This enables the presentation of interactive content in the video conference and improves the interactivity of the video conference.

[0074] The following combination Figure 2 The embodiments of the present invention are exemplarily described as follows:

[0075] The visual network client collects the conference audio and video streams of the host or participants, and then sends the conference audio and video streams to the core server. The core server then determines the interactive data generation rules corresponding to the conference audio and video streams, and sends the conference audio and video streams and the corresponding interactive data generation rules to the computing power server.

[0076] The computing server can decode the received conference audio and video streams, extract the audio data, and perform speech recognition on the audio data to obtain the target words.

[0077] For example, the spokesperson repeatedly mentioned visual Internet of Things technology, and the audio data contained target terms of visual Internet of Things technology such as "visual Internet of Things", "visual power", "V2V protocol", and "new computing power network".

[0078] The computing power server can generate interactive multimedia data corresponding to the target words according to the interactive data generation rules. When the target words are included in the audio data, the corresponding interactive multimedia data, such as animations, pictures, and emoticons, can be superimposed and played in the visual network client that plays the conference audio and video stream.

[0079] Reference Figure 3 , shows a flowchart of another video conference processing step provided by some embodiments of the present invention, which may specifically include the following steps:

[0080] Step 301: During a video conference, a conference audio and video stream sent by a first conference client is received.

[0081] After starting the video conference, the first conference client can collect the conference audio and video streams through devices such as cameras and microphones, and then send the collected conference audio and video streams to the server side.

[0082] Step 302: Perform speech recognition on the conference audio and video stream to obtain a speech recognition result.

[0083] In a specific implementation, the server side may first preprocess the conference audio and video streams. The preprocessing process is to perform real-time speech recognition on the audio data in the conference audio and video streams to obtain speech recognition results, such as the speech recognition results being the text content appearing in the audio data.

[0084] Step 303, using a preset big data model, analyze the speech recognition result to obtain an analysis result including target words; wherein the target words include any one or more of the following: high-frequency words, keywords.

[0085] In actual applications, a big data model can be pre-set or a third-party big data model can be called. After obtaining the speech recognition result, the big data model can be used to analyze the speech recognition result to obtain an analysis result including the target word. In some examples, the speech recognition result is the text content appearing in the audio data, and the big data model can be used to extract the target word from the text content.

[0086] In some examples, the target words may be high-frequency words or keywords. For example, the big data model may determine high-frequency words based on the frequency of the words appearing in the text content. Alternatively, the big data model may determine keywords in the text content based on the contextual semantics of the text content.

[0087] Step 304: Generate interactive multimedia data according to the analysis result.

[0088] In actual applications, the server side can analyze the conference audio and video streams to obtain analysis results, and then generate corresponding interactive multimedia data based on the analysis results. The interactive multimedia data corresponds to the analysis results. In some examples, the interactive multimedia data can include the analysis results. For example, if the analysis results include target words, the interactive multimedia data can carry the target words in the form of multimedia.

[0089] Step 305: Send the conference audio and video stream and the interactive multimedia data to the second conference client, so that the conference audio and video stream and the interactive multimedia data are presented on the second conference client.

[0090] In actual applications, the conference audio and video stream and interactive multimedia data can be sent to the second conference client, and then the conference audio and video stream and interactive multimedia data can be superimposed and played in the second conference client. In some examples, the superimposed playback can be to synchronize the interactive multimedia data and the conference audio and video stream according to the timestamp and play them in the same interface, such as playing the interaction and multimedia data corresponding to the target word when the target word is mentioned.

[0091] In an embodiment of the present invention, by superimposing and playing the conference audio and video streams and interactive multimedia data, it is achieved that during the video conference, by monitoring specific audio words or sentences, some pre-made logo images or animation videos or real-time synthesized video clips will appear on the screen of the conference client, thereby solving the problem of poor expansion function of video conferencing and inability to interact in real time according to keyword rules.

[0092] Reference Figure 4 , shows a flowchart of another video conference processing step provided by some embodiments of the present invention, which may specifically include the following steps:

[0093] Step 401: During a video conference, a conference audio and video stream sent by a first conference client is received.

[0094] After starting the video conference, the first conference client can collect the conference audio and video streams through devices such as cameras and microphones, and then send the collected conference audio and video streams to the server side.

[0095] Step 402: Analyze the conference audio and video streams to obtain analysis results.

[0096] Step 403, receiving an interactive control instruction, and parsing the interactive control instruction to obtain an interactive data generation rule; wherein the interactive data generation rule includes candidate words and multimedia data types corresponding to the candidate words.

[0097] In actual applications, the server side can obtain interactive control instructions for audio and video streams, and then obtain interactive data generation rules by parsing the interactive control instructions, and then convert the analysis results into interactive multimedia data according to the instructions of the interactive data generation rules.

[0098] Step 404, when the target word in the analysis result matches the candidate word in the interactive data generation rule, generating interactive multimedia data for the target word according to the multimedia data type corresponding to the candidate word in the interactive data generation rule.

[0099] In practical applications, the interactive data generation rules include multiple candidate words and multimedia data types corresponding to each candidate word, such as multimedia data types including any one or more of the following: video, picture, emoticon. After obtaining the analysis results, the analysis results carry the target word, and the target word can be matched in the candidate words. When the candidate word matching the target word is found, the multimedia data type corresponding to the candidate word matching the target word can be determined.

[0100] After obtaining the multimedia data type, the target words in the analysis results can be converted into interactive multimedia data corresponding to the multimedia data type, such as videos, pictures, and emoticons.

[0101] Step 405: Send the conference audio and video stream and the interactive multimedia data to the second conference client, so that the conference audio and video stream and the interactive multimedia data are presented on the second conference client.

[0102] In actual applications, the conference audio and video stream and interactive multimedia data can be sent to the second conference client, and then the conference audio and video stream and interactive multimedia data can be superimposed and played in the second conference client. In some examples, the superimposed playback can be to synchronize the interactive multimedia data and the conference audio and video stream according to the timestamp and play them in the same interface, such as playing the interaction and multimedia data corresponding to the target word when the target word is mentioned.

[0103] In an embodiment of the present invention, by superimposing and playing the conference audio and video streams and interactive multimedia data, it is achieved that during the video conference, by monitoring specific audio words or sentences, some pre-made logo images or animation videos or real-time synthesized video clips will appear on the screen of the conference client, thereby solving the problem of poor expansion function of video conferencing and inability to interact in real time according to keyword rules.

[0104] Reference Figure 5 , shows a flowchart of another video conference processing step provided by some embodiments of the present invention, which may specifically include the following steps:

[0105] Step 501: During a video conference, a conference audio and video stream sent by a first conference client is received.

[0106] After starting the video conference, the first conference client can collect the conference audio and video streams through devices such as cameras and microphones, and then send the collected conference audio and video streams to the server side.

[0107] Step 502: Perform speech recognition on the conference audio and video stream to obtain a speech recognition result.

[0108] In a specific implementation, the server side may first preprocess the conference audio and video streams. The preprocessing process is to perform real-time speech recognition on the audio data in the conference audio and video streams to obtain speech recognition results, such as the speech recognition results being the text content appearing in the audio data.

[0109] Step 503, using a preset big data model, analyze the speech recognition result to obtain an analysis result including target words; wherein the target words include any one or more of the following: high-frequency words, keywords.

[0110] In actual applications, a big data model can be pre-set or a third-party big data model can be called. After obtaining the speech recognition result, the big data model can be used to analyze the speech recognition result to obtain an analysis result including the target word. In some examples, the speech recognition result is the text content appearing in the audio data, and the big data model can be used to extract the target word from the text content.

[0111] In some examples, the target words may be high-frequency words or keywords. For example, the big data model may determine high-frequency words based on the frequency of the words appearing in the text content. Alternatively, the big data model may determine keywords in the text content based on the contextual semantics of the text content.

[0112] Step 504, receiving an interactive control instruction, and parsing the interactive control instruction to obtain an interactive data generation rule; wherein the interactive data generation rule includes candidate words and multimedia data types corresponding to the candidate words.

[0113] In actual applications, the server side can obtain interactive control instructions for audio and video streams, and then obtain interactive data generation rules by parsing the interactive control instructions, and then convert the analysis results into interactive multimedia data according to the instructions of the interactive data generation rules.

[0114] Step 505, when the target word in the analysis result matches the candidate word in the interactive data generation rule, generating interactive multimedia data for the target word according to the multimedia data type corresponding to the candidate word in the interactive data generation rule.

[0115] In practical applications, the interactive data generation rules include multiple candidate words and multimedia data types corresponding to each candidate word, such as multimedia data types including any one or more of the following: video, picture, emoticon. After obtaining the analysis results, the analysis results carry the target word, and the target word can be matched in the candidate words. When the candidate word matching the target word is found, the multimedia data type corresponding to the candidate word matching the target word can be determined.

[0116] After obtaining the multimedia data type, the target words in the analysis results can be converted into interactive multimedia data corresponding to the multimedia data type, such as videos, pictures, and emoticons.

[0117] Step 506: Send the conference audio and video stream and the interactive multimedia data to the second conference client, so that the conference audio and video stream and the interactive multimedia data are presented on the second conference client.

[0118] In actual applications, the conference audio and video stream and interactive multimedia data can be sent to the second conference client, and then the conference audio and video stream and interactive multimedia data can be superimposed and played in the second conference client. In some examples, the superimposed playback can be to synchronize the interactive multimedia data and the conference audio and video stream according to the timestamp and play them in the same interface, such as playing the interaction and multimedia data corresponding to the target word when the target word is mentioned.

[0119] In an embodiment of the present invention, by superimposing and playing the conference audio and video streams and interactive multimedia data, it is achieved that during the video conference, by monitoring specific audio words or sentences, some pre-made logo images or animation videos or real-time synthesized video clips will appear on the screen of the conference client, thereby solving the problem of poor expansion function of video conferencing and inability to interact in real time according to keyword rules.

[0120] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0121] Reference Figure 6 , shows a schematic diagram of the structure of a video conference processing device provided by some embodiments of the present invention, which may specifically include the following modules:

[0122] The data receiving module 601 is used to receive the conference audio and video stream sent by the first conference client during the video conference;

[0123] A data generation module 602, configured to generate interactive multimedia data according to the conference audio and video streams;

[0124] The data sending module 603 is used to send the conference audio and video stream and the interactive multimedia data to the second conference client, so that the conference audio and video stream and the interactive multimedia data are presented on the second conference client.

[0125] In some embodiments of the present invention, generating interactive multimedia data according to the conference audio and video streams includes:

[0126] Analyze the conference audio and video streams to obtain analysis results;

[0127] Interactive multimedia data is generated according to the analysis result.

[0128] In some embodiments of the present invention, the analyzing the conference audio and video stream to obtain the analysis result includes:

[0129] Performing speech recognition on the conference audio and video stream to obtain a speech recognition result;

[0130] The speech recognition result is analyzed to obtain an analysis result.

[0131] In some embodiments of the present invention, analyzing the speech recognition result to obtain the analysis result includes:

[0132] The speech recognition results are analyzed using a preset big data model to obtain analysis results including target words; wherein the target words include any one or more of the following: high-frequency words, keywords.

[0133] In some embodiments of the present invention, generating interactive multimedia data according to the analysis result includes:

[0134] Receive interactive control instructions;

[0135] Parsing the interactive control instructions to obtain interactive data generation rules,

[0136] According to the interactive data generation rule, interactive multimedia data for the analysis result is generated.

[0137] In some embodiments of the present invention, the interactive data generation rule includes candidate words and multimedia data types corresponding to the candidate words, and generating interactive multimedia data for the analysis result according to the interactive data generation rule includes:

[0138] When the target word in the analysis result matches the candidate word in the interactive data generation rule, interactive multimedia data for the target word is generated according to the type of multimedia data corresponding to the candidate word in the interactive data generation rule.

[0139] In an embodiment of the present invention, during a video conference, a conference audio and video stream sent by a first conference client is received, interactive multimedia data is generated based on the conference audio and video stream, and the conference audio and video stream and the interactive multimedia data are sent to a second conference client so that the conference audio and video stream and the interactive multimedia data are presented on the second conference client. This enables the presentation of interactive content in the video conference and improves the interactivity of the video conference.

[0140] In some embodiments of the present invention, the multimedia data type includes any one or more of the following: video, picture, emoticon.

[0141] Some embodiments of the present invention further provide an electronic device, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the above method is implemented when the computer program is executed by the processor.

[0142] Some embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, and the computer program implements the above method when executed by a processor.

[0143] Some embodiments of the present invention further provide a computer program product, including a computer program, which implements the above method when executed by a processor.

[0144] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0145] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0146] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0147] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0148] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0149] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0150] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0151] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0152] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the above elements.

[0153] The above is a detailed introduction to the provided method, device, equipment and medium for video conferencing processing. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for video conference processing, characterized in that: The method comprises: During the video conference, receiving the conference audio and video stream sent by the first conference client; Generate interactive multimedia data according to the conference audio and video streams; The conference audio and video stream and the interactive multimedia data are sent to the second conference client so that the conference audio and video stream and the interactive multimedia data are presented on the second conference client.

2. The method according to claim 1, characterized in that The step of generating interactive multimedia data according to the conference audio and video streams includes: Analyze the conference audio and video streams to obtain analysis results; Interactive multimedia data is generated according to the analysis result.

3. The method according to claim 2, characterized in that The analyzing the conference audio and video stream to obtain the analysis result includes: Performing speech recognition on the conference audio and video stream to obtain a speech recognition result; The speech recognition result is analyzed to obtain an analysis result.

4. The method according to claim 3, characterized in that The step of analyzing the speech recognition result to obtain the analysis result includes: The speech recognition results are analyzed using a preset big data model to obtain analysis results including target words; wherein the target words include any one or more of the following: high-frequency words, keywords.

5. The method according to any one of claims 2 to 4, characterized in that: The step of generating interactive multimedia data according to the analysis result includes: Receive interactive control instructions; Parsing the interactive control instructions to obtain interactive data generation rules, According to the interactive data generation rule, interactive multimedia data for the analysis result is generated.

6. The method according to claim 5, characterized in that The interactive data generation rule includes candidate words and multimedia data types corresponding to the candidate words, and generating interactive multimedia data for the analysis result according to the interactive data generation rule includes: When the target word in the analysis result matches the candidate word in the interactive data generation rule, interactive multimedia data for the target word is generated according to the type of multimedia data corresponding to the candidate word in the interactive data generation rule.

7. The method according to claim 6, characterized in that The multimedia data type includes any one or more of the following: video, picture, emoticon.

8. A device for video conferencing processing, characterized in that: The device comprises: A data receiving module, used to receive the conference audio and video stream sent by the first conference client during the video conference; A data generation module, used to generate interactive multimedia data according to the conference audio and video streams; The data sending module is used to send the conference audio and video stream and the interactive multimedia data to the second conference client, so that the conference audio and video stream and the interactive multimedia data are presented on the second conference client.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images