Conference coordination method, device, system and electronic equipment based on intelligent audio

Abstract

The embodiments of the present invention provide a conference coordination method, device, system and electronic device based on intelligent audio, including: obtaining audio data during the meeting, the audio data including environmental audio data and human voice audio data; processing the environmental audio data and human voice audio data based on a preset AI big model, and determining the key meeting data during the meeting; performing AI-assisted processing on the key meeting data. By intelligently identifying the audio data during the meeting and intelligently processing the identified key meeting data through a preset AI big model, the demand for AI-assisted processing of key meeting data under complex meeting scenarios is met, the efficiency and convenience of information transmission in the meeting are improved, and the AI-assisted function can be selected for flexible processing of the key meeting data in the meeting, thereby improving the efficiency and quality of the meeting.

Description

Technical Field

[0001] The present invention relates to the field of smart conferences, and in particular to a conference coordination method, device, system, electronic device and storage medium thereof based on smart speakers. Background Art

[0002] With the increase in office and meeting needs, the market demand for audio equipment with powerful computing power and intelligent functions is growing. In the existing intelligent audio system technology, since traditional intelligent audio systems usually only have the basic functions of audio playback, voice recognition and voice control, they are unable to automatically and intelligently recognize the audio during the meeting in the meeting scenario, and intelligently process the corresponding meeting content based on the recognition results, so as to meet many meeting needs in the meeting scenario. Therefore, in the existing intelligent audio system technology, there is a problem that it is unable to intelligently recognize the audio during the meeting in the meeting scenario, and intelligently process the corresponding meeting content based on the recognition results. Summary of the invention

[0003] The embodiment of the present invention provides a conference coordination method based on intelligent audio to solve the problem that the audio in the conference process cannot be intelligently recognized and the corresponding conference content cannot be intelligently processed according to the recognition result.

[0004] In a first aspect, an embodiment of the present invention provides a conference coordination method based on an intelligent speaker, the method comprising the following steps:

[0005] Acquire audio data during the conference, wherein the audio data includes environmental audio data and human voice audio data;

[0006] Processing the environmental audio data and the human voice audio data based on a preset AI big model, and determining at least the key meeting data during the meeting;

[0007] At least AI-assisted processing is performed on the key meeting data.

[0008] Optionally, the method for acquiring audio data during the conference further includes:

[0009] Preprocessing the audio data acquired during the conference by using a preset microphone array to obtain target audio data;

[0010] The target audio data is subjected to feature classification processing based on a preset AI large model to obtain the environmental audio data and the human voice audio data.

[0011] Optionally, the processing of the human voice audio data based on a preset AI big model to at least determine key meeting data during the meeting includes:

[0012] Performing timbre feature recognition processing on the human voice audio data through a preset AI large model to obtain target human voice timbre feature data;

[0013] Determining the emotional characteristics of the corresponding person based on the target human voice and tone characteristic data;

[0014] Based on the emotional characteristics, key meeting data is determined during the meeting.

[0015] Optionally, the processing of the environmental audio data based on a preset AI big model to at least determine key meeting data during the meeting includes:

[0016] Performing environmental timbre content recognition processing on the environmental audio data through a preset AI large model to obtain target environmental timbre feature data;

[0017] Determining the meeting action of the corresponding person based on the target environment timbre content characteristic data;

[0018] Based on the conference actions, conference key data is determined during the conference.

[0019] Optionally, performing AI-assisted processing on at least the key conference data includes:

[0020] Performing attribute analysis on the key conference data to determine the conference demand type corresponding to the key conference data;

[0021] Based on the type of meeting demand, the corresponding preset AI assistance function is used to perform AI-assisted processing on the corresponding meeting key data.

[0022] Optionally, the performing attribute analysis on the key conference data to determine the conference requirement type corresponding to the key conference data includes:

[0023] Based on the preset AI big model, the audio data corresponding to the collected meeting key data is transcribed to obtain the text data corresponding to the meeting key data;

[0024] By using a preset sentiment model, a direction analysis is performed on the content of the text data to determine the direction of the meeting demand;

[0025] Based on the conference demand direction, the conference demand type is determined.

[0026] In a second aspect, an embodiment of the present invention further provides a conference coordination device based on an intelligent speaker, wherein the conference coordination device based on an intelligent speaker comprises:

[0027] An acquisition module, used to acquire audio data during a conference, wherein the audio data includes environmental audio data and human voice audio data;

[0028] A determination module, used to process the environmental audio data and the human voice audio data based on a preset AI big model, and determine the key meeting data during the meeting;

[0029] The auxiliary module is used to perform AI-assisted processing on the key data of the meeting.

[0030] In a third aspect, an embodiment of the present invention further provides a conference coordination system based on an intelligent speaker, and the conference coordination system based on an intelligent speaker comprises: a conference coordination device based on an intelligent speaker, a server and an intelligent speaker.

[0031] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the smart speaker-based conference coordination method provided in an embodiment of the present invention when executing the computer program.

[0032] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the conference coordination method based on smart speakers provided in the embodiment of the invention are implemented.

[0033] In an embodiment of the present invention, audio data is obtained during a meeting, the audio data including environmental audio data and human voice audio data; the environmental audio data and human voice audio data are processed based on a preset AI big model, and key meeting data is determined during the meeting; AI-assisted processing is performed on the key meeting data. By intelligently identifying the audio data during the meeting and intelligently processing the identified key meeting data through a preset AI big model, the need for AI-assisted processing of key meeting data in complex meeting scenarios is met, the efficiency and convenience of information transmission in the meeting is improved, and the AI-assisted function can be selected for flexible processing of key meeting data in the meeting, thereby improving the efficiency and quality of the meeting. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 This is an architecture diagram of a conference coordination system based on intelligent audio provided by an embodiment of the present invention;

[0036] Figure 2 is a flow chart of a conference coordination method based on intelligent audio provided by an embodiment of the present invention;

[0037] Figure 3 yes Figure 2 The specific flow chart of step 201;

[0038] Figure 4 yes Figure 2 Specific flow chart of step 202;

[0039] Figure 5 yes Figure 2 Another specific flow chart of step 202;

[0040] Figure 6 yes Figure 2 The specific flow chart of step 203;

[0041] Figure 7 yes Figure 6 The specific flow chart of step 2031;

[0042] Figure 8 is a structural diagram of a conference coordination device based on intelligent audio provided in an embodiment of the present invention;

[0043] Fig. 9 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] like Figure 1 As shown, Figure 1It is an architecture diagram of a conference coordination system 100 based on smart speakers provided by an embodiment of the present invention. The conference coordination based on smart speakers includes: a conference coordination device 800 based on smart speakers, a server 101, and a smart speaker 102. Among them, the above-mentioned conference coordination device 800 based on smart speakers also includes an acquisition module, which can be used to acquire audio data during the meeting; a determination module, which can be used to process environmental audio data and human voice audio data based on a preset AI large model, and determine the key data of the meeting during the meeting; an auxiliary module, which can be used to perform AI-assisted processing on the key data of the meeting.

[0046] Specifically, the audio data during the above conference may include but is not limited to environmental audio data and human voice audio data. Generally speaking, the above audio data can be obtained by collecting all-round sound through a microphone installed in the smart speaker. The above environmental audio data may include but is not limited to noise data in the environment, natural audio data in the environment, and interactive audio data of people in the environment.

[0047] More specifically, since various natural audio data in nature will affect the human voice audio data in the meeting, it is necessary to collect and identify the natural audio data, and perform denoising or clearing processing. The above-mentioned noise data can be the pickup noise caused by signal interference between electronic devices used in the meeting, which will also affect the human voice audio data in the meeting. The above-mentioned interactive audio data of the characters can be the sound generated by the interaction between characters, or between characters and objects. For example, when a character uses a keyboard to record, the sound data generated by the interaction between the character and the keyboard can also be attributed to the interactive audio data of the characters in the environment.

[0048] The above-mentioned human voice audio data can be language audio data that can be recognized by a preset AI big model to derive specific content and meaning. It is understandable that the human voice audio data can be audio data of human text language. Therefore, the human voice audio data can also include but is not limited to the language and text pronunciation of different races and countries. Generally speaking, audio data that can be converted from speech recognition to text through a preset AI big model and has substantial meaning after conversion can be used as the above-mentioned human voice audio data.

[0049] In one embodiment, the audio data obtained during the meeting can be obtained by collecting audio through a microphone array set in an intelligent speaker. The microphone array can collect audio data in the meeting by arranging and combining the microphones in different ways, or using different types of microphones, so as to achieve the effect of all-round sound pickup. Among them, before the microphone array collects audio data during the meeting, it is also necessary to judge whether it is in a meeting state. Specifically, it can be judged by the conference participants pressing the conference button corresponding to the intelligent speaker. The microphone array can also recognize the voice of the conference participants. When words or short sentences that can describe the start of a meeting, such as "meeting", "meeting" or "start meeting", are recognized, the intelligent speaker is triggered to collect audio data during the meeting.

[0050] More specifically, it is also possible to determine whether the voice of the conference participants is continuous, and to compare the continuous duration of the sound source coordinates of the current voice with the preset continuous duration to determine whether it is in a meeting state. Among them, if the continuous duration of the sound source coordinates of the current voice is greater than or equal to the preset continuous duration, it means that the audio data of the person corresponding to the sound source coordinates is continuous, and the continuous duration is long, and it should be in a statement state, rather than an ordinary chat state. Therefore, it is judged that it is in a meeting state at this time, so as to activate the smart speaker to collect audio data during the meeting. If the continuous duration of the sound source coordinates of the current voice is less than the preset continuous duration, it means that the audio data of the person corresponding to the sound source coordinates can be continuous or discontinuous, and the continuous duration is short, and it should be in a chat state, rather than a statement state. Therefore, it is judged that it is not in a meeting state at this time, and the smart speaker is not activated to collect audio data during the meeting.

[0051] In another optional solution, the judgment of whether the above is in a meeting state can also be determined by whether the voice of the conference participants is continuous, combined with the sound source coordinates of the conference participants. As a specific example of marking the sound source coordinates, the collected human voice audio data is first located to determine the sound source coordinates corresponding to each person in the current meeting, and the sound source coordinates are detected in real time within a preset time period. When the number of sound source coordinates detected to remain in the original coordinates within the preset time period is less than the preset ratio, it is judged that some of the conference participants have changed their positions. It can be understood that in a meeting state, people generally do not move at will. Therefore, the above-mentioned intelligent audio-based conference coordination system can judge that the above-mentioned scene is not in a meeting state. When the number of sound source coordinates detected to remain in the original coordinates within the preset time period is greater than or equal to the preset ratio, it is judged that the above-mentioned scene is in a meeting state. It should be noted that the above-mentioned preset ratio can be adjusted and set according to the scale of the meeting, that is, the number of participants.

[0052] The above-mentioned preset AI big model may be a deep learning model that can recognize and process the above-mentioned audio data, so as to determine the key data in the meeting process through the audio data. The model can also transcribe the above-mentioned audio data to complete the transformation of audio data into text data, that is, the text data corresponding to the content of the audio data can be obtained by transcribing the audio data into text. Generally speaking, the above-mentioned preset AI big model may include but is not limited to the GPT (Generative Pre-trained Transformer) series, BERT (Bidirectional EncoderRepresentations from Transformers) series, and Wenxin Yiyan and other big models. The above-mentioned preset AI big model can also provide functions such as speech recognition, natural language processing, and meeting minutes generation for meeting content.

[0053] In another possible embodiment, the above-mentioned smart audio-based conference coordination system recognizes and processes the above-mentioned collected environmental audio data and life audio data through a built-in preset AI large model to determine the key meeting data during the meeting.

[0054] The above-mentioned meeting key data may be the meeting content corresponding to the moment with substantial characteristics during the meeting. Specifically, the above-mentioned meeting key data may be the moment with guiding or substantial characteristics of the meeting during the meeting, or the meeting content corresponding to that moment. For example, it may be the meeting content that aroused discussion among the meeting participants, or it may be the moment when the most complex audio data was collected.

[0055] More specifically, the above-mentioned meeting key data can be data used to summarize or refine the meeting, obtain meeting abstracts or meeting minutes and other documents, so as to facilitate the meeting participants to summarize the meeting content. It is understandable that the above-mentioned meeting key data can be obtained through prompts or emphasis by leaders during the meeting, or through the preset AI large model attached to the above-mentioned smart audio-based conference coordination system to recognize and process the audio data during the meeting, extract information about summarizing or refining the meeting content, and generate text data based on the information, which is stored in the database corresponding to the current meeting in the above-mentioned smart audio-based conference coordination system.

[0056] The above-mentioned AI-assisted processing can determine the requirements of the above-mentioned key meeting data according to the preset AI large model attached to the above-mentioned smart audio-based conference coordination system, and use the corresponding functions to complete the processing of the key meeting data according to the corresponding requirements. Specifically, when the above-mentioned key meeting data is analyzed as an issue that people are discussing, a meeting text file corresponding to the issue can be generated, and key points can be marked to facilitate determination after subsequent meetings. The functions corresponding to the above-mentioned AI-assisted processing may include but are not limited to any functions that can meet the needs of complex meeting scenarios, such as conference audio file generation, conference text file generation, conference summary generation and email sending.

[0057] In a possible embodiment, the above-mentioned smart audio-based conference coordination system intelligently identifies audio data during the meeting, and intelligently processes the identified key meeting data through a preset AI large model to meet the needs of AI-assisted processing of key meeting data under complex meeting scenarios, thereby improving the efficiency and convenience of information transmission in the meeting, and being able to flexibly select AI-assisted functions for key meeting data in the meeting, thereby improving meeting efficiency and quality.

[0058] like Figure 2 As shown, Figure 2 1 is a flow chart of a conference coordination method based on an intelligent speaker provided by an embodiment of the present invention, and the conference coordination method based on an intelligent speaker comprises the following steps:

[0059] 201. Obtain audio data during the conference.

[0060] In an embodiment of the present invention, the above-mentioned smart speaker-based conference coordination method can be applied to the above-mentioned smart speaker-based conference coordination system. The above-mentioned smart speaker-based conference coordination system has functions such as audio data processing, audio data transmission and reception, and audio data storage, and can be constructed based on a server or a server cluster. The above-mentioned server or server cluster can be an electronic device with audio recognition and processing capabilities.

[0061] The above audio data may include but is not limited to environmental audio data and human voice audio data. Generally speaking, the above audio data can be obtained by collecting all-round sound through a microphone installed in the smart speaker. The above environmental audio data may include but is not limited to noise data in the environment, natural audio data in the environment, and interactive audio data of people in the environment.

[0062] More specifically, since various natural audio data in nature will affect the human voice audio data in the meeting, it is necessary to collect and identify the natural audio data, and perform denoising or clearing processing. The above-mentioned noise data can be the pickup noise caused by signal interference between electronic devices used in the meeting, which will also affect the human voice audio data in the meeting. The above-mentioned interactive audio data of the characters can be the sound generated by the interaction between characters, or between characters and objects. For example, when a character uses a keyboard to record, the sound data generated by the interaction between the character and the keyboard can also be attributed to the interactive audio data of the characters in the environment.

[0063] The above-mentioned human voice audio data can be language audio data that can be recognized by a preset AI big model to derive specific content and meaning. It is understandable that the human voice audio data can be audio data of human text language. Therefore, the human voice audio data can also include but is not limited to the language and text pronunciation of different races and countries. Generally speaking, audio data that can be converted from speech recognition to text through a preset AI big model and has substantial meaning after conversion can be used as the above-mentioned human voice audio data.

[0064] 202. Process the environmental audio data and human voice audio data based on the preset AI big model, and at least determine the key data of the meeting during the meeting.

[0065] In an embodiment of the present invention, the above-mentioned preset AI big model may be a deep learning model that can recognize and process the above-mentioned audio data, so as to determine the key data in the meeting process through the audio data. The model can also transcribe the above-mentioned audio data to complete the transformation of audio data into text data, that is, the text data corresponding to the content of the audio data can be obtained by transcribing the audio data into text. Generally speaking, the above-mentioned preset AI big model may include but is not limited to the GPT (Generative Pre-trained Transformer) series, BERT (Bidirectional Encoder Representations from Transformers) series, and Wenxin Yiyan and other big models. The above-mentioned preset AI big model can also provide functions such as speech recognition, natural language processing, and meeting minutes generation for the content of the meeting.

[0066] The above-mentioned meeting key data can be the meeting content corresponding to the moment with substantial characteristics during the meeting. Specifically, the above-mentioned meeting key data can be the moment in the meeting that has the guidance or substantial characteristics of the meeting, or it can be the meeting content corresponding to that moment. For example, it can be the meeting content that caused the meeting participants to discuss, or it can be the moment when the most complex audio data is collected. In this embodiment, the above-mentioned meeting key data can be obtained by performing intelligent recognition processing on the above-mentioned audio data through the above-mentioned preset AI large model.

[0067] More specifically, the above-mentioned meeting key data can be data used to summarize or refine the meeting, obtain meeting abstracts or meeting minutes and other documents, so as to facilitate the meeting participants to summarize the meeting content. It is understandable that the above-mentioned meeting key data can be obtained through prompts or emphasis by leaders during the meeting, or through the preset AI large model attached to the above-mentioned smart audio-based conference coordination system to recognize and process the audio data during the meeting, extract information about summarizing or refining the meeting content, and generate text data based on the information, which is stored in the database corresponding to the current meeting in the above-mentioned smart audio-based conference coordination system.

[0068] 203. Perform AI-assisted processing on at least the key meeting data.

[0069] The above-mentioned AI-assisted processing can determine the requirements of the above-mentioned key meeting data according to the preset AI large model attached to the above-mentioned smart audio-based conference coordination system, and use the corresponding functions to complete the processing of the key meeting data according to the corresponding requirements. Specifically, when the above-mentioned key meeting data is analyzed as an issue that people are discussing, a meeting text file corresponding to the issue can be generated, and key points can be marked to facilitate determination after subsequent meetings. The functions corresponding to the above-mentioned AI-assisted processing may include but are not limited to any functions that can meet the needs of complex meeting scenarios, such as conference audio file generation, conference text file generation, conference summary generation and email sending.

[0070] In a possible embodiment, the above-mentioned intelligent audio-based conference coordination system can intelligently identify the audio data during the meeting, and intelligently process the identified key meeting data through a preset AI big model to meet the needs of AI-assisted processing of key meeting data under complex meeting scenarios, thereby improving the efficiency and convenience of information transmission in the meeting, and being able to select AI-assisted functions for flexible processing of key meeting data in the meeting, thereby improving the efficiency and quality of the meeting. It should be noted that the above-mentioned AI-assisted processing can select different AI big models to change the corresponding functions according to the specific implementation plan, and the AI ​​big models can cooperate through multiple models to complete the collective realization of a certain function.

[0071] In an embodiment of the present invention, audio data is obtained during a meeting, and the audio data may include but is not limited to environmental audio data and human voice audio data; the environmental audio data and human voice audio data are processed based on a preset AI big model, and key meeting data are determined during the meeting; AI-assisted processing is performed on the key meeting data. By intelligently identifying the audio data during the meeting and intelligently processing the identified key meeting data through a preset AI big model, the need for AI-assisted processing of key meeting data under complex meeting scenarios is met, the efficiency and convenience of information transmission in the meeting is improved, and the AI-assisted function can be selected for flexible processing of key meeting data in the meeting, thereby improving the efficiency and quality of the meeting.

[0072] Optional, such as Figure 3 As shown, in step 201, the step of obtaining audio data during the conference also includes 2011-2012, wherein:

[0073] 2011. Preprocess the audio data acquired during the conference by using a preset microphone array to obtain target audio data.

[0074] In an embodiment of the present invention, the preset microphone array may be a type composed of multiple microphones, and a microphone system capable of sound source localization and signal enhancement may be formed by selecting a specific geometric shape or a specific microphone type. Specifically, the preset microphone array may include but is not limited to a linear microphone array, a spherical microphone array, a directional microphone array, and a circular microphone array, etc. In the preset microphone array, microphone arrays of different geometric shapes may be selected for conference pickup by determining a specific conference scene. More specifically, in the preset microphone array, the type of microphone may be selected to cooperate with different preset microphone arrays to enhance the collection effect of audio data. The types of microphones also include electronic microphones and tiny microphones.

[0075] By selecting different preset microphone arrays and different microphone types, the audio data during the meeting can be accurately collected, and the above preset microphone arrays and microphone types can be matched to complete functions such as accurate speech recognition, noise suppression, far-field pickup, sound source localization and sound field reconstruction.

[0076] The above-mentioned preprocessing may include but is not limited to performing operations such as denoising and audio enhancement on the collected audio data. It is understandable that in the process of collecting audio data through the above-mentioned preset microphone array, the above-mentioned intelligent audio-based conference coordination system can also preprocess the collected audio data through the preset microphone array so as to obtain clear audio data, and use the clear audio data as the above-mentioned target audio data.

[0077] 2012. Based on the preset AI big model, the target audio data is feature classified and processed to obtain environmental audio data and human voice audio data.

[0078] The above-mentioned feature classification processing can perform feature recognition on the above-mentioned target audio data through the above-mentioned preset AI large model, and extract the recognized features, and distinguish audio data with different features according to the extraction results. Specifically, the above-mentioned features may include but are not limited to human voice features and environmental sound features, wherein the above-mentioned preset AI large model can recognize the human voice features in the above-mentioned collected target audio data through built-in AI. It can be understood that the human voice features in the above-mentioned target audio data can be recognized through models such as AI semantic language models and AI semantic recognition models that can recognize human voice features, and the human voice audio data can be extracted accordingly. It is also possible to exclude the human voice audio data, exclude the audio data such as noise signals, and use the remaining target audio data as environmental audio data.

[0079] It should be noted that the above-mentioned environmental audio data may include but is not limited to noise data in the environment, natural audio data in the environment, and interactive audio data of characters in the environment.

[0080] More specifically, since various natural audio data in nature will affect the human voice audio data in the meeting, it is necessary to collect and identify the natural audio data, and perform denoising or clearing processing. The above-mentioned noise data can be the pickup noise caused by signal interference between electronic devices used in the meeting, which will also affect the human voice audio data in the meeting. The above-mentioned interactive audio data of the characters can be the sound generated by the interaction between characters, or between characters and objects. For example, when a character uses a keyboard to record, the sound data generated by the interaction between the character and the keyboard can also be attributed to the interactive audio data of the characters in the environment.

[0081] The above-mentioned human voice audio data can be language audio data that can be recognized by a preset AI big model to derive specific content and meaning. It is understandable that the human voice audio data can be audio data of human text language. Therefore, the human voice audio data can also include but is not limited to the language and text pronunciation of different races and countries. Generally speaking, audio data that can be converted from speech recognition to text through a preset AI big model and has substantial meaning after conversion can be used as the above-mentioned human voice audio data.

[0082] Optional, such as Figure 4 As shown, in step 202, the human voice audio data is processed based on the preset AI big model, and at least the step of determining the key meeting data during the meeting also includes 2021-2023, wherein:

[0083] 2021. Use the preset AI big model to perform timbre feature recognition and processing on human voice audio data to obtain the target human voice timbre feature data.

[0084] In an embodiment of the present invention, the above-mentioned timbre feature recognition processing can perform timbre feature recognition on the above-mentioned human voice audio data through the human voice timbre model in the above-mentioned preset AI large model, thereby determining the target human voice timbre feature data of the current person. Specifically, the above-mentioned timbre features may include but are not limited to speaking speed, pitch, pitch change rate, timbre sharpness, and speaking habits and other speaking characteristics, and the above-mentioned speaking habits may include but are not limited to stuttering, intermittent speaking, and the habit of mixing modal particles.

[0085] The above-mentioned target human voice timbre feature data may include but is not limited to the timbre characteristics of the corresponding person obtained after the timbre feature recognition processing of the above-mentioned human voice audio data is performed by the emotion model in the above-mentioned preset AI large model.

[0086] 2022. Determine the emotional characteristics of the corresponding person based on the target person's voice and tone feature data.

[0087] The above-mentioned emotional characteristics may include but are not limited to excitement, tension, confusion, boredom, etc., which can express the emotions of the meeting participants regarding their understanding of the meeting content during the meeting. Specifically, during the meeting, understanding the emotional characteristics of the meeting participants is of great significance for improving communication efficiency, enhancing teamwork and resolving conflicts. Among them, the above-mentioned target human voice and color feature data can be analyzed by the emotional model in the above-mentioned preset AI large model to obtain the analysis results. Generally speaking, the above-mentioned target human voice and color feature data can be analyzed and processed according to the natural language processing and deep learning modules in the emotional model to obtain the emotional characteristics of the above-mentioned corresponding persons.

[0088] The corresponding person can also be determined by acquiring the continuous duration of the person's voiceprint and sound source coordinates. Specifically, the voiceprint data of the corresponding person and the continuous duration of the sound source coordinates are acquired, and in the process of comparing the continuous duration of the sound source coordinates with the preset continuous duration, the current voiceprint is continuously matched to determine whether the current voiceprint data matches the current sound source coordinates. If the continuous duration of the sound source coordinates is greater than or equal to the preset continuous duration when the voiceprint data matches the sound source coordinates, the person of the current voiceprint data is determined to be the corresponding person. If the continuous duration of the sound source coordinates is less than the preset continuous duration when the voiceprint data matches the sound source coordinates, the person of the current voiceprint data is determined to be a non-corresponding person, that is, the person currently speaking has been changed. In another possible embodiment, when the current voiceprint is matched and it is determined that the current voiceprint data matches the current sound source coordinates, if the current voiceprint data does not match the collected voiceprint data, the matching operation of the current voiceprint data and the sound source coordinates is suspended, and the newly collected voiceprint data is matched with the newly collected sound source coordinates.

[0089] 2023. Based on emotional characteristics, key meeting data is determined during the meeting.

[0090] In a possible embodiment, the above-mentioned smart speaker-based conference coordination system can determine the degree of understanding of the meeting content by the meeting participants during the meeting by analyzing the emotional characteristics of the above-mentioned corresponding personnel, and thereby determine the key meeting data during the meeting.

[0091] Optional, such as Figure 5 As shown, in step 202, the human voice audio data is processed based on the preset AI big model, and at least the step of determining the key data of the meeting during the meeting also includes 2024-2026, wherein:

[0092] 2024. Perform environmental timbre content recognition and processing on environmental audio data through a preset AI large model to obtain target environmental timbre feature data.

[0093] In an embodiment of the present invention, the above-mentioned environmental sound content recognition process can perform environmental sound feature recognition on the above-mentioned environmental audio data through the environmental sound recognition model in the above-mentioned preset AI large model, thereby determining the target environmental sound feature data in the current meeting. Specifically, the above-mentioned environmental sound features may include but are not limited to the sound features emitted by the interaction with environmental objects during the meeting, such as typing on the keyboard and taking notes.

[0094] The above-mentioned target environment sound feature data may include but is not limited to the sound characteristics emitted by the corresponding personnel interacting with the environmental objects during the meeting after the environmental interaction model in the above-mentioned preset AI large model performs environmental sound content recognition processing on the above-mentioned target environment sound content feature data.

[0095] 2025. Determine the meeting actions of the corresponding personnel based on the target environment sound content feature data.

[0096] The above-mentioned meeting action can be to identify and process the above-mentioned target environment sound content feature data through the above-mentioned environment interaction model, so as to judge the current action of the corresponding person through the environment sound. Specifically, since certain special sounds are always emitted during the interaction between the human body and the environment, the above-mentioned special sounds can be captured by the above-mentioned preset microphone array, and the above-mentioned special sounds can be identified and processed by the environment interaction model in the preset AI large model, so as to judge the current action of the corresponding person. For example, the sound of typing on the keyboard indicates that the corresponding person is using the keyboard and may be recording, or the sound of PPT switching indicates that the meeting page may be turned, etc.

[0097] 2026. Based on the meeting actions, key meeting data is determined during the meeting.

[0098] In a possible embodiment, the above-mentioned smart audio-based conference coordination system can determine the degree of importance that the conference participants attach to the meeting content during the meeting by analyzing the meeting actions of the above-mentioned corresponding personnel. For example, when the sound of typing is recognized, it means that the corresponding personnel attaches great importance to the content of the current meeting or has doubts, etc., and needs to record it. Therefore, the recorded meeting content can be used as the key meeting data during the meeting.

[0099] In another possible embodiment, the above-mentioned intelligent audio-based conference coordination system can also simultaneously analyze and process the emotional characteristics and conference actions of the above-mentioned corresponding personnel, and combine the emotional characteristics and conference actions to determine more specific meeting focus data. For example, when the corresponding personnel is in an excited state and is typing on the keyboard to record the content of the meeting, it means that the corresponding personnel attaches great importance to the current meeting content, so it can be used as the meeting focus data. It can also be that the corresponding personnel is in a state of doubt, and when it is recognized that they are in a daze state of tapping the table with a pen, it means that the corresponding personnel do not understand the content of the current meeting very well. At this time, the above-mentioned intelligent audio-based conference coordination system will record the content of the current meeting and send it to the corresponding personnel, so that the corresponding personnel can conduct post-meeting analysis.

[0100] Optional, such as Figure 6As shown, in step 203, the step of performing AI-assisted processing on at least the key conference data also includes 2031-2032, wherein:

[0101] 2031. Analyze the attributes of the conference key data to determine the conference demand type corresponding to the conference key data.

[0102] In an embodiment of the present invention, the above-mentioned attribute analysis may be to analyze the attributes such as the theme, content and direction of the meeting key data, so as to determine the main direction and demand of the current meeting. Specifically, the meeting key data may be a section of content during the meeting, which may include a discussion or understanding of the meeting content. Therefore, the meeting demand type corresponding to the meeting key data can be obtained by summarizing and refining the attributes of the above-mentioned meeting key data.

[0103] More specifically, the natural language processing model in the preset AI big model can be used to analyze and process the above-mentioned key meeting data, and the final meeting demand type can be determined through voice sentiment analysis and text content analysis.

[0104] 2032. Based on the type of meeting requirements, use the corresponding preset AI assistance function to perform AI-assisted processing on the corresponding meeting key data.

[0105] The above conference requirement types may include, but are not limited to, conference audio file generation, conference text file generation, conference summary generation, and email sending, etc., functional requirements determined based on attribute analysis of key conference data.

[0106] In a possible embodiment, after performing attribute analysis on the above-mentioned conference key data, the above-mentioned smart speaker-based conference coordination system determines the functions that may be required corresponding to the current conference key data according to the conference demand type obtained by the analysis, and performs AI-assisted processing on the current conference key data through the AI-assisted module of the corresponding function.

[0107] Optional, such as Figure 7 As shown, in step 2031, the step of performing attribute analysis on the conference key data to determine the conference demand type corresponding to the conference key data also includes steps 20311-20313, wherein:

[0108] 20311. Based on the preset AI big model, the collected audio data corresponding to the key meeting data are transcribed into text to obtain the text data corresponding to the key meeting data.

[0109] In an embodiment of the present invention, the above-mentioned intelligent audio-based conference coordination system performs text transcription processing on the collected audio data through a text transcription model to obtain corresponding text data, and corrects the content of the text data to ensure that the transcribed text data has no semantic or sentence errors. It is understandable that the above-mentioned text transcription processing can also perform audio transcription processing under different voices and different semantics on the collected audio data through the above-mentioned text transcription model.

[0110] 20312. By using the preset sentiment model, the direction of the text data content is analyzed to determine the direction of the meeting needs.

[0111] In a possible embodiment, the above-mentioned smart speaker-based conference coordination system performs sentiment analysis on the content of the above-mentioned text data through the above-mentioned preset sentiment model, determines the sentiment score corresponding to the text data, and determines whether the sentiment of the current text data is positive or negative based on the sentiment score, and determines the direction of the meeting needs based on the positive and negative sentiments.

[0112] For example, when the content demand for a company's listing in the text data is processed by the above preset sentiment model and the sentiment of the text data is positive, it means that most people in the meeting agree with the company's listing, so it is judged that the direction of the meeting demand is the company's listing.

[0113] 20313. Determine the type of meeting demand based on the direction of the meeting demand.

[0114] In a possible embodiment, the above-mentioned smart speaker-based conference coordination system determines the conference demand direction of the current conference after analyzing the content of the text data through the above-mentioned preset emotion model, and determines the corresponding conference demand type according to the conference demand direction. For example, when the conference demand direction is the company's listing, the corresponding conference demand type can be to organize the company's listing materials and send them to the corresponding personnel.

[0115] like Figure 8 As shown, the embodiment of the present invention further provides a conference coordination device 800 based on an intelligent speaker, and the conference coordination device based on an intelligent speaker includes:

[0116] An acquisition module 801 is used to acquire audio data during a conference, wherein the audio data includes environmental audio data and human voice audio data;

[0117] A determination module 802 is used to process the environmental audio data and the human voice audio data based on a preset AI big model, and determine the key meeting data during the meeting;

[0118] The auxiliary module 803 is used to perform AI-assisted processing on the key conference data.

[0119] Optionally, the above device further comprises:

[0120] A microphone module is used to pre-process the audio data acquired during the conference by using a preset microphone array to obtain target audio data;

[0121] A processing module is used to perform feature classification processing on the target audio data based on a preset AI large model to obtain the environmental audio data and the human voice audio data.

[0122] Optionally, the determination module 802 includes:

[0123] The first acquisition submodule is used to perform timbre feature recognition processing on the human voice audio data through a preset AI large model to obtain target human voice timbre feature data;

[0124] A first determination submodule, configured to determine the emotional characteristics of a corresponding person based on the target person's voice and tone characteristic data;

[0125] The second determination submodule is used to determine the key meeting data during the meeting based on the emotional characteristics.

[0126] Optionally, the determining module 802 further includes:

[0127] The second acquisition submodule is used to perform environmental timbre content recognition processing on the environmental audio data through a preset AI large model to obtain target environmental timbre feature data;

[0128] A third determination submodule is used to determine the meeting action of the corresponding person based on the target environment timbre content feature data;

[0129] The fourth determination submodule is used to determine conference key data during the conference based on the conference action.

[0130] Optionally, the auxiliary module 803 includes:

[0131] A fifth determination submodule is used to perform attribute analysis on the conference key data to determine the conference requirement type corresponding to the conference key data;

[0132] The processing submodule is used to perform AI-assisted processing on the corresponding conference key data based on the conference demand type using the corresponding preset AI-assisted function.

[0133] Optionally, the fifth determining submodule includes:

[0134] A transcription unit, used to perform text transcription processing on the collected audio data corresponding to the key meeting data based on a preset AI large model to obtain text data corresponding to the key meeting data;

[0135] A first determining unit is used to perform direction analysis on the content of the text data by using a preset sentiment model to determine the direction of the conference demand;

[0136] The second determining unit is used to determine the conference requirement type based on the conference requirement direction.

[0137] like Fig. 9 As shown, an embodiment of the present invention further provides an electronic device 900, including a processor, and the processor can execute any one of the above-mentioned conference coordination methods based on smart speakers.

[0138] Specifically, it includes a processor 901 and a memory 902, and a computer program for executing a conference coordination method based on an intelligent speaker, which is stored in the memory 902 and can be run on the processor 901, wherein:

[0139] The processor 901 runs the computer program of the conference coordination method based on the intelligent speaker stored in the memory 902, and performs the following steps:

[0140] Acquire audio data during the conference, wherein the audio data includes environmental audio data and human voice audio data;

[0141] Processing the environmental audio data and the human voice audio data based on a preset AI big model, and determining at least the key meeting data during the meeting;

[0142] At least AI-assisted processing is performed on the key meeting data.

[0143] Optionally, the processor 901 executes the step of acquiring audio data during the conference, and the method further includes:

[0144] Preprocessing the audio data acquired during the conference by using a preset microphone array to obtain target audio data;

[0145] The target audio data is subjected to feature classification processing based on a preset AI large model to obtain the environmental audio data and the human voice audio data.

[0146] Optionally, the processor 901 executes the processing of the human voice audio data based on the preset AI big model to at least determine key meeting data during the meeting, including:

[0147] Performing timbre feature recognition processing on the human voice audio data through a preset AI large model to obtain target human voice timbre feature data;

[0148] Determining the emotional characteristics of the corresponding person based on the target human voice and tone characteristic data;

[0149] Based on the emotional characteristics, key meeting data is determined during the meeting.

[0150] Optionally, the processor 901 executes the processing of the environmental audio data based on the preset AI big model to at least determine key meeting data during the meeting, including:

[0151] Performing environmental timbre content recognition processing on the environmental audio data through a preset AI large model to obtain target environmental timbre feature data;

[0152] Determining the meeting action of the corresponding person based on the target environment timbre content characteristic data;

[0153] Based on the conference actions, conference key data is determined during the conference.

[0154] Optionally, the processor 901 performs the AI-assisted processing at least for the conference key data, including:

[0155] Performing attribute analysis on the key conference data to determine the conference demand type corresponding to the key conference data;

[0156] Based on the type of meeting demand, the corresponding preset AI assistance function is used to perform AI-assisted processing on the corresponding meeting key data.

[0157] Optionally, the processor 901 further performs attribute analysis on the conference key data to determine the conference requirement type corresponding to the conference key data, including:

[0158] Based on the preset AI big model, the audio data corresponding to the collected meeting key data is transcribed to obtain the text data corresponding to the meeting key data;

[0159] By using a preset sentiment model, a direction analysis is performed on the content of the text data to determine the direction of the meeting demand;

[0160] Based on the conference demand direction, the conference demand type is determined.

[0161] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the conference coordination method based on smart speakers or the conference coordination method based on smart speakers on the application side provided in the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0162] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0163] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A conference coordination method based on intelligent audio, characterized in that: include: Acquire audio data during the conference, wherein the audio data includes environmental audio data and human voice audio data; Processing the environmental audio data and the human voice audio data based on a preset AI big model, and determining at least the key meeting data during the meeting; Perform AI-assisted processing on at least the key meeting data; The AI-assisted processing of at least the key conference data includes: Performing attribute analysis on the key conference data to determine the conference demand type corresponding to the key conference data; Based on the type of meeting requirements, use the corresponding preset AI assistance function to perform AI-assisted processing on the corresponding meeting key data; The performing of attribute analysis on the key conference data to determine the conference requirement type corresponding to the key conference data includes: Based on the preset AI big model, the audio data corresponding to the collected meeting key data is transcribed to obtain the text data corresponding to the meeting key data; By using a preset sentiment model, a direction analysis is performed on the content of the text data to determine the direction of the meeting demand; Based on the conference demand direction, the conference demand type is determined.

2. The conference coordination method based on intelligent audio according to claim 1, characterized in that: The method for obtaining audio data during the conference further includes: Preprocessing the audio data acquired during the conference by using a preset microphone array to obtain target audio data; The target audio data is subjected to feature classification processing based on a preset AI large model to obtain the environmental audio data and the human voice audio data.

3. The conference coordination method based on intelligent audio according to claim 1, characterized in that: The processing of the human voice audio data based on the preset AI big model to at least determine the key meeting data during the meeting includes: Performing timbre feature recognition processing on the human voice audio data through a preset AI large model to obtain target human voice timbre feature data; Determining the emotional characteristics of the corresponding person based on the target human voice and tone characteristic data; Based on the emotional characteristics, key meeting data is determined during the meeting.

4. The conference coordination method based on intelligent audio according to claim 1, characterized in that: The processing of the environmental audio data based on the preset AI big model to at least determine the key meeting data during the meeting includes: Performing environmental timbre content recognition processing on the environmental audio data through a preset AI large model to obtain target environmental timbre feature data; Determining the meeting action of the corresponding person based on the target environment timbre characteristic data; Based on the conference actions, conference key data is determined during the conference.

5. A conference coordination device based on intelligent audio, characterized in that: include: An acquisition module, used to acquire audio data during a conference, wherein the audio data includes environmental audio data and human voice audio data; A determination module, used to process the environmental audio data and the human voice audio data based on a preset AI big model, and determine the key meeting data during the meeting; An auxiliary module, used for performing AI-assisted processing on the key meeting data; The auxiliary module is further used to perform attribute analysis on the key conference data to determine the conference requirement type corresponding to the key conference data; based on the conference requirement type, use the corresponding preset AI auxiliary function to perform AI-assisted processing on the corresponding key conference data; The auxiliary module is also used to perform text transcription processing on the audio data corresponding to the collected meeting key data based on the preset AI large model to obtain the text data corresponding to the meeting key data; perform direction analysis on the content of the text data through the preset emotional model to determine the direction of the meeting demand; and determine the type of meeting demand based on the direction of the meeting demand.

6. A conference coordination system based on intelligent audio, characterized in that: The conference coordination system based on smart speakers includes: a conference coordination device based on smart speakers; The intelligent audio-based conference coordination device is determined by an intelligent audio-based conference coordination method as shown in any one of claims 1-4.

7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the conference coordination method based on smart speakers as described in any one of claims 1 to 4 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the conference coordination method based on smart speakers as described in any one of claims 1 to 4 are implemented.

Images