Communication assistance method, device, equipment and medium based on online conference

Abstract

The present invention relates to the field of artificial intelligence and provides a communication assistance method, apparatus, device, and medium for online conferencing. These methods first maximize the smoothness and security of voice recognition-to-text conversion using a local model, further filter out audio that is less accurately recognized or has a low recognition rate for secondary recognition to ensure better voice recognition results and reliable backup. The methods then concatenate the first and second text sequences to obtain a target text, which is then displayed in real time to facilitate assisted communication in online conferencing using artificial intelligence. Furthermore, the present invention relates to blockchain technology, and the local and target voice recognition models can be stored in blockchain nodes.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a communication assistance method, device, equipment and medium based on online conferences. Background Art

[0002] Many service outlets (such as bank service outlets) have multiple large TV screens in their negotiation rooms, allowing outlet staff (such as financial managers) to call other people (such as remote investment advisors) to provide customers with more professional services (such as investment services) and conduct multi-party online audio and video calls.

[0003] However, in current online meetings, there are usually the following disadvantages:

[0004] 1. Conventional audio and video conferencing does not provide real-time audio-to-text conversion, which can lead to a poor experience for users with hearing impairments or when the recipient is unable to turn up the volume.

[0005] 2. Conventional use of public network voice clouds for real-time speech-to-text conversion requires that the voice stream be uploaded to a public network server in real time, parsed into text, and then transmitted back to the local server. This requires frequent interaction with third-party network requests, which is not only prone to lag but also poses a certain risk of leakage. Frequent network requests can also easily cause delays in subtitle display. Summary of the Invention

[0006] In view of the above content, it is necessary to provide a communication assistance method, device, equipment and medium based on online meetings, which can first maximize the conversion of speech recognition into text through local models, with fluency and security, and further filter out audio that is not accurately recognized or has a low recognition rate for secondary recognition to ensure better speech recognition results and a reliable guarantee, so as to achieve assisted communication in online meetings based on artificial intelligence.

[0007] A communication assistance method based on an online conference, the communication assistance method based on an online conference comprising:

[0008] In response to the communication assistance instruction, determining a target voice acquisition device according to the communication assistance instruction, and starting the target voice acquisition device to acquire voice;

[0009] Preprocessing the collected speech to obtain a first audio to be recognized;

[0010] Inputting the first audio to be recognized into a local speech recognition model to obtain a local recognition result;

[0011] Obtaining a matching degree and a recognition rate from the local recognition result;

[0012] Splitting the first audio to be recognized into recognized audio and second audio to be recognized according to the matching degree and the recognition rate, and obtaining a text recognition result corresponding to the recognized audio from the local recognition result as a first text sequence;

[0013] Connecting to a target server, inputting the second audio to be recognized into a target speech recognition model in the target server, and obtaining a text recognition result from an output of the target speech recognition model as a second text sequence;

[0014] concatenating the first character sequence and the second character sequence to obtain a target text;

[0015] The target text is displayed in real time.

[0016] According to a preferred embodiment of the present invention, determining the target voice acquisition device according to the communication assistance instruction includes:

[0017] Parsing the method body of the communication assistance instruction to obtain information carried by the communication assistance instruction;

[0018] Get the preset label;

[0019] Constructing a regular expression according to the preset tag;

[0020] Using the regular expression to traverse the information carried in the communication assistance instruction, and determining the traversed data as the conference identifier;

[0021] Establishing a connection with a target conference room corresponding to the conference identifier, and obtaining a voice acquisition device of the target conference room as the target voice acquisition device; and / or

[0022] When it is detected that a terminal device is remotely connected to the target conference room, the voice collection device of the terminal device is determined as the target voice collection device.

[0023] According to a preferred embodiment of the present invention, splitting the first audio to be recognized into recognized audio and second audio to be recognized according to the matching degree and the recognition rate includes:

[0024] Acquire, from the first audio to be recognized, the audio corresponding to the text recognition result whose matching degree is less than or equal to the configured matching degree as the first audio;

[0025] Acquire, from the first audio to be recognized, an audio whose recognition rate is less than or equal to the configured recognition rate as the second audio;

[0026] Integrating the first audio and the second audio to obtain the second audio to be recognized;

[0027] The second audio to be recognized is deleted from the first audio to be recognized to obtain the recognized audio.

[0028] According to a preferred embodiment of the present invention, the method further includes:

[0029] Regularly detecting the resource occupancy rate of the target server;

[0030] When it is detected that the resource occupancy rate of the target server is less than or equal to the configured resource occupancy rate, acquiring new data;

[0031] Performing word frequency sampling on the newly added data to obtain a sampling result;

[0032] Get historical data;

[0033] When the similarity between the sampling result and the historical data is less than or equal to the configuration similarity, the target speech recognition model is optimized and trained using the newly added data.

[0034] According to a preferred embodiment of the present invention, the method further includes:

[0035] When the update condition is met, connecting to the target server and obtaining all speech recognition models stored in the target server;

[0036] Obtaining update times of all the speech recognition models, and identifying the latest speech recognition model according to the update times;

[0037] Converting the identified speech recognition model using the WebAssembly mode, and determining the converted model as the local speech recognition model;

[0038] Deploy the local speech recognition model.

[0039] According to a preferred embodiment of the present invention, the step of concatenating the first character sequence and the second character sequence to obtain the target text includes:

[0040] Obtaining a first word order of the first word sequence, and obtaining a second word order of the second word sequence;

[0041] The first character sequence and the second character sequence are concatenated according to the first word order and the second word order to obtain the target text.

[0042] According to a preferred embodiment of the present invention, the real-time display of the target text includes:

[0043] Obtaining a text format corresponding to the display of the target conference room, and / or obtaining a text format corresponding to the display of the terminal device;

[0044] Converting the target text into a text format corresponding to the display of the target conference room, and / or converting the target text into a text format corresponding to the display of the terminal device;

[0045] The converted target text is displayed on a corresponding display in real time.

[0046] A communication assistance device based on an online conference, the communication assistance device based on an online conference comprising:

[0047] a collection unit, configured to respond to a communication assistance instruction, determine a target voice collection device according to the communication assistance instruction, and start the target voice collection device to collect voice;

[0048] A preprocessing unit, configured to preprocess the collected speech to obtain a first audio to be recognized;

[0049] An input unit, configured to input the first audio to be recognized into a local speech recognition model to obtain a local recognition result;

[0050] An acquisition unit, configured to acquire a matching degree and a recognition rate from the local recognition result;

[0051] an acquisition unit, configured to split the first audio to be recognized into recognized audio and second audio to be recognized according to the matching degree and the recognition rate, and acquire a text recognition result corresponding to the recognized audio from the local recognition result as a first text sequence;

[0052] The input unit is further configured to connect to a target server, input the second audio to be recognized into a target speech recognition model in the target server, and obtain a text recognition result from an output of the target speech recognition model as a second text sequence;

[0053] a concatenation unit, configured to concatenate the first character sequence and the second character sequence to obtain a target text;

[0054] A display unit is used to display the target text in real time.

[0055] An electronic device, comprising:

[0056] a memory storing at least one instruction; and

[0057] The processor executes the instructions stored in the memory to implement the communication assistance method based on the online conference.

[0058] A computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the communication assistance method based on online conference.

[0059] It can be seen from the above technical solution that the present invention can respond to a communication assistance instruction, determine the target voice acquisition device according to the communication assistance instruction, and start the target voice acquisition device to collect voice, pre-process the collected voice to obtain a first audio to be recognized, input the first audio to be recognized into a local voice recognition model, obtain a local recognition result, first maximize the voice recognition to text conversion through the local model, with fluency and security, obtain the matching degree and recognition rate from the local recognition result, split the first audio to be recognized into recognized audio and second audio to be recognized according to the matching degree and the recognition rate, and obtain the text recognition result corresponding to the recognized audio from the local recognition result as a first text sequence, connect to the target server, input the second audio to be recognized into the target voice recognition model in the target server, and obtain the text recognition result from the output of the target voice recognition model as a second text sequence, and then filter out the audio that is not recognized accurately or has a low recognition rate for secondary recognition to ensure better voice recognition effect and a bottom-line reliability, splice the first text sequence and the second text sequence to obtain the target text, and display the target text in real time, so as to realize assisted communication for online meetings based on artificial intelligence means. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flow chart of a preferred embodiment of the communication assistance method based on online conference of the present invention.

[0061] Figure 2 It is a functional module diagram of a preferred embodiment of the communication assistance device based on online conference of the present invention.

[0062] Figure 3 It is a structural diagram of an electronic device of a preferred embodiment of the present invention for implementing a communication assistance method based on an online conference. DETAILED DESCRIPTION

[0063] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0064] like Figure 1 FIG. 1 is a flow chart of a preferred embodiment of the communication assistance method based on online conference of the present invention. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

[0065] The online conference-based communication assistance method is applied to one or more electronic devices, which are devices that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Their hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0066] The electronic device may be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0067] The electronic device may further include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0068] The network where the electronic device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0069] S10 , in response to a communication assistance instruction, determining a target voice acquisition device according to the communication assistance instruction, and starting the target voice acquisition device to acquire voice.

[0070] In at least one embodiment of the present invention, the communication assistance instruction may be triggered by a branch financial manager, investment consultant or other related staff, and the present invention is not limited thereto.

[0071] In at least one embodiment of the present invention, determining the target voice collection device according to the communication assistance instruction includes:

[0072] Parsing the method body of the communication assistance instruction to obtain information carried by the communication assistance instruction;

[0073] Get the preset label;

[0074] Constructing a regular expression according to the preset tag;

[0075] Using the regular expression to traverse the information carried in the communication assistance instruction, and determining the traversed data as the conference identifier;

[0076] Establishing a connection with a target conference room corresponding to the conference identifier, and obtaining a voice acquisition device of the target conference room as the target voice acquisition device; and / or

[0077] When it is detected that a terminal device is remotely connected to the target conference room, the voice collection device of the terminal device is determined as the target voice collection device.

[0078] Specifically, the communication assistance instruction is essentially a code. In the communication assistance instruction, according to the code writing principle, the content between {} is called the method body.

[0079] The preset tag can be customized and has a one-to-one correspondence with the address. For example, the preset tag can be ID, and the regular expression ID() is further established with the preset tag, and traversal is performed with ID().

[0080] Through the above implementation, the target voice collection device can be quickly determined based on the regular expression and the preset label, so that real-time voice collection can be performed using the target voice collection device.

[0081] For example, when a customer manager holds an online meeting with a customer, the target voice acquisition device may be the voice acquisition device on the customer manager's and the customer's terminal devices respectively. At this time, if the customer manager is holding a meeting in a conference room, the target voice acquisition device may also include the voice acquisition device in the conference room.

[0082] S11, pre-processing the collected speech to obtain a first audio to be recognized.

[0083] It is understandable that since the collected speech may contain noise, the collected speech must first be preprocessed to improve the efficiency and accuracy of recognition.

[0084] Specifically, the pre-processing may include, but is not limited to: pre-emphasis and windowing and framing.

[0085] S12: Input the first audio to be recognized into a local speech recognition model to obtain a local recognition result.

[0086] The local speech recognition model is a recognition model deployed on the client. When the local recognition model is used for speech recognition, there is no need for frequent network connections, thus effectively avoiding delays in speech recognition.

[0087] In addition, the local speech recognition model is deployed on the local client and does not need to be connected to an external server or network, which avoids the occurrence of lag. This not only reduces the cost of speech recognition, but also effectively avoids information leakage and improves the security of the speech recognition process.

[0088] Through the above implementation, speech recognition to text conversion can be performed to the maximum extent through the local model, with fluency and security.

[0089] S13, obtaining a matching degree and a recognition rate from the local recognition result.

[0090] Specifically, the local recognition result may include: text obtained after speech recognition (ie, text recognition result), the matching degree between the text obtained after speech recognition and the original speech, and the recognition rate of speech recognition.

[0091] S14: Split the first audio to be recognized into recognized audio and second audio to be recognized according to the matching degree and the recognition rate, and obtain a text recognition result corresponding to the recognized audio from the local recognition result as a first text sequence.

[0092] It is understandable that since the local speech recognition model is deployed on the local client and is not convenient for real-time updating, the recognition effect may be subject to certain limitations. In order to ensure the comprehensiveness and accuracy of speech recognition, it is necessary to further split the first audio to be recognized according to the matching degree and recognition rate of the local recognition results to screen out the audio that requires secondary speech recognition.

[0093] Specifically, the splitting of the first audio to be recognized into recognized audio and second audio to be recognized according to the matching degree and the recognition rate includes:

[0094] Acquire, from the first audio to be recognized, the audio corresponding to the text recognition result whose matching degree is less than or equal to the configured matching degree as the first audio;

[0095] Acquire, from the first audio to be recognized, an audio whose recognition rate is less than or equal to the configured recognition rate as the second audio;

[0096] Integrating the first audio and the second audio to obtain the second audio to be recognized;

[0097] The second audio to be recognized is deleted from the first audio to be recognized to obtain the recognized audio.

[0098] The configuration matching degree and the configuration recognition rate can be configured according to specific recognition requirements.

[0099] For example, if the recognition result of an audio segment shows that the final output text matches the original speech at 85%, which is lower than the configured matching degree of 95%, it means that the recognition is not accurate and the audio segment needs to be re-recognized.

[0100] Alternatively, when a section of audio is not recognized successfully, or only a few words are recognized, it means that the recognition rate of this section of audio is low, and this section of audio also needs to be recognized again.

[0101] Through the above implementation, audio that is not recognized accurately or has a low recognition rate can be screened out for secondary recognition to ensure better speech recognition effect.

[0102] S15, connecting to a target server, inputting the second audio to be recognized into a target speech recognition model in the target server, and obtaining a text recognition result from an output of the target speech recognition model as a second text sequence.

[0103] The target server may be a private server to avoid information leakage when connecting to the target server for voice recognition.

[0104] The target speech recognition model may be trained using professional terms to improve the accuracy of speech recognition in professional scenarios.

[0105] Through the above implementation, audio that cannot be recognized locally or is recognized inaccurately can be transmitted to the server for recognition, and the recognition result is returned with guaranteed reliability.

[0106] In at least one embodiment of the present invention, the method further comprises:

[0107] Regularly detecting the resource occupancy rate of the target server;

[0108] When it is detected that the resource occupancy rate of the target server is less than or equal to the configured resource occupancy rate, acquiring new data;

[0109] Performing word frequency sampling on the newly added data to obtain a sampling result;

[0110] Get historical data;

[0111] When the similarity between the sampling result and the historical data is less than or equal to the configuration similarity, the target speech recognition model is optimized and trained using the newly added data.

[0112] The configuration resource occupancy rate and the configuration similarity can be customized.

[0113] By comparing with the configured resource occupancy rate, it is possible to avoid optimizing the model during the business period, thereby preventing the execution of normal business.

[0114] By comparing with the configuration similarity, repeated training using similar data can be avoided.

[0115] Through the above implementation, the speech recognition model can be continuously optimized and trained using newly added data during idle time to continuously improve the recognition effect and coverage of the model.

[0116] Furthermore, the method further comprises:

[0117] When the update condition is met, connecting to the target server and obtaining all speech recognition models stored in the target server;

[0118] Obtaining update times of all the speech recognition models, and identifying the latest speech recognition model according to the update times;

[0119] Converting the identified speech recognition model using the WebAssembly mode, and determining the converted model as the local speech recognition model;

[0120] Deploy the local speech recognition model.

[0121] The update condition may be detecting that a pre-configured trigger period is met to implement regular updates to the local model, or detecting that a pre-configured update control is triggered to perform updates according to actual needs, which is not limited in the present invention.

[0122] Through the above implementation, the latest speech recognition model can be obtained from the server each time and converted and deployed using the WebAssembly model, which solves the problem in the existing technology that the offline speech recognition solution is mainly based on iOS or Android App, there is no adapted PC (personal computer) browser and the recognition scenario is relatively simple, resulting in low recognition rate.

[0123] S16: Concatenate the first character sequence and the second character sequence to obtain a target text.

[0124] It can be understood that the first character sequence and the second character sequence are equivalent to the character fragments obtained after recognition. Therefore, in order to form a complete text, the first character sequence and the second character sequence need to be spliced ​​together.

[0125] Specifically, the step of concatenating the first character sequence and the second character sequence to obtain a target text includes:

[0126] Obtaining a first word order of the first word sequence, and obtaining a second word order of the second word sequence;

[0127] The first character sequence and the second character sequence are concatenated according to the first word order and the second word order to obtain the target text.

[0128] For example, the first text sequence is "weekend" and "play", and the second text sequence is "where to go". After splicing them together according to the word order, the final target text is "Where to play on the weekend".

[0129] S17, displaying the target text in real time.

[0130] Specifically, the real-time display of the target text includes:

[0131] Obtaining a text format corresponding to the display of the target conference room, and / or obtaining a text format corresponding to the display of the terminal device;

[0132] Converting the target text into a text format corresponding to the display of the target conference room, and / or converting the target text into a text format corresponding to the display of the terminal device;

[0133] The converted target text is displayed on a corresponding display in real time.

[0134] Through the above implementation, after the target text is identified, personalized display can be performed according to the text format requirements of different displays, thereby improving the adaptability of text display.

[0135] For example, the target meeting display will be displayed in "Songti, Highlight" and the terminal device will be displayed in "Fangsongti, Size 4".

[0136] It should be noted that in order to further ensure the security of the data, the local speech recognition model and the target speech recognition model can be stored in the blockchain to prevent the data from being maliciously tampered with.

[0137] It can be seen from the above technical solution that the present invention can respond to a communication assistance instruction, determine the target voice acquisition device according to the communication assistance instruction, and start the target voice acquisition device to collect voice, pre-process the collected voice to obtain a first audio to be recognized, input the first audio to be recognized into a local voice recognition model, obtain a local recognition result, first maximize the voice recognition to text conversion through the local model, with fluency and security, obtain the matching degree and recognition rate from the local recognition result, split the first audio to be recognized into recognized audio and second audio to be recognized according to the matching degree and the recognition rate, and obtain the text recognition result corresponding to the recognized audio from the local recognition result as a first text sequence, connect to the target server, input the second audio to be recognized into the target voice recognition model in the target server, and obtain the text recognition result from the output of the target voice recognition model as a second text sequence, and then filter out the audio that is not recognized accurately or has a low recognition rate for secondary recognition to ensure better voice recognition effect and a bottom-line reliability, splice the first text sequence and the second text sequence to obtain the target text, and display the target text in real time, so as to realize assisted communication for online meetings based on artificial intelligence means.

[0138] like Figure 2 , which is a functional module diagram of a preferred embodiment of the online conference-based communication assistance device of the present invention. The online conference-based communication assistance device 11 includes an acquisition unit 110, a preprocessing unit 111, an input unit 112, an acquisition unit 113, a splicing unit 114, and a display unit 115. The modules / units referred to herein are a series of computer program segments that can be executed by the processor 13 and perform fixed functions, and are stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0139] In response to the communication assistance instruction, the collection unit 110 determines a target voice collection device according to the communication assistance instruction, and starts the target voice collection device to collect voice.

[0140] In at least one embodiment of the present invention, the communication assistance instruction may be triggered by a branch financial manager, investment consultant or other related staff, and the present invention is not limited thereto.

[0141] In at least one embodiment of the present invention, the acquisition unit 110 determines the target voice acquisition device according to the communication assistance instruction, including:

[0142] Parsing the method body of the communication assistance instruction to obtain information carried by the communication assistance instruction;

[0143] Get the preset label;

[0144] Constructing a regular expression according to the preset tag;

[0145] Using the regular expression to traverse the information carried in the communication assistance instruction, and determining the traversed data as the conference identifier;

[0146] Establishing a connection with a target conference room corresponding to the conference identifier, and obtaining a voice acquisition device of the target conference room as the target voice acquisition device; and / or

[0147] When it is detected that a terminal device is remotely connected to the target conference room, the voice collection device of the terminal device is determined as the target voice collection device.

[0148] Specifically, the communication assistance instruction is essentially a code. In the communication assistance instruction, according to the code writing principle, the content between {} is called the method body.

[0149] The preset tag can be customized and has a one-to-one correspondence with the address. For example, the preset tag can be ID, and the regular expression ID() is further established with the preset tag, and traversal is performed with ID().

[0150] Through the above implementation, the target voice collection device can be quickly determined based on the regular expression and the preset label, so that real-time voice collection can be performed using the target voice collection device.

[0151] For example, when a customer manager holds an online meeting with a customer, the target voice acquisition device may be the voice acquisition device on the customer manager's and the customer's terminal devices respectively. At this time, if the customer manager is holding a meeting in a conference room, the target voice acquisition device may also include the voice acquisition device in the conference room.

[0152] The preprocessing unit 111 preprocesses the collected speech to obtain a first audio to be recognized.

[0153] It is understandable that since the collected speech may contain noise, the collected speech must first be preprocessed to improve the efficiency and accuracy of recognition.

[0154] Specifically, the pre-processing may include, but is not limited to: pre-emphasis and windowing and framing.

[0155] The input unit 112 inputs the first audio to be recognized into a local speech recognition model to obtain a local recognition result.

[0156] The local speech recognition model is a recognition model deployed on the client. When the local recognition model is used for speech recognition, there is no need for frequent network connections, thus effectively avoiding delays in speech recognition.

[0157] In addition, the local speech recognition model is deployed on the local client and does not need to be connected to an external server or network, which avoids the occurrence of lag. This not only reduces the cost of speech recognition, but also effectively avoids information leakage and improves the security of the speech recognition process.

[0158] Through the above implementation, speech recognition to text conversion can be performed to the maximum extent through the local model, with fluency and security.

[0159] The acquisition unit 113 acquires the matching degree and the recognition rate from the local recognition result.

[0160] Specifically, the local recognition result may include: text obtained after speech recognition (ie, text recognition result), the matching degree between the text obtained after speech recognition and the original speech, and the recognition rate of speech recognition.

[0161] The acquisition unit 113 splits the first audio to be recognized into recognized audio and second audio to be recognized according to the matching degree and the recognition rate, and acquires a text recognition result corresponding to the recognized audio from the local recognition result as a first text sequence.

[0162] It is understandable that since the local speech recognition model is deployed on the local client and is not convenient for real-time updating, the recognition effect may be subject to certain limitations. In order to ensure the comprehensiveness and accuracy of speech recognition, it is necessary to further split the first audio to be recognized according to the matching degree and recognition rate of the local recognition results to screen out the audio that requires secondary speech recognition.

[0163] Specifically, the acquiring unit 113 splits the first audio to be recognized into recognized audio and second audio to be recognized according to the matching degree and the recognition rate, including:

[0164] Acquire, from the first audio to be recognized, the audio corresponding to the text recognition result whose matching degree is less than or equal to the configured matching degree as the first audio;

[0165] Acquire, from the first audio to be recognized, an audio whose recognition rate is less than or equal to the configured recognition rate as the second audio;

[0166] Integrating the first audio and the second audio to obtain the second audio to be recognized;

[0167] The second audio to be recognized is deleted from the first audio to be recognized to obtain the recognized audio.

[0168] The configuration matching degree and the configuration recognition rate can be configured according to specific recognition requirements.

[0169] For example, if the recognition result of an audio segment shows that the final output text matches the original speech at 85%, which is lower than the configured matching degree of 95%, it means that the recognition is not accurate and the audio segment needs to be re-recognized.

[0170] Alternatively, when a section of audio is not recognized successfully, or only a few words are recognized, it means that the recognition rate of this section of audio is low, and this section of audio also needs to be recognized again.

[0171] Through the above implementation, audio that is not recognized accurately or has a low recognition rate can be screened out for secondary recognition to ensure better speech recognition effect.

[0172] The input unit 112 is connected to a target server, inputs the second audio to be recognized into a target speech recognition model in the target server, and obtains a text recognition result from an output of the target speech recognition model as a second text sequence.

[0173] The target server may be a private server to avoid information leakage when connecting to the target server for voice recognition.

[0174] The target speech recognition model may be trained using professional terms to improve the accuracy of speech recognition in professional scenarios.

[0175] Through the above implementation, audio that cannot be recognized locally or is recognized inaccurately can be transmitted to the server for recognition, and the recognition result is returned with guaranteed reliability.

[0176] In at least one embodiment of the present invention, the resource occupancy rate of the target server is periodically detected;

[0177] When it is detected that the resource occupancy rate of the target server is less than or equal to the configured resource occupancy rate, acquiring new data;

[0178] Performing word frequency sampling on the newly added data to obtain a sampling result;

[0179] Get historical data;

[0180] When the similarity between the sampling result and the historical data is less than or equal to the configuration similarity, the target speech recognition model is optimized and trained using the newly added data.

[0181] The configuration resource occupancy rate and the configuration similarity can be customized.

[0182] By comparing with the configured resource occupancy rate, it is possible to avoid optimizing the model during the business period, thereby preventing the execution of normal business.

[0183] By comparing with the configuration similarity, repeated training using similar data can be avoided.

[0184] Through the above implementation, the speech recognition model can be continuously optimized and trained using newly added data during idle time to continuously improve the recognition effect and coverage of the model.

[0185] Further, when the update condition is met, connecting to the target server and obtaining all speech recognition models stored in the target server;

[0186] Obtaining update times of all the speech recognition models, and identifying the latest speech recognition model according to the update times;

[0187] Converting the identified speech recognition model using the WebAssembly mode, and determining the converted model as the local speech recognition model;

[0188] Deploy the local speech recognition model.

[0189] The update condition may be detecting that a pre-configured trigger period is met to implement regular updates to the local model, or detecting that a pre-configured update control is triggered to perform updates according to actual needs, which is not limited in the present invention.

[0190] Through the above implementation, the latest speech recognition model can be obtained from the server each time and converted and deployed using the WebAssembly model, which solves the problem in the existing technology that the offline speech recognition solution is mainly based on iOS or Android App, there is no adapted PC (personal computer) browser and the recognition scenario is relatively simple, resulting in low recognition rate.

[0191] The concatenation unit 114 concatenates the first character sequence and the second character sequence to obtain a target text.

[0192] It can be understood that the first character sequence and the second character sequence are equivalent to the character fragments obtained after recognition. Therefore, in order to form a complete text, the first character sequence and the second character sequence need to be spliced ​​together.

[0193] Specifically, the splicing unit 114 splices the first character sequence and the second character sequence to obtain a target text including:

[0194] Obtaining a first word order of the first word sequence, and obtaining a second word order of the second word sequence;

[0195] The first character sequence and the second character sequence are concatenated according to the first word order and the second word order to obtain the target text.

[0196] For example, the first text sequence is "weekend" and "play", and the second text sequence is "where to go". After splicing them together according to the word order, the final target text is "Where to play on the weekend".

[0197] The display unit 115 displays the target text in real time.

[0198] Specifically, the display unit 115 displays the target text in real time including:

[0199] Obtaining a text format corresponding to the display of the target conference room, and / or obtaining a text format corresponding to the display of the terminal device;

[0200] Converting the target text into a text format corresponding to the display of the target conference room, and / or converting the target text into a text format corresponding to the display of the terminal device;

[0201] The converted target text is displayed on a corresponding display in real time.

[0202] Through the above implementation, after the target text is identified, personalized display can be performed according to the text format requirements of different displays, thereby improving the adaptability of text display.

[0203] For example, the target meeting display will be displayed in "Songti, Highlight" and the terminal device will be displayed in "Fangsongti, Size 4".

[0204] It should be noted that in order to further ensure the security of the data, the local speech recognition model and the target speech recognition model can be stored in the blockchain to prevent the data from being maliciously tampered with.

[0205] It can be seen from the above technical solution that the present invention can respond to a communication assistance instruction, determine the target voice acquisition device according to the communication assistance instruction, and start the target voice acquisition device to collect voice, pre-process the collected voice to obtain a first audio to be recognized, input the first audio to be recognized into a local voice recognition model, obtain a local recognition result, first maximize the voice recognition to text conversion through the local model, with fluency and security, obtain the matching degree and recognition rate from the local recognition result, split the first audio to be recognized into recognized audio and second audio to be recognized according to the matching degree and the recognition rate, and obtain the text recognition result corresponding to the recognized audio from the local recognition result as a first text sequence, connect to the target server, input the second audio to be recognized into the target voice recognition model in the target server, and obtain the text recognition result from the output of the target voice recognition model as a second text sequence, and then filter out the audio that is not recognized accurately or has a low recognition rate for secondary recognition to ensure better voice recognition effect and a bottom-line reliability, splice the first text sequence and the second text sequence to obtain the target text, and display the target text in real time, so as to realize assisted communication for online meetings based on artificial intelligence means.

[0206] like Figure 3 , which is a structural diagram of an electronic device of a preferred embodiment of the present invention for implementing a communication assistance method based on an online conference.

[0207] The electronic device 1 may include a memory 12, a processor 13 and a bus, and may also include a computer program stored in the memory 12 and executable on the processor 13, such as a communication assistance program based on an online conference.

[0208] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 1 and does not constitute a limitation on the electronic device 1. The electronic device 1 may have either a bus structure or a star structure. The electronic device 1 may also include more or less other hardware or software than shown in the figure, or a different arrangement of components. For example, the electronic device 1 may also include input and output devices, network access devices, etc.

[0209] It should be noted that the electronic device 1 is only an example. Other existing or future electronic products that are adaptable to the present invention should also be included in the protection scope of the present invention and included herein by reference.

[0210] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 12 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Furthermore, the memory 12 can also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 12 can not only be used to store application software and various types of data installed on the electronic device 1, such as the code of a communication assistance program based on an online meeting, but can also be used to temporarily store data that has been output or is to be output.

[0211] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 13 is the control core (Control Unit) of the electronic device 1, and utilizes various interfaces and lines to connect the various components of the entire electronic device 1. It executes or executes programs or modules stored in the memory 12 (for example, executing a communication assistance program based on an online meeting, etc.), and calls data stored in the memory 12 to perform various functions of the electronic device 1 and process data.

[0212] The processor 13 executes the operating system of the electronic device 1 and various installed applications. The processor 13 executes the applications to implement the steps in the above-mentioned embodiments of the communication assistance method based on online conferences, for example Figure 1 Steps shown.

[0213] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to implement the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of performing specific functions, which describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into an acquisition unit 110, a preprocessing unit 111, an input unit 112, an acquisition unit 113, a splicing unit 114, and a display unit 115.

[0214] The above-mentioned integrated unit implemented in the form of a software functional module can be stored in a computer-readable storage medium. The above-mentioned software functional module stored in a storage medium includes a number of instructions for causing a computer device (which can be a personal computer, computer device, or network device, etc.) or a processor to execute the portion of the online conference-based communication assistance method described in various embodiments of the present invention.

[0215] If the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present invention can also implement all or part of the processes in the above-mentioned method embodiments by instructing relevant hardware devices through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments.

[0216] The computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory, etc.

[0217] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0218] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0219] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The diagram is represented by only one arrow, but it does not mean that there is only one bus or one type of bus. The bus is configured to implement connection and communication between the memory 12 and at least one processor 13, etc.

[0220] Although not shown, the electronic device 1 may also include a power source (such as a battery) to power various components. Preferably, the power source may be logically connected to the at least one processor 13 via a power management device, thereby implementing functions such as charging management, discharging management, and power consumption management through the power management device. The power source may also include one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be detailed here.

[0221] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0222] Optionally, the electronic device 1 may further include a user interface, which may be a display or an input unit (such as a keyboard). Optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0223] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0224] Figure 3 Only the electronic device 1 having components 12-13 is shown, and it can be understood by those skilled in the art that Figure 3 The structure shown does not constitute a limitation on the electronic device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0225] Combine Figure 1 The memory 12 in the electronic device 1 stores a plurality of instructions to implement a communication assistance method based on an online conference, and the processor 13 can execute the plurality of instructions to implement:

[0226] In response to the communication assistance instruction, determining a target voice acquisition device according to the communication assistance instruction, and starting the target voice acquisition device to acquire voice;

[0227] Preprocessing the collected speech to obtain a first audio to be recognized;

[0228] Inputting the first audio to be recognized into a local speech recognition model to obtain a local recognition result;

[0229] Obtaining a matching degree and a recognition rate from the local recognition result;

[0230] Splitting the first audio to be recognized into recognized audio and second audio to be recognized according to the matching degree and the recognition rate, and obtaining a text recognition result corresponding to the recognized audio from the local recognition result as a first text sequence;

[0231] Connecting to a target server, inputting the second audio to be recognized into a target speech recognition model in the target server, and obtaining a text recognition result from an output of the target speech recognition model as a second text sequence;

[0232] concatenating the first character sequence and the second character sequence to obtain a target text;

[0233] The target text is displayed in real time.

[0234] Specifically, the specific implementation method of the processor 13 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0235] In the several embodiments provided herein, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical functional division, and actual implementation may employ other division methods.

[0236] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0237] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0238] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0239] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0240] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in a system embodiment may also be implemented by a single unit or device through software or hardware. Terms such as first and second are used to indicate names and do not imply any particular order.

[0241] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A communication assistance method based on online conference, characterized in that: The communication assistance method based on online conference includes: In response to the communication assistance instruction, determining a target voice acquisition device according to the communication assistance instruction, and starting the target voice acquisition device to acquire voice; Preprocessing the collected speech to obtain a first audio to be recognized; Inputting the first audio to be recognized into a local speech recognition model to obtain a local recognition result; Obtaining a matching degree and a recognition rate from the local recognition result; Splitting the first audio to be recognized into recognized audio and second audio to be recognized according to the matching degree and the recognition rate, and obtaining a text recognition result corresponding to the recognized audio from the local recognition result as a first text sequence; Connecting to a target server, inputting the second audio to be recognized into a target speech recognition model in the target server, and obtaining a text recognition result from an output of the target speech recognition model as a second text sequence; concatenating the first character sequence and the second character sequence to obtain a target text; The target text is displayed in real time.

2. The communication assistance method based on online conference according to claim 1, characterized in that: Determining the target voice acquisition device according to the communication assistance instruction includes: Parsing the method body of the communication assistance instruction to obtain information carried by the communication assistance instruction; Get the preset label; Constructing a regular expression according to the preset tag; Using the regular expression to traverse the information carried in the communication assistance instruction, and determining the traversed data as the conference identifier; Establishing a connection with a target conference room corresponding to the conference identifier, and obtaining a voice acquisition device of the target conference room as the target voice acquisition device; and / or When it is detected that a terminal device is remotely connected to the target conference room, the voice collection device of the terminal device is determined as the target voice collection device.

3. The communication assistance method based on online conference according to claim 1, characterized in that: The step of splitting the first audio to be recognized into recognized audio and second audio to be recognized according to the matching degree and the recognition rate includes: Acquire, from the first audio to be recognized, the audio corresponding to the text recognition result whose matching degree is less than or equal to the configured matching degree as the first audio; Acquire, from the first audio to be recognized, an audio whose recognition rate is less than or equal to the configured recognition rate as the second audio; Integrating the first audio and the second audio to obtain the second audio to be recognized; The second audio to be recognized is deleted from the first audio to be recognized to obtain the recognized audio.

4. The communication assistance method based on online conference according to claim 1, characterized in that: The method further comprises: Regularly detecting the resource occupancy rate of the target server; When it is detected that the resource occupancy rate of the target server is less than or equal to the configured resource occupancy rate, acquiring new data; Performing word frequency sampling on the newly added data to obtain a sampling result; Get historical data; When the similarity between the sampling result and the historical data is less than or equal to the configuration similarity, the target speech recognition model is optimized and trained using the newly added data.

5. The communication assistance method based on online conference according to claim 1, characterized in that: The method further comprises: When the update condition is met, connecting to the target server and obtaining all speech recognition models stored in the target server; Obtaining update times of all the speech recognition models, and identifying the latest speech recognition model according to the update times; Converting the identified speech recognition model using the WebAssembly mode, and determining the converted model as the local speech recognition model; Deploy the local speech recognition model.

6. The communication assistance method based on online conference according to claim 1, characterized in that: The step of concatenating the first character sequence and the second character sequence to obtain a target text includes: Obtaining a first word order of the first word sequence, and obtaining a second word order of the second word sequence; The first character sequence and the second character sequence are concatenated according to the first word order and the second word order to obtain the target text.

7. The communication assistance method based on online conference according to claim 2, characterized in that: The real-time display of the target text comprises: Obtaining a text format corresponding to the display of the target conference room, and / or obtaining a text format corresponding to the display of the terminal device; Converting the target text into a text format corresponding to the display of the target conference room, and / or converting the target text into a text format corresponding to the display of the terminal device; The converted target text is displayed on a corresponding display in real time.

8. A communication assistance device based on online conference, characterized in that: The communication assistance device based on online conference includes: a collection unit, configured to respond to a communication assistance instruction, determine a target voice collection device according to the communication assistance instruction, and start the target voice collection device to collect voice; A preprocessing unit, configured to preprocess the collected speech to obtain a first audio to be recognized; An input unit, configured to input the first audio to be recognized into a local speech recognition model to obtain a local recognition result; An acquisition unit, configured to acquire a matching degree and a recognition rate from the local recognition result; an acquisition unit, configured to split the first audio to be recognized into recognized audio and second audio to be recognized according to the matching degree and the recognition rate, and acquire a text recognition result corresponding to the recognized audio from the local recognition result as a first text sequence; The input unit is further configured to connect to a target server, input the second audio to be recognized into a target speech recognition model in the target server, and obtain a text recognition result from an output of the target speech recognition model as a second text sequence; a concatenation unit, configured to concatenate the first character sequence and the second character sequence to obtain a target text; A display unit is used to display the target text in real time.

9. An electronic device, characterized in that: The electronic device comprises: a memory storing at least one instruction; and A processor executes instructions stored in the memory to implement the communication assistance method based on online conference as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the communication assistance method based on an online conference as described in any one of claims 1 to 7.

Images