Meeting minutes generation method and electronic device

Abstract

The application provides a conference minutes generation method and an electronic device, relates to the technical field of terminals, and can enable a member newly joining a conference to quickly obtain key information of the conference, reduce the probability of the conference being interrupted, and improve the efficiency of the conference. The method comprises the following steps: an electronic device determines a target domain sentence pattern library corresponding to a current conference, the target domain sentence pattern library is used to indicate the arrangement of words in a target text, the target text is a text corresponding to a target domain, and the target domain is a conference domain to which the current conference belongs. The electronic device acquires audio data in the current conference, generates conference minutes corresponding to the current conference according to the audio data and the target domain sentence pattern library, and outputs the conference minutes.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a method for generating meeting minutes and an electronic device. Background Technology

[0002] Currently, online meetings (hereinafter referred to as meetings) and other multi-person dialogue scenarios are becoming increasingly common, and users often use meetings to communicate in various aspects such as work and study.

[0003] During a meeting, new members may have various concerns upon joining, such as not knowing the topics for discussion or the current progress of the meeting. Existing meeting systems often fail to address these concerns and help new members quickly integrate into the meeting agenda. Furthermore, when previously joined members attempt to answer new members' questions, it disrupts the meeting, leading to inefficiency. Summary of the Invention

[0004] This application provides a method for generating meeting minutes and an electronic device that enables newly joined meeting members to quickly obtain key meeting information, reduces the probability of meeting interruptions, and improves meeting efficiency.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, this application provides a method for generating meeting minutes, applied to an electronic device. The method includes: determining a target domain sentence structure library corresponding to the current meeting, wherein the target domain sentence structure library is used to indicate the arrangement of words in the target text, the target text is the text corresponding to the target domain, and the target domain is the meeting domain to which the current meeting belongs; acquiring audio data from the current meeting; generating meeting minutes corresponding to the current meeting based on the audio data and the target domain sentence structure library; and outputting the meeting minutes.

[0007] Based on the above technical solution, electronic devices can generate and output meeting minutes of the current meeting using a domain-specific sentence structure library and audio data from the meeting. This allows newly joined members to quickly obtain key information from the output minutes, reducing the probability of interruptions and improving meeting efficiency. Furthermore, by incorporating a domain-specific sentence structure library, the minutes' expression adapts to the expression habits of the corresponding meeting domain (e.g., professional terms and jargon), enhancing readability and professionalism, and improving the overall quality of the meeting minutes.

[0008] In one possible design, determining the target domain phrase library corresponding to the current meeting includes: obtaining prior information corresponding to the current meeting, which includes one or more of the following: meeting schedule information, chat information, and email information; the prior information includes multiple words; determining the target domain phrase library based on the probability of the target word in at least one domain phrase library; the target word consists of any e consecutive words followed by f consecutive words, where e and f are preset positive integers; wherein each domain phrase library includes the probability of any e consecutive words followed by f consecutive words in the meeting minutes of the corresponding meeting domain.

[0009] In one possible design, determining the target domain sentence library corresponding to the current meeting includes: receiving target domain input from the user; and determining the target domain sentence library based on the target domain and at least one domain sentence library.

[0010] In one possible design, the meeting minutes corresponding to the current meeting are generated based on audio data and a target domain sentence structure library, including: acquiring the voice data of the meeting members in the audio data; converting the voice data into text data; and generating the meeting minutes based on the text data and the target domain sentence structure library.

[0011] In one possible design, the target domain sentence structure library includes the probabilities of any e consecutive words followed by f consecutive words in the target text, where e and f are preset positive integers. Generating meeting minutes based on text data and the target domain sentence structure library includes: inputting text data into a preset model; obtaining the first probability of candidate words corresponding to the f consecutive words to be output in the meeting minutes through the preset model; determining the third probability of candidate words based on the first probability of candidate words and the second probability of candidate words corresponding to them in the target domain sentence structure library; wherein the second probability is the probability of candidate words following the e words already output in the meeting minutes; and generating the meeting minutes based on the third probability of candidate words.

[0012] In one possible design, the text data includes punctuation marks used to segment the text data. These punctuation marks include one or more of the following: Chinese period, question mark, exclamation mark, and English period. Before inputting the text data into a preset model, the method further includes determining whether the size of the text data meets a preset threshold using the punctuation marks. Based on this design, a corresponding meeting summary can be generated from a quantitative amount of text data, resulting in a high degree of real-time performance.

[0013] In one possible design, the meeting minutes include a target summary, which is generated based on the text data corresponding to the target punctuation marks in the punctuation marks. The method also includes: obtaining the text data corresponding to the target punctuation marks based on the punctuation marks; and generating the target summary based on the text data corresponding to the target punctuation marks.

[0014] In one possible design, the text data corresponding to the target punctuation mark includes at least one statement corresponding to the target punctuation mark; generating a target summary based on the text data corresponding to the target punctuation mark includes: determining at least one target statement from the statements corresponding to at least one target punctuation mark based on at least one domain sentence pattern library, wherein the at least one target statement corresponds to a target domain; and generating a target summary based on the at least one target statement.

[0015] In one possible design, at least one target statement includes multiple overlapping target statements, and the similarity between the multiple overlapping target statements satisfies a preset condition; generating a target summary based on at least one target statement includes: generating a target summary based on one or more of the multiple overlapping target statements, and target statements in at least one target statement other than the multiple overlapping target statements.

[0016] In one possible design, the meeting minutes also include time information and / or meeting member information; the time information is determined based on the time endpoint information corresponding to the voice data, which includes the start time and / or end time of the voice data; the meeting member information is determined based on the meeting member information corresponding to the voice data; the meeting minutes corresponding to the current meeting are generated based on the audio data and the target domain sentence library, including: generating a meeting summary based on the audio data and the domain sentence library; and matching the time information and / or meeting member information with the meeting summary to generate the meeting minutes.

[0017] In one possible design, before determining the target domain sentence structure library corresponding to the current meeting, the method includes: obtaining the target text; and constructing the target domain sentence structure library based on the target text.

[0018] In one possible design, outputting meeting minutes includes: displaying the meeting minutes.

[0019] In one possible design, after displaying the meeting minutes, the method further includes: receiving a user's target action, which initiates the meeting minutes editing function; and displaying the meeting minutes editing interface in response to the target action. Based on this design, the meeting minutes can be edited online by users and can be directly used as meeting records after the meeting ends. This avoids the need for additional editing of the meeting minutes after the meeting to generate meeting records, saving editing time and manpower, and improving the efficiency of meeting recording.

[0020] In one possible design, the conference field includes one or more of the following: clinical medicine, forensic medicine, preventive medicine, rehabilitation medicine, health care medicine, psychology, pathology, genetics, human histology, surgery, internal medicine, stomatology, oncology, respiratory diseases, human immunology, medical imaging, banking, securities, insurance, trust, investment, finance, accounting, leasing, stocks, funds, trade, big data, artificial intelligence, Internet of Things, the Internet, cloud computing, connected vehicles, machine learning, autonomous driving, mobile communication, optical communication, microwave communication, satellite communication, WiFi, Bluetooth, radio frequency, Ethernet, NLP, ASR, petroleum, natural gas, coal, horticulture, animal husbandry, breeding, cultivation, aquaculture, soil and fertilizer, marine, agricultural product processing, vehicles, textiles, chemicals, metallurgy, electrical appliances, water supply and drainage, civil engineering, road transportation, bridges, and river crossing.

[0021] Secondly, this application provides an electronic device having the function of implementing the method described in the first aspect and any of the designs described above. This function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described function.

[0022] Thirdly, this application provides an electronic device including a processor that is coupled to a memory and, after reading computer instructions stored in the memory, executes the method described in accordance with the instructions as described in any of the foregoing aspects and any of the designs thereof.

[0023] Fourthly, this application provides an electronic device including one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory and, when executed by the one or more processors, cause the electronic device to perform the methods as described in the first aspect and any of the designs therein.

[0024] Fifthly, this application provides an electronic device including a processor, a memory, and a display screen, wherein the display screen, the memory, and the processor are coupled together, the memory is used to store computer program code, the computer program code including computer instructions, and the processor reads the computer instructions from the memory to cause the electronic device to perform the method described in the first aspect and any of the designs described above.

[0025] In one possible design, the electronic device also includes a communication interface that allows the electronic device to communicate with other devices. For example, this communication interface could be a transceiver, an input / output interface, an interface circuit, an output circuit, an input circuit, a pin, or related circuitry.

[0026] In a sixth aspect, this application provides a computer-readable storage medium comprising a computer program or instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect and any of the designs therein.

[0027] In a seventh aspect, this application provides a computer program product comprising: a computer program or instructions that, when executed on a computer, cause the computer to perform the method described in the first aspect and any of the designs therein.

[0028] Eighthly, this application provides a chip system including at least one processor and at least one interface circuit, the at least one interface circuit being used to perform transceiver functions and send instructions to at least one processor, and when at least one processor executes instructions, at least one processor performs the method as described in the first aspect and any of the designs therein.

[0029] It should be noted that the technical effects of any of the designs in the second to eighth aspects mentioned above can be found in the technical effects of the corresponding designs in the first aspect, and will not be repeated here. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of an existing process for generating meeting minutes;

[0031] Figure 2 This is a schematic diagram of another existing process for generating meeting minutes;

[0032] Figure 3 A schematic diagram of the structure of a conference system provided in this application embodiment;

[0033] Figure 4 This is a schematic diagram of the structure of another conference system provided in an embodiment of this application;

[0034] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0035] Figure 6 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this application;

[0036] Figure 7 A software structure block diagram of an electronic device provided in an embodiment of this application;

[0037] Figure 8 A set of interface schematic diagrams provided for embodiments of this application;

[0038] Figure 9 Another set of interface schematic diagrams provided for embodiments of this application;

[0039] Figure 10 A schematic diagram illustrating the process of generating meeting minutes provided in this application embodiment;

[0040] Figure 11 A schematic diagram of voice data and time endpoints provided in an embodiment of this application;

[0041] Figure 12 This is a schematic flowchart of a method for obtaining text data provided in an embodiment of this application;

[0042] Figure 13 A schematic diagram illustrating the process of constructing a domain-specific sentence structure library provided in this application embodiment;

[0043] Figure 14 A schematic diagram of a domain-specific syntax library provided for an embodiment of this application;

[0044] Figure 15 A schematic diagram of prior information provided in an embodiment of this application;

[0045] Figure 16 Another set of interface schematic diagrams provided for embodiments of this application;

[0046] Figure 17 Another set of interface schematic diagrams provided for embodiments of this application;

[0047] Figure 18 Another set of interface schematic diagrams provided for embodiments of this application;

[0048] Figure 19 A schematic diagram illustrating the matching probability of prior information in different fields, provided for embodiments of this application;

[0049] Figure 20 A schematic diagram of the structure of a machine model provided in an embodiment of this application;

[0050] Figure 21 A schematic diagram illustrating the calculation of the probability of candidate words, provided as an embodiment of this application;

[0051] Figure 22 A schematic diagram of a meeting summary provided for an embodiment of this application;

[0052] Figure 23 A schematic diagram illustrating time endpoint information and meeting member information provided in an embodiment of this application;

[0053] Figure 24 A schematic diagram of a meeting minutes provided for an embodiment of this application;

[0054] Figure 25 Another set of interface schematic diagrams provided for embodiments of this application;

[0055] Figure 26 Another set of interface schematic diagrams provided for embodiments of this application;

[0056] Figure 27 Another set of interface schematic diagrams provided for embodiments of this application;

[0057] Figure 28 This is a schematic flowchart illustrating another method for acquiring text data provided in an embodiment of this application;

[0058] Figure 29 A schematic diagram illustrating the matching probability of a question in different fields, provided as an embodiment of this application;

[0059] Figure 30 A schematic diagram illustrating a problem summary provided for an embodiment of this application;

[0060] Figure 31 A schematic diagram of yet another type of meeting minutes provided for an embodiment of this application;

[0061] Figure 32 Another set of interface schematic diagrams provided for embodiments of this application;

[0062] Figure 33 A schematic diagram of yet another type of meeting minutes provided for an embodiment of this application;

[0063] Figure 34 A flowchart illustrating a method for generating meeting minutes is provided in this application embodiment;

[0064] Figure 35 A schematic flowchart illustrating another method for generating meeting minutes provided in this application embodiment;

[0065] Figure 36 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this application;

[0066] Figure 37 This is a schematic diagram of a chip system provided in an embodiment of this application. Detailed Implementation

[0067] The meeting minutes generation method and electronic device provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0068] The terms “comprising” and “having”, and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0069] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0070] In the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone.

[0071] Currently, multi-person dialogue scenarios such as meetings are becoming increasingly common, with users frequently using meetings for communication in various aspects such as work and study. However, during a meeting, if a new member who has just joined has doubts, the ongoing meeting may be interrupted, leading to low meeting efficiency.

[0072] Based on this, an existing technology provides a solution that generates meeting minutes during the meeting, so that new members joining the meeting can obtain key information from the generated meeting minutes and quickly integrate into the current meeting agenda.

[0073] For example, Figure 1 This paper presents an existing scheme for generating meeting minutes. It obtains meeting invitation information, which includes multiple meeting topics. Then, it matches keywords from the meeting topics to obtain meeting process keywords. Simultaneously, it acquires audio information from attendees and extracts keywords from it. Then, it matches the keywords from the audio information with the obtained meeting process keywords, and uses the matched meeting process keywords and the text information converted from the audio information to generate meeting minutes. This scheme generates meeting minutes solely by matching keywords, resulting in low accuracy. Furthermore, converting audio data into text data and using this large amount of text data as the content of the meeting minutes leads to poor readability. Therefore, the quality of meeting minutes generated by this scheme is poor.

[0074] Figure 2This paper presents another existing scheme for generating meeting minutes. It collects each participant's speaking time, number of times they spoke, and chat content to generate meeting minutes. However, this scheme directly uses the participants' complex chat content as the meeting minutes, resulting in poor readability. Therefore, the quality of the meeting minutes generated by this scheme is also poor.

[0075] To address the aforementioned technical issues, this application provides a method for generating meeting minutes, which improves the quality of the generated meeting minutes, enabling newly joined members to quickly obtain key meeting information based on the generated minutes, reducing the probability of meeting interruptions, and improving meeting efficiency.

[0076] The meeting minutes generation method provided in this application can be applied to various meeting systems, including but not limited to meetings conducted using meeting software, conference room systems, and other methods.

[0077] Figure 3 An embodiment of the present application provides a conference system 300. For example... Figure 3 As shown, the conference system 300 includes multiple electronic devices 301. These electronic devices 301 can be of the same type or different types. For example, these electronic devices 301 include, but are not limited to, mobile phones, tablets, personal computers (PCs), personal digital assistants (PDAs), smartwatches, netbooks, wearable electronic devices, and in-vehicle devices. This application does not impose any special restrictions on the specific form of the electronic device 301. The electronic devices can communicate directly or indirectly, for example, through communication forwarding via a server 302.

[0078] The electronic device 301 has one or more conferencing applications (or conferencing software, conferencing plugins, etc.) installed on it, which can enable multi-person conversations. Users can conduct meetings through the conferencing applications on the electronic device. In this embodiment, the meeting can be a telephone conference or a web conference, and web conferences include, but are not limited to, voice conferences and video conferences.

[0079] During the meeting, electronic device 301 can acquire relevant information from the meeting (e.g., audio data of meeting participants) and generate meeting minutes based on this information. Optionally, electronic device 301 can also send the acquired meeting information to server 302, which will then generate the meeting minutes. Optionally, server 302 can also send the generated meeting minutes to each electronic device 301.

[0080] Figure 4An embodiment of this application illustrates a conference system 400. For example... Figure 4 As shown, the conference system 400 includes one or more electronic devices 301 and one or more conference room systems 401. For a description of the electronic devices 301, please refer to [reference needed]. Figure 3 Related information from China.

[0081] The conference room system 401 includes one or more electronic devices 402. Exemplary examples include, but are not limited to, various devices such as tablets, personal computers (PCs), personal digital assistants (PDAs), and netbooks. Electronic devices 300 and 402 can be of the same type or different types; this application does not impose any special limitations on the specific form of electronic devices 402. Communication between the conference room systems 401, between the electronic devices 301, and between the online conference room system 401 and the electronic devices 301 can be direct or indirect, for example, through communication forwarding via server 302.

[0082] One or more conference applications are installed on both electronic devices 301 and 402. These applications can be used to enable online meetings, allowing members of the conference room system to conduct meetings with users of other electronic devices 301 through the conference application installed on electronic device 402.

[0083] During the meeting, electronic devices 301 and 402 can acquire relevant information from the meeting (e.g., audio data of meeting participants) and generate meeting minutes based on this information. Optionally, electronic devices 301 and 402 can also send the acquired meeting information to server 302, which will then generate the meeting minutes. Optionally, server 302 can also send the generated meeting minutes to electronic devices 301 and 402.

[0084] Optionally, the conference system 400 may also include one or more display devices 403, such as, but not limited to, projectors, motorized screens, and other devices with display functions. These display devices 403 can be used to display conference images on a large screen. In some embodiments of this application, the display device 403 can also be used to display generated meeting minutes.

[0085] It should be understood that Figure 3 , Figure 4 The above diagram is a simplified example for ease of understanding. In actual applications, the above conference system may include other devices, which are not shown in the diagram.

[0086] The solutions provided in this application can be applied to various electronic devices, such as... Figure 3 The electronic device 301 shown Figure 4 In the electronic device 402 shown. Taking a mobile phone as an example, Figure 5 A schematic diagram of the electronic device is shown.

[0087] The electronic device may include a processor 510, an external memory interface 520, an internal memory 521, a universal serial bus (USB) interface 530, a charging management module 540, a power management module 541, a battery 542, an antenna 1, an antenna 2, a mobile communication module 550, a wireless communication module 560, an audio module 570, a sensor module 580, a button 590, a motor 591, an indicator 592, a camera 593, a display screen 594, and a subscriber identification module (SIM) card interface 595, etc.

[0088] Processor 510 may include one or more processing units, such as application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0089] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0090] The processor 510 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 510 is a cache memory. This memory can store instructions or data that the processor 510 has just used or that are used repeatedly. If the processor 510 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 510, and thus improves the efficiency of the system.

[0091] In some embodiments, processor 510 may include one or more interfaces.

[0092] The charging management module 540 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger.

[0093] The power management module 541 is used to connect the battery 542, the charging management module 540, and the processor 510. The power management module 541 receives input from the battery 542 and / or the charging management module 540 to power the processor 510, internal memory 521, display 594, camera 593, and wireless communication module 560, etc.

[0094] The wireless communication function of electronic devices can be realized through antenna 1, antenna 2, mobile communication module 550, wireless communication module 560, modem processor and baseband processor, etc.

[0095] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0096] The mobile communication module 550 can provide solutions for wireless communication applications, including 2G / 3G / 4G / 5G, in electronic devices. The mobile communication module 550 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc.

[0097] The wireless communication module 560 can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0098] In some embodiments, the antenna 1 of the electronic device is coupled to the mobile communication module 550, and the antenna 2 is coupled to the wireless communication module 560, enabling the electronic device to communicate with networks and other devices via wireless communication technology.

[0099] In some embodiments of this application, each electronic device in the conference system can establish a communication connection through the aforementioned mobile communication module 550 and / or wireless communication module 560, etc.

[0100] Electronic devices implement display functions through a GPU, a display screen 594, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 594 and the application processor. The GPU performs mathematical and geometric calculations and is used for graphics rendering. The processor 510 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0101] Display screen 594 is used to display images, videos, etc. Display screen 594 includes a display panel. In some embodiments, the electronic device may include one or N displays screens 594, where N is a positive integer greater than 1. In some embodiments of this application, display screen 594 can be used to display generated meeting minutes.

[0102] Electronic devices can achieve shooting functions through ISP, camera 593, video codec, GPU, display 594 and application processor.

[0103] Camera 593 is used to capture still images or videos. In some embodiments, the electronic device may include one or N cameras 593, where N is a positive integer greater than 1.

[0104] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0105] The external storage interface 520 can be used to connect external memory cards, such as Micro SD cards, to expand the storage capacity of electronic devices.

[0106] Internal memory 521 can be used to store executable program code, including instructions. Internal memory 521 may include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, phonebook, etc.). Processor 510 executes various functional applications and data processing of the electronic device by running instructions stored in internal memory 521 and / or instructions stored in memory located within the processor.

[0107] Electronic devices can implement audio functions through audio modules 570 and application processors, such as music playback and recording.

[0108] The audio module 570 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 570 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 570 may be located in the processor 510, or some functional modules of the audio module 570 may be located in the processor 510.

[0109] Optionally, the sensor module 580 may include one or more of the following: pressure sensor 180A, gyroscope sensor 180B, barometric pressure sensor 180C, magnetic sensor 180D, accelerometer sensor 180E, distance sensor 180F, proximity sensor 180G, fingerprint sensor 180H, temperature sensor 180J, touch sensor 180K, ambient light sensor 180L, bone conduction sensor 180M, etc.

[0110] Buttons 590 include a power button, volume buttons, etc. Buttons 590 can be mechanical buttons or touch-sensitive buttons. Electronic devices can receive button input and generate key signal inputs related to user settings and function control.

[0111] Motor 591 can generate vibration alerts.

[0112] Indicator 592 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0113] The SIM card interface 595 is used to connect a SIM card.

[0114] For example, the above description uses a mobile phone as an example to illustrate the structure of the electronic device in this application embodiment, but it does not constitute a limitation on the structure or form of the electronic device. This application embodiment does not limit the structure or form of the electronic device. For example, Figure 6 Another exemplary structure of an electronic device is shown. For example... Figure 6 As shown, the electronic device includes: a processor 601, a memory 602, a transceiver 603, and an input / output device 604. The implementation of the processor 601 and memory 602 can be found in the implementation of a mobile phone's processor and memory. The transceiver 603 is used for interaction between the electronic device and other devices. The transceiver 603 can be a device based on protocols such as Wi-Fi, Bluetooth, or other communication protocols. The input / output device 604 can be used to input information to the electronic device or to output information to the electronic device. For example, the input / output device 604 can be a display, an audio player, a speaker, etc., which can be used to output meeting minutes to the electronic device.

[0115] In other embodiments of this application, the electronic device may include a ratio Figure 5 , Figure 6 The diagram shows more or fewer components, or combinations of components, or splitting of components, or replacement of components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of both.

[0116] The software system of electronic devices can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This invention embodiment uses a layered architecture. Taking the system as an example, the software structure of the electronic device is illustrated.

[0117] Figure 7 This is a software structure block diagram of an electronic device according to an embodiment of the present invention.

[0118] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0119] The application layer can include a series of application packages.

[0120] like Figure 7 As shown, the application package can include applications such as calendar, map, WLAN, SMS, gallery, and navigation.

[0121] Optionally, in some embodiments of this application, the above-mentioned application package may further include a conferencing application, which can be used to implement online conferencing functionality. Optionally, the conferencing application may be a pre-installed application or an application downloaded from a third-party app store. The embodiments of this application do not limit the specific implementation of the conferencing application.

[0122] In some embodiments of this application, during a meeting using a conferencing application, the conferencing application can output or input audio data, and an electronic device can use this audio data to generate meeting minutes.

[0123] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0124] like Figure 7As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0125] Optionally, in some embodiments of this application, the framework layer may further include modules for generating meeting minutes, such as, but not limited to, a voice activity detection (VAD) module, an automatic speech recognition (ASR) module, a summary generation module, and an information arrangement module. The technical implementation and purpose of each module can be found below. It should be noted that the software layers to which the aforementioned modules for generating meeting minutes belong are merely illustrative examples and may also be located in other software layers, such as the kernel layer; this application does not specifically limit their location.

[0126] Optionally, the framework layer may also include other interfaces or modules required to implement the technical solutions of the embodiments of this application.

[0127] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0128] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0129] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0130] A phone manager is used to provide communication functions for electronic devices. For example, it manages call status (including connection and disconnection).

[0131] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0132] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, and flashing indicator lights.

[0133] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0134] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0135] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0136] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0137] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0138] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0139] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0140] A 2D graphics engine is a graphics engine for 2D drawing.

[0141] The kernel layer is the layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver. In some embodiments of this application, after the meeting minutes are generated, the display driver can be used to drive the screen of an electronic device to display the meeting minutes.

[0142] The technical solutions involved in the following embodiments can all be implemented in a manner that allows for the application of [missing information - likely related to technology or technology]. Figure 5 ,like Figure 6 , Figure 7 Implemented in the device with the structure shown.

[0143] Taking a scenario where a user conducts a meeting using a conferencing application installed on their mobile phone as an example, this application provides a method for generating meeting minutes. The mobile phone can obtain relevant information about the meeting and generate corresponding meeting minutes based on that information. Optionally, the mobile phone can also control the display screen to show the meeting minutes.

[0144] Figure 8 This illustrates a process for a user to start a meeting using a meeting application installed on their mobile phone. For example, such as... Figure 8 As shown in (1), the mobile phone displays a main interface 800, which includes one or more applications. Different applications can be used to achieve the same or different functions, including a meeting application 801, which can be used to conduct online meetings.

[0145] Users can directly join the meeting using their meeting account or password. For example, if the phone detects an action such as the user clicking the icon of the meeting application 801 to launch the application, it will respond to that action as follows: Figure 8 As shown in (2), the mobile phone can display a meeting login interface 810. The meeting login interface 810 includes a meeting account input area 811 and a user information input area 812. The meeting account input area 811 can be used to input the meeting account, and the user information input area 812 can be used to input the user's identity identifier, such as, but not limited to, name, employee ID, meeting application account name, nickname, and other types of user information, so that meeting members can identify each other. Optionally, the meeting login interface 810 may also include a join button 813. When the mobile phone detects that the user has entered the meeting account "123456" and user information "Xiaoming" in the meeting login interface 810 and clicked the join button 813, it responds to this operation as follows: Figure 8 As shown in (3), the mobile phone displays the meeting interface 820. Optionally, in some embodiments, before the mobile phone displays the meeting interface 820, the user may need to enter a meeting password before joining the meeting.

[0146] The meeting interface 820 includes one or more of the following: a main display area 821, a menu bar 822, and a topic bar 823. In some embodiments, the main display area 821 can be used to display relevant information of the current meeting members, such as: image information (e.g., facial information captured by a mobile phone in a video conferencing scenario), user information (e.g., name, employee ID, meeting application account name, nickname, etc.). Figure 8 The user information entered in the user information input area 812 of (2) is such as user information. In some other embodiments, the main display area 821 can also be used to display materials shared by meeting members through their mobile phone screens. The menu bar 822 includes some commonly used settings, such as one or more of the following: mute, video, share, participants, more. The implementation and function of these commonly used settings can be referred to the prior art. The topic bar 823 can be used to display the meeting account of the current meeting, the user information of the creator of the current meeting, etc.

[0147] Optional, at the beginning Figure 8 Before the meeting shown in (3), the user may need to book the meeting through the meeting application 801. For example, Figure 9 This shows some of the interfaces related to users booking meetings through the 801 conferencing application. For example... Figure 9 As shown in (1), the mobile phone displays an interface 900, which includes a meeting reservation option 901. Users can reserve meetings through the meeting reservation option 901. Optionally, the meeting reservation interface may also include other options, such as one or more of the following: start a meeting 902, join a meeting 903, and meeting room 904. Among them, start a meeting 902 can be used to start a meeting immediately. Join a meeting 903 can be used to enter a meeting, for example, by entering a meeting account to enter the meeting corresponding to that meeting account. Meeting room 904 can be used to reserve a meeting location, etc. The mobile phone detects operations such as the user clicking on the meeting reservation option 901 and responds to the operation as follows: Figure 9 As shown in (2), the mobile phone displays a meeting reservation interface 910. Optionally, the meeting reservation interface 910 can select one or more of the following: meeting type (e.g., video conference, audio conference, offline conference, etc.), meeting start time, meeting duration, meeting account, participants, and meeting location. Optionally, the meeting reservation interface 910 may also include a meeting reservation button 911. The mobile phone detects, for example, user actions on the meeting reservation button 911 and responds to the action, such as, for example, Figure 9 As shown in (3), the mobile phone can display a meeting reservation success interface 920, which may include meeting records 921 that the user has successfully reserved. Optionally, the meeting records 921 may include one or more of the following: meeting date, meeting start time, meeting end time, and meeting reservation person. For a description of other content in the meeting reservation success interface 920, please refer to the corresponding description in interface 900.

[0148] like Figure 10As shown, during a meeting conducted by a user through the conferencing application 801, the mobile phone can acquire audio data (or audio stream) from the meeting, which includes the voice data of the meeting participants. Optionally, the mobile phone can acquire the audio data from the meeting in real time or periodically. In some embodiments, the mobile phone can input the acquired audio data from the meeting into the VAD module in real time. In other embodiments, the mobile phone can also first store the acquired audio data from the meeting in a storage module (e.g., memory, cache, etc.) and then send the audio data from the storage module to the VAD module later. Optionally, the VAD module may not be used to process the audio data.

[0149] Accordingly, after receiving the audio data from the meeting, the VAD module processes it. Since the audio data may include silence data, the VAD module can use VAD technology to locate the start and end points of the voice data, separating the voice data from the silence data to obtain the voice data and its corresponding time endpoint information. For example, the voice data and corresponding time endpoint information obtained by the VAD module can be as follows: Figure 10 As shown, for example, the time endpoint information for voice data 1 is: start time: 10:11, end time: 10:20. The time endpoint information for voice data 2 is: start time: 10:21, end time: 10:30. The time endpoint information for voice data 3 is: start time: 10:35, end time: 10:45, and so on. For instance, the VAD module can use a neural network VAD model to process audio data. Currently, the VAD module can also use other VAD algorithms to process audio data, and this application does not specifically limit this.

[0150] In some embodiments, voice data is associated with (or corresponding to) meeting members (or user identifiers (IDs)). Figure 11 As shown, voice data 1 belongs to meeting member 1, voice data 2 belongs to meeting member 2, voice data 3 belongs to meeting member 1, and so on. For example: Figure 8 Taking the meeting interface 820 shown in (3) as an example, each meeting member corresponds to a meeting ID in the meeting. When the mobile phone obtains the audio data in the meeting, it can obtain the audio data under each user ID. In this way, the obtained audio data is associated with the user ID. The voice data obtained later based on the audio data is also associated with the user ID. Since the user ID is associated with the meeting member, the voice data is also associated with the meeting member.

[0151] The VAD module sends the acquired speech data to the ASR module, which then processes the speech data using ASR technology to obtain text data. Optionally, the text data may also carry time endpoint information generated by the VAD module. For example, the ASR module may use a neural network ASR model to process the speech data; of course, the ASR module may also use other ASR algorithms to process the speech data, and this application does not specifically limit this.

[0152] For example, the ASR module supports Figure 11 The voice data 1 shown can be processed to obtain the text data "Currently, many parameters have not yet met the acceptance criteria. There is still a lot of work to be arranged before the delivery time. What are the detailed plans for each department?". Optionally, the text data can also carry time endpoint information, which is the time endpoint information of voice data 1, such as: start time: 10:11 and / or end time: 10:20, etc.

[0153] In some embodiments, the ASR module can send the generated text data to the summary generation module in real time, and correspondingly, the summary generation module can receive the text data sent by the ASR module in real time. In this embodiment, after the summary generation module receives the text data, it can first store it. When the stored text data meets certain conditions (e.g., the quantity meets certain conditions, the storage time meets certain conditions, etc.), the meeting summary is then generated using the text data that meets the conditions.

[0154] In other embodiments, the ASR module can first store the generated text data, for example, in a text cache module. Then, the cached text data in the text cache module can be input into the summary generation module for generating a conference summary. Optionally, the text cache module can input the cached text data into the summary generation module using methods such as Method 1 or Method 2. Optionally, after this text data is input into the summary generation module, the data in the text cache module can also be cleared.

[0155] Method 1: The text data cached in the text caching module can be sent to the summarization module in a quantitative manner, and correspondingly, the summarization module can receive the text data from the text caching module in a quantitative manner. For example, when the text data stored in the text caching module meets a preset threshold, this text data is input into the summarization module.

[0156] In some embodiments, the number of statements in the text data in the text caching module can be used to determine whether the text data meets a preset threshold. For example, the text data cached in the text caching module may include sentence break symbols, including but not limited to ".。!?", etc. These sentence break symbols can be used to break the text data cached in the text caching module into sentences, so as to determine whether the number of statements in the text data in the text caching module meets the preset threshold. Optionally, such as... Figure 10 As shown, this sentence segmentation operation can be performed by a sentence segmentation module, which can be located between the text caching module and the summary generation module. Of course, this sentence segmentation module can also be integrated with the text caching module or the summary generation module into a single module; this application does not limit this approach.

[0157] For example, Figure 12 This diagram illustrates a process of inputting text data from a text caching module into a summary generation module using sentence segmentation symbols. Optionally, this process can be performed by the sentence segmentation module.

[0158] S1201, Search for the next character in the text data cached by the text cache module.

[0159] The sentence segmentation module can sequentially traverse each character in the text data cached in the text caching module.

[0160] For example, the character includes text, punctuation marks, etc.

[0161] S1202. Determine whether the current character is a sentence break symbol.

[0162] If yes, proceed to step S1203; otherwise, proceed to step S1201.

[0163] S1203. Add the entire sentence corresponding to the character to the list.

[0164] It can be understood that the sentence between two punctuation marks is the complete sentence (or sentence) corresponding to the subsequent punctuation mark. This list can be located inside the text cache module, or inside the punctuation module, etc.

[0165] For example, taking the text data cached in the text caching module as "Every animal has its own language, and so do machines! NLP is the bridge between machine language and human language to achieve human-machine communication. Do machines also have their own ways of communicating?" as an example, Table 1 shows some complete sentences in the list.

[0166] Table 1

[0167] Every animal has its own language, and so do machines! NLP acts as a bridge between machine language and human language to achieve human-computer interaction. Do machines have their own ways of communicating?

[0168] S1204. Determine whether the number of complete sentences in the list meets the preset threshold.

[0169] If yes, proceed to step S1205. If no, proceed to step S1201.

[0170] Optionally, the preset threshold can be set by the developers according to actual needs, such as 8, 10, 15, 20, etc. This application does not make specific limitations on this.

[0171] S1205. Input the complete sentences from the list into the summary generation module.

[0172] It should be noted that the text data in the input summary generation module can be in the form of a list or other forms, and this application does not make any specific restrictions on this.

[0173] In other embodiments, the text data of the text cache module can be determined by judging the capacity of the text cache area (e.g., used capacity, remaining capacity, etc.).

[0174] It should be noted that this application does not impose any limitations on the specific method for determining whether text data meets the preset threshold.

[0175] Similarly, in an embodiment where the ASR module sends the generated text data to the summary generation module in real time, the text data cached in the summary generation module can also be judged to meet the preset conditions according to the method described in Method 1.

[0176] Method 2: The text caching module can periodically send cached text data to the summary generation module, and correspondingly, the summary generation module can periodically receive text data from the text caching module. For example, the text caching module can send text data to the summary generation module at regular intervals, and the sent text data can be the text data cached by the text caching module during that period.

[0177] It should be noted that this application does not limit the timing, quantity, or method of the text caching module sending text data to the summary generation module.

[0178] Accordingly, the summary generation module generates a conference summary based on the domain sentence structure library determined by the prior knowledge extraction module and the text data obtained from the text caching module. A conference summary can be understood as text data that has been simplified, key points extracted, and summarized. For example, summarizing a text with 50 sentences into 5 sentences.

[0179] The following is an introduction to the domain-specific syntax library.

[0180] In the embodiments of the present application, the domain sentence pattern library refers to a sentence pattern library composed of sentence patterns corresponding to the meeting domain. A sentence pattern can refer to the arrangement of words, words (such as nouns, verbs, etc.), words and short sentences, short sentences, etc. The arrangement can include the order of arrangement, as well as the number of times and probability of the order of arrangement. Exemplarily, the domain sentence pattern library can be in the form of an entity list or in the form of a language model (such as a statistical language model, a neural network language model, etc.), and the present application does not limit this.

[0181] Table 2 shows an example of the domain sentence pattern library represented in the form of an entity list.

[0182] Table 2

[0183]

[0184] Among them, the meaning of Table 2 is how many times the subsequent words of a word appear. For example, the number of times "的" appears after "是" is 2 times, and the number of times "气" appears after "天" is 5 times, etc. The subsequent words can be understood as one or more consecutive words located after and adjacent to this word.

[0185] Table 3 shows an example of the domain sentence pattern library represented in the form of a language model.

[0186] Table 3

[0187]

[0188] Among them, the meaning of Table 3 is what the probability of the subsequent words of a word is. For example, "P(的|是)=0.40" means that the probability of "的" appearing after "是" is 0.40, and "P(气|天)=0.08" means that the probability of "气" appearing after "天" is 0.08, etc. <当该词语后接词语出现概率为多少,比如:“P(的|是)=0.40表示“是”后接“的”出现概率为0.40,“P(气|天)=0.08”表示“天”后接“气”出现概率为0.08等。It can be understood that different meeting domains may have different proper nouns, terms, sentence expression methods, etc. Therefore, different meeting domains can have different domain sentence pattern libraries. It can be understood that the meeting domain can refer to the domain to which the content discussed in a meeting belongs.

[0190] Exemplarily, the meeting domain can include general domain, medical domain, financial domain, electronic technology, energy domain, agricultural domain, mechanical manufacturing domain, construction domain, etc. It can be understood that if the meeting domain corresponding to a meeting is the general domain, it means that the content discussed in this meeting can belong to various domains (such as other domains except the general domain), that is, these contents can be included in each domain.

[0191] The medical field includes: clinical medicine, forensic medicine, preventive medicine, rehabilitation medicine, health care medicine, psychology, pathology, genetics, human histology, surgery, internal medicine, stomatology, oncology, respiratory diseases, human immunology, and medical imaging.

[0192] The financial sector includes: banking, securities, insurance, trust, investment, finance, accounting, leasing, stocks, funds, and trade.

[0193] The field of electronic technology includes big data, artificial intelligence, the Internet of Things (IoT), the Internet, cloud computing, connected vehicles, communications, machine learning, and autonomous driving. The communications field includes mobile communications, optical communications, microwave communications, satellite communications, WiFi, Bluetooth, radio frequency (RF), Ethernet, natural language processing (NLP), and automatic speech recognition (ASR).

[0194] The energy sector includes: oil, natural gas, and coal.

[0195] The agricultural sector includes: horticulture, animal husbandry, breeding, cultivation, aquaculture, soil and fertilizer, marine industries, and agricultural product processing.

[0196] The machinery manufacturing sector includes: vehicle manufacturing, textile manufacturing, chemical manufacturing, metallurgy, electrical manufacturing, etc.

[0197] The construction field includes: water supply and drainage, civil engineering, road and transportation, bridges, and river crossings.

[0198] It should be noted that the above-mentioned fields are merely illustrative examples for the purpose of understanding this application. In practical applications, other fields may also be included, and the above-mentioned fields may be further classified.

[0199] It should be noted that, in this embodiment, the operation of building the domain syntax library can be performed by a mobile phone. Alternatively, it can be performed by other devices (such as servers), and then the other devices send the built domain syntax library to the mobile phone. The following example illustrates the operation of building the domain syntax library using a mobile phone.

[0200] Optionally, the domain-specific syntax library can be pre-built. For example, the domain-specific syntax library can be built before the conference begins, or it can be built before the summary generation module starts working; this application does not limit this. Optionally, the domain-specific syntax library can also be updated periodically; this application also does not limit the timing of such updates.

[0201] Taking the domain syntax library as an example, which uses a language model to represent the domain syntax library, the following is an implementation method for building such a domain syntax library.

[0202] Take the n-gram language model (or Chinese language model, CLM) at the Chinese character level as an example.

[0203] like Figure 13 As shown, assume the number of meeting domains is K, where K is a positive integer. For each meeting domain, the mobile phone acquires a large amount of text data within that domain. For example, this text data can be in various forms such as reports, documents, professional books, project specifications, meeting minutes, articles, manuals, and newspapers; this application does not impose any special restrictions on the type of text data. For example, the mobile phone can acquire the text data corresponding to each domain from the network, other devices, the cloud, databases, etc. Optionally, the text data acquired by the mobile phone can be directly in text format, or it can be text data obtained by converting other formats (such as audio data, video data, etc.) by the mobile phone; this application does not impose any limitations on this. This application also does not specifically limit the technical means used by the mobile phone to convert non-text data into text data.

[0204] For example, Table 4 shows some text data corresponding to some conference areas.

[0205] Table 4

[0206]

[0207]

[0208] It should be noted that the text data corresponding to the above conference fields are merely illustrative examples for the purpose of understanding this application and do not constitute a limitation on this application.

[0209] For each meeting domain, assuming the ASR module in this embodiment supports N Chinese characters, that is, the text data converted by the ASR module may contain N types of Chinese characters, where N is a positive integer. For these N Chinese characters, such as... Figure 13As shown, the mobile phone counts the probability of the subsequent Chinese characters of each Chinese character among some or all of the N Chinese characters according to the obtained text data corresponding to the meeting field. It can be understood that the subsequent Chinese characters of a Chinese character refer to one or more consecutive Chinese characters adjacent to and following the Chinese character. Exemplarily, taking the text data corresponding to the meeting field as "Coordinated development of assets and liabilities, continuous deepening of strategic transformation" as an example, assuming that the aforementioned N Chinese characters include "war", the subsequent Chinese characters of "war" can include "strategy", "strategy transformation", "strategy transformation continuously", etc. The present application does not limit the specific number of subsequent Chinese characters of a Chinese character. Optionally, the types of Chinese characters included in the field sentence pattern library can also be more than N or less than N, and the present application does not limit this.

[0210] Generate the field sentence pattern library corresponding to the meeting field according to the subsequent Chinese characters of each Chinese character and the probability corresponding to each subsequent Chinese character.

[0211] Exemplarily, as Figure 14 shown, in a meeting field, taking the Chinese character "is" as an example, the mobile phone counts the probability of the subsequent Chinese characters of "is" appearing in the text data corresponding to the meeting field. The mobile phone counts that the subsequent Chinese characters of "is" are "of", "me", "true", "no", etc., which appear 40 times, 32 times, 8 times, and etc. 7 times respectively. The mobile phone calculates the probability of the subsequent Chinese character appearing according to the number of times the subsequent Chinese character appears.

[0212] Exemplarily, the probability P of each subsequent Chinese character in the subsequent Chinese characters of a Chinese character can be calculated according to Equation 1.1.

[0213]

[0214] Among them, a is the number of times the subsequent Chinese character appears, and b is the total number of times all the subsequent Chinese characters of the Chinese character appear. For example: taking the Chinese character "I" as an example, the subsequent Chinese characters of "I" include "we", "of", "you", "like", "hobby" as examples. Among them, "we", "of", "you", "like", "hobby" appear 5 times, 6 times, 3 times, 2 times, and 1 time respectively. Then the probability of "we" appearing can be calculated according to Equation 1.1 The probability of "of" appearing can be calculated according to Equation 1.1 etc.

[0215] The field sentence pattern library generated by the mobile phone according to the subsequent Chinese characters of each Chinese character and the probability corresponding to each subsequent Chinese character can be as Figure 14As shown in 1400. Among them, "P(is|of)=0.40" means that the probability of "of" following "is" is 0.04, "P(me|is)=0.32" means that the probability of "me" following "is" is 0.32, "P(true|is)=0.08" means that the probability of "true" following "is" is 0.08, "P(no|is)=0.32" means that the probability of "no" following "is" is 0.07, etc.

[0216] Figure 14 The examples in it are illustrated by taking the number of Chinese characters following a Chinese character as one. The number of Chinese characters following can also be multiple, and this application does not limit this. For example: when the number of Chinese characters following is two, "P(of me|is)=0.02" means that the probability of "of me" following "is" is 0.02. Another example: when the number of Chinese characters following is three, "P(of us|is)=0.01" means that the probability of "of us" following "is" is 0.01.

[0217] It should be noted that this application does not specifically limit the number of Chinese characters to be followed either. For example: [[ID=⑧]] Figure 14 In the example of, "is" is called the Chinese character to be followed. This example is illustrated by taking the number of Chinese characters to be followed as 1. The number of Chinese characters to be followed can also be multiple. For example: when the number of Chinese characters to be followed is two, "P(me|you are)=0.02" means that the probability of "me" following "you are" is 0.02. Another example: "P(of me|you are)=0.02" means that the probability of "of me" following "you are" is 0.02.

[0218] The above implementation method of constructing the domain sentence pattern library is illustrated by taking the language model as a Chinese character-level n-gram language model. It should be noted that this n-gram language model can also be a model of other languages, such as: English, Japanese, Korean, etc., and this application does not limit this.

[0219] Optionally, the domain sentence pattern library can be represented in the form of a matrix (or an array). For example: for the probability of Chinese characters following the N Chinese characters statistically in the meeting domain, it can be saved as an N-dimensional array. Each row in this array represents the probability of Chinese characters following the corresponding Chinese character. This N-dimensional array is the domain sentence pattern library corresponding to this domain. Optionally, the dimension of this array can be N×N. For K meeting domains, K N×N-dimensional arrays can be formed. Optionally, the domain sentence pattern library can also be represented in the form of a set, and this application is not limited to this.

[0220] Next, the working principle of the prior knowledge extraction module will be introduced.

[0221] Such as Figure 15As shown, the prior knowledge extraction module can determine the domain phrase library corresponding to the meeting based on prior information related to the meeting obtained from the mobile phone. For example, the prior information includes one or more of the following: schedule information, chat information, email information, etc. For example, the schedule information can be as follows: Figure 15 The information shown in 1501 indicates that chat messages can be as follows: Figure 15 The information shown in 1502, email information can be as follows: Figure 15 The information shown is 1503, etc. Optionally, this prior information may have been generated before the meeting began, for example, based on user input... Figure 8 The relevant operations in the meeting application 801 shown in (1) are generated.

[0222] For example, with Figure 8 Taking the meeting application 801 shown in (1) as an example, for Figure 15 The schedule information in the example is illustrated below. Figure 16 As shown in Figure (1), the mobile phone displays the message interface 1600 of the meeting application 801. The message interface 1600 includes a schedule creation button 1601, which can be used by the user to create a meeting schedule. Optionally, the message interface 1600 may also include one or more contacts. Optionally, the message interface 1600 may also include other buttons, options, etc., such as: function button 1602, address book, business, knowledge, etc. Since these buttons, options, etc. are existing technologies, their specific implementation and functions can refer to existing technologies. The mobile phone detects the user's operation to create a meeting schedule, such as: the user's click operation on the schedule creation button 1601, and responds to the operation as follows: Figure 16 As shown in (2), the mobile phone displays a schedule creation interface 1610, which includes a calendar 1611 and a schedule creation button 1612. The calendar 1611 is used by the user to select the date for which the schedule is to be created. The mobile phone detects actions such as the user clicking the schedule creation button 1612 and responds to the action as follows: Figure 16 As shown in (3), the mobile phone displays a schedule creation interface 1620, which includes one or more of the following related to the meeting: topic, type, start time, end time, participants, and content. Users can set these parameters according to their actual needs. Optionally, the schedule creation interface 1620 also includes a complete button 1621 and / or a cancel button 1622. The complete button 1621 can be used to complete the operation of creating a meeting schedule, and the cancel button 1622 can be used to cancel the operation of creating the meeting schedule. When the mobile phone detects an operation such as the user clicking the complete button 1621, the mobile phone completes the operation of creating the meeting schedule. For example, Figure 15 The schedule information in the middle can be Figure 16The information contained in the meeting schedule created by the user shown in (3) (e.g., meeting content, meeting topic, etc.).

[0223] Or with Figure 8 Taking the meeting application 801 shown in (1) as an example, for Figure 15 The following example illustrates the chat information provided. Figure 17 As shown in (1), the mobile phone displays a message interface 1600, which includes one or more contacts. Users can send and receive messages with these contacts. For a description of other content in the message interface 1600, please refer to [reference needed]. Figure 16 The relevant introduction. Suppose a user wants to discuss meeting matters with Xiaohong. The user can click on Xiaohong's displayed location. The phone detects this action, such as initiating a chat with a contact, and responds to the user's action, such as... Figure 17 As shown in (2), the mobile phone displays a chat interface 1700, which can be used to display chat information between the user and Xiaohong. For example, Figure 15 The chat information in the chat interface 1700 can be the chat information included in the chat interface.

[0224] Or with Figure 8 Taking the meeting application 801 shown in (1) as an example, for Figure 15 The email information in the example is explained below. Figure 18 As shown in (1), the mobile phone displays the email interface 1800 of the meeting application 801, which includes multiple emails. Optionally, these emails may include emails sent by the user and / or emails received by the user. Optionally, the email interface 1800 may also include a new button 1801, which can be used by the user to create emails, etc. The mobile phone detects operations such as user clicks on the display location of email 1603, and responds to the operation as follows: Figure 18 As shown in (2), the mobile phone can display an email details interface 1810, which may include the content of email 1603. For example, Figure 15 The email information in the email details page 1810 can be the email content included in the email details page.

[0225] It's understandable that the above processes for creating a schedule, launching the chat interface, and viewing emails are just examples; other processes can also be used. Figure 8 The meeting application 801 shown in (1) can create schedules, start chat interfaces, and view emails, but this application does not limit this.

[0226] It should be noted that this prior information can be obtained from... Figure 8The information can be obtained from the meeting application 801 shown in (1), or from other applications installed on the mobile phone, or from other devices (such as servers, cloud, etc.). This application does not limit the source of this prior information or the specific method of obtaining it.

[0227] The following describes a specific implementation of a prior knowledge extraction module that determines the domain syntax library corresponding to a meeting based on prior information related to the meeting.

[0228] In some embodiments, the prior knowledge extraction module matches the prior information of the meeting with some or all of the domain sentence structures in the K meeting domains mentioned above, thereby determining the domain sentence structure library corresponding to the meeting. For example, the prior knowledge extraction module calculates the matching probability between the prior information of the meeting and the domain sentence structure library, and uses the domain sentence structure library corresponding to the matching probability that meets the preset conditions as the domain sentence structure library corresponding to the meeting.

[0229] For example, the matching probability between the prior information of the meeting and the domain phrase library can be determined by taking the probabilities of any number of consecutive words (or characters, or terms, etc.) included in the prior information of the meeting in the domain phrase library. Optionally, the matching probability can be obtained by multiplying the probabilities of these multiple words in the domain phrase library.

[0230] Based on the prior information of the meeting Figure 15 Taking information 1501 as an example, where two consecutive words are present, any two adjacent words in the prior information of this meeting include "communication," "connection," "side contact," "side words," ... "investment," etc. For example, the probability of "communication" in a domain phrase library can be expressed as the probability of "communication" followed by "connection" in that domain phrase library, which can be represented as "P(communication) = P(connection|channel)." The probability of "connection" in a domain phrase library is the probability of "connection" followed by "connection" in that domain phrase library. The meaning of the probabilities of "side contact," "side words," ... "investment," etc., in a domain phrase library is similar.

[0231] like Figure 19 As shown, the matching probability of the prior information of this meeting in meeting domain 1 is: P(communication|channel)*P(terminal|communication)*…=0.05. The matching probability in meeting domain 2 is: P(communication|channel)*P(terminal|communication)*…=0.04. The matching probability in meeting domain k is: P(communication|channel)*P(terminal|communication)*…=0.01, and so on. For example, the domain sentence structure library corresponding to the highest matching probability can be used as the domain sentence structure library for this meeting. Figure 19 In the example, it is assumed that the matching probability is highest for conference domain 1, so the domain sentence library corresponding to conference domain 1 is used as the domain sentence library for this conference.

[0232] Figure 19The example shown uses two consecutive words as illustration. The number of consecutive words can also take other values, for example, three consecutive words. Assuming the prior information of the meeting is... Figure 15 Information 1501 shows that any three adjacent words in the prior information of the meeting include "communication end", "communication side", "end-side domain", ... "effort input", etc. For example, the probability of "communication end" in a domain phrase library can be the probability of "channel" followed by "communication end" in that domain phrase library, which can be expressed as "P(communication end) = P(communication end|channel)", or it can be the probability of "communication" followed by "communication end" in that domain phrase library, which can be expressed as "P(communication end) = P(end|communication)". The meaning of the probability of "communication side", "end-side domain", ... "effort input" in a domain phrase library is similar.

[0233] It should be noted that the probability of any number of consecutive words included in the prior information corresponding to any given number of words in the domain phrase library depends on the specific implementation of the domain phrase library. For example, assuming the domain phrase library has *e* words (refer to the meaning of the Chinese characters following the *e* words) and *f* words (refer to the meaning of the Chinese characters following the *f* words), then the number of consecutive words should be the sum of *e* and *f*. Furthermore, the probability of such consecutive words in the domain phrase refers to the probability of any *e* consecutive words being followed by *f* consecutive words. *e* and *f* are both positive integers, and their specific values ​​can be set by the developers according to actual needs.

[0234] The following describes the specific process by which the summary generation module generates conference summaries based on the domain sentence structure library determined by the prior knowledge extraction module and the text data obtained from the ASR module.

[0235] In some embodiments, the summary generation module includes a machine model. The summary generation module can input received text data into the machine model, and then obtain the corresponding conference summary based on the output of the machine model and a domain syntax library. For example, the machine model can be a neural network model, or other types of machine models; this application does not limit this.

[0236] Optionally, the machine model can be obtained through training. For example, the machine model can be trained by taking text data from a meeting as input and a meeting summary (e.g., the meeting summary includes, but is not limited to, a manually summarized meeting summary, or a meeting summary automatically generated by a device, etc.) as output. Optionally, the text data can be obtained by converting audio data from the meeting, for example, by converting it through a VAD module or an ASR module, or it can be obtained directly; this application is not limited to these methods.

[0237] For example, Figure 20A structural schematic diagram of the machine model is shown. The machine model includes an encoding module (encoder) and a decoding module (decoder). Among them, the encoding module is used to encode text data to obtain an encoding vector corresponding to the text data. Exemplarily, the encoding vector can be a one-hot type vector or other forms of vectors, which are not specifically limited in this application. Taking the text data "NLP is the bridge between machine language and human language" as an example, its corresponding encoding vector can be [0, 1, 1, 1, 1, 1, 0, 0], and this encoding vector is output to the decoding module. The decoding module is used to decode the encoding vector from the encoding module to obtain the meeting summary corresponding to the text data.

[0238] In some embodiments, after the decoding module decodes the encoding vector from the encoding module, the probability of the meeting summary corresponding to the text data can be obtained. The probability of the meeting summary includes the probability of the candidate words corresponding to each word included in the meeting summary, and the probability of the candidate words corresponding to each word is not related to the words before that word.

[0239] In some other embodiments, after the decoding module decodes the encoding vector from the encoding module, it can output the probability of the candidate words corresponding to p words included in the meeting summary each time. Among them, these p words are related to the q words in the previously determined meeting summary, that is, the decoding module can predict the subsequent p words based on the previously output q words in the meeting summary. Both q and p are positive integers. Optionally, the q words can be the words adjacent to the p words and consecutive before the p words.

[0240] For example: as Figure 21 shown, taking the values of both p and q as 1 for example, assuming the word in the previously determined meeting summary is "is", then the probability that the decoding module determines the next word as "me" according to "is" is 0.40, that is, P(me) = 0.40; the probability of being "of" is 0.32, P(me) = 0.40; the probability of being "true" is 0.08, P(true) = 0.08; the probability of being "not" is 0.07, that is, P(not) = 0.07, etc. Among them, "me", "of", "true", "not" are all candidate words corresponding to the next word.

[0241] Another example: taking the value of p as "2" and the value of q as 1 for example, assuming the word in the previously determined meeting summary is "is", then the probability that the decoding module determines the next word as "mine" according to "is" is 0.05, that is, P(mine) = 0.05; the probability of being "true's" is 0.3, that is, P(true's) = 0.3, etc.

[0242] The abstract generation module updates the probabilities of the candidate words corresponding to the p words output by the decoding module, the e consecutive words in the conference abstract that are adjacent to and precede the p words, and the probabilities of the candidate words corresponding to the p words in the domain sentence structure library. For example, it performs a dot product between the probability of each candidate word corresponding to the p words output by the decoding module and the probability of the corresponding candidate word in the domain sentence structure library.

[0243] Taking the case where p and e both have a value of 1 as an example, such as Figure 21 As shown, the summary generation module performs a dot product between the P(I) = 0.40 output by the decoding module and the P(I|Is) = 0.32 in the domain sentence pattern; it performs a dot product between the P(True) = 0.08 output by the decoding module and the P(True|Is) = 0.08 in the domain sentence pattern library; and it performs a dot product between the P(Not) = 0.07 output by the decoding module and the P(Not|Is) = 0.07 in the domain sentence pattern library, and so on. Finally, the updated probabilities of P(I) are 0.128, P(True) are 0.064, and P(Not) are 0.049, etc.

[0244] It should be noted that the value of p is the same as the value of f mentioned above. The values ​​of p and q can be the same or different. The value of q can be the same or different from the value of e mentioned above. The value of q can be set by the developers according to actual needs.

[0245] The summary generation module can determine the final output word based on the probability of all candidate words; for example, it can select the candidate word with the highest probability as the final output word. For example: suppose... Figure 21 In the example, "I" has the highest probability, so the output is "I".

[0246] The abstract generation module can generate a conference abstract based on all the words in the final output. For example, the conference abstract generated by the abstract generation module can be like this: Figure 22 As shown in (1). For example, the conference abstracts generated by the abstract generation module can also be like... Figure 22 As shown in (2), Figure 22 The meeting summary shown in (2) is generated by the summary generation module. Figure 22 The meeting summary shown in (1) is generated by combining it with the meeting template. In one possible implementation, the meeting module can be a preset template. For example, the format of the preset module can be set by the developer according to actual needs, and this application does not specifically limit it. In another possible implementation, the meeting template can be based on the prior information of the meeting (such as...). Figure 15 The information shown (e.g., 1501) is generated.

[0247] It is understood that, in this embodiment of the application, during the meeting, the summary generation module can generate a meeting summary corresponding to that time period based on the text data obtained from the audio data over a certain period. For example, the summary generation module can generate a corresponding meeting summary from the text data between 10:00 and 10:05.

[0248] Optional, Figure 22 The time information included in the meeting summary shown can be based on the time endpoint information generated by the VAD module (such as...). Figure 11 The time endpoint information shown is determined. Optional, Figure 22 The meeting summary shown may not include time information, but it is arranged in the order of its generation to facilitate the subsequent generation of meeting minutes. For example, during the meeting, the audio data acquired by the VAD module is obtained in chronological order. Therefore, optionally, the text data generated by the ASR module can also be generated in chronological order. Since the summary generation module can receive the text data generated by the ASR module, it can also generate the meeting summary in chronological order.

[0249] The summary generation module inputs the meeting summary into the information orchestration module, which in turn receives the meeting summary from the summary generation module. The information orchestration module can then use the time endpoint information and meeting member information output by the VAD module (e.g., ...) Figure 8 The user information entered in the user information input area 812 shown in (2) is used to generate meeting minutes, along with a meeting summary. For example, the time endpoint information and meeting member information output by the VAD module can be as follows: Figure 23 As shown. Among them, regarding Figure 23 The time endpoint information and meeting member information shown are obtained by combining them with the meeting template. Alternatively, the meeting template can be used without a template. For an introduction to the meeting template, please refer to the relevant introduction in the meeting summary. It will not be repeated here.

[0250] Optionally, the information orchestration module can obtain the time endpoint information directly from the VAD module, or indirectly from other modules, such as the ASR module. This application does not limit this.

[0251] In some embodiments, the information encoding module can convert the time endpoint information output by the VAD module (such as...) Figure 23 )and Figure 22 The meeting minutes are generated by matching the time information in the meeting summary shown.

[0252] In other embodiments, if the meeting summary generated by the summary generation module does not contain time information but is arranged in chronological order, the information arrangement module can also match the order of the meeting summary with the chronological order of the time endpoint information input by the VAD module to generate meeting minutes.

[0253] For example, meeting minutes generated by the information orchestration module can be like this: Figure 24 As shown. It should be noted that, Figure 24 The meeting minutes shown are generated in conjunction with a meeting template. However, they may not be required. For more information on the meeting template, please refer to the relevant section on meeting templates in the meeting summary.

[0254] Optionally, the meeting minutes generated by the information orchestration module can also be output to the mobile phone screen for display. For example, Figure 25 This shows a diagram of meeting minutes displayed on a mobile phone. Figure 25 As shown, the meeting minutes 2500 can be displayed. Figure 8 The meeting interface 820 shown in (3) is optional. Optionally, the meeting interface 820 may also include a drop-down button 2501, which can be used to view details of the meeting minutes 2500. Optionally, the mobile phone can output the meeting minutes generated by the information arrangement module to the display screen in real time.

[0255] Optionally, meeting minutes can also be displayed on other interfaces on the phone. Figure 26 This shows a schematic diagram of yet another type of meeting minutes. (For example...) Figure 26 As shown in (1) of the table, in Figure 8 Based on the meeting interface 820 shown in (3), the mobile phone detects the user's click operation on the more button 2601, and responds to the operation as follows: Figure 26 As shown in (2), the mobile phone can display other functions, such as: meeting minutes 2602, chat 2603, invite attendees 2604, settings 2605, etc. Among them, meeting minutes 2602 can be used by the user to view the current meeting minutes, chat 2603 can be used to send messages to the current meeting members, invite attendees 2604 can be used to invite other people to join the current meeting, and settings 2605 can be used to set other aspects of the current meeting, such as: camera, etc. The mobile phone detects the user's click operation on meeting minutes 2602 and responds to the operation, such as Figure 26 As shown in (3), the mobile phone displays a meeting minutes display interface 2606, which includes the meeting minutes.

[0256] It should be noted that the above display location of the meeting minutes is only an example, and this application does not limit the specific display location of the meeting minutes.

[0257] Optionally, some or all meeting participants can also make changes to the meeting minutes displayed on their mobile phones. Figure 26 For example, the meeting minutes display interface 2606 shown in (3) is used as an example. Figure 27 As shown in (1), the meeting minutes display interface 2606 may also include an edit button 2700, which can be used by the user to modify the currently displayed meeting minutes. For example, if the mobile phone detects the user's click on the edit button 2700, in response to the operation, as shown in Figure (1), the user can make changes to the meeting minutes. Figure 27 As shown in Figure (2), the mobile phone can display an editing interface 2710, in which the user can edit the meeting minutes. Optionally, the editing interface 2710 may also include a cancel button 2711 and / or a complete button 2712. The cancel button 2711 can be used to return to the meeting minutes display interface 2606, and the complete button 2712 can be used to save the meeting minutes edited by the user.

[0258] In some embodiments, only a subset of meeting members have permission to modify the meeting minutes; for example, a member specifically designated to record the minutes has this permission, while other members do not. For instance, the meeting minutes displayed for the authorized members are in an editable state. Figure 27 The edit button 2700 in (1) is operable. Meeting minutes displayed for meeting members without permission are not editable, for example: Figure 27 The edit button 2700 in (1) is in an inoperable state (e.g., grayed out), or the edit button 2700 is not displayed.

[0259] Optionally, the mobile phone can also send the meeting minutes output by the information arrangement module to other devices for display, which is not limited in this application.

[0260] In some embodiments, the mobile phone can also generate different summaries based on the text data generated by the ASR module. For example... Figure 10 As shown, the mobile phone may also include a question extraction module, which can be used to extract questions from the text data generated by the ASR module and generate a question summary based on the extracted questions.

[0261] It should be noted that the processes of generating question summaries and generating meeting summaries can be performed at the same or different times. The way (quantitative, periodic, etc.) the question extraction module extracts questions from the text data generated by the ASR module, as well as the timing, can be the same or different from those of the summary generation module.

[0262] Optional, such as Figure 28 As shown, Figure 12 The process shown may also include the following steps:

[0263] S1206. Determine whether the current character is the punctuation mark corresponding to a question.

[0264] If yes, proceed to step S1207. If no, proceed to step S1201. For example, the punctuation mark corresponding to the question can be "?", etc.

[0265] Optionally, if step S1202 is "yes", this step can also be placed after step S1202.

[0266] S1207. Input the entire sentence corresponding to the current character into the problem extraction module.

[0267] It is understandable that the problem extraction module has undergone [processing / processing]. Figure 28 The process shown may extract one or more repeated questions, and / or questions that are not relevant to the current meeting topic.

[0268] Therefore, in some embodiments, the question extraction module may include an associated question extraction module, which can be used to determine questions related to the current meeting domain from all questions extracted by the question extraction module.

[0269] In some embodiments, a domain-specific sentence structure library can be used to determine whether a question is relevant to the current conference domain.

[0270] In one possible implementation, the question extraction module calculates the matching probability of the question with some or all of the domain phrase libraries. If the domain phrase library corresponding to the matching probability that meets the preset condition is the domain phrase library corresponding to the current meeting, the question is determined to be relevant to the current meeting. If the domain phrase library corresponding to the matching probability that meets the preset condition is not the domain phrase library corresponding to the current meeting, the question is determined to be irrelevant to the current meeting. Optionally, the preset condition can be the maximum matching probability.

[0271] For example, the matching probability between a question and a domain phrase library can be determined by the probabilities of any number of consecutive words included in the question in the domain phrase library. Optionally, the matching probability can be obtained by multiplying the probabilities of these multiple words in the domain phrase library.

[0272] Taking the question "When will the edge speech recognition model be deployed?" as an example, with two consecutive words, any two consecutive words in this question include "edge", "side speech", "speech", ... "deployment", etc. For example, the probability of "edge" in a domain sentence pattern library can be represented as the probability of "edge" followed by "connected" in that domain sentence pattern library, which can be expressed as "P(edge) = P(side|edge)". The probability of "side speech" in a domain sentence pattern library can be represented as the probability of "side" followed by "speech" in that domain sentence pattern library, which can be expressed as "P(side speech) = P(speech|side)". The meanings of "speech", "deployment", etc. in a domain sentence pattern library are similar.

[0273] like Figure 29 As shown, the matching probability of this question in conference domain 1 is: P(side|end)*P(language|side)*…=0.04. The matching probability in the current conference domain is: P(side|end)*P(language|side)*…=0.05, and the matching probability in conference domain K is P(side|end)*P(language|side)*…=0.01, etc.

[0274] The related question extraction module determines whether the matching probability of the current meeting domain is the highest among all meeting domains. If so, it determines that the question is related to the current meeting and extracts the question; otherwise, it determines that the question is not related to the current meeting domain and does not extract the question.

[0275] Optionally, the questions extracted by the question extraction module may include one or more duplicate or similar questions. Therefore, in some embodiments, the questions extracted by the question extraction module can also be deduplicated. Optionally, the question extraction module may also include a question deduplication module, which can be used to deduplicat the questions extracted by the question extraction module.

[0276] The following describes an implementation method for the question deduplication module to remove duplicate questions extracted by the question extraction module.

[0277] For example, for a question, the question deduplication module can first determine the sentence vector corresponding to the question. For instance, the question deduplication module can encode the words included in a question to obtain the encoding vector corresponding to each word. For example, the encoding vector can be in the form of a one-hot vector or other vector forms; this application does not specifically limit this. Then, the sentence vector of the question is determined based on the encoding vectors corresponding to all words in the question. Optionally, the question deduplication module can calculate the average vector of the encoding vectors corresponding to all words and use this average vector as the sentence vector. Of course, other methods can also be used to determine the sentence vector; this application does not limit this.

[0278] Then, the question deduplication module can cluster these questions based on the sentence vectors corresponding to all questions extracted by the associated question extraction module and a clustering algorithm. For questions in the same category, one or more questions are extracted to complete the question deduplication.

[0279] Clustering algorithms can understandably divide data into several categories. Data within the same category share similar characteristics. Based on the similarity between data points, they group data that are more similar and have smaller differences into one category, ultimately forming multiple categories. This results in high similarity within the same category and high differences between different categories. For example, this clustering algorithm could be the k-means clustering algorithm, where k is the number of categories to be obtained. Optionally, the value of k can be determined using methods such as the elbow method or the Calinski-Harabaz index method; this application is not limited to these. Of course, other clustering algorithms can also be used, and this application does not specifically limit them.

[0280] It should be noted that the question deduplication module can also deduplicat questions in other ways, such as by calculating the similarity between two questions. This application is not limited to this method.

[0281] It should be noted that this application does not limit the execution order of the above operations of determining whether the question is related to the current meeting domain and the operation of deduplicating the question.

[0282] Finally, the question extraction module can generate a question summary based on the questions obtained after deduplication by the question deduplication module. For example, the question summary generated by the question extraction module can be as follows: Figure 30 As shown. It should be noted that, Figure 30 The problem summary shown is obtained by combining it with the meeting template. It can also be obtained without using the meeting template. For an introduction to the meeting template, please refer to the relevant introduction to the meeting template in the meeting summary.

[0283] Similar to the conference summary, this issue summary may not include time information, but it should be arranged in chronological order. Alternatively, it may include time information, which can be based on time endpoint information generated by the VAD module (such as...). Figure 11 The time endpoint information shown is determined.

[0284] Optionally, other types of statements besides questions can be extracted from the text data generated by the ASR module to generate summaries for those types of statements. The extraction method can refer to the question extraction process, and the corresponding summary generation method can refer to the question summary generation process.

[0285] Optionally, after the questions are extracted from the text data generated by the ASR module, the summary generation module can also generate a conference summary based on other text data besides the questions. The specific process of generating the conference summary can be found above.

[0286] Optionally, the question extraction module can also input the generated question summary into the information arrangement module for use in generating meeting minutes.

[0287] Correspondingly, the information orchestration module can receive question summaries from the question extraction module. Based on the question summaries generated by the question extraction module, the meeting summary generated by the summary generation module, and the time endpoint information and meeting member information output by the VAD module, the information orchestration module can generate meeting minutes. The specific generation process can be referenced in the process by which the information orchestration module generates meeting minutes based on the meeting summary generated by the summary generation module and the time endpoint information and meeting member information output by the VAD module.

[0288] For example, meeting minutes generated by the information orchestration module can be like this: Figure 31 As shown. It should be noted that, Figure 31 The meeting minutes shown are also generated in conjunction with a meeting template. However, they can also be generated without a meeting template. For an introduction to the meeting template, please refer to the relevant introduction to the meeting template in the meeting summary.

[0289] Optionally, in some scenarios, the meeting domain may change during the meeting. If the original domain syntax library is used to generate the meeting minutes, the generated minutes may be inaccurate. Therefore, in some embodiments, meeting members can also change the meeting domain. For example, a user can change the current meeting domain by inputting prior information and / or the meeting domain itself. Correspondingly, the summary generation module can generate a meeting summary based on the updated domain syntax library, and / or the question extraction module can generate a question summary based on the updated domain syntax library. Furthermore, the information arrangement module can generate meeting minutes based on the newly generated meeting summary and / or question summary. Optionally, only some meeting members may have permission to change the meeting domain. For example, a member specifically responsible for recording meeting minutes may have permission, while other members may not. Of course, all meeting members may also have permission to change the meeting domain; this application does not limit this.

[0290] For example, Figure 32 This diagram illustrates the process of a user changing the meeting area. (For example...) Figure 32 As shown in (1), in Figure 26Based on the meeting display interface 2606 shown in (3), the meeting display interface 2606 may further include a meeting area selection area 3200, which can be used by the user to reselect the meeting area. For example, the mobile phone detects an action such as a user clicking on the meeting area selection area 3200, such as... Figure 32 As shown in (2), the mobile phone can display one or more meeting areas, and the user can select the meeting area according to the actual needs.

[0291] It should be noted that, Figure 32 The method of changing the meeting area shown is merely illustrative and can also be used in other ways. Figure 26 The meeting area can be changed by inputting prior information in the meeting area display interface 2606 shown in (3). Of course, the meeting area can also be changed in other interfaces. This application is not limited to this.

[0292] For example, suppose the current meeting domain changes from the ASR meeting domain to the VAD technology domain, the user can reselect the VAD meeting domain. For instance, after the user resets the meeting domain to the VAD meeting domain, the meeting minutes generated by the information orchestration module can be as follows: Figure 33 As shown. From Figure 33 The meeting minutes show that the content belongs to the ASR meeting domain and the VAD meeting domain, respectively, that is, they belong to different meeting domains.

[0293] It should be noted that the above example illustrates the operation of generating meeting minutes using a mobile phone. In some possible embodiments, the mobile phone can also send the collected audio data to other devices (e.g., a server), which then perform the operation of generating meeting minutes and send the minutes back to the mobile phone. In this embodiment, the VAD module, ASR module, question extraction module, summary generation module, and information arrangement module can be located in other devices. In some other possible embodiments, multiple devices (e.g., a mobile phone and a server) can collaboratively implement the operation of generating meeting minutes. That is, the multiple operation steps involved in generating meeting minutes can be executed by different devices. In this embodiment, the modules involved in this application can also be located in different devices. For example, the VAD module and ASR module are located in the mobile phone, and the summary generation module and information arrangement module are located in the server. This application does not impose any restrictions on the specific location of each module.

[0294] It should be noted that the solutions involved in the above embodiments are for online meeting scenarios. The technical solutions provided in this application are also applicable to offline meeting scenarios. For example, audio data in an offline meeting can be collected by the electronic device provided in this application, and meeting minutes can be generated according to the technical solutions provided in this application.

[0295] Optionally, voiceprint technology can be combined with the technical solutions provided in the embodiments of this application. Voiceprint technology can be used to identify the meeting members to whom the audio data belongs; however, this application is not limited to this. The combined technical solution is applicable to both online and offline meeting scenarios, and this application will not describe it in detail further.

[0296] It should be noted that in the embodiments of this application, each interface is merely a schematic diagram. In actual applications, each interface may include more or less content, or may include more or less interfaces. This application does not impose any specific limitations on this.

[0297] For example, Figure 34 This paper illustrates a flowchart of a meeting minutes generation method provided in an embodiment of this application. The method includes the following steps:

[0298] S3401. The electronic device determines the target domain sentence structure library corresponding to the current meeting.

[0299] The target domain sentence structure library is used to indicate the arrangement of words in the target text, where the target text is the text in the target domain, which is the conference domain to which the current meeting belongs.

[0300] It is understandable that the arrangement of words can refer to the order in which the words are arranged, the number of times that order appears, the probability, and so on.

[0301] Optionally, the target domain sentence structure library may include the probabilities of any e consecutive words followed by f consecutive words in the target text, where e and f are preset positive integers. That is, the target domain sentence structure library can be represented in the form of a language model.

[0302] In some embodiments, the electronic device can acquire prior information about the current meeting, including one or more of schedule creation information, chat information, and email information. Then, a target domain phrase library is determined based on the prior information and at least one domain phrase library.

[0303] In one possible implementation, the electronic device can determine the matching probability between prior information and at least one domain phrase library, and determine the target domain phrase library based on this matching probability. For example, the domain phrase library with the highest matching probability can be used as the target domain phrase library. Optionally, the prior information includes multiple words, and the electronic device can determine the target domain phrase library based on the probability of the target word among the multiple words corresponding to the corresponding words in at least one domain phrase library. The target word consists of any e consecutive words followed by f consecutive words, where e and f are preset positive integers. For example, for a domain phrase library, the electronic device can multiply the probabilities of the target words corresponding to the domain phrases to obtain the matching probability between the prior information and the domain phrase library. Each domain phrase library includes the probabilities of any e consecutive words followed by f consecutive words in the meeting minutes of the corresponding meeting domain.

[0304] In another possible implementation, the electronic device can first determine the target meeting domain based on prior information, and then determine the target domain syntax library based on the domain syntax library corresponding to the target meeting domain. Optionally, if the target meeting domain corresponds to multiple domain syntax libraries, one of them can be used as the target domain syntax library; if the target meeting domain corresponds to only one domain syntax library, the domain syntax library corresponding to that target meeting domain can be used as the target domain syntax library.

[0305] It should be noted that this application does not limit the specific method of determining the meeting area based on prior information.

[0306] For example, this prior information can be as follows: Figure 15 The information shown is 1501, 1502, 1503, etc.

[0307] Optionally, the prior information may be input by the user or automatically obtained by the electronic device; this application does not limit this.

[0308] In other embodiments, the electronic device may receive a target domain input by a user and determine a target domain sentence library based on the target domain and at least one domain sentence library. For example, Figure 32 As shown, users can input the target field through conference field 3200, etc.

[0309] S3402. Electronic devices acquire audio data from the current meeting.

[0310] It should be noted that steps S3401 and S3402 can be performed sequentially or simultaneously, and this application does not limit this.

[0311] S3403. The electronic device generates meeting minutes corresponding to the current meeting based on audio data and the target domain sentence structure library.

[0312] Optionally, the meeting minutes may also include time information and / or meeting member information; the time information is determined based on the time endpoint information corresponding to the voice data, and the time endpoint information includes the start time and / or end time of the voice data; the meeting member information is determined based on the meeting member information corresponding to the voice data. For example, the time endpoint information and meeting member information may be as follows: Figure 23 As shown, the meeting minutes can be such as Figure 24 As shown.

[0313] Electronic devices can generate meeting summaries based on audio data and a domain-specific syntax library (e.g., meeting summaries generated by a summary generation module), and then match time information and / or meeting participant information with the meeting summaries to generate meeting minutes. For example, [the following is an example:] Figure 22 The meeting summary shown is Figure 23 The time endpoint information and meeting member information shown can be matched to obtain Figure 23 The meeting minutes shown.

[0314] Optionally, if the meeting minutes do not include time information and meeting member information, the meeting summary generated by the summary generation module in the above embodiments can be directly used as the meeting minutes of the current meeting.

[0315] Optionally, meeting minutes can also be generated using a preset format, that is, in combination with a template. Please refer to the above text for an introduction to templates.

[0316] Optionally, prior to this step, the electronic device can acquire the target text and construct a target domain sentence structure library based on the target text. For example, the target text can be in various forms such as reports, documents, professional books, project specifications, meeting minutes, articles, manuals, and newspapers; this application does not impose any special restrictions on the type of target text. For a detailed implementation of constructing the domain sentence structure library, please refer to the above description.

[0317] S3404, Electronic equipment outputs meeting minutes.

[0318] Optionally, electronic devices may output meeting minutes using methods such as display screens or voice broadcasts.

[0319] Optionally, the user can also modify the meeting minutes. For example, the electronic device can receive a target action from the user to initiate the editing function of the meeting minutes. Figure 27 As shown in (1), the target operation is such as a user clicking the edit button 2700. In response to this target operation, the electronic device can display a meeting minutes editing interface, for example, such as... Figure 27The interface 2710 shown in (2) allows users to modify the meeting minutes. Based on this design, the meeting minutes can be edited online and used directly as meeting records after the meeting ends. This avoids the need for additional editing of the meeting minutes to generate meeting records after the meeting ends, saving editing time and manpower, and improving the efficiency of meeting records.

[0320] Based on the above technical solution, newly joined meeting members can quickly obtain key meeting information from the output meeting minutes, reducing the probability of meeting interruptions and improving meeting efficiency. Furthermore, by incorporating a domain-specific sentence structure library, the meeting minutes' expression adapts to the expression habits of the corresponding meeting domain (e.g., professional terms and jargon), enhancing the readability and professionalism of the meeting minutes and improving their overall quality.

[0321] Optional, such as Figure 35 As shown, Figure 34 The method shown can be implemented in the following steps:

[0322] S3405. Electronic devices acquire voice data of conference members from audio data.

[0323] Optionally, the audio data may also include silence data.

[0324] In some embodiments, electronic devices can also directly obtain text chat information from meeting members in the current meeting.

[0325] S3406. Electronic devices convert voice data into text data.

[0326] Optionally, the text data may include punctuation marks used to segment the text data. These punctuation marks may include one or more of the following: Chinese period, question mark, exclamation mark, and English period. Of course, the text data may also exclude these punctuation marks, and this application does not impose any restrictions on this.

[0327] S3407. Electronic devices generate meeting minutes based on text data and a target domain sentence structure library.

[0328] Optionally, the electronic device can input text data into a preset model (i.e., the machine model mentioned above); then obtain the first probability of candidate words corresponding to the f consecutive words to be output in the meeting minutes through the preset model; and then determine the third probability of candidate words based on the first probability of candidate words and the second probability of candidate words corresponding to the sentence patterns in the target domain; such as Figure 21 As shown, the third probability is obtained by multiplying the first probability by the second probability.

[0329] The second probability is the probability of candidate words following the e words already output in the meeting minutes; finally, the electronic device generates the meeting minutes based on the third probability of the candidate words.

[0330] It is understandable that the number of candidate words corresponding to the f consecutive words to be output may be one or more. Optionally, each candidate word is also a combination of f words. For example, if f is 1, the candidate words corresponding to the f consecutive words may be "I", "you", "we", etc., and each candidate word is 1 word. If f is 2, the candidate words corresponding to the f consecutive words may be "my", "your", "we", etc., and each candidate word is 2 words.

[0331] In some embodiments, the first probability is related to q consecutive words that have been output in the meeting minutes, and the explanation of q can be found above.

[0332] In other embodiments, the first probability is unrelated to the words already output in the meeting minutes.

[0333] Optionally, the size of the input file data for the preset model can meet a preset threshold. This allows the summary generation module to generate corresponding conference summaries based on this quantitative text data, resulting in highly real-time conference summaries.

[0334] Optionally, the meeting minutes may also include a target summary. The target summary is generated based on the text data corresponding to the target punctuation marks. Electronic devices can obtain the text data corresponding to the target punctuation marks based on the punctuation marks; then, they can generate the target summary based on the text data corresponding to the target punctuation marks.

[0335] For example: the target punctuation mark is a question mark, and the target summary can be a summary of the questions mentioned above, such as... Figure 30 , Figure 31 The problem summary is shown below. Optional. The target punctuation mark can be of one or more types, and correspondingly, the target summary can also be of one or more types.

[0336] Optionally, the text data corresponding to the target punctuation mark includes at least one statement corresponding to the target punctuation mark. The electronic device can determine at least one target statement from the statements corresponding to at least one target punctuation mark based on at least one domain sentence structure library, and the at least one target statement corresponds to (or is associated with) the target domain. The electronic device can generate a target summary based on at least one target statement. For example, the association question extraction operation described above.

[0337] Optionally, at least one target statement may include multiple overlapping target statements, and the similarity between the multiple overlapping target statements meets a preset condition. The electronic device can generate a target summary based on one of the multiple overlapping target statements, as well as the target statements in at least one target statement other than the multiple overlapping target statements. For example, a deduplication operation as described above.

[0338] For example: Suppose the target statements are statements 1 to 7, and the similarity between statements 1, 2, and 3 meets the preset condition, that is, statements 1, 2, and 3 are overlapping statements. Then the electronic device can generate the target minutes based on statement 1 (or statement 2, or statement 3, or statement 1 and statement 2, or statement 2 and statement 3, or statement 1 and statement 3, etc.) and statements 4 to 7.

[0339] For example: Suppose the target statements are statements 1 to 7. The similarity between statements 1 and 2 meets the condition, and the similarity between statements 3 and 4 meets the preset condition. Then statements 1 and 2 are overlapping statements, and statements 3 and 4 are overlapping statements. The electronic device can generate the target minutes based on statements 1 (or 2), 3 (or 4), and statements 5 to 7.

[0340] It should be noted that the electronic device can first perform the operation of determining whether it is relevant to the meeting area, or it can first perform the deduplication operation, or it can perform only one of them.

[0341] The above primarily describes the solutions provided by the embodiments of this application from a methodological perspective. It is understood that, in order to achieve the above functions, the electronic device includes hardware structures and / or software modules corresponding to the execution of each function. Based on the units and algorithm steps of the various examples described in the embodiments disclosed in this application, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by a computer driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solutions of the embodiments of this application.

[0342] This application provides embodiments for dividing an electronic device into functional modules based on the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into a single processing unit. The integrated unit can be implemented in hardware or as a software functional module. It should be noted that the unit division in this application embodiment is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used.

[0343] like Figure 36 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. This electronic device 3600 can be used to implement the methods described in the above method embodiments. For example, the electronic device 3600 may specifically include: a processing unit 3601, an acquisition unit 3602, and an output unit 3603.

[0344] The processing unit 3601 is used to perform actions that support the electronic device 3600. Figure 34 Steps S3401 and S3403 in the process. And / or, the processing unit 3601 is used to perform the actions of the supporting electronic device 3600. Figure 35 Steps S3401, S3406 to S3407 in the process. And / or, the processing unit 3601 is also used to support the electronic device 3600 in performing other steps performed by the electronic device in the embodiments of this application.

[0345] The acquisition unit 3602 is used to perform actions that support the electronic device 3600. Figure 34 Step S3402 in the process. And / or, the acquisition unit 3602 is used to perform the execution of the supporting electronic device 3600. Figure 35 Steps S3402 and S3405 in the process. And / or, the acquisition unit 3602 is also used to support the electronic device 3600 in performing other steps performed by the electronic device in the embodiments of this application.

[0346] Output unit 3603 is used to perform actions that support electronic device 3600. Figure 34 Step S3404 in the process. And / or, the output unit 3603 is used to perform the execution of the supporting electronic device 3600. Figure 35 Step S3404 in the process. And / or, the output unit 3603 is also used to support the electronic device 3600 in performing other steps performed by the electronic device in the embodiments of this application.

[0347] Optional, Figure 36 The output unit 3603 shown can be specifically implemented as a display unit, which can be used to support the electronic device 3600 in performing display operations, such as displaying meeting minutes. Optionally, Figure 36 The output unit 3603 shown can also be specifically implemented as a voice broadcasting unit, which can be used to support the electronic device 3600 to perform voice broadcasting operations, such as: voice broadcasting meeting minutes, etc.

[0348] Optional, Figure 36 The electronic device 3600 shown may also include a communication unit ( Figure 36 (Not shown in the image), this communication unit is used to support electronic device 3600 in performing the steps of communication between electronic device and other electronic devices in the embodiments of this application.

[0349] Optional, Figure 36 The electronic device 3600 shown may also include a storage unit ( Figure 36 (not shown in the image), this storage unit stores a program or instruction. When the processing unit 3601 executes the program or instruction, it causes... Figure 36 The electronic device 3600 shown can perform Figure 34 and / or Figure 35 The method shown.

[0350] Figure 36 The technical effects of the electronic device 3600 shown can be referenced. Figure 34 , Figure 35 The technical effects of the method shown will not be elaborated here. Figure 36 The processing unit 3601 involved in the illustrated electronic device 3600 can be implemented by a processor or processor-related circuit components, and can be a processor or processing module. The communication unit can be implemented by a transceiver or transceiver-related circuit components, and can be a transceiver or transceiver module. The display unit can be implemented by display screen-related components.

[0351] This application also provides a chip system, such as... Figure 37 As shown, the chip system includes at least one processor 3701 and at least one interface circuit 3702. The processor 3701 and the interface circuit 3702 are interconnected via lines. For example, the interface circuit 3702 can be used to receive signals from other devices. As another example, the interface circuit 3702 can be used to send signals to other devices (e.g., the processor 3701). Exemplarily, the interface circuit 3702 can read instructions stored in memory and send those instructions to the processor 3701. When the instructions are executed by the processor 3701, the electronic device can perform the various steps performed by the electronic device in the above embodiments. Of course, the chip system may also include other discrete components, which are not specifically limited in this application embodiment.

[0352] Optionally, the chip system may contain one or more processors. These processors can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0353] Optionally, the chip system may contain one or more memories. The memory may be integrated with the processor or disposed separately from it; this application does not limit this. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or disposed separately on different chips. This application does not specifically limit the type of memory or the arrangement of the memory and processor.

[0354] For example, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0355] It should be understood that each step in the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0356] This application also provides a computer storage medium storing computer instructions, which, when executed on an electronic device, cause the electronic device to perform the methods described in the above-described method embodiments.

[0357] This application provides a computer program product, which includes a computer program or instructions that, when run on a computer, cause the computer to perform the methods described in the above-described method embodiments.

[0358] In addition, this application also provides an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory. The memory is used to store computer execution instructions. When the apparatus is running, the processor can execute the computer execution instructions stored in the memory to cause the apparatus to perform the methods in the above-described method embodiments.

[0359] In this embodiment, the electronic device, computer storage medium, computer program product or chip are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding method provided above, and will not be repeated here.

[0360] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0361] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The embodiments can be combined with or referenced to each other without conflict. The apparatus embodiments described above are merely illustrative; for example, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0362] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0363] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0364] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0365] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for generating meeting minutes, characterized in that, Applied to electronic devices, the method includes: A target domain sentence pattern library is determined for the current meeting. The target domain sentence pattern library is used to indicate the arrangement of words in the target text, where the target text is the text corresponding to the target domain, and the target domain is the meeting domain to which the current meeting belongs. The target domain sentence pattern library is constructed based on the target text and a language model. The target domain sentence pattern library includes the probability that any e consecutive words in the target text are followed by f consecutive words, where e and f are preset positive integers. Obtain the audio data from the current meeting; The meeting minutes corresponding to the current meeting are generated based on the audio data and the target domain sentence structure library; wherein, the meeting minutes are generated based on the third probability of candidate words, the candidate words correspond to the f consecutive words to be output in the meeting minutes, the third probability is determined based on the first probability of the candidate words and the second probability of the candidate words in the target domain sentence structure library, the first probability is the probability obtained by processing the text data corresponding to the voice data of the meeting members in the audio data based on a preset model, and the second probability is the probability of the candidate words following the e words already output in the meeting minutes; Output the meeting minutes.

2. The method according to claim 1, characterized in that, The process of determining the target domain sentence structure library corresponding to the current meeting includes: Obtain prior information corresponding to the current meeting, the prior information including one or more of meeting schedule information, chat information, and email information; the prior information includes multiple words; The target domain sentence structure library is determined based on the probability of the target word among the plurality of words corresponding to at least one domain sentence structure library; the target word is composed of any e consecutive words followed by f consecutive words, where e and f are both preset positive integers; Each domain sentence structure library includes the probability of any e consecutive words followed by f consecutive words in the meeting minutes of the corresponding meeting domain.

3. The method according to claim 1, characterized in that, The process of determining the target domain sentence structure library corresponding to the current meeting includes: Receive the target area as input by the user; The target domain sentence library is determined based on the target domain and at least one domain sentence library.

4. The method according to any one of claims 1-3, characterized in that, The step of generating the meeting minutes corresponding to the current meeting based on the audio data and the target domain sentence structure library includes: Obtain the voice data of the meeting participants from the audio data; Convert the voice data into the text data; The meeting minutes are generated based on the text data and the target domain sentence structure library.

5. The method according to any one of claims 1-3, characterized in that, The text data includes sentence break symbols, which are used to break the text data into sentences. The sentence break symbols include one or more of the following: Chinese period, question mark, exclamation mark, and English period. The method further includes: The size of the text data is determined to meet a preset threshold by using the punctuation marks.

6. The method according to claim 5, characterized in that, The meeting minutes include a target summary, which is generated based on the text data corresponding to the target punctuation marks in the punctuation marks. The method further includes: Obtain the text data corresponding to the target punctuation mark based on the punctuation mark; The target summary is generated based on the text data corresponding to the target punctuation marks.

7. The method according to claim 6, characterized in that, The text data corresponding to the target punctuation mark includes at least one sentence corresponding to the target punctuation mark; The step of generating the target summary based on the text data corresponding to the target punctuation marks includes: At least one target statement is determined from the statements corresponding to the at least one target sentence segmentation symbol based on at least one domain sentence pattern library, wherein the at least one target statement corresponds to the target domain; The target summary is generated based on the at least one target statement.

8. The method according to claim 7, characterized in that, The at least one target statement includes multiple overlapping target statements, and the similarity between the multiple overlapping target statements satisfies a preset condition. The step of generating the target summary based on the at least one target statement includes: The target summary is generated based on one or more of the plurality of overlapping target statements, and the target statements other than the plurality of overlapping target statements in the at least one target statement.

9. The method according to any one of claims 1-3, characterized in that, The meeting minutes also include time information and / or meeting member information; the time information is determined based on the time endpoint information corresponding to the voice data, the voice data is related to the meeting members in the audio data, and the time endpoint information includes the start time and / or end time of the voice data; the meeting member information is determined based on the meeting member information corresponding to the voice data. The step of generating the meeting minutes corresponding to the current meeting based on the audio data and the target domain sentence structure library includes: A meeting summary is generated based on the audio data and the domain phrase library; The meeting minutes are generated by matching the time information and / or meeting member information with the meeting summary.

10. The method according to any one of claims 1-3, characterized in that, Before determining the target domain sentence structure library corresponding to the current meeting, the method includes: Obtain the target text; Construct the target domain sentence structure library based on the target text.

11. The method according to any one of claims 1-3, characterized in that, Output the meeting minutes, including: The meeting minutes are displayed.

12. The method according to claim 11, characterized in that, After displaying the meeting minutes, the method further includes: Receive the user's target operation, which is used to initiate the editing function of the meeting minutes; In response to the target operation, the editing interface for the meeting minutes is displayed.

13. The method according to any one of claims 1-3, characterized in that, The conference covers one or more of the following fields: clinical medicine, forensic medicine, preventive medicine, rehabilitation medicine, health care medicine, psychology, pathology, genetics, human histology, surgery, internal medicine, stomatology, oncology, respiratory diseases, human immunology, medical imaging, banking, securities, insurance, trust, investment, finance, accounting, leasing, stocks, funds, trade, big data, artificial intelligence, Internet of Things, the Internet, cloud computing, connected vehicles, machine learning, autonomous driving, mobile communication, optical fiber communication, microwave communication, satellite communication, WiFi, Bluetooth, radio frequency, Ethernet, NLP, ASR, petroleum, natural gas, coal, horticulture, animal husbandry, breeding, cultivation, aquaculture, soil and fertilizer, marine, agricultural product processing, vehicles, textiles, chemicals, metallurgy, electrical appliances, water supply and drainage, civil engineering, road transportation, bridges, and river crossing.

14. An electronic device, characterized in that, include: One or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored on the memory, and when the computer programs are executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1-13.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer program or instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-13.

16. A computer program product, characterized in that, The computer program product includes: a computer program or instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-13.

Images