Speech analysis system and method

Abstract

Embodiments of the present specification provide a speech analysis system and method, the system comprising: a first storage device configured to store a plurality of large language models corresponding to a plurality of fields; the first server side is in communication connection with the first user terminal and the first storage device, and the first server side is configured to obtain target recording in response to a recording analysis request sent by the first user terminal; processing the target recording to determine a target field corresponding to the target recording from a plurality of fields; and processing the target recording by using the target large language model corresponding to the target field to generate a listening report corresponding to the target recording.

Description

Technical Field

[0001] This specification relates to the technical field of voice information processing, and particularly to a voice analysis system and method. Background Art

[0002] In the process of business operation, enterprises usually need to analyze the voice messages uploaded by users, understand their feedback information such as needs, complaints, suggestions, etc., and form corresponding analysis reports or solutions. The method of manually collecting and analyzing user voices often requires a large amount of human resources and time costs, and the efficiency is low. At the same time, it is difficult to ensure the accuracy of the analysis reports obtained based on manual experience. With the development of information technology, artificial intelligence technology has been more widely applied to the field of voice processing. However, the training, updating, etc. of artificial intelligence models also often require a lot of time and effort. There are also certain differences in the voice analysis requirements for different fields, different application scenarios or tasks.

[0003] Therefore, a voice analysis system and method are provided, which can automatically analyze user voices, thereby generating targeted analysis reports. In addition, the large language models for different fields and / or tasks are automatically iteratively updated, saving labor training costs while improving the accuracy and diversity of the analysis reports. Summary of the Invention

[0004] One embodiment of this specification provides a voice analysis system, including: a first storage device configured to store multiple large language models corresponding to multiple fields; and a first server side, the first server side being communicatively connected to a first user terminal and the first storage device, wherein the first server side is configured to: in response to a recording analysis request sent by the first user terminal, obtain a target recording; process the target recording to determine a target field corresponding to the target recording from the multiple fields; and use the target large language model corresponding to the target field to process the target recording to generate a listening report corresponding to the target recording.

[0005] One embodiment of this specification provides a voice analysis method, which is executed by the first server side of a voice analysis system. The voice analysis system includes a first storage device and the first server side. The first storage device is configured to store multiple large language models corresponding to multiple fields. The first server side is communicatively connected to a first user terminal and the first storage device. The method includes: in response to a recording analysis request sent by the first user terminal, obtain a target recording; process the target recording to determine a target field corresponding to the target recording from the multiple fields; and use the target large language model corresponding to the target field to process the target recording to generate a listening report corresponding to the target recording. Brief Description of the Drawings

[0006] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same reference numerals represent the same structures, where:

[0007] Figure 1 is an exemplary schematic diagram of a speech analysis system shown in some embodiments of this specification;

[0008] Figure 2 is an exemplary flowchart of a method for generating a listening report shown in some embodiments of this specification;

[0009] Figure 3 is an exemplary flowchart of a method for updating a large language model shown in some embodiments of this specification;

[0010] Figure 4 is an exemplary schematic diagram of a training method for a large language model shown in some embodiments of this specification;

[0011] Figure 5 is an exemplary schematic diagram of a training method for a dedicated large language model shown in some embodiments of this specification;

[0012] Figure 6 is a schematic diagram of the hardware and / or software of a speech system shown in some embodiments of this specification. Detailed implementation manners

[0013] To more clearly illustrate the technical solutions of the embodiments of this specification, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some examples or embodiments of this specification. For those of ordinary skill in the art, without creative efforts, this specification can also be applied to other similar scenarios based on these drawings. Unless obvious from the language context or otherwise stated, the same reference numerals in the figures represent the same structures or operations.

[0014] It should be understood that the "system", "device", "unit" and / or "module" used herein is a way to distinguish different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the said words can be replaced by other expressions.

[0015] Unless the context clearly indicates otherwise, the words "a," "an," "one," and / or "the" are not intended to refer to the singular and may also include the plural. In general, the terms "comprising" and "including" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive listing. A method or device may also include other steps or elements.

[0016] Flowcharts are used in this specification to illustrate the operations performed by a system according to an embodiment of this specification. It should be understood that the operations before or after do not necessarily have to be executed precisely in sequence. On the contrary, they can be executed in reverse order or processed simultaneously. Also, other operations can be added to these processes, or one or more operations can be removed from these processes.

[0017] For the solutions described in this specification, if they involve the processing of personal information, they will be processed on the premise of having a legal basis (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract, etc.), and will only be processed within the specified or agreed scope. If a user refuses to allow us to process personal information other than the necessary information required for basic functions, it will not affect the user's use of the basic functions.

[0018] Figure 1 is an exemplary schematic diagram of a speech analysis system according to some embodiments of this specification.

[0019] As Figure 1 shown, the speech analysis system 100 may include a first server end 110, a first storage device 120, a second storage device 130, a first user terminal 140, a second server end 150, and a second user terminal 160.

[0020] The server end in this specification may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the server end may be local or remote. In some embodiments, the server end may be implemented on a cloud platform. By way of example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, etc. or any combination thereof.

[0021] The first server 110 can be used for speech analysis. In some embodiments, the first server 110 can be communicatively connected to the first user terminal 140 and the first storage device 120 respectively to process the data and / or information obtained from the first user terminal 140 and the first storage device 120. For example, the first server 110 can respond to a recording analysis request sent by the first user terminal 140, obtain the target recording, and process the target recording to determine the target field corresponding to the target recording from multiple fields. Among them, the target recording is obtained by the first server 110 from the second server 150. In some embodiments, the first server 110 can also obtain the target large language model corresponding to the target field from the first storage device 120, and use the target large language model to process the target recording to generate a listening report corresponding to the target recording. For more content on determining the target field corresponding to the target recording and generating the listening report corresponding to the target recording, see Figure 2 and its description.

[0022] In some embodiments, the first server 110 can monitor multiple knowledge bases in the second storage device 130 and update multiple large language models in the first storage device 120 based on the new data in the multiple knowledge bases. For more content on updating the large language model, see Figure 3 and its description.

[0023] The storage device can store data and / or instructions. In some embodiments, the storage device can include the first storage device 120 and the second storage device 130. The first storage device 120 can be configured to store multiple large language models corresponding to multiple fields. For example, it can store the general large language model in the field of two-wheeled vehicles (such as electric bicycles, motorcycles), the general large language model in the field of four-wheeled vehicles (such as electric vehicles, fuel vehicles), etc. In some embodiments, the first storage device 120 can also store large language models for different application scenarios or tasks. For example, it can store a dedicated large language model for the power-off scenario during riding in the field of two-wheeled electric bicycles, and a dedicated large language model for the traffic accident scenario in the field of two-wheeled motorcycles. For the relevant content of the general large language model and the dedicated large language model in the field, see Figure 2 and its description.

[0024] The second storage device 130 can be configured to store multiple knowledge bases corresponding to multiple fields. For example, it can store the knowledge base in the field of two-wheeled vehicles, the knowledge base in the field of four-wheeled vehicles, etc. In some embodiments, the knowledge base can include, but is not limited to, a database (such as a structured database), a file system (such as a distributed file system), etc., or any combination thereof. Only as an example, the knowledge base in the field of two-wheeled vehicles can store relevant text data (such as knowledge documents, historical case documents, historical listening reports, etc.) of two-wheeled vehicles such as electric bicycles and motorcycles.

[0025] In some embodiments, the first storage device 120 and the second storage device 130 may be independently deployed or integrated in one storage device. In some embodiments, the first storage device and / or the second storage device may be part of the first server side 110. In some embodiments, the first storage device 120 and / or the second storage device 130 may include a mass storage device, a removable storage device, a volatile read-write memory, a read-only memory (ROM), etc., or any combination thereof. In some embodiments, the first storage device 120 and / or the second storage device 130 may be implemented on a cloud platform.

[0026] The first user terminal 140 refers to the terminal device used by the first type of users, which may include, but is not limited to, a mobile device 140-1, a tablet computer 140-2, a laptop computer 140-3, etc. The first type of users refers to users with voice analysis requirements. Taking a taxi service enterprise as an example, the first type of users may include employees of enterprise departments, department heads, listening analysts, etc., who need to analyze the voice feedback information such as recordings or other forms (such as voice texts) uploaded by consumers on the taxi platform. In some embodiments, the first user terminal 140 may include software or application programs for voice analysis (such as Figure 6 the shown listening assistant), which can be used to obtain the recording analysis requests of the first type of users and generate a listening report via the first server 110 and present it to the first type of users.

[0027] The second server side 150 can be used to collect and / or store the recordings of the second type of users. The second type of users refers to the users who issue user voices. Taking a taxi service enterprise as an example, the second type of users may include, but is not limited to, vehicle rental personnel, passengers, drivers, etc., who can upload voice feedback information such as recordings or other forms (such as voice texts) on the taxi platform. Only as an example, a passenger can record the problems (such as vehicle problems, accident situations, etc.) occurring during the vehicle use process in voice and send them to the second server side 150 through the second user terminal 160. In some embodiments, the second server side 150 may interact with other components in the voice analysis system 100 (for example, the second user terminal 160 and the first server side 110). In some embodiments, the second server side 150 may be configured to collect the recordings uploaded by the second type of users through the second user terminal 160. In some embodiments, the second server side 150 may be configured as part of the first server side 110.

[0028] In some embodiments, the second server 150 may include an Application Programming Interface (API) for obtaining data in different fields (such as user feedback information in different fields) to obtain data in different fields from the second user terminal 160 or other data channels (such as third-party platforms). In some embodiments, the second server 150 may also be communicatively connected to the first server 110 to send data in different fields to the first server 110 and store them in the knowledge base corresponding to the second storage device 130. Taking a taxi service enterprise as an example, the second server 150 may include APIs related to the field of two-wheeled vehicles (such as electric bicycles and motorcycles) to obtain data in the field of two-wheeled vehicles (such as recordings of questions, complaints, suggestions, etc. related to taking a taxi) uploaded by users (such as the second type of users) through the second user terminal 160.

[0029] The second user terminal 160 may be configured as a terminal device used by the second type of users. Similar to the first user terminal 140, it may also be a mobile device 160-1, a tablet computer 160-2, a laptop computer 160-3, etc. The second type of users may upload user voices to the second server 150 through the second user terminal 160.

[0030] The voice analysis system 100 may further include a network, which may include any suitable network that facilitates the exchange of information and / or data of the voice analysis system 100. In some embodiments, one or more components of the voice analysis system 100 (for example, the first server 110, the first storage device 120, the second storage device 130, the first user terminal 140, the second server 150, and the second user terminal 160) may transmit information and / or data to one or more other components of the voice analysis system 100 via the network. For example, the first server 110 may send the listening report corresponding to the target recording to the first user terminal 140 through the network. In some embodiments, the network may be any one or more of a wired network or a wireless network. In some embodiments, the network may be various topological structures such as point-to-point, shared, centralized, etc. or a combination of multiple topological structures.

[0031] The above description is for illustrative purposes only, and the voice analysis system 100 may have various variations.

[0032] It should be noted that the voice analysis system 100 is provided only for illustrative purposes and is not intended to limit the scope of this specification. For those of ordinary skill in the art, various modifications or changes can be made according to the description of this specification. However, these changes and modifications will not depart from the scope of this application.

[0033] Figure 2It is an exemplary flowchart of a method for generating a listening report according to some embodiments of this specification.

[0034] In some embodiments, the first server 110 may generate a listening report based on the following steps 210 to 250.

[0035] Step 210, in response to a recording analysis request sent by the first user terminal, obtain a target recording.

[0036] The recording may reflect feedback information (such as consultations, complaints, suggestions, etc.) of the second type of user in various business scenarios. The target recording refers to the recording to be analyzed and for which a listening report needs to be generated.

[0037] In some embodiments, the first server 110 may obtain the target recording from the second server 150. For example, the recording analysis request (such as an Http network request) sent by the first type of user through the first user terminal 140 may include keyword information. After receiving the recording analysis request, the first server 110 may obtain the target recording from the second server 150 based on the keyword information. Among them, the keyword information may include one or more keywords or key phrases used to retrieve or match the target recording. For example, it may include but is not limited to the recording name, upload date (such as a timestamp), field of belonging, business scenario, etc.

[0038] In some embodiments, the first server 110 may obtain the target recording from the first user terminal 140. For example, the first type of user obtains the target recording from the second server 150 through the first user terminal 140 (for example, by clicking or keyword searching), and sends a recording analysis request containing the target recording to the first server 110. The first server 110 may obtain the target recording from the recording analysis request.

[0039] It should be noted that the first server 110 may also obtain the target recording through other means. For example, the first user terminal 140 may present one or more candidate recordings (such as recordings obtained from the local cache of the second server 150 and / or the first user terminal 140), and the first type of user may select one or more of the candidate recordings as the target recording and send it to the first server 110, thereby enabling the first server 110 to obtain the target recording.

[0040] In some embodiments, the first server 110 may process the target recording to determine the target field corresponding to the target recording from multiple fields, and use the target large language model corresponding to the target field to process the target recording to generate a listening report corresponding to the target recording, which may be implemented based on the following steps 220 to 250.

[0041] Step 220: Process the target recording using a speech recognition model to generate a transcription text corresponding to the target recording. The speech recognition model is a trained machine learning model.

[0042] The speech recognition model can be used to convert the target recording into text information. This text information can be referred to as the transcription text, which may include the speech content related to the business scenario described by the user in the target recording. For example, the transcription text may include the business scenarios (such as car rental scenarios, repair scenarios), problems (such as the problem that the electric bicycle cannot be locked), requirements (such as refunding fees), etc. feedback by the user (such as the second type of user).

[0043] In some embodiments, the speech recognition model is a trained machine learning model. For example, it can be one or a combination of a Hidden Markov Model (HMM), a Recurrent Neural Network (RNN), or a Convolutional Neural Networks (CNN), etc.

[0044] Step 230: Process the transcription text using a domain discrimination model to determine the target domain corresponding to the target recording. The domain discrimination model is a trained machine learning model.

[0045] The domain can be various classifications preset according to actual needs. For example, the domain can include the two-wheeled vehicle domain, the four-wheeled vehicle domain, etc. Each domain can also be divided into sub-domains. For example, the two-wheeled electric bicycle domain, the two-wheeled vehicle accident handling domain, etc. The target domain refers to the domain to which the currently processed target recording belongs.

[0046] The domain discrimination model refers to a classification model used to determine the domain to which the text belongs, and it can be a trained machine learning model. For example, it can be a Recurrent Neural Network (RNN), a Convolutional Neural Network (CNN), or other deep neural network models.

[0047] In some embodiments, the input of the domain discrimination model includes the transcription text, and the output includes the target domain corresponding to the transcription text. In some embodiments, the initial domain discrimination model can be iteratively trained using supervised learning to obtain a trained domain discrimination model. The training samples can include the sample transcription texts corresponding to the historical recordings, and the label of each training sample is the domain corresponding to the sample transcription text, which can be labeled manually or by other feasible methods.

[0048] The first server 110 can also determine the target field based on the transcribed text in other ways. In some embodiments, the first server 110 can obtain the keyword information in the transcribed text and determine the target field based on the keyword information in the transcribed text and the preset field keywords.

[0049] In some embodiments, the preset field keywords can be used to represent each field in multiple preset fields. For example, the field keywords corresponding to the two-wheeler field can include electric bicycles, shared electric bicycles, motorcycles, etc.; the field keywords corresponding to the four-wheeler field can include electric vehicles, shared cars, fuel vehicles, etc.

[0050] In some embodiments, the preset field keywords can also include scene keywords (or task keywords) within each field. Among them, the scene keywords can be used to distinguish different fields and different scenes or tasks under different fields. For example, the borrowing scene of the two-wheeler field can include scene keywords such as electric bicycles, scanning codes, unlocking, locking, orders, and payments. Another example is that the borrowing and riding failure scene of the two-wheeler field can include scene keywords such as scanning codes, unlocking, low battery, and power outage during the journey.

[0051] In some embodiments, the first server 110 can extract the keyword information (such as a combination of multiple field keywords) in the transcribed text and perform keyword comparison and analysis with the preset field keywords to determine the target field corresponding to the target recording. Exemplarily, when the keyword information in the transcribed text includes electric bicycle keywords, the target field can be determined as the two-wheeler field; another example is that when the keyword information in the transcribed text includes keywords such as electric bicycles, scanning codes, unlocking, and riding, the similarity with the preset field keywords corresponding to the borrowing scene of the two-wheeler field is relatively high, and the target field is the two-wheeler field.

[0052] Step 240, determine the target large language model based on the target field and multiple large language models.

[0053] Multiple large language models can be stored in the first storage device 120. For more information about the first storage device 120, see Figure 1 and its description.

[0054] The target large language model refers to the large language model in multiple large language models that matches the target field. In some embodiments, the large language model corresponding to each field can be a trained large language model (LLM), which can be a deep learning model trained using a large amount of text data.

[0055] Such as Figure 2As shown, at least one large language model stored in the first storage device 120 may include a large language model for domain D1, a large language model for domain D2... a large language model for domain D n When the domain to which a certain large language model (such as the large language model for domain D1) belongs (such as domain D1) is the target domain, the first server 110 may obtain the large language model from the first storage device 120 and use it as the target large language model.

[0056] In some embodiments, the large language model corresponding to each domain may include a general large language model corresponding to that domain. Hereinafter, the general large language model corresponding to a certain domain may be referred to as the domain general model, which can be trained with a large amount of training data related to that domain (such as domain D1, domain D2, etc.). In some embodiments, the domain general model may be a GPT (Generative Pre-Trained Transformer) model. For more content on the training of the general large language model corresponding to each domain, see Figure 4 and its description.

[0057] In some embodiments, the large language model corresponding to each domain may include multiple dedicated large language models corresponding to multiple tasks in that domain. Hereinafter, the dedicated large language model corresponding to a task in a certain domain may be referred to as the task-specific model. A task in a certain domain may correspond to a preset application scenario / business scenario within that domain. For example, the riding interruption scenario in the two-wheeler domain, the traffic accident scenario in the two-wheeler domain, etc.

[0058] In some embodiments, the task-specific model may include a BERT (Bidirectional Encoder Representations from Transformer) model, which can be trained with a large amount of training data related to that task. It should be understood that the domain general model and the task-specific model can be of any type, and their types can be the same or different. For more content on dedicated training samples and the training of dedicated large language models corresponding to multiple tasks, see Figure 5 and its description.

[0059] In some embodiments of this specification, considering the commonalities of multiple scenarios within each domain and using the general large language model of each domain to perform the general tasks of that domain can save training time and resources; in addition, introducing dedicated large language models can fully consider the particularities of specific tasks to customize the listening report, making the listening report more targeted.

[0060] In some embodiments, the target large language model can also be a preset or default pre-trained general large language model. Exemplarily, when the field corresponding to the target recording does not belong to any of the preset fields, steps 230 and 240 can be omitted. At this time, this default general large language model can be used as the target large language model for subsequent processing.

[0061] Step 250, use the target large language model to process the transcribed text to generate a listening report corresponding to the target recording.

[0062] A listening report refers to an analysis document generated based on the content described by the user in the target recording. It can be a file in forms including but not limited to Word, PDF, etc. For example, the listening report can be an analysis report on the power outage during the user's borrowing of an electric two-wheeler.

[0063] In some embodiments, the listening report can include a domain-general report and a task-specific report. Among them, the domain-general report refers to the listening report generated by the domain-general model corresponding to the target domain, and the task-specific report refers to the listening report generated by the task-specific model corresponding to the target task.

[0064] In some embodiments, the listening report has a preset document structure. Among them, the preset document structure can be determined based on a preset listening report template. The content of the preset listening report template can be set according to actual needs. For example, it can include one or more preset headings or content classifications. Only as an example, the listening report template in the two-wheeler field can include headings such as "Background", "Speculated Reasons", "Key Points", "Solutions", etc. Only as an example, the scenario described by the user in the target recording is "I was riding the company's bicycle, and because the brakes were broken and couldn't stop, I hit a telegraph pole and got injured while going downhill. I demand compensation!". Based on the content of the target recording, the target large language model can generate a listening report as follows: "1. Background: The user couldn't stop the bike and hit the telegraph pole; 2. Speculated Reasons: The brakes were broken and the vehicle wasn't retrieved in time, resulting in the user hitting the telegraph pole; 3. Key Points: The user used a faulty vehicle; 4. Solutions: First, retrieve the faulty vehicle in time; Second, when the user rides the vehicle, there should be means to monitor whether the vehicle is damaged; Third, provide corresponding compensation to the user".

[0065] In some embodiments, the preset listening report template may include multiple first listening templates for general domain reports and multiple second listening templates for task-specific reports. The first listening template and the second listening template may be different, and the second listening templates corresponding to different tasks may also be different. For example, a task-specific report for a preset task (such as a traffic accident) needs to include more information related to the task, and its corresponding second listening template may be more detailed. Exemplarily, a task-specific report for a traffic accident scenario in the two-wheeler field may include information such as "accident time", "location", "casualty information", "accident severity level", etc.

[0066] In some embodiments of this specification, by setting different listening report templates for different domains and different tasks, it is possible to guide a target large language model (such as a general domain model, a task-specific model) to generate listening reports that meet different domains and / or different tasks, improving the diversity and pertinence of the listening reports, and providing detailed, accurate, and valuable intelligence for the enterprise's operation team.

[0067] In some embodiments, the first server 110 may also send the listening report or a part thereof to the first user terminal 140 or the terminals of other target users (such as administrators, employees, etc.) through various communication services (such as email service, SMS / MMS service).

[0068] In some embodiments of this specification, introducing the artificial intelligence algorithm of the large language model can perform logical reasoning on the context described in the recording uploaded by the user (such as the second type of user), making the generated listening report more in line with the semantic scenario. In addition, by setting large language models corresponding to different domains and task scenarios in different domains, the listening report is also made more accurate and targeted.

[0069] Figure 3 It is an exemplary flowchart of a method for updating a large language model according to some embodiments of this specification.

[0070] In some embodiments, process 300 may be executed by the first server 110. As Figure 3 shown, process 300 includes the following steps.

[0071] Step 310, for each of multiple domains, based on the new data in the knowledge base corresponding to the domain, determine whether the large language model corresponding to the domain meets the model update condition.

[0072] The new data refers to the data that has newly entered the knowledge base as of the current moment after the large language model of the domain was last updated (such as 20 newly added literature / documents related to the domain)

[0073] In some embodiments, the first server 110 may also perform a hashing operation (such as MD5 encryption) on each uploaded data, generate a corresponding hash value, and mark the new data based on the hash value of each file.

[0074] The model update condition refers to a preset condition that triggers an update to the large language model. In some embodiments, the model update condition may include that the number of new data reaches a preset number threshold (such as adding n pieces), or the capacity reaches a preset capacity threshold (such as adding 300 MB). The first server 110 may set different preset number thresholds and / or preset capacity thresholds according to different fields. Only as an example, for fields with a high degree of attention or a large number of users, the preset number threshold can be set higher (such as 500 pieces), and vice versa, it can be set lower (such as 100 pieces).

[0075] In some embodiments, the model update condition may include that the current time reaches a preset update time point. The preset update time point can be determined based on a preset update period or update frequency. For example, the preset update period is weekly or bi - weekly. Only as an example, for fields with a high degree of attention or a large number of users, the preset update period can be set shorter to ensure that the large language model in this field can learn the characteristics of new data in a timely manner.

[0076] In some embodiments, the first server 110 may continuously monitor the addition situation and update time points of new data in each knowledge base to determine whether the large language model corresponding to the field meets the model update conditions.

[0077] Step 320, in response to determining that the large language model corresponding to the field meets the model update conditions, update the large language model corresponding to the field based on the new data in the knowledge base corresponding to the field.

[0078] It should be noted that the large language model in this field may include the general large language model corresponding to this field and / or multiple dedicated large language models corresponding to multiple tasks. For the content of the general large language model and dedicated large language models, please refer to Figure 2 and its description.

[0079] Taking the general large language model corresponding to a certain field D i as an example, the first server 110 may, based on a method similar to step 420, perform automated pre - processing on the new data in the knowledge base of field D i to obtain new training samples for field D i The first server 110 may, through a method similar to step 430, train the general large language model of this field based on multiple new training samples of field D i to obtain the updated field D iGeneral large language models. In some embodiments, the large language model can be updated based on transfer learning algorithms.

[0080] For a certain domain D i and a certain task T i The corresponding dedicated large language model is updated in a similar way to the general large language model in this domain. The difference is that the training samples of the dedicated large language model are determined based on the new data corresponding to the task T in this domain D i under the task T i corresponding to the new data determined.

[0081] In some embodiments, the first server 110 can verify the updated large language models corresponding to each domain based on the following steps 321 to 323.

[0082] Step 321, determine whether the updated large language model corresponding to the domain meets the model verification conditions.

[0083] The model verification conditions refer to the conditions for determining whether the updated large language model needs to be verified. The model verification conditions can include, but are not limited to, whether the model parameter difference between the updated large language model and the large language model before the update exceeds a threshold (such as 1%), whether the current time reaches a preset verification time (such as weekly, monthly), etc.

[0084] Step 322, in response to determining that the updated large language model corresponding to the domain meets the model verification conditions, use the updated large language model to process the new data to generate a verification report corresponding to the new data.

[0085] The verification report can be used to evaluate the effect (such as accuracy) of the listening reports generated by the updated large language model. In some embodiments, the first server 110 can select multiple new data (such as randomly selecting or the latest multiple new data), and use the updated large language model to process them to generate multiple listening reports as verification reports.

[0086] Step 323, send the verification report to the verifier terminal to verify the updated large language model.

[0087] The verifier terminal can be various terminal devices capable of viewing verification reports (such as mobile phones, tablets, personal computers, etc.). In some embodiments, the verifier terminal can be the first user terminal 140. For example, the first user terminal 140 can be used to present the verification report for the first type of users (such as administrators, listening analysts, etc.) to view. The first server 110 can also send the verification report to the first type of users in the form of an email through communication services such as email. Optionally, the first server 110 can send the original document and the verification report in the new data to the verifier terminal together. The first type of users can view the verification report based on the verifier terminal and evaluate it to determine whether the accuracy of the updated large language model meets the expected standard. For example, the first type of users can evaluate whether the format and content of each listening report in the verification report are correct. If the first type of users evaluate that the updated large language model does not meet the expected standard, the first server 110 can re-execute step 320 to update the large language model again; or the first server 110 can continue to monitor the new data and update the model again when the model update condition is met.

[0088] Step 330, in response to determining that the large language model corresponding to the domain does not meet the model update condition, continue to monitor the knowledge base corresponding to the domain.

[0089] In some embodiments, the first server 110 can execute step 310 based on a preset time interval (such as every 12h, every 24h, etc.) to monitor the knowledge base corresponding to the domain. When at least one of the model update conditions is met, steps 320 to 323 can be executed again to update and verify the large language model corresponding to the domain.

[0090] In some embodiments of this specification, by monitoring the new data in the knowledge bases of each domain, it is possible to automate the update of the large language models in each domain, reduce the time and energy consumption brought by manual inspection, and improve the efficiency of model update.

[0091] Figure 4 It is an exemplary schematic diagram of the training method of the large language model shown in some embodiments of this specification.

[0092] In some embodiments, the first server 110 trains and generates a general large language model for each domain based on the following steps 410 to 430.

[0093] Step 410, obtain training data from the knowledge bases corresponding to each domain.

[0094] Training data can be used to generate training samples for each general large language model for training and / or testing the model. It can include, but is not limited to, knowledge documents, reference documents, historical listening reports, etc. in various fields.

[0095] As Figure 4 shown, the first server 110 can obtain training data from the knowledge bases of domain D1, domain D2... domain D n respectively. Among them, the training data in the knowledge base of domain D1 can be used to train the general large language model of domain D1. Similarly, the training data in the knowledge base of domain D n can be used to train the general large language model of domain D n . For more information about the knowledge base, see Figure 1 and its description.

[0096] Step 420: Automatically preprocess the training data to obtain multiple training samples.

[0097] Preprocessing refers to various processes of analyzing and / or processing the training data to construct training samples. For example, preprocessing can include extracting text information from documents in the knowledge base corresponding to the target domain to obtain semantic text.

[0098] In some embodiments, the preprocessing includes word segmentation processing, stop word removal processing, data cleaning processing, vectorization processing, tagging processing, etc. of the semantic text. In some embodiments, the first server 110 can perform automatic preprocessing on the training data based on various automatic algorithms. For example, it can be implemented based on the algorithms provided by the NLTK library and Scikit-learn library in Python. Among them, word segmentation processing can be used to split long sentences in the semantic text, etc. Stop word removal processing can be used to remove preset redundant words (such as meaningless modal particles, etc.) in the semantic text. Data cleaning processing is used to remove incorrect data, content that does not conform to language norms, etc. Vectorization processing is used to map the text into a digital vector form. Tagging processing is used to tag keywords to determine the preset domain keywords and / or task keywords to which the keyword belongs. For example, a bicycle of a certain brand can be tagged as "two-wheeled vehicle field" or other preset domain keywords.

[0099] In some embodiments, the first server 110 can construct training samples corresponding to the general large language models of each field based on the preprocessed semantic text according to the structure of the preset general interaction template. As Figure 4 shown, the first server 110 can generate training samples for domain D1, training samples for domain D2... domain D based on the preprocessed training data of domain D1, training data of domain D2... domain D n respectively.n Training samples.

[0100] The general interaction template can be used to guide general large language models in various fields to generate corresponding listening reports (i.e., domain-general reports) in various fields. For example, the general interaction template can guide the domain-general model to generate a listening report that is consistent with the document structure of the listening report template (such as the first listening template). Exemplarily, the first server 110 can construct a general interaction template for the electric bicycle field according to the first listening template, which can include guiding the domain-general model corresponding to the electric bicycle field to generate a listening report with an outline consisting of four major parts: background, speculated reasons, key points, and solutions. For the relevant content of the listening report template, see Figure 2 And its description.

[0101] Step 430, use an automatic hyperparameter tuning algorithm to determine the model hyperparameters, and based on multiple training samples and the model hyperparameters, obtain the general large language models corresponding to various fields by training the initial model.

[0102] Model hyperparameters refer to the training parameters configured before model training to guide or control the training process or training effect (such as the convergence speed, etc.). For example, before training the general large language model for domain D1, the first server 110 can pre-determine the model hyperparameters for this training, which can include one or more combinations of, but are not limited to, learning rate, batch size, etc. Among them, the learning rate can be used to control the update amplitude of the model parameters in each iteration, and the batch size can be used to control the number of training samples used in each iteration. It should be noted that the model hyperparameters corresponding to the training of general large language models in different fields can be the same or different.

[0103] In some embodiments, the first server 110 can determine the model hyperparameters based on the training data or the quantity of training samples of the knowledge base corresponding to each field, and the computing resources of the first server 110 (such as the memory size, etc.). For example, if the quantity of training data or training samples is large, the learning rate can be set smaller; if the memory is lower than the capacity threshold, the batch size can be set smaller.

[0104] In some embodiments, the first server 110 can automatically determine the model hyperparameters based on various preset hyperparameter tuning algorithms. Among them, the preset hyperparameter tuning algorithms can include one or a combination of, but are not limited to, grid search algorithm, random search algorithm, Bayesian optimization algorithm, objective function algorithm.

[0105] In some embodiments of this specification, through the automatic hyperparameter tuning algorithm, the model hyperparameters can be automatically adjusted, and the optimal combination of each model hyperparameter and its parameter value can be effectively determined. The efficiency of subsequent model training is improved.

[0106] The initial model can be used for training to obtain the base model of the general large language model corresponding to each field. In some embodiments, the initial model can be a pre-trained model. For example, the initial model can be a pre-trained GPT model, and the initial models in different fields can be the same or different. By fine-tuning the pre-trained model, labor and time costs can be saved, and the training efficiency can be improved.

[0107] After determining the model hyperparameters, as Figure 4 shown, the first server 110 can input the training samples of field D1, the training samples of field D2... the training samples of field D n into the initial model of field D1, the initial model of field D2... the initial model of field D n respectively to perform the training process.

[0108] In some embodiments, the first server 110 can perform automated verification of the model parameters of the initial model during or after the training process based on an automated model verification algorithm. Among them, the automated model verification algorithm can include cross-validation algorithms such as simple cross-validation and K-fold cross-validation. In some embodiments, the first server 110 can implement cross-validation based on the cross_val_score method in the Scikit-learn library.

[0109] In some embodiments, the first server 110 can also monitor the model performance of the initial model during training. For example, when the model performance of the initial model stagnates during training or the error on the validation set increases, this round of training can be stopped, and the parameters of the previous iteration can be used as the model parameters. In some embodiments, the first server 110 can implement model performance monitoring based on methods such as early stopping in Scikit-learn.

[0110] In some embodiments, the first server 110 can also perform automated fine-tuning of the initial model based on an automated model fine-tuning algorithm. For example, during the training process, the first server 110 can adjust the model hyperparameters through the Bayesian optimization algorithm to adjust the convergence speed, model performance, etc.

[0111] In some embodiments of this specification, through the methods of integrating automated data preprocessing, automated hyperparameter adjustment, automated model verification and debugging, and automated model fine-tuning, the automated training of the model is realized. Thus, the training efficiency is greatly improved, and the model training is made easier to manage and optimize.

[0112] Figure 5 is an exemplary schematic diagram of the training method of the dedicated large language model shown in some embodiments of this specification.

[0113] In some embodiments, the first server 110 trains and generates a dedicated large language model corresponding to each task based on the following steps 510 to 530.

[0114] Step 510: Determine a dedicated training sample set corresponding to the task based on multiple training samples.

[0115] The dedicated training sample set refers to a set of multiple training samples used to train a dedicated model for a certain task. For example, it can be a set of multiple training samples for a dedicated model corresponding to the riding interruption scenario in the field of two-wheel vehicles, a set of multiple training samples for a dedicated model corresponding to the traffic accident scenario in the field of two-wheel vehicles, etc.

[0116] In some embodiments, the first server 110 can determine whether each training sample (or the training data corresponding to the training sample) contains one or more preset scenario keywords to determine the target task (or target scenario) to which the training sample belongs, and then classify it into the dedicated training sample set corresponding to the target task. For more information about scenario keywords and dedicated models, see Figure 2 and its description.

[0117] As Figure 5 shown, the first server 110 determines the dedicated training samples for task T1, the dedicated training samples for task T2... the dedicated training samples for task T i based on the training samples in multiple fields D k .

[0118] Step 520: Determine an initial dedicated model corresponding to the field.

[0119] The initial dedicated model can be used for training to obtain the basic model of each task dedicated model. In some embodiments, the initial dedicated model can be a pre-trained BERT model.

[0120] In some embodiments, the initial dedicated model can be obtained through transfer learning based on the general large language models of each field. Exemplarily, the first server 110 can use the model parameters of the trained general large language model of field D i as the initialization parameters of the initial dedicated model, thereby obtaining the initial dedicated model.

[0121] In some embodiments, the initial dedicated models corresponding to multiple task dedicated models in the same field can be the same. As Figure 5 shown, the dedicated large language models for task T1, the dedicated large language models for task T2, the dedicated large language models for task T k can all be trained and obtained via the same initial dedicated model.

[0122] It is understandable that for the domain D i The task-specific model corresponding to the preset task (such as task T1) (such as the dedicated large language model for task T1) and the domain D i The data processed by the domain-general model have certain commonalities. Through the transfer learning method, the task-specific model can, on the basis of retaining the learning results of the domain-general model (i.e., the model parameters of the domain-general model), further learn the characteristics of the dedicated training samples, thereby improving the training efficiency.

[0123] Step 530: Train the initial dedicated model based on the dedicated training sample set to obtain the dedicated large language model corresponding to the task.

[0124] In some embodiments, before training each task-specific model, the first server 110 can use an automatic hyperparameter tuning algorithm to determine the model hyperparameters for training each task-specific model, and based on the dedicated training sample set and the model hyperparameters, train the initial dedicated model to obtain the task-specific model corresponding to each preset task. The training process is similar to the training method of the domain-general large language model in this domain and will not be elaborated here. For more content related to model hyperparameters and the training of general large language models, see Figure 4 And its description.

[0125] In some embodiments of this specification, for multiple different tasks in each domain, customizing the dedicated large language model corresponding to each task can form a personalized listening report for a specific application scenario.

[0126] Figure 6 is a schematic diagram of the hardware and / or software of the voice system shown in some embodiments of this specification.

[0127] For example purposes, the following takes a taxi service enterprise as an example to describe the construction method of the voice system that provides voice analysis services for the taxi service enterprise.

[0128] Such as Figure 6 As shown, the voice system 600 may include a listening assistant 610, a speech recognition model 620, a large language model 630, a data server 640, a knowledge base 650, a listening system server 660, and data 670 from other domains. The listening system server 660 may also be referred to as the second server. The data server 640 may also be referred to as the first server.

[0129] The listening system server 660 is used to collect the recordings uploaded by users (such as passengers, drivers, etc.) who use the taxi service. For example, passengers or drivers can upload recordings through the taxi-hailing APP to file complaints, provide suggestions, etc. about the current ride experience or service.

[0130] The listening assistant 610 can be configured as software or an application for processing a target recording, which can be installed on a terminal device (such as the first user terminal 140).

[0131] The listening assistant 610 can be configured to receive a recording analysis request from a first type of user, obtain the target recording from the listening system server 660, and then convert the target recording into a transcribed text through the speech recognition model 620. The transcribed text is processed by the large language model 630 to generate a listening report. Among them, the large language model 630 can be a domain-general model or a task-specific model in the target domain corresponding to the target recording. In some embodiments, the speech recognition model 620 and / or the large language model 630 can be integrated into the listening assistant 610 or stored on the server side (such as the data server 640 or other servers).

[0132] The listening assistant 610 can be further configured to receive the listening report corresponding to the recording analysis request and send the listening report to the employee 680 (such as the first type of user) based on the communication service 611. The communication service 611 can include email, text message, or other internal communication services (such as the communication service of an internal communication tool).

[0133] In some embodiments, the data server 640 can also be configured to send the listening report to the employee 680. In some embodiments, the data server 640 is configured to update the data in the knowledge base 650 based on a preset scheduled task. For example, the data server 640 can periodically obtain newly generated listening reports and update them to the knowledge base 650.

[0134] As Figure 6 shown, the knowledge base 650 can include multiple knowledge bases in different fields (such as the two-wheeler field and the four-wheeler field), such as the knowledge base 650-1, the knowledge base 650-2,..., the knowledge base 650-n. Each knowledge base can be used to store, including but not limited to, product documents, technical solution documents, analysis documents and solution documents of historical problems, etc.

[0135] In some embodiments, the listening assistant 610 further includes an open capability service 612, which can be configured to provide service interfaces such as FTP file upload. The listening assistant 610 is further configured to obtain data 670 in other fields through the open capability service 612 and can send it to the data server 640. The data 670 in other fields can include data in the fields corresponding to multiple departments.

[0136] In some embodiments, the data server 640 can also receive the data 670 in other fields from multiple departments based on the open capability service 612 and update the knowledge base 650.

[0137] In some embodiments, the data server 640 may periodically provide new training data for the large language model 630 based on the data in the knowledge base 650 to update the large language model 630 based on the new data. For more content on the update of the large language model, see Figure 3 and its description.

[0138] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only an example and does not constitute a limitation to this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification, so such modifications, improvements, and corrections still fall within the spirit and scope of the exemplary embodiments of this specification.

[0139] At the same time, this specification uses specific terms to describe the embodiments of this specification. Such as "one embodiment", "an embodiment", and / or "some embodiments" mean a certain feature, structure, or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that the "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more at different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0140] In addition, unless clearly stated in the claims, the order of the processing elements and sequences, the use of numbers and letters, or the use of other names in this specification are not used to limit the order of the processes and methods in this specification. Although some currently considered useful inventive embodiments are discussed through various examples in the above disclosure, it should be understood that such details only serve the purpose of illustration. The appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that conform to the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only through software solutions, such as installing the described system on existing servers or mobile devices.

[0141] Similarly, it should be noted that in order to simplify the expression of the disclosure of this specification and thus help the understanding of one or more inventive embodiments, in the previous description of the embodiments of this specification, sometimes multiple features are merged into one embodiment, drawing, or description thereof. However, this disclosure method does not mean that the features required by the subject of this specification are more than those mentioned in the claims. In fact, the features of the embodiments are less than all the features of the individual embodiments disclosed above.

[0142] In some embodiments, numbers are used to describe components and the quantity of attributes. It should be understood that such numbers used in the description of embodiments are, in some examples, modified by the modifiers "about", "approximately" or "substantially". Unless otherwise specified, "about", "approximately" or "substantially" indicate that the said numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may vary according to the characteristics required by individual embodiments. In some embodiments, the numerical parameters should consider the specified significant digits and adopt the method of retaining the general number of digits. Although the numerical ranges and parameters used in some embodiments of this specification to confirm the breadth of their scope are approximate values, in specific embodiments, such numerical settings are as precise as possible within the feasible range.

[0143] For each patent, patent application, patent application publication and other materials cited in this specification, such as articles, books, specifications, publications, documents, etc., their entire contents are hereby incorporated into this specification by reference. Except for the application history documents that are inconsistent with or conflict with the content of this specification, and also except for the documents that limit the broadest scope of the claims of this specification (currently or subsequently appended to this specification). It should be noted that if there are inconsistencies or conflicts between the descriptions, definitions, and / or uses of terms in the supplementary materials of this specification and the content described in this specification, the descriptions, definitions, and / or uses of terms in this specification shall prevail.

[0144] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification may be considered to be consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A voice analysis system, characterized in that, Comprising: A first storage device configured to store multiple large language models corresponding to multiple fields; And A first server - side, the first server - side is communicatively connected to a first user terminal and the first storage device, wherein, the first server - side is configured to: In response to a recording analysis request sent by the first user terminal, obtain a target recording; Process the target recording to determine a target field corresponding to the target recording from the multiple fields; Process the target recording using the target large language model corresponding to the target field to generate a listening report corresponding to the target recording.

2. The system according to claim 1, further comprising: A second server - side, the second server - side is used to collect recordings uploaded through a second user terminal, and the target recording is obtained by the first server - side from the second server - side.

3. The system according to claim 2, wherein the recording analysis request includes keyword information, and the target recording is obtained by the first server - side from the second server - side based on the keyword information.

4. The system according to claim 1, wherein The processing of the target recording to determine a target field corresponding to the target recording from the multiple fields includes: Process the target recording using a speech recognition model to generate a transcription text corresponding to the target recording, the speech recognition model being a trained machine - learning model; Process the transcription text using a field discrimination model to determine the target field corresponding to the target recording, the field discrimination model being a trained machine - learning model.

5. The system according to claim 1, characterized in that Further comprising: A second storage device configured to store multiple knowledge bases corresponding to the multiple fields; The first server - side is further configured to monitor the multiple knowledge bases and update the multiple large language models in the first storage device based on new data in the multiple knowledge bases.

6. The system according to claim 5, wherein The updating of at least one large language model in the first storage device based on new data in the knowledge bases corresponding to the multiple fields includes: For each field in the multiple fields, Based on the new data in the knowledge base corresponding to the field, determine whether the large language model corresponding to the field meets the model update condition; In response to determining that the large language model corresponding to the field meets the model update condition, update the large language model corresponding to the field based on the new data in the knowledge base corresponding to the field; In response to determining that the large language model corresponding to the field does not meet the model update condition, continue to monitor the knowledge base corresponding to the field.

7. The system according to claim 6, characterized in that, In response to determining that the large language model corresponding to the field meets the model update condition, the first server - side is further configured to: Determine whether the updated large language model corresponding to the field meets the model verification condition; In response to determining that the updated large language model corresponding to the field meets the model verification condition, process the new data using the updated large language model to generate a verification report corresponding to the new data; Send the verification report to the first user terminal to verify the updated large language model.

8. The system according to claim 5, characterized in that, The large language model corresponding to each of the said fields includes a general large language model, The general large language model corresponding to each of the said fields is generated by the first server side through the following training process: Obtain training data from the knowledge base corresponding to the field; Automatically preprocess the training data to obtain a plurality of training samples; Use an automatic hyperparameter tuning algorithm to determine model hyperparameters; Based on the plurality of training samples and the model hyperparameters, obtain the general large language model corresponding to the field by training an initial model.

9. The system according to claim 8, wherein The large language model corresponding to each of the said fields further includes a plurality of dedicated large language models corresponding to a plurality of tasks, The dedicated large language model corresponding to each of the said tasks is generated through the following training process: Based on the plurality of training samples, determine a dedicated training sample set corresponding to the task; Determine an initial dedicated model corresponding to the field; Train the initial dedicated model based on the dedicated training sample set to obtain the dedicated large language model corresponding to the task.

10. A voice analysis method is executed by a first server end of a voice analysis system. The voice analysis system includes a first storage device and the first server end. The first storage device is configured to store multiple large language models corresponding to multiple fields. The first server end is communicatively connected to a first user terminal and the first storage device. It is characterized in that, The method includes: In response to a recording analysis request sent by a first user terminal, obtain a target recording; Process the target recording to determine a target field corresponding to the target recording from the plurality of fields; Use the target large language model corresponding to the target field to process the target recording to generate a listening report corresponding to the target recording.

Images