Information extraction method for bulk commodity market investigation voice

Abstract

The invention belongs to the technical field of artificial intelligence, and relates to an information extraction method for bulk commodity market investigation voice, which comprises the following steps: extracting an original MP3 audio file into an effective voice segment set by adopting a VAD model; constructing a speech recognition model, recognizing the effective speech segment set, and generating an original text set; performing sentence division and punctuation recovery on the original text set based on a PUNC model to form a standard text; carrying out speaker clustering according to the voiceprint features of the voice segments by utilizing a Speaker model, and carrying out association through timestamps to form a dialogue text; constructing and training a large language model, removing irrelevant information in a correct text in combination with basic attributes of samples, and supplementing key tags; according to the basic attribute and the dynamic attribute of the sample, a cue word of a fine-tuning large language model is constructed, a thinking chain is introduced to perfect the cue word, and key information is extracted; and sorting the extracted key information by utilizing the credibility, and taking a result with the highest credibility as output to realize the extraction of the voice information.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for extracting information from voice recordings used in commodity market research. Background Technology

[0002] Currently, routine research on commodity-related data primarily relies on manual transcription. The specific process involves researchers and traders communicating daily by phone to manually input and record key research data, such as prices and price fluctuations. This method is highly dependent on manpower, which is not only inefficient but also prone to human error, affecting data quality and timeliness.

[0003] Invention patent CN114491010B proposes an information extraction model through active learning and iterative training for different domains and dimensions. While this method improves the model's extraction capabilities for specific domains, it still heavily relies on manual annotation, and the model's generalization ability is significantly insufficient. When facing the same research task across domains, it is often necessary to retrain a dedicated information extraction model, leading to increased application costs and timelines. Invention patent CN114372454B employs a multi-model collaborative mechanism, first identifying relationships and subjects, and then extracting objects to improve the accuracy of information extraction in multi-relationship scenarios. However, the relationship and entity extraction models used in this method are typically trained on written corpora, making it difficult to adapt to the complex dialogue contexts in commodity research tasks. Actual research dialogues often involve multiple entities and objects, making it difficult for the model to make accurate judgments in one go, thus affecting the extraction results.

[0004] Furthermore, the aforementioned patents all focus on text information extraction, rather than voice information extraction. Compared to text information extraction tasks, voice data may contain errors and omissions during the transcription process, and researchers' expression logic on the phone is often less standardized than written language, which further increases the difficulty of information extraction.

[0005] In the current field of text information extraction, mainstream methods generally rely on Large Language Models (LLMs) to achieve automated extraction. However, directly transferring these technologies to speech information extraction, especially in the context of voice-based research on bulk commodities, faces numerous challenges.

[0006] First, speech information extraction is highly dependent on the accuracy of speech recognition. However, researchers are geographically dispersed with significant accent differences, and commodity research involves a large amount of technical terminology, which places higher demands on the adaptability and accuracy of speech recognition models. Although the recognition effect can be improved by fine-tuning the speech recognition model, this process relies heavily on a large amount of high-quality labeled data, which is difficult to obtain in practical applications. Furthermore, while large language models can be used to post-process and correct transcribed text, alleviating the dependence on high-quality labeled data to some extent, large models typically require regenerating the entire dialogue content, resulting in a slow overall correction speed that fails to meet the practical needs of efficient information extraction.

[0007] Secondly, conversations between researchers often exhibit characteristics such as loose logic, high levels of colloquialism, and ambiguous subject-object relationships. For example, in some short dialogues, a single subject may correspond to multiple objects, meaning one research category may be associated with multiple prices (where the subject is the research category and the object is the price of that category). This complex subject-object relationship significantly increases the difficulty of information extraction, posing a greater challenge to large language models in the discrimination and extraction process. Especially in cases of colloquial expressions and ambiguous semantics, the model struggles to accurately identify and associate various entities and attributes, thus affecting the accuracy and completeness of the extraction results. Furthermore, bulk commodities, such as iron ore, are usually subdivided by category (e.g., 66% grade iron concentrate, 65% grade iron concentrate, etc.), while actual surveys often cover multiple categories simultaneously. This further exacerbates the complexity and ambiguity of category division, significantly increasing the difficulty of information extraction.

[0008] Therefore, there is a need for a speech information extraction method that can overcome practical problems such as insufficient accuracy of speech recognition in commodity market research, difficulty in terminology recognition, low efficiency of text correction, and complex information extraction logic. Summary of the Invention

[0009] To address the aforementioned technical problems, this invention provides a method for extracting information from audio recordings used in commodity market research. The invention first converts the research audio into complete dialogue text; then, by correcting the dialogue text, it improves the accuracy of information extraction. A large language model is constructed and trained, and information extraction is performed using this model. The result with the highest credibility is used as the final extraction result. This invention's method integrates Voice Activity Detection (VAD), Automatic Speech Recognition (ASR), Punctuation Recovery (PUNC), and Speaker Recognition models. Through timestamp alignment and data integration, it ensures the output of accurate dialogue-style text.

[0010] The technical means employed in this invention are as follows: A method for extracting information from speech in commodity market research includes: converting the original MP3 audio file into a standardized WAV format and extracting a set of effective speech segments using a VAD model; constructing a speech recognition model and training it using historical data; recognizing the effective speech segments using the speech recognition model and generating an original text set; segmenting the original text set into sentences and restoring punctuation based on a PUNC model to form standardized text; clustering speakers based on the voiceprint features of the speech segments using a Speaker model and forming dialogue text by associating them with timestamps; constructing and training a large language model to correct the dialogue text into correct text; removing irrelevant information from the correct text and adding key labels based on the basic attributes of the samples; constructing prompt words for the fine-tuned large language model based on the basic and dynamic attributes of the samples, and introducing thought chains to improve the prompt words and extract key information; ranking the extracted key information based on credibility, and using the result with the highest credibility as the output to achieve the extraction of speech information.

[0011] Furthermore, the process of converting the original MP3 audio file to a standardized WAV format and extracting a set of valid speech segments using a VAD model specifically includes: through... The tool library converts raw MP3 audio files into a standardized WAV format, generating a uniform audio byte stream:

[0012] in, Represents the original audio data matrix. Represents the set of real numbers. This represents the total number of sample points in the original audio. Indicates the number of original audio channels. This represents the preprocessed audio data. This represents the total number of sampling points after resampling; The VAD model is used to extract a set of valid speech segments:

[0013] in, This represents the set of valid speech segments output. This represents a speech activity detection function based on the VAD model. Indicates the first One valid speech segment, This indicates the total number of detected speech segments.

[0014] Furthermore, the construction of the speech recognition model and the training of the speech recognition model using historical data specifically includes: based on professional corpus, using a speech cloning model to clone the voice corpus of the surveyors and generate audio files; for easily misspelled professional terms, using a large language model to randomly generate colloquial corpus containing easily misspelled professional terms, and then using the speech cloning model to clone the voice corpus and generate audio files.

[0015]

[0016] in, This refers to the cloned audio. This represents randomly sampled audio. This refers to the text used for audio synthesis. This represents the augmented training dataset. This refers to the original audio recordings of the research staff. , They represent The corresponding text and timestamp, This refers to the cloning process for professional audio. This indicates the speech-to-text process. This indicates the process of manually calibrating the data; The enhanced training dataset was used to fine-tune the speech recognition model:

[0017] in, This represents the fine-tuned speech recognition model parameters. Indicates the parameters of the original speech recognition model. This describes the process of fine-tuning the speech recognition model.

[0018] Furthermore, the process of generating the original text set specifically includes: using the constructed speech recognition model to identify the effective speech segment set and generate the original text set.

[0019] in, This represents the original set of text output by the speech recognition model. Indicates the first The original text corresponding to each speech segment This indicates the word sequence with the highest probability of selection. , Represents the conditional probability of a word sequence given features. The feature extraction function is derived from the first... Acoustic feature vectors extracted from each valid speech segment.

[0020] Furthermore, the process of forming the standardized text specifically includes: using the PUNC model to segment sentences and restore punctuation from the original text set, and forming a complete standardized text.

[0021] in, This indicates that the output should include the text sequence with punctuation. Indicates the first A text string with punctuation. This represents the punctuation conditional probability given a context. Indicates an embedded function, from The context vector extracted from it.

[0022] Furthermore, the process of forming the dialogue text specifically includes: for the set of valid speech segments, using a Speaker model to cluster speakers based on the voiceprint features of the speech segments, and outputting the speaker ID corresponding to each sentence:

[0023] in, This indicates that the speaker label sequence should be output. Indicates the first Speaker tags for each speech segment Represents the clustering function. Indicates the embedding extraction function, Indicates the first One valid speech segment.

[0024] By using timestamps to associate the outputs of the VAD model, speech recognition model, PUNC model, and Speaker model, a complete speech semantic parsing result is constructed, forming the dialogue text.

[0025] Furthermore, the construction and training of the large language model specifically includes: generating a training dataset for fine-tuning by collecting a large amount of erroneous text and its corresponding correct text from real-world commodity survey speech transcriptions. This indicates the text to be corrected. Indicates the index position corresponding to the misspelled word. The correct text corresponding to the misspelled words is represented by the following formula for generating the training dataset:

[0026] in, This represents the training dataset used for fine-tuning a large language model. This represents the function that generates the training dataset. Indicates the corresponding index position for correction; The large language model is fine-tuned using the training dataset to enable it to locate and correct misspellings; a loss function is constructed that combines misspelling location index prediction and corrected character prediction.

[0027] in, This indicates a correction for character prediction loss. This represents the error location index prediction loss. These are the weighting coefficients for the two parts of the loss, respectively; Using the training dataset Combined with loss function The fine-tuned large language model was obtained. :

[0028] For the dialogue text to be corrected Using the fine-tuned large language model Obtain the index of the misspelled text and the correct text to be corrected. ;by As input, use the misspelling replacement algorithm The completion of dialogue text correction can be expressed by the formula:

[0029] in, This is the corrected text.

[0030] Furthermore, the step of removing irrelevant information from the correct text and supplementing key tags by combining the basic attributes of the sample specifically includes: combining the basic attributes of the survey sample... As supplementary input, combined with the corrected dialogue text The large language model is used to remove information irrelevant to the survey and to add key tags, which can be expressed by the formula:

[0031] in, This refers to the text after information refinement and completion.

[0032] Furthermore, the extraction of key information specifically includes: based on the basic attributes of the survey sample. and dynamic attributes Initially constructing prompt words for the large language model to improve its information understanding and extraction capabilities:

[0033] in, Prompt words indicating initial construction, This represents a prompt construction function that integrates attributes S and H based on a preset prompt framework; In the information extraction stage, for complex scenarios where a basic attribute in the dialogue content corresponds to multiple dynamic attributes, a thought chain is introduced to improve the prompt words, guiding the large language model to fully cover and extract all potentially correct data:

[0034] in, The prompt word indicates the improved version. This indicates that the attributes S and H are fused based on the preset prompt word framework. The prompt word construction function.

[0035] Furthermore, the extracted key information is sorted based on credibility, and the result with the highest credibility is taken as the output. Specifically, this includes: the improved prompt words. It includes a credibility score for each dynamic attribute, expressed by the formula:

[0036] in, This represents the possible results of extracting a basic attribute. Indicates the number of possible outcomes; This indicates the confidence score corresponding to the possible results. This indicates the text after information refinement and completion. This indicates the specific extraction process using a large model; The text is refined and completed using a large language model. Market judgments are made, with results categorized as positive, neutral, or negative; the upward or downward trend is assumed to be... The sentiment assessment results are compared with the trend; if market sentiment matches the upward or downward trend, the corresponding credibility is positively weighted; if they do not match, a negative correction is made; if the sentiment assessment result is neutral, the credibility is not affected; the credibility correction formula is as follows:

[0037] in, For indicator functions, This represents a weighted score that incorporates market sentiment to determine credibility. This represents the self-evaluation score of the large model used to extract information for the extracted results; , These are weighting coefficients; Sort all extracted results according to their confidence level, and select the result with the highest confidence level as the final output.

[0038] Compared with the prior art, the present invention has the following advantages: This invention innovatively proposes a data synthesis method based on speech cloning and a large language model. By cloning the voice of researchers and combining it with colloquial text containing error-prone professional terms generated by a large model, a large-scale, high-quality synthetic training dataset is automatically constructed. This effectively overcomes the problems of reliance on manual annotation and high data acquisition costs in traditional methods, enabling the ASR model to be deeply fine-tuned for specific terms in the commodity field (such as the name of iron ore of a specific grade) and colloquial expressions, significantly improving the recognition accuracy of speech transcription.

[0039] This invention employs a method of training and fine-tuning a large model, abandoning the inefficient mode of traditional large models that require regenerating the entire dialogue. While ensuring the quality of error correction, it significantly improves the reasoning speed of text processing, meeting the timeliness requirements of market research and data collection tasks.

[0040] To address the complexities of logical jumps, ambiguous referencing, and multiple objects corresponding to a single subject (e.g., multiple prices for a single commodity category) in research dialogues, this invention designs a multi-turn dialogue extraction strategy based on the CoT (Co-Linguistics of Thought) framework. This strategy significantly reduces information omissions and illusions, and improves the usability of structured data.

[0041] This invention integrates a complete technology stack, from audio preprocessing, role separation, text transcription, error correction to the extraction of final structured information (such as dynamic attribute information like survey category prices and price fluctuations). The method automates the previously cumbersome process of manually listening to and recording survey data, significantly shortening the time from survey to data entry and avoiding errors caused by human recording. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart of the information extraction method for voice-based research on bulk commodity markets in this invention.

[0044] Figure 2 A flowchart for generating the training dataset for the speech recognition model in this invention.

[0045] Figure 3 This is a flowchart illustrating the audio-to-dialogue text conversion process in this invention.

[0046] Figure 4 This is a sample dialogue text in an embodiment of the present invention.

[0047] Figure 5 This is a flowchart of the dialogue text correction process based on a large language model in this invention.

[0048] Figure 6 This is a flowchart of the information extraction and output results in this invention. Detailed Implementation

[0049] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0052] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following figures denote similar items; therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0053] like Figure 1 As shown, this invention provides a method for extracting information from speech during commodity market research, including: converting the original MP3 audio file into a standardized WAV format (16kHz sampling rate, mono, PCM_S16LE encoding), and using a VAD model to extract a set of effective speech segments; in a preferred embodiment of this invention, through... The tool library converts the original MP3 audio file into a standardized WAV format, generating a uniform audio byte stream, providing a consistent data input foundation for subsequent models.

[0054] in, Represents the original audio data matrix. Represents the set of real numbers. This represents the total number of sample points in the original audio. Indicates the number of original audio channels. This represents the preprocessed audio data. This represents the total number of sampling points after resampling; The survey audio contains a large amount of non-dialogue noise. To effectively improve recognition accuracy, it is necessary to detect valid speech (i.e., dialogue speech) in the original audio and extract valid speech segments. A VAD model (FSMN network) is used to extract a set of valid speech segments:

[0055] in, This represents the set of valid speech segments output. This represents a speech activity detection function based on the VAD model. Indicates the first One valid speech segment, This indicates the total number of detected speech segments.

[0056] Because the survey audio consists of informal professional dialogues with numerous simplified and specialized expressions, existing publicly available models cannot meet the required conversion accuracy. Therefore, it is necessary to train a professional speech recognition model (i.e., the ASR model mentioned above) to convert such audio into text.

[0057] A speech recognition model (ASR model Paraformer-Large) is constructed and trained using historical data. In a preferred embodiment, based on professional corpus, a speech cloning model is used to clone the voice data of researchers and generate audio files. For error-prone professional vocabulary, a large language model is used to randomly generate colloquial corpus containing these error-prone terms, and then the speech cloning model is used to clone the voice data to generate audio files.

[0058]

[0059] in, This refers to the cloned audio. This represents randomly sampled audio. This refers to the text used for audio synthesis (such as research reports, news articles, and other professional corpora in the field, as well as colloquial corpora generated from easily misspelled words). This represents the augmented training dataset (including audio files and their corresponding timestamps and text). This refers to the original audio recordings of the research staff. , They represent The corresponding text and timestamp, This refers to the cloning process for professional audio. This indicates the speech-to-text process. This describes the process of manually calibrating data (audio, text, timestamps); The enhanced training dataset was used to fine-tune the speech recognition model:

[0060] in, This represents the fine-tuned speech recognition model parameters. Indicates the parameters of the original speech recognition model. This describes the process of fine-tuning the speech recognition model. The specific steps are as follows: Figure 2 As shown. Specifically, model parameters These are the parameters used in the ASR model mentioned above.

[0061] A speech recognition model is used to identify a set of valid speech segments and generate an original text set. Specifically, in a preferred embodiment of this invention, the constructed speech recognition model is used to identify the set of valid speech segments and generate the original text set.

[0062] in, This represents the original set of text output by the speech recognition model. Indicates the first The original text corresponding to each speech segment This indicates the word sequence with the highest probability of selection. , Represents the conditional probability of a word sequence given features. The feature extraction function is derived from the first... Acoustic feature vectors (Mel spectrum coefficients) extracted from each effective speech segment.

[0063] The original text set is segmented into sentences and punctuation restored using the PUNC model (CT-Transformer) to form a standardized text. Specifically, as a preferred embodiment of this invention, the PUNC model is used to segment the original text set into sentences and restore punctuation, thus forming a complete standardized text.

[0064] in, This indicates that the output should include the text sequence with punctuation. Indicates the first A text string with punctuation. This represents the punctuation conditional probability given a context. Indicates an embedded function, from The context vector extracted from it.

[0065] For the set of valid speech segments, the Speaker Model (CAM++) is used to cluster speakers based on the voiceprint features of the speech segments, and then the dialogue text is formed by associating them through timestamps. The dialogue text conversion process is as follows: Figure 3 As shown, the dialogue text is as follows: Figure 4 As shown; in a preferred embodiment of the present invention, the speaker ID (spk0, spk1, etc.) corresponding to each sentence is output:

[0066] in, This indicates that the speaker label sequence should be output. Indicates the first Speaker tags for each speech segment Represents the clustering function. Indicates the embedding extraction function, Indicates the first One valid speech segment.

[0067] VAD identifies "when to speak," ASR recognizes "what was said," PUNC annotates "how to break sentences," and the Speaker model identifies "who is speaking." By using timestamps to correlate the outputs of the VAD model, speech recognition model, PUNC model, and Speaker model, a complete speech semantic analysis result is constructed, forming the dialogue text. Spoken dialogues are prone to noise interference; for example, when speaker A speaks a long sentence, speaker B may interfere. Short sentences from the same speaker are merged based on a minimum sentence length threshold to eliminate interference.

[0068] Using the obtained dialogue text for information extraction still fails to achieve the desired accuracy. This is because colloquial expressions and rare technical terms lead to errors in the dialogue text. For example, "refined powder" is recognized as "gold powder", "New powder" is recognized as "cow powder", etc.

[0069] Construct and train a large language model to correct the dialogue text into correct text, as Figure 5 shown; specifically in implementation, as a preferred implementation manner of the present invention, by collecting a large number of error texts of voice transcriptions of real commodity research and their corresponding correct texts, a training data set for fine-tuning is generated, making represent the text to be corrected, represent the index position corresponding to the misspelled character, represent the correct character corresponding to the misspelled character, and the generation of the training data set is expressed by the formula:

[0070] Among them, represents the training data set for fine-tuning the large language model, represents the training data set generation function, represents the index position corresponding to the correction; Fine-tune the large language model with the training data set to enable the large language model to have the ability to locate and correct misspelled characters; construct a loss function, and the loss function combines two parts: prediction of the misspelled character position index and prediction of the corrected character:

[0071] Among them, represents the loss of corrected character prediction, represents the loss of misspelled character position index prediction, are the weight coefficients of the losses of the two parts respectively; Using the training data set combined with the loss function , obtain the fine-tuned large language model :

[0072] For the dialogue text to be corrected , using the fine-tuned large language model , obtain the misspelled character position index and the correct text to be corrected ; taking as the input, use the misspelled character replacement algorithm to complete the correction of the dialogue text, which is expressed by the formula:

[0073] Among them, This is the corrected text. This method accurately outputs the list of correct characters to be replaced and their exact positions in the original text, improving text correction efficiency.

[0074] Extracting useful information from dialogue texts presents two main challenges. First, the texts contain a large amount of content irrelevant to the research, interfering with information extraction. Second, the highly refined, conversational language of the dialogues lacks key subjects and modifiers (key tags), reducing the accuracy of information extraction.

[0075] Based on the basic attributes of the sample (such as product category, region, manufacturer, etc.), irrelevant information is removed from the correct text, and key tags are added; in specific implementation, as a preferred embodiment of the present invention, the basic attributes of the survey sample are... As supplementary input, combined with the corrected dialogue text The large language model is used to remove information irrelevant to the survey and to add key tags, which can be expressed by the formula:

[0076] in, This refers to the text after information refinement and completion.

[0077] Based on the basic and dynamic attributes of the sample (such as historical prices, fluctuations, etc.), prompt words for fine-tuning the large language model are constructed, and thought chains are introduced to improve the prompt words and extract key information. In specific implementation, as a preferred embodiment of the present invention, the basic attributes of the survey sample are used to construct prompt words. and dynamic attributes Initially constructing prompt words for the large language model to improve its information understanding and extraction capabilities:

[0078] in, Prompt words indicating initial construction, This represents a prompt construction function that integrates attributes S and H based on a preset prompt framework; In the information extraction stage, for complex scenarios where a basic attribute in the dialogue content corresponds to multiple dynamic attributes, a Chain of Thought (COT) is introduced to refine the prompt words, guiding the large language model to fully cover and extract all potentially correct data:

[0079] in, The prompt word indicates the improved version. This indicates that the attributes S and H are fused based on the preset prompt word framework. The prompt word construction function.

[0080] The extracted key information is sorted by credibility, and the result with the highest credibility is used as the output to extract voice information. In a preferred embodiment of this invention, when a basic attribute is extracted into multiple dynamic attributes (e.g., the basic attribute "iron ore" corresponds to five prices), the improved prompt words are... It includes a credibility score for each dynamic attribute, expressed by the formula:

[0081] in, This represents the possible results of extracting a basic attribute. Indicates the number of possible outcomes; This indicates the confidence score corresponding to the possible results. This indicates the text after information refinement and completion. This indicates the specific extraction process using a large model; To further improve the accuracy of the collected information, constraints are established based on historical information to assist in calculating the reliability score. For example, when extracting price information, a large language model is used to refine and complete the text. Market judgments are made, with results categorized as positive, neutral, or negative; the upward or downward trend is assumed to be... The sentiment assessment results are compared with the trend; if market sentiment matches the upward or downward trend, the corresponding credibility is positively weighted; if they do not match, a negative correction is made; if the sentiment assessment result is neutral, the credibility is not affected; the credibility correction formula is as follows:

[0082] in, For indicator functions, This represents a weighted score that incorporates market sentiment to determine credibility. This represents the self-evaluation score of the large model used to extract information for the extracted results; , These are weighting coefficients; All extracted results are sorted according to their confidence level, and the result with the highest confidence level is selected as the final output. The extraction process is as follows: Figure 6 As shown, it achieves multi-dimensional fusion, intelligent weighting, and highly reliable information extraction.

[0083] Example The results of information extraction using the large language model in this invention are shown in Table 1.

[0084] Table 1. Results of large language model extraction from iron concentrate samples

[0085] The experimental data in Table 1 clearly demonstrate the superiority of this invention in practical applications, leading to the following conclusions: 1. The overall technical solution significantly improves performance. Compared to Group D, which relies solely on basic speech and text, the "perfect match rate" in each market increased by an average of 20 to 30 percentage points after adopting the complete method of this invention (Group A) (for example, the Chengde market jumped from 46.7% to 76.7%). This directly proves that the combined approach of "speech cloning fine-tuning + LLM text correction + thought chain extraction" used in this invention can effectively solve the problem of low recognition rate caused by heavy accents and numerous technical terms in traditional methods, and greatly improve the usability of unstructured speech data.

[0086] 2. The necessity of the core modules has been validated; observing the data decline in Group B (excluding market sentiment analysis) and Group C (excluding historical information fusion), it can be seen that: The introduction of historical information is crucial: the significant drop in accuracy of Group C indicates that when facing the complex logic of "multiple categories and multiple prices" in commodity research, using historical data to establish constraints can effectively assist model reasoning and avoid misattribution.

[0087] The sentiment verification mechanism is effective: the decline in Group B data shows that ranking market sentiment by comparing it with the upward and downward trends can indeed filter out some logically contradictory erroneous information, further improving the accuracy of the final output.

[0088] 3. The method is highly robust; in tests across four different regions, the data from Group A maintained a high level of stability, with no drastic fluctuations in recognition rate due to regional accent differences. This indirectly confirms that the method of fine-tuning the ASR model by synthesizing speech clone data in this invention effectively overcomes the challenges posed by the wide geographical distribution and significant accent differences among researchers.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting information from voice recordings used in commodity market research, characterized in that, include: The original MP3 audio file is converted to the standardized WAV format, and the effective speech segment set is extracted using the VAD model; Construct a speech recognition model and train the speech recognition model using historical data; The speech recognition model is used to identify the set of valid speech segments and generate the original text set. Based on the PUNC model, the original text set is segmented into sentences and punctuation restored to form a standardized text. For the set of effective speech segments, the Speaker Model is used to cluster speakers based on the voiceprint features of the speech segments, and the dialogue text is formed by associating them with timestamps. Build and train a large language model to correct the dialogue text into the correct text; Based on the basic attributes of the sample, irrelevant information is removed from the correct text and key tags are added; Based on the basic and dynamic attributes of the sample, the prompt words of the fine-tuned large language model are constructed, and the thought chain is introduced to improve the prompt words and extract key information. The extracted key information is sorted by credibility, and the result with the highest credibility is used as the output to extract voice information.

2. The method for extracting information from voice recordings in commodity market research according to claim 1, characterized in that, The process of converting the original MP3 audio file to a standardized WAV format and extracting a set of valid speech segments using the VAD model specifically includes: pass The tool library converts raw MP3 audio files into a standardized WAV format, generating a uniform audio byte stream: in, Represents the original audio data matrix. Represents the set of real numbers. This represents the total number of sample points in the original audio. Indicates the number of original audio channels. This represents the preprocessed audio data. This represents the total number of sampling points after resampling; The VAD model is used to extract a set of valid speech segments: in, This represents the set of valid speech segments output. This represents a speech activity detection function based on the VAD model. Indicates the first One valid speech segment, This indicates the total number of detected speech segments.

3. The method for extracting information from voice data in commodity market research according to claim 1, characterized in that, The construction of the speech recognition model and the training of the speech recognition model using historical data specifically includes: Based on the professional corpus, a speech cloning model was used to clone the researchers' voice data and generate audio files. For easily misspelled professional terms, a large language model was used to randomly generate colloquial corpus containing these terms, and then the speech cloning model was used to clone the voice data to generate audio files. in, This refers to the cloned audio. This represents randomly sampled audio. This refers to the text used for audio synthesis. This represents the augmented training dataset. This refers to the original audio recordings of the research staff. , They represent The corresponding text and timestamp, This refers to the cloning process for professional audio. This indicates the speech-to-text process. This indicates the process of manually calibrating the data; The enhanced training dataset was used to fine-tune the speech recognition model: in, This represents the fine-tuned speech recognition model parameters. Indicates the parameters of the original speech recognition model. This describes the process of fine-tuning the speech recognition model.

4. The method for extracting information from voice recordings in commodity market research according to claim 1, characterized in that, The process of generating the original text set specifically includes: Using the constructed speech recognition model, the set of valid speech segments is identified and the original text set is generated: in, This represents the original set of text output by the speech recognition model. Indicates the first The original text corresponding to each speech segment This indicates the word sequence with the highest probability of selection. , Represents the conditional probability of a word sequence given features. The feature extraction function is derived from the first... Acoustic feature vectors extracted from each valid speech segment.

5. The method for extracting information from voice data in commodity market research according to claim 4, characterized in that, The process of forming the standard text specifically includes: The PUNC model is used to segment sentences and restore punctuation from the original text set, forming a complete and standardized text: in, This indicates that the output should include a sequence of text with punctuation. Indicates the first A text string with punctuation. This represents the punctuation conditional probability given a context. Indicates an embedded function, from The context vector extracted from it.

6. The method for extracting information from voice data in commodity market research according to claim 1, characterized in that, The process of forming the dialogue text specifically includes: For the valid set of speech segments, the Speaker Model is used to cluster speakers based on the voiceprint features of the speech segments, and the speaker ID corresponding to each sentence is output: in, This indicates that the speaker label sequence should be output. Indicates the first Speaker tags for each speech segment Represents the clustering function. Indicates the embedding extraction function, Indicates the first One valid speech segment. By using timestamps to associate the outputs of the VAD model, speech recognition model, PUNC model, and Speaker model, a complete speech semantic parsing result is constructed, forming the dialogue text.

7. The method for extracting information from voice data in commodity market research according to claim 1, characterized in that, The construction and training of the large language model specifically includes: By collecting a large amount of erroneous text and its corresponding correct text from real-world commodity survey speech transcriptions, a training dataset for fine-tuning is generated. This indicates the text to be corrected. Indicates the index position corresponding to the misspelled word. The correct text corresponding to the misspelled words is represented by the following formula for generating the training dataset: in, This represents the training dataset used for fine-tuning a large language model. This represents the function that generates the training dataset. Indicates the corresponding index position for correction; The large language model is fine-tuned using the training dataset to enable it to locate and correct misspellings; a loss function is constructed that combines misspelling location index prediction and corrected character prediction. in, This indicates a correction for character prediction loss. This represents the error location index prediction loss. These are the weighting coefficients for the two parts of the loss; Using the training dataset Combined with loss function The fine-tuned large language model was obtained. : For the dialogue text to be corrected Using the fine-tuned large language model Obtain the index of the misspelled text and the correct text to be corrected. ;by As input, use the misspelling replacement algorithm The completion of dialogue text correction can be expressed by the formula: in, This is the corrected text.

8. The method for extracting information from voice data in commodity market research according to claim 7, characterized in that, The process of removing irrelevant information from the correct text and adding key tags based on the basic attributes of the samples specifically includes: Basic attributes of the survey sample As supplementary input, combined with the corrected dialogue text The large language model is used to remove information irrelevant to the survey and to add key tags, which can be expressed by the formula: in, This refers to the text after information refinement and completion.

9. The method for extracting information from voice recordings in commodity market research according to claim 1, characterized in that, The extracted key information specifically includes: Based on the basic attributes of the survey sample and dynamic attributes Initially constructing prompt words for the large language model to improve its information understanding and extraction capabilities: in, Prompt words indicating initial construction, This represents a prompt construction function that integrates attributes S and H based on a preset prompt framework; In the information extraction stage, for complex scenarios where a basic attribute in the dialogue content corresponds to multiple dynamic attributes, a thought chain is introduced to improve the prompt words, guiding the large language model to fully cover and extract all potentially correct data: in, The prompt word indicates the improved version. This indicates that the attributes S and H are fused based on the preset prompt word framework. The prompt word construction function.

10. The method for extracting information from voice recordings in commodity market research according to claim 9, characterized in that, The process of ranking the extracted key information based on credibility and outputting the result with the highest credibility includes: Improved prompt words It includes a credibility score for each dynamic attribute, expressed by the formula: in, This represents the possible results of extracting a basic attribute. Indicates the number of possible outcomes; This indicates the confidence score corresponding to the possible results. This indicates the text after information refinement and completion. This indicates the specific extraction process using a large model; The text is refined and completed using a large language model. Market judgments are made, with results categorized as positive, neutral, or negative; the upward or downward trend is assumed to be... The sentiment assessment results are compared with the trend; if market sentiment matches the upward or downward trend, the corresponding credibility is positively weighted; if they do not match, a negative correction is made; if the sentiment assessment result is neutral, the credibility is not affected; the credibility correction formula is as follows: in, For indicator functions, This represents a weighted score that incorporates market sentiment to determine credibility. This represents the self-evaluation score of the large model used to extract information for the extracted results; , These are weighting coefficients; Sort all extracted results according to their confidence level, and select the result with the highest confidence level as the final output.

Images