A voiceprint optimization method and system based on time length weighting fusion and quality optimization

Abstract

The application relates to a voiceprint optimization method and system based on time length weighted fusion and quality optimization, an electronic device and a computer readable storage medium. The voiceprint feature vector, the timestamp and the subsegment time length are obtained by acquiring the audio subsegments after the talk audio is divided, the voiceprint feature vector is compared and matched with the speaker archive of an offline single terminal, and the target archive to which the audio subsegment belongs is determined; the time length weighted fusion is performed by using the subsegment time length as a weight, and the updated fusion vector and the accumulated time length are calculated; when the accumulated time length reaches the preset growth target time length, the updated fusion vector is submitted to replace the main voiceprint vector of the target archive. The problems that the short segment voiceprint is easily disturbed in the related art, the recognition is unstable, the time correlation is not utilized, the archive is redundant and expanded, and the resources of the offline single terminal are limited are solved, the voiceprint of a speaker in a meeting is smoothly, dynamically and lightly evolved, and the voiceprint recognition stability and the local archive management standard degree under the condition that the computing power is limited are improved.

Description

Technical Field

[0001] This application relates to the field of audio recognition and processing, and in particular to a voiceprint optimization method and system based on time-weighted fusion and quality selection. Background Technology

[0002] With the widespread application of intelligent voice technology, intelligent conversation terminals are widely deployed in various scenarios such as meetings, interrogations, and face-to-face interviews. In these application scenarios, voiceprint recognition technology, as the core means of distinguishing, confirming, and recording the identities of different speakers, directly affects the overall transcription quality and personnel attribution determination due to its accuracy and stability. However, in related technologies, especially in the operating environment of offline standalone intelligent terminals with limited computing power and storage, there are many significant technical bottlenecks in voiceprint recognition and dynamic optimization during meetings: First, in real-time dialogue scenarios, the voiceprint vectors extracted from short audio clips are easily affected by background noise and truncation operations. If a strategy of directly covering the main voiceprint with a single short clip is adopted, it is easy to cause frequent oscillations or incorrect coverage of the recognition results, resulting in distortion of voiceprint features.

[0003] Secondly, existing voiceprint update mechanisms mostly rely on a single matching threshold, which cannot effectively filter out higher quality voiceprint samples. As a result, the speaker's main voiceprint often remains in the low-quality state of the initial file and is difficult to evolve to a higher stability as the meeting progresses.

[0004] The existing technologies lack the ability to analyze the temporal correlation of speakers' continuous speaking during meetings, processing each audio segment in isolation and wasting the contribution of adjacent segments to the construction of stable voiceprints. Furthermore, the existence of automatic in-meeting archiving mechanisms can easily lead to an expansion of the internal voiceprint archives as the meeting duration increases. Additionally, existing solutions often use the same archive merging threshold and online recognition threshold interchangeably, which can easily lead to erroneous merging of different speaker archives and compromise identity isolation.

[0005] In addition, cloud-based long-term voiceprint fusion solutions rely on network transmission, making it impossible to deploy offline standalone conversation terminals; local simple voiceprint updates only cover a single segment directly, and short noise clips can easily damage the baseline voiceprint. Furthermore, the threshold reuse mechanism has a two-way defect: if the threshold is too low, different speakers will be mistakenly grouped into the same file; if the threshold is too high, the same speaker's short sentence will generate multiple independent files, causing file expansion, speaker identification fragmentation, and extremely high manual cleaning costs. In addition, traditional solutions involve multiple people interrupting and strong noise segments participating in voiceprint iteration, which continuously lowers the overall quality of the main voiceprint. Furthermore, there is no historical version archive, making it impossible to retrospectively review after an identification error occurs, which does not meet the audit requirements for conversation scenarios.

[0006] Finally, offline standalone smart terminal devices face dual limitations in computing power and storage space, making them unable to support massive cloud-based voiceprint clustering or complex model update mechanisms. Stable and optimized voiceprint management needs to be achieved without interrupting the core transcription process or significantly increasing resource consumption.

[0007] There is currently no effective solution to the above problems. Summary of the Invention

[0008] This application provides a voiceprint optimization method, system, electronic device, and computer-readable storage medium based on time-weighted fusion and quality selection, so as to at least solve the problem of poor accuracy of voiceprint recognition in related technologies.

[0009] In a first aspect, embodiments of this application provide a voiceprint optimization method based on time-weighted fusion and quality selection. The method operates on an offline, standalone terminal without cloud dependency, and includes: Obtain audio segments after segmentation of the conversation audio, extract the voiceprint feature vector of the audio segment, and obtain the timestamp and duration of the audio segment; Furthermore, the audio segments are formatted and, optionally, uniformly converted to 16kHz mono PCM16 format; for audio segments with a duration of ≥600ms, the lightweight speaker clustering algorithm of the offline standalone terminal is called to perform single-person probability determination. If it is determined to be a mixed multi-person or high uncertainty, the growth buffer is cleared and this voiceprint optimization is skipped, without affecting the main ASR transcription process. The voiceprint feature vector is matched with the local speaker archive to determine the target archive to which the audio segment belongs. Specifically, the 128-dimensional voiceprint feature vector is matched with the local speaker archive based on a 0.5 recognition and filing threshold. The continuity between the audio segment and the previous audio segment in the target file is determined based on the timestamp; In the case of continuous speaking, the historical fusion vector in the growth buffer of the target file is obtained, and the sub-segment duration is used as the weight to perform duration-weighted fusion of the voiceprint feature vector and the historical fusion vector to obtain the updated fusion vector and cumulative duration. When the accumulated duration reaches the preset growth target duration of 20,000 ms, the updated fusion vector is submitted to overwrite the main voiceprint vector of the target file; wherein, this application sets two sets of independent similarity thresholds that are completely decoupled, the identification and filing threshold (configurable to 0.5) is used to create / match speaker files, and the merging threshold (configurable to 0.62) is used to merge redundant files; the two sets of thresholds are configured independently to solve the two-way contradiction of "missed identification and incorrect merging" caused by the reuse of thresholds in traditional solutions.

[0010] In some embodiments, the step of using the segment duration as weight to perform duration-weighted fusion of the voiceprint feature vector and the historical fusion vector in the growth buffer of the target file includes: Obtain the historical fusion vector and historical cumulative duration from the growth buffer; Calculate the unnormalized weighted average vector based on the historical cumulative duration, the historical fusion vector, the sub-segment duration, and the voiceprint feature vector; The unnormalized weighted average vector is subjected to L2 normalization to obtain the updated fusion vector; The cumulative duration is obtained by adding the historical cumulative duration to the sub-segment duration.

[0011] In some embodiments, determining the continuity between the audio segment and the previous audio segment in the target file based on the timestamp includes: Calculate the time interval between the start time of the audio segment and the end time of the previous audio segment; If the time gap is less than or equal to the preset maximum allowable time gap, and the start time of the audio segment is not less than the end time of the previous audio segment, it is determined to be continuous speech. If the time gap is greater than the maximum allowed time gap, or if the start time of the audio segment is less than the end time of the previous audio segment, it is determined to be discontinuous speech, and the growth buffer of the target file is cleared.

[0012] In some embodiments, the method further includes a quality selection step, specifically: When it is determined that the speech is discontinuous and the offline single-machine terminal voiceprint quality optimization switch is turned on, the current quality score of the voiceprint feature vector is calculated based on the sub-segment duration, including: configuring the current quality score to be equal to the sub-segment duration; or, the current quality score is a score obtained by weighted summation based on the sub-segment duration, the signal-to-noise ratio of the audio sub-segment, and the speech recognition confidence. If the current quality score is greater than the historical best quality score of the target file, the voiceprint feature vector is used to cover the main voiceprint vector of the target file; The historical best quality score is updated to the current quality score, the overwritten main voiceprint vector is archived to the history database, and the merging process for redundant files in the same meeting is triggered.

[0013] In some embodiments, triggering the merging of redundant files within the same meeting includes: Using the updated main voiceprint vector as the anchor point, calculate the cosine similarity between the voiceprint vector of the subordinate files in the archive of the offline single-machine terminal under the same meeting and the main voiceprint vector; If the cosine similarity is greater than or equal to a preset merging similarity threshold, the subordinate file is merged into the target file; The merging similarity threshold (default 0.62) is independent of the in-meeting identification and filing threshold (default 0.5) used to match the target files. The two thresholds are configured independently and do not affect each other.

[0014] In some embodiments, the method further includes: If the accumulated duration is less than the preset growth target duration of 20,000 ms, determine whether the duration corresponding to the main voiceprint vector of the target file has reached the growth target duration, and perform hierarchical update processing based on the current stability of the target file: If the target file is not yet stable (i.e., cumulative duration < 20000ms), the updated fusion vector will be synchronously updated to the main voiceprint vector of the target file for real-time preview updates, thereby supporting real-time recognition of the current meeting; If the target file has reached a stable state (i.e., cumulative duration ≥ 20000ms), then only the growth buffer is updated to accumulate subsequent data, keeping the main voiceprint vector of the target file unchanged, effectively avoiding potential disturbances to the constructed stable and high-quality voiceprint by subsequent new audio segments.

[0015] In some embodiments, after obtaining the segmented audio data of the conversation and before matching the voiceprint feature vector with a local speaker database, the method further includes: If the duration of the sub-segment is greater than or equal to the preset minimum interception duration, the speaker clustering algorithm of the offline standalone terminal is invoked to determine the single-person probability of the audio sub-segment. If the target file is determined to be not a single person or the determination result has high uncertainty, the growth buffer of the target file is cleared and the voiceprint recognition optimization process for the current audio segment is skipped.

[0016] Secondly, this application also provides a voiceprint optimization system based on time-weighted fusion and quality selection. The system runs on an offline standalone terminal and includes: an acquisition module, a promiscuous detection module, a matching module, a temporal continuity judgment module, a voiceprint growth and fusion module, a quality selection module, a redundant file merging module, a historical archiving module, and an update and overwrite module, wherein: The acquisition module is used to acquire audio segments after the conversation audio is segmented, acquire the timestamp and duration of the audio segments, and perform audio standardization preprocessing on the audio segments. The promiscuous detection module is used to, when the duration of the sub-segment is greater than or equal to the preset minimum interception duration, call the speaker clustering algorithm of the offline stand-alone terminal to determine the single-person probability of the standardized preprocessed audio sub-segment; when it is determined to be non-single-person or the determination result has high uncertainty, clear the growth buffer used for accumulating speech in the offline stand-alone terminal and skip the voiceprint recognition optimization processing for the current audio sub-segment; when it is determined that the audio sub-segment is a single-person speech, extract the voiceprint feature vector of the audio sub-segment. The matching module is used to match the voiceprint feature vector with the speaker archive of the offline standalone terminal to determine the target archive to which the audio segment belongs; The temporal continuity determination module is used to determine the continuity between the audio segment and the previous audio segment in the target file based on the timestamp. The voiceprint growth and fusion module is used to obtain the historical fusion vector in the growth buffer of the target file when it is determined that the speech is continuous, and use the sub-segment duration as a weight to perform duration-weighted fusion of the voiceprint feature vector and the historical fusion vector to obtain the updated fusion vector and cumulative duration. The quality optimization module is used to calculate the current quality score of the voiceprint feature vector based on the duration of the sub-segment when the segment is determined to be discontinuous and optimization is allowed; when the current quality score is greater than the historical best quality score of the target file, the main voiceprint vector of the target file is overwritten with the voiceprint feature vector, and the historical best quality score is updated to the current quality score. The redundant file merging module is used to traverse and merge redundant subordinate files under the same conference based on an independently configured merging similarity threshold after the main voiceprint vector is updated. The historical archiving module is used to archive the old version of the main voiceprint vector and its related data to the historical database before the main voiceprint vector is replaced or overwritten, so as to support audit backtracking and error rollback. The update overlay module is used to submit the updated fusion vector as the main voiceprint vector of the target file when the cumulative duration reaches the preset growth target duration.

[0017] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0019] Compared to related technologies, the embodiments of this application provide a voiceprint optimization method, system, electronic device, and computer-readable storage medium based on duration-weighted fusion and quality selection. This method involves acquiring audio segments after conversation audio is segmented and extracting voiceprint feature vectors, timestamps, and segment durations. The voiceprint feature vectors are then compared and matched with a speaker archive in an offline single-machine terminal to determine the target archive to which the audio segment belongs. Based on the timestamp, the continuity between the current audio segment and the previous audio segment in the target archive is determined. If continuous speaking is determined, the historical fusion vector in the target archive's growth buffer is obtained, and duration-weighted fusion is performed using the segment duration as a weight to calculate the updated fusion vector and cumulative duration. When the cumulative duration reaches a preset growth target duration, the updated fusion vector is submitted to cover the main voiceprint vector of the target archive. This invention addresses the issues in related technologies, such as the susceptibility of short-segment voiceprints to interference leading to recognition oscillations, lack of utilization of temporal correlation, redundant expansion of archives, and limited resources of offline standalone terminals. It enables a smooth, dynamic, and lightweight evolution of the speaker's voiceprint during meetings, improving the stability of voiceprint recognition and the standardization of local archive management under computing power constraints.

[0020] The overall processing flow of this application is implemented locally on an offline single-machine terminal with lightweight execution. Combined with multi-person mixed segment interception and failure degradation isolation, it ensures that the optimization process does not interrupt the core audio transcription process while meeting the constraints of the computing power-limited environment, and takes into account the audit traceability requirements of high-fidelity voiceprint iteration and offline scenarios. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a voiceprint optimization method based on time-weighted fusion and quality selection according to an embodiment of this application; Figure 2 This is a structural block diagram of a voiceprint optimization method system based on time-weighted fusion and quality selection according to an embodiment of this application; Figure 3 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0023] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0024] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0025] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0026] The methods and systems described in this application are applicable to intelligent conversation hardware and software integrated terminals deployed offline. This type of offline standalone terminal possesses real-time or near real-time speech transcription, speaker profiling, and voiceprint updating capabilities. All audio preprocessing, voiceprint feature extraction, clustering determination, and data storage are performed locally on the offline standalone terminal, without relying on cloud network interaction. The offline standalone terminal hardware platform can be based on an ARM architecture or include low-power chips such as digital signal processors (DSPs), and its computing power and storage resources are subject to objective constraints.

[0027] This embodiment provides a voiceprint optimization method based on time-weighted fusion and quality selection. Figure 1 This is a flowchart of a voiceprint optimization method based on time-weighted fusion and quality selection according to an embodiment of this application, as shown below. Figure 1 As shown, the process includes the following steps: S101, obtain the audio segments after the conversation audio is segmented, extract the voiceprint feature vector of the audio segment, and obtain the timestamp and duration of the audio segment.

[0028] Specifically, the offline standalone intelligent conversation terminal continuously collects conversation audio during the meeting. The offline standalone terminal has a locally pre-installed Voice Activity Detection (VAD) module, which segments the continuous audio sequence to obtain several audio segments containing independent speech.

[0029] In some embodiments, for each segmented audio sub-segment, local format normalization is performed to convert it to an audio format with a preset sampling rate (e.g., 16kHz mono PCM16). After format conversion, the specific sub-segment duration Δt (milliseconds) corresponding to the audio sub-segment is calculated. Subsequently, a voiceprint feature extraction model deployed locally on an offline single-machine terminal is invoked to extract voiceprint feature vectors with fixed dimensions for the normalized audio sub-segments. Specifically, the fixed dimension is set to 128 dimensions to adapt to lightweight model inference on offline standalone terminals.

[0030] Additionally, it should be noted that during the audio segmentation and feature extraction process, the system simultaneously obtains the absolute start timestamp of the audio segment within the entire recording track using the global recording clock. and absolute end timestamp Understandably, the introduction of this absolute timestamp provides a data basis for subsequent determination of the time interval between consecutive statements. The entire preprocessing feature analysis is stored in the offline single-machine terminal's memory pool, ensuring extremely low latency feedback for the core process.

[0031] S102, match the voiceprint feature vector with the speaker archive of the offline standalone terminal to determine the target archive to which the audio segment belongs.

[0032] First, considering the high likelihood of noisy audio segments in conference scenarios where multiple people interrupt simultaneously, directly introducing these segments into the voiceprint update process would severely contaminate the main voiceprint data. Therefore, a noisy segment interception mechanism is implemented after feature extraction. The duration Δt of the aforementioned audio segment is read, and it is determined whether this duration is greater than or equal to the preset minimum interception duration (default configuration is 600 milliseconds). Audio segments shorter than 600 milliseconds skip the noisy detection and proceed to the subsequent matching process, reducing the computational overhead of the offline single-machine terminal. If the minimum interception duration condition is met, the lightweight speaker clustering algorithm built into the offline single-machine terminal is triggered.

[0033] It should be noted that this embodiment differs from computationally intensive global clustering. Here, the algorithm only performs fast processing on the PCM data of the current segment, outputting a probability determination of whether it belongs to a single speaker. If it is determined that the current audio segment contains features of multiple speakers, or if the confidence level of the model output is lower than a preset threshold indicating high uncertainty, the growth buffer data of the target file to which the audio segment belongs is actively cleared, and all subsequent steps of voiceprint recognition optimization for the current audio segment are skipped, ending this round of algorithm cycle. This step limits the computational cost to a controllable range through front-end interception, ensuring the purity of the data source entering the subsequent optimization branch.

[0034] Furthermore, for single-person audio segments that are not blocked by promiscuous detection, the entire set of speaker profiles established for the current meeting is retrieved from the local storage of the offline single-machine terminal for matching. Specifically, each profile contains a standard main voiceprint vector, profile identifier, speaking duration, and profile status.

[0035] Before calculating similarity, a uniform dimension verification is performed to confirm the voiceprint feature vectors to be matched. The dimensions of the voiceprint vectors must be consistent with those in the archive. If the dimensions are abnormal, the matching is considered a failure and the process is skipped. During the matching process, the standard master voiceprint vectors built into each archive are retrieved sequentially. The similarity between the two is calculated using cosine similarity as the metric:

[0036] Furthermore, the classification is determined based on the calculated similarity score: if the highest similarity score is greater than or equal to the preset recognition and filing threshold, the current audio segment is determined to be a successful match and is assigned to the target speaker file with the highest similarity.

[0037] If the similarity scores calculated from all files are lower than the identification and filing threshold, it is determined that there is a completely new unknown speaker. Then, a blank speaker file is automatically created on the offline standalone terminal, a new and unique file identifier is assigned, and the current voiceprint feature vector is directly confirmed as the initial main voiceprint of the target file.

[0038] In the fuzzy matching range where scores are close to the threshold, if multiple files have similar scores, a lightweight time-series constraint rule is activated on the offline single-machine terminal. This rule combines the speaking order and time interval of each target file to assist in locking and avoid mismatching. In addition, the matching process in this step follows a preset priority rule, prioritizing the matching of stable files with high-frequency speaking and long cumulative duration within the meeting. Redundant subordinate files that have been merged are no longer included in the main matching comparison.

[0039] S103, determine the continuity between the audio segment and the previous audio segment in the target file based on the timestamp.

[0040] After determining the target file, this embodiment proceeds to the core steps of the voiceprint growth branch, specifically including: Read the absolute end timestamp of the previous audio segment from the growth buffer corresponding to the target file on the offline single-player terminal. By the absolute start timestamp of the current audio segment. and Subtracting them, we can calculate the time gap between the two.

[0041] In this embodiment, optionally, the preset maximum allowed time gap is configured to be 1000 milliseconds by default. If the calculated time gap is less than or equal to the maximum allowed time gap, and the absolute start timestamp of the current audio segment is not less than the end timestamp of the previous audio segment (i.e., there is no time reversal anomaly), and the dimension of the current feature vector is consistent with the dimension of the existing vector in the buffer, the current speaker is determined to be in a continuous speaking state, and the subsequent duration-weighted fusion process is entered.

[0042] Additionally, if the time gap exceeds the preset maximum value, or if time reversal occurs, it is determined to be discontinuous speaking. Upon such determination, the current growth buffer of the target file is immediately cleared to prevent the accumulation of erroneous time. This step explicitly models the strong correlation attributes of the speaking time sequence in the meeting, which can effectively connect scattered voiceprint segments.

[0043] S104. If the speech is determined to be continuous, the historical fusion vector in the growth buffer of the target file is obtained, and the sub-segment duration is used as the weight to perform duration-weighted fusion of the voiceprint feature vector and the historical fusion vector to obtain the updated fusion vector and cumulative duration.

[0044] For audio segments of continuous speech, obtain the historical fusion vector maintained by the target file in the growth buffer. (The initial state before the arrival of a new segment is a zero vector) and the historical cumulative duration (Initial state is 0). The time-weighted averaging principle is applied to calculate the unnormalized weighted average vector. The calculation process is achieved through the following formula:

[0045] This fusion mechanism ensures that the longer the audio segment occupies, the greater the proportion of its corresponding voiceprint vector in the final fusion result. By smoothing the feature distribution of multiple data segments, it approaches the extraction effect of a long speech window under ideal conditions.

[0046] Furthermore, L2 normalization is performed on the unnormalized weighted average vector to project it onto a unit sphere, ensuring the scale consistency of the feature vectors in the cosine distance calculation, thus obtaining the updated fused vector. ;

[0047] Simultaneously, the cumulative execution time is updated, including the historical cumulative execution time. Add the current segment duration The updated cumulative duration is calculated. The updated fusion vector, the current segment end timestamp, and the spliced ​​PCM audio are then written back to the local growth buffer on the offline single-machine terminal.

[0048] S105, when the cumulative duration reaches the preset growth target duration, submit the updated fusion vector to overwrite the main voiceprint vector of the target file.

[0049] After updating the cumulative duration, perform a growth submission check. If the current cumulative duration... Less than the preset growth target duration (Default configuration is 20000ms) Execute preview-level update logic: Determine whether the duration corresponding to the current main voiceprint vector of the target file has met the stability requirement (i.e., reached the growth target duration); if the stability requirement has not been met, the updated fusion vector is synchronously written to the main voiceprint field of the target file for the recognition system to call and display, and the current sub-segment optimization ends; if the stability requirement has been met, only the data update of the growth buffer is maintained, and no modification operation is performed on the main voiceprint.

[0050] If the current cumulative duration If the duration is greater than or equal to the preset growth target duration, the final submission action is triggered. The updated fusion vector generated in this round completely overwrites the main voiceprint vector stored in the target file, and the cumulative duration at this point is recorded as the quality score data corresponding to the main file. Before the overwrite occurs, to ensure data traceability, the system timestamps the original main voiceprint vector, the historical best quality score, and related feature data, and archives them in the local historical database of the offline single-machine terminal. After submission, the corresponding growth buffer is cleared.

[0051] Additionally, it should be noted that, in cases where the segment is determined to be discontinuous and optimal selection is allowed, the duration of the segment is used as the current quality score of the voiceprint feature vector, and if the current quality score is greater than the historical best quality score of the target file, the voiceprint feature vector is used to cover the main voiceprint vector of the target file.

[0052] In the basic embodiment of this application, the current quality score Q is directly equal to the segment duration Δt. Since longer audio segments provide richer voiceprint decoupling features and are more robust to sudden truncation noise, using duration to directly represent voiceprint quality can significantly reduce the computational overhead of offline single-machine terminals. In other optional embodiments, the current quality score can also be calculated using a weighted mapping algorithm, the calculation formula of which can be expressed as: ,in , , The preset weighting coefficients, The signal-to-noise ratio of the audio segment. This represents the confidence score for speech recognition.

[0053] This step belongs to the quality-optimized replacement branch. It is executed when an audio segment fails to meet the continuity condition, resulting in the growth branch not being triggered, or when the growth buffer is cleared and enters a stateless isolated judgment. A local lightweight quality evaluation rule based on duration is adopted, directly using the acquired segment duration. Current quality score used as the current voiceprint feature vector Retrieve the target file's best historical quality score recorded locally. (This value is 0 if the file is newly created). The calculated current quality score is strictly greater than the historical best quality score. In the event of a situation where the current single-segment voiceprint feature vector is overwritten with the main voiceprint vector of the target file, the replacement logic is allowed to be triggered. This directly uses the current single-segment voiceprint feature vector to overwrite the main voiceprint vector written to the target file, and then updates the historical best quality score recorded in the target file to the current value. Similarly, in accordance with transaction processing principles, the overwritten old version's voiceprint vector and related information are properly archived in the local history database. If the stricter comparison criteria are not met, the replacement process is skipped to ensure a high-standard iterative trend for the archives.

[0054] It should also be noted that after updating the main voiceprint vector of the target file, a merging process for redundant files within the same meeting is triggered. Regardless of whether the main voiceprint update is triggered by continuous duration-weighted fusion or by quality-selective overlay, a merging branch is triggered after the update operation is written to disk to address the problem of excessive automatic file creation caused by unidentified files during the meeting. Using the updated main voiceprint vector as the comparison anchor point, all subordinate speaker files within the same meeting on the offline single-machine terminal are traversed, and voiceprint cosine similarity calculation is performed. A dedicated merging threshold is set (default configuration is 0.62).

[0055] It should be specifically noted that this merging similarity threshold is logically independent of the threshold used for online identification, to avoid the adjustment of the identification threshold affecting the merging stability. If the cosine similarity between the voiceprint vector of a subordinate file and the main voiceprint of the current anchor point is found to be greater than or equal to the merging similarity threshold, the subordinate file and its speaking duration information are merged into the current target file, and the underlying storage relationship is updated, thus achieving automated cleanup of redundant identity files within the meeting.

[0056] Furthermore, this embodiment also deploys failure degradation and transaction consistency control strategies during the entire optimization process described above. Specifically, if any abnormal situation occurs at any stage (such as abnormal feature dimensions, local flash memory read / write failure, or merging module operation error), the program immediately suspends the current process and generates an alarm log containing the target file identifier, sub-segment error timestamp, and exception type only on the offline single-machine terminal. After an exception occurs, the current optimization request is skipped and the system safely reverts to the original file state before processing, without interrupting the system's main speech-to-text service. A series of related actions, such as voiceprint replacement, historical file dumping, and file relationship merging, are all executed in the same transaction on the offline single-machine terminal to prevent data inconsistency intermediate states caused by abnormal downtime.

[0057] Through the steps S101 to S105 above, this application relies entirely on the local computing power of offline single-machine terminals, uses continuous duration weighted fusion to simulate long speech feature extraction to suppress short audio noise, and combines a quality replacement strategy with duration as a hard indicator to establish an iterative path for steadily improving voiceprint files; and designs an independent threshold control merging mechanism to prevent disorderly expansion of files, and ensures the foundation of transcription business through a degradation processing design that does not block the entire link. Thus, high-fidelity voiceprint recognition capability and file management optimization are achieved under limited hardware resources.

[0058] This application also provides a voiceprint optimization method system based on time-weighted fusion and quality selection. Figure 2 This is a structural block diagram of a voiceprint optimization method system based on time-weighted fusion and quality selection according to an embodiment of this application, such as... Figure 2 As shown, the system runs on an offline standalone terminal and includes: an acquisition module 20, a promiscuous detection module 21, a matching module 22, a temporal continuity judgment module 23, a voiceprint growth and fusion module 24, a quality optimization module 25, a redundant file merging module 26, a historical archiving module 27, and an update overwrite module 28, wherein: The acquisition module 20 is used to acquire audio segments after the conversation audio is segmented, acquire the timestamp and duration of the audio segments, and perform audio standardization preprocessing on the audio segments; The promiscuous detection module 21 is used to call the speaker clustering algorithm of the offline single terminal to determine the single-person probability of the standardized preprocessed audio segment when the segment duration is greater than or equal to the preset minimum interception duration; when it is determined to be non-single or the determination result has high uncertainty, it clears the local growth buffer used for accumulating speech and skips the voiceprint recognition optimization processing for the current audio segment; when it is determined that the audio segment is a single person's speech, it extracts the voiceprint feature vector of the audio segment. Matching module 22 is used to match the voiceprint feature vector with the speaker archive of the offline standalone terminal to determine the target archive to which the audio segment belongs; The temporal continuity judgment module 23 is used to determine the continuity between an audio segment and the previous audio segment in the target file based on the timestamp. The voiceprint growth fusion module 24, when it is determined that the speech is continuous, obtains the historical fusion vector in the growth buffer of the target file, and uses the sub-segment duration as the weight to perform duration-weighted fusion of the voiceprint feature vector and the historical fusion vector to obtain the updated fusion vector and cumulative duration. The quality selection module 25 is used to calculate the current quality score of the voiceprint feature vector based on the duration of the sub-segment when the segment is determined to be discontinuous and selection is allowed; when the current quality score is greater than the historical best quality score of the target file, the voiceprint feature vector is used to cover the main voiceprint vector of the target file, and the historical best quality score is updated to the current quality score. The redundant file merging module 26 is used to traverse and merge redundant subordinate files under the same conference based on the independently configured merging similarity threshold after the main voiceprint vector is updated. The historical archiving module 27 is used to archive the old version of the main voiceprint vector and its related data to the historical database before the main voiceprint vector is replaced or overwritten, so as to support audit backtracking and error rollback. The update overlay module 28 is used to submit the updated fusion vector as the main voiceprint vector of the target file when the cumulative duration reaches the preset growth target duration.

[0059] The system addresses the issues in related technologies, such as the susceptibility of short-segment voiceprints to interference leading to recognition oscillations, lack of utilization of temporal correlation, redundant expansion of archives, and limited resources of offline single-machine terminals. It enables the smooth, dynamic, and lightweight evolution of the speaker's voiceprint during meetings, improving the stability of voiceprint recognition and the standardization of local archive management under computing power constraints.

[0060] To further verify the beneficial effects of the technical solution provided in this application and its significant progress compared to the prior art, the following description is provided in conjunction with specific quantitative effect test examples: This test was conducted using the same offline single-machine terminal hardware (ARM Cortex-A53 quad-core, 1GB RAM, 8GB flash memory) and the same real meeting recording (60 minutes in total, including 8 speakers, frequent interruptions, background air conditioning noise, and alternating long and short speaking segments). Three comparison schemes were deployed, and five core indicators were uniformly collected for quantitative comparison. The indicators are defined as follows: Voiceprint recognition oscillation rate: the number of times a speaker does not match the previous segment in a single audio segment / the total number of segments. The higher the value, the more unstable the recognition. Final number of meeting files: The total number of speaker files automatically generated by the system after the meeting. A higher number indicates a greater expansion of the file count. Voiceprint matching accuracy: The percentage of speakers who are correctly matched across all audio segments throughout the entire process; Offline standalone terminal average CPU utilization: The average CPU load of the entire machine during the voiceprint optimization process, reflecting the computing power consumption; Historical error correction count: The number of times a mistake has been corrected manually using historical archived vectors after it has been identified, reflecting the audit traceability capability.

[0061] 1. Setting up three sets of comparative test schemes Comparative Example 1 (Traditional stand-alone direct voiceprint overlay scheme): No single-person mixed segment interception, no continuous speech duration weighted fusion, recognition / merging uses a single threshold of 0.5, no quality-optimized replacement, no historical vector archiving; after extracting features from each audio segment, the main voiceprint vector is directly overlaid.

[0062] Comparative Example 2 (Simplified Scheme of Single Threshold Weighted Fusion): It has basic time-weighted fusion logic, but it does not set up pre-interception for single-person mixed data, share the same similarity threshold (0.5) for identification and file merging, lack stable hierarchical update logic of 20000ms, and lack historical vector archiving audit function.

[0063] In this embodiment of the application, the configuration parameters include a minimum interception time of 600ms, a maximum interval between consecutive messages of 1000ms, a growth target duration of 20000ms, an identification and filing threshold of 0.5, and a redundant file merging threshold of 0.62; and enable single-person mixed detection, time-weighted fusion, hierarchical preview update, quality-selective replacement, independent dual-threshold merging, full archiving of historical vectors, and abnormal transaction rollback mechanism.

[0064] 2. The quantitative test results are shown in Table 1 below: Table 1: Quantitative Test Results Data Table

[0065] 3. Dimensional Effect Analysis (1) Identification stability: It solves the problem of oscillation in short noise segments and has an unexpected optimization amplitude. Comparative Example 1 directly covered the main voiceprint with short noise clips. The short noise clips and truncated speech continuously interfered with the baseline features, and the oscillation rate exceeded 30%. Comparative Example 2 only added weighted fusion but did not filter out multi-person mixed segments. A large number of interruptions polluted the fusion vector, and the oscillation rate was still as high as 14.2%. This application employs a dual mechanism of pre-emptive single-person mixed interception and continuous speech time-weighted smoothing, resulting in a significant reduction in oscillation rate to 4.1%, far exceeding the optimization effect achievable by conventional weighted algorithms. This is an unexpected technical achievement.

[0066] (2) Redundancy control of archives: dual independent thresholds to suppress archive expansion at the source Comparative Example 1 and Comparative Example 2 both use a single threshold. If the threshold is too low, different speakers will be merged; if the threshold is too high, a large number of files will be created for the same person's short sentences. In the test, the traditional solution generated 27 / 18 redundant files per meeting, and the manual cleaning cost was extremely high. This application completely decouples the two sets of thresholds for identification and filing, and independently configures differentiated thresholds. It automatically merges redundant files of speakers from the same source, and finally retains only 6 valid files, which greatly reduces the cost of manual operation and maintenance. This combination of dual threshold collaborative weighted fusion cannot be easily derived through existing technology.

[0067] (3) Improved recognition accuracy: Multi-module collaboration creates superimposed gain The accuracy of Comparative Example 2, which relies solely on weighted fusion, only improved by 11.2 percentage points. This application combines three methods: mixed filtering, sequential continuous fusion, and high-quality sample selection and replacement, which improves the accuracy by 20.5 percentage points compared to the traditional approach. The combined effect of multiple technical methods is not a conventional optimization based on a single algorithm improvement.

[0068] (4) The computing power cost is controllable and it is compatible with offline low-cost single-machine terminals. After adding a complete set of optimization modules, the CPU usage of this application only increases by 0.7%, which is far lower than the computing power consumption of existing cloud clustering and global complex clustering algorithms. It achieves lightweight operation in ARM embedded offline single-machine terminal scenarios without interrupting the main thread of ASR speech transcription, thus resolving the technical contradiction of existing technologies that "optimizing voiceprints consumes too much computing power and affects the real-time performance of transcription".

[0069] (5) Audit traceability capability fills the existing technological gap. The existing two types of comparison models lack historical vector archiving mechanisms. After an identification error, there is no historical data to review and verify, which cannot meet the strict compliance requirements for auditing in conversation scenarios such as interrogation and interviews. This application automatically archives old version vectors for each main voiceprint coverage. In the test, a total of 19 identification deviations were corrected by rolling back through the historical database, forming an additional compliance effect that existing technologies do not have.

[0070] 4. Comparison of experimental conclusions Existing technologies only employ one method from time-weighted fusion, single threshold matching, and single-segment voiceprint replacement, which cannot simultaneously solve the five major technical defects of "identification oscillation, file expansion, noise pollution, limited offline computing power, and inability to trace back audits". This application organically combines six innovative methods: single-person mixed pre-filtering, time-series continuous time-weighted fusion, hierarchical stable update, dual independent similarity thresholds, quality-optimized iteration, and historical vector archiving. It achieves optimization effects far exceeding existing improvement schemes in five dimensions: identification stability, matching accuracy, lightweight file management, offline computing power adaptation, and audit traceability.

[0071] In one embodiment, Figure 3 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 3 As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 3 As shown, the electronic device includes a processor, a network interface, internal memory, and non-volatile memory connected via an internal bus. The non-volatile memory stores the operating system, computer programs, and a database. The processor provides computing and control capabilities, the network interface communicates with external terminals via a network, the internal memory provides an environment for the operating system and computer programs to run, the computer programs are executed by the processor to implement a voiceprint optimization method based on time-weighted fusion and quality selection, and the database stores data.

[0072] Those skilled in the art will understand that Figure 3 The structures shown are merely block diagrams of a portion of the structure related to the present application and do not constitute a limitation on the electronic devices to which the present application is applied. Specific electronic devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. It should be specifically noted that when the computer program stored in the non-volatile memory in the embodiments of this application is executed by the processor, the entire process of voiceprint optimization, including audio segmentation, voiceprint feature extraction, multi-person intermingling detection, continuous duration weighted fusion, and redundant file merging, is completed collaboratively and in a closed loop within the local computing resources of the electronic device, without relying on a network interface for any network interaction with an external cloud server. Figure 3 The network interface shown is only optional for offline export of device local logs or local upgrade of device firmware when not in a meeting, and does not participate in the core voiceprint calculation and dynamic file update process during a meeting.

[0073] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0074] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0075] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A voiceprint optimization method based on time-weighted fusion and quality selection, characterized in that, The method operates on an offline, standalone terminal without cloud dependency, including: Obtain audio segments after segmentation of the conversation audio, obtain the timestamp and duration of the audio segments, and perform audio standardization preprocessing on the audio segments; If the duration of the sub-segment is greater than or equal to the preset minimum interception duration, the speaker clustering algorithm of the offline standalone terminal is invoked to determine the single-person probability of the standardized preprocessed audio sub-segment. If the judgment is not made by a single person or the judgment result has high uncertainty, clear the growth buffer used for accumulating speech in the offline single terminal and skip the voiceprint recognition optimization processing for the current audio segment. If the audio segment is determined to be spoken by a single person, the voiceprint feature vector of the audio segment is extracted; The voiceprint feature vector is matched with the speaker archive of the offline standalone terminal to determine the target archive to which the audio segment belongs; The continuity between the audio segment and the previous audio segment in the target file is determined based on the timestamp; In the case of continuous speaking, the historical fusion vector in the growth buffer of the target file is obtained, and the sub-segment duration is used as the weight to perform duration-weighted fusion of the voiceprint feature vector and the historical fusion vector to obtain the updated fusion vector and cumulative duration. When the accumulated duration reaches the preset growth target duration, the updated fusion vector is submitted to overwrite the main voiceprint vector of the target file.

2. The method according to claim 1, characterized in that, The step of using the sub-segment duration as weight to perform duration-weighted fusion of the voiceprint feature vector and the historical fusion vector in the growth buffer of the target file includes: Obtain the historical fusion vector and historical cumulative duration from the growth buffer; Calculate the unnormalized weighted average vector based on the historical cumulative duration, the historical fusion vector, the sub-segment duration, and the voiceprint feature vector; The unnormalized weighted average vector is subjected to L2 normalization to obtain the updated fusion vector; The cumulative duration is obtained by adding the historical cumulative duration to the sub-segment duration.

3. The method according to claim 1, characterized in that, The step of determining the continuity between the audio segment and the previous audio segment in the target file based on the timestamp includes: Calculate the time interval between the start time of the audio segment and the end time of the previous audio segment; If the time gap is less than or equal to the preset maximum allowable time gap, and the start time of the audio segment is not less than the end time of the previous audio segment, it is determined to be continuous speech. If the time gap is greater than the maximum allowed time gap, or if the start time of the audio segment is less than the end time of the previous audio segment, it is determined to be discontinuous speech, and the growth buffer of the target file is cleared.

4. The method according to claim 3, characterized in that, The method further includes a quality selection step, specifically: In cases where discontinuity is determined and optimal selection is allowed, the current quality score of the voiceprint feature vector is calculated based on the duration of the sub-segment, including: configuring the current quality score to be equal to the duration of the sub-segment; or, the current quality score is a score obtained by weighted summation based on the duration of the sub-segment, the signal-to-noise ratio of the audio sub-segment, and the confidence level of speech recognition. If the current quality score is greater than the historical best quality score of the target file, the voiceprint feature vector is used to cover the main voiceprint vector of the target file; The historical best quality score is updated to the current quality score, the overwritten main voiceprint vector is archived to the history database, and the merging process for redundant files in the same meeting is triggered.

5. The method according to claim 4, characterized in that, The triggering of the merging process for redundant files within the same meeting includes: Using the updated main voiceprint vector as the anchor point, calculate the cosine similarity between the voiceprint vector of the subordinate file in the archive of the offline single terminal under the same meeting and the main voiceprint vector; If the cosine similarity is greater than or equal to a preset merging similarity threshold, the subordinate file is merged into the target file; The merged similarity threshold is independent of the online identification similarity threshold used to match the target file.

6. The method according to claim 1, characterized in that, The method further includes: If the cumulative duration is less than the preset growth target duration, determine whether the duration corresponding to the main voiceprint vector of the target file has reached the growth target duration. If not, the updated fusion vector is synchronously updated to the main voiceprint vector of the target file for preview update; If the growth target duration has been reached, only the growth buffer is updated, while the main voiceprint vector of the target file remains unchanged.

7. The method according to claim 1, characterized in that, Matching the voiceprint feature vector with the speaker archive of the offline standalone terminal to determine the target archive to which the audio segment belongs also includes: When the similarity scores of the voiceprint feature vector and multiple speaker profiles in the speaker database are close to the preset recognition threshold and the score difference is less than the preset difference threshold, the timing constraint rules of the offline single-machine terminal are activated. Combining the speaking order and speaking time interval of the audio segments in the same meeting, a unique existing speaker profile is locked as the target profile.

8. A voiceprint optimization system based on time-weighted fusion and quality selection, characterized in that, The system operates on an offline standalone terminal without cloud dependency. The system is used to execute and implement the method of any one of claims 1 to 7. The system includes: an acquisition module, a promiscuous detection module, a matching module, a temporal continuity judgment module, a voiceprint growth and fusion module, a quality optimization module, a redundant file merging module, a historical archiving module, and an update and overwrite module, wherein: The acquisition module is used to acquire audio segments after the conversation audio is segmented, acquire the timestamp and duration of the audio segments, and perform audio standardization preprocessing on the audio segments. The promiscuous detection module is used to, when the duration of the sub-segment is greater than or equal to the preset minimum interception duration, call the speaker clustering algorithm of the offline single-machine terminal to determine the single-person probability of the standardized preprocessed audio sub-segment; when it is determined to be non-single-person or the determination result has high uncertainty, clear the local growth buffer used for accumulating speech and skip the voiceprint recognition optimization processing for the current audio sub-segment; when it is determined that the audio sub-segment is a single-person speech, extract the voiceprint feature vector of the audio sub-segment. The matching module is used to match the voiceprint feature vector with the speaker archive of the offline standalone terminal to determine the target archive to which the audio segment belongs; The temporal continuity determination module is used to determine the continuity between the audio segment and the previous audio segment in the target file based on the timestamp. The voiceprint growth and fusion module is used to obtain the historical fusion vector in the growth buffer of the target file when it is determined that the speech is continuous, and use the sub-segment duration as a weight to perform duration-weighted fusion of the voiceprint feature vector and the historical fusion vector to obtain the updated fusion vector and cumulative duration. The quality optimization module is used to calculate the current quality score of the voiceprint feature vector based on the duration of the sub-segment when the segment is determined to be discontinuous and optimization is allowed; when the current quality score is greater than the historical best quality score of the target file, the main voiceprint vector of the target file is overwritten with the voiceprint feature vector, and the historical best quality score is updated to the current quality score. The redundant file merging module is used to traverse and merge redundant subordinate files under the same conference based on an independently configured merging similarity threshold after the main voiceprint vector is updated. The historical archiving module is used to archive the old version of the main voiceprint vector and its related data to the historical database before the main voiceprint vector is replaced or overwritten, so as to support audit backtracking and error rollback. The update overlay module is used to submit the updated fusion vector as the main voiceprint vector of the target file when the cumulative duration reaches the preset growth target duration.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images