Role speech distinguishing and modeling method and device based on multi-dimensional features, equipment, medium and system

Abstract

The application provides a role voice distinguishing and modeling method and device based on multi-dimensional features, electronic equipment, a computer readable storage medium and a system. A source role voice segment in initial audio-visual data is obtained, and acoustic multi-dimensional features are extracted, which are bound to a preset role mark to generate a feature association sequence. A role-specific feature cluster is generated according to structural similarity and a preset approximation threshold. A basic role acoustic network is generated through initial acoustic topology network training. Finally, a final role acoustic network is generated by detecting a representation deviation state and adjusting node parameters with a verification voice segment.

Description

Technical Field

[0001] This application relates to the field of speech processing technology, and more specifically, to a method, apparatus, electronic device, computer-readable storage medium, and system for role speech differentiation and modeling based on multi-dimensional features. Background Technology

[0002] In applications such as film and television dubbing, animation dubbing, virtual character voice generation, and multi-character voice material management, the voices of different characters need to maintain relatively stable acoustic recognizability in terms of timbre, intonation, speech rate, and emotional expression. As the sources of audiovisual data continue to increase, the same character may appear in different scenes, under different recording conditions, and in different emotional states. The differences and similarities in character voices are increasing simultaneously, placing higher demands on character voice differentiation, attribution identification, and subsequent modeling processing.

[0003] Existing character voice processing solutions typically use static voiceprint templates to categorize multiple audio segments. This approach first segments the input audio, then extracts the average spectrum or a fixed voiceprint template from the segmented audio, and finally matches the fixed voiceprint template with existing character tags to complete subsequent voice attribution processing. This solution mainly relies on the matching relationship between a single template and a fixed tag, making the processing chain relatively simple.

[0004] However, the above-mentioned scheme has certain technical shortcomings. Because static voiceprint templates cannot simultaneously reflect the multidimensional acoustic changes of the same character in different contexts, and it is also difficult to continuously distinguish the structural differences between similar character voices, the character voice samples are prone to classification confusion. At the same time, after completing the initial modeling, there is a lack of a correction step to detect the deviation between the output distribution and the original acoustic performance using independent speech data, which makes it easy for the subsequently generated character acoustic expressions to deviate from the real character voices, resulting in low adaptability of character voice differentiation and modeling. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, computer-readable storage medium, and system for character voice differentiation and modeling based on multi-dimensional features, in order to at least alleviate the aforementioned technical problems.

[0006] A method for character voice differentiation and modeling based on multi-dimensional features includes: Obtain source character voice segments from initial audiovisual data, extract acoustic multidimensional features from the source character voice segments, and bind the acoustic multidimensional features to a preset character tag used to identify the source character voice segment's affiliation, so as to generate a feature association sequence; The structural similarity of the acoustic multidimensional features within the feature association sequence is analyzed, and based on the structural similarity, the acoustic multidimensional features that reach a preset approximation threshold are subjected to feature clustering processing to generate a character-specific feature cluster. The acoustic multidimensional features within the character-specific feature cluster are imported into the initial acoustic topology network to update the node parameters of the initial acoustic topology network until the output distribution of the initial acoustic topology network approximates the feature distribution of the character-specific feature cluster, thereby generating the basic character acoustic network. A verification speech segment is obtained and imported into the basic character acoustic network to generate expected distribution features. The representation offset state between the expected distribution features and the original features of the verification speech segment is detected. Then, the node parameters of the basic character acoustic network are adjusted according to the representation offset state to generate the final character acoustic network.

[0007] A device for character voice differentiation and modeling based on multi-dimensional features, comprising: The feature association sequence generation module is used to obtain source character voice segments in the initial audiovisual data, extract the acoustic multidimensional features of the source character voice segments, and bind the acoustic multidimensional features to a preset character tag used to identify the source character voice segment to generate a feature association sequence. The character-specific feature cluster generation module is used to analyze the structural similarity of the acoustic multidimensional features within the feature association sequence, and based on the structural similarity, to perform feature clustering processing on the acoustic multidimensional features that reach a preset approximation threshold to generate character-specific feature clusters. The basic character acoustic network generation module is used to import the acoustic multidimensional features within the character-specific feature cluster into the initial acoustic topology network to update the node parameters of the initial acoustic topology network until the output distribution of the initial acoustic topology network approximates the feature distribution of the character-specific feature cluster, thereby generating the basic character acoustic network. The final character acoustic network generation module is used to acquire a verification speech segment, import the verification speech segment into the basic character acoustic network to generate expected distribution features, detect the representation offset state between the expected distribution features and the original features of the verification speech segment, and then adjust the node parameters of the basic character acoustic network according to the representation offset state to generate the final character acoustic network.

[0008] An electronic device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the character voice differentiation and modeling method based on multi-dimensional features as described above.

[0009] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the character voice differentiation and modeling method based on multi-dimensional features as described above.

[0010] A role-based voice discrimination and modeling system based on multi-dimensional features includes a feature association processing end, a feature aggregation processing end, an acoustic network training end, and an acoustic network verification end; The feature association processing end is used to obtain source character voice segments in the initial audiovisual data, extract the acoustic multidimensional features of the source character voice segments, and bind the acoustic multidimensional features to a preset character tag used to identify the source character voice segment to generate a feature association sequence. The feature clustering processing end is used to analyze the structural similarity of the acoustic multidimensional features in the feature association sequence, and based on the structural similarity, to perform feature clustering processing on the acoustic multidimensional features that reach a preset approximation threshold to generate a character-specific feature cluster. The acoustic network training terminal is used to import the acoustic multidimensional features within the character-specific feature cluster into the initial acoustic topology network to update the node parameters of the initial acoustic topology network until the output distribution of the initial acoustic topology network approximates the feature distribution of the character-specific feature cluster, so as to generate a basic character acoustic network. The acoustic network verification terminal is used to acquire verification speech segments, import the verification speech segments into the basic character acoustic network to generate expected distribution features, detect the representation offset state between the expected distribution features and the original features of the verification speech segments, and then adjust the node parameters of the basic character acoustic network according to the representation offset state to generate the final character acoustic network.

[0011] The technical advantages of the technical solution provided in this application are: This application presents a multi-dimensional feature-based method for character speech differentiation and modeling. Addressing the shortcomings of existing static voiceprint templates in reflecting the multi-dimensional acoustic variations of the same character across different contexts, this method first acquires source character speech segments from initial audiovisual data, extracts their acoustic multi-dimensional features, and binds these features to preset character tags used to identify the source character's speech segment affiliation, thereby generating a feature association sequence. Compared to traditional methods that directly map audio processing results to fixed templates, this application integrates the acoustic multi-dimensional features of speech segments and character affiliation tags into the same processing chain through feature association sequences. This allows subsequent processing to move beyond relying solely on a single static template and instead continue structural differentiation and modeling based on the established acoustic multi-dimensional features, thus alleviating the problem of inconsistently handling cross-scene speech variations for the same character.

[0012] This application further analyzes the structural similarity of acoustic multidimensional features within the feature association sequence, and based on structural similarity, performs feature clustering processing on acoustic multidimensional features that reach a preset approximation threshold to generate character-specific feature clusters. Compared to traditional methods that directly match using fixed voiceprint templates, this application, after binding acoustic multidimensional features with preset character labels, continues to use structural similarity as the criterion for determining whether acoustic multidimensional features are close, and filters acoustic multidimensional features that need feature clustering processing through a preset approximation threshold, so that similar acoustic performances can be included in character-specific feature clusters. Therefore, the character speech differentiation process no longer stops at the correspondence between a single sample and a single template, but can form a feature set more suitable for subsequent training around character-specific feature clusters, effectively mitigating the confusion between similar character speech.

[0013] This application also imports the acoustic multidimensional features within the character-specific feature cluster into the initial acoustic topology network to update the node parameters of the initial acoustic topology network until the output distribution of the initial acoustic topology network approximates the feature distribution of the character-specific feature cluster, thus generating the basic character acoustic network. Compared to the traditional approach of simply saving static voiceprint templates and repeatedly calling them in subsequent processing, this application drives the node parameter update of the initial acoustic topology network through the character-specific feature cluster, making the network output distribution gradually approach the feature distribution of the character-specific feature cluster. In this way, the generation process of the basic character acoustic network forms a continuous succession relationship with the preceding character-specific feature cluster, enabling the character speech modeling results to reflect the acoustic multidimensional features that have been aggregated in the character-specific feature cluster, rather than just reflecting the acoustic performance of a certain audio segment or a fixed template, thereby improving the adaptability of the character's acoustic expression.

[0014] This application finally obtains a verification speech segment, imports it into the basic character acoustic network to generate expected distribution features, and detects the representational offset between the expected distribution features and the original features of the verification speech segment. Based on this offset, the node parameters of the basic character acoustic network are adjusted to generate the final character acoustic network. Compared to traditional solutions that lack an independent verification and correction step after initial template establishment, this application uses the verification speech segment as an external check on the output of the basic character acoustic network. The representational offset between the expected distribution features and the original features is used as the basis for node parameter adjustment, ensuring that network parameter updates continue to be performed around the actual offset. Therefore, after basic training, the character acoustic network can be corrected using the verification speech segment, reducing the deviation between the output distribution and the original acoustic performance, making the final character acoustic network more suitable for multi-character speech discrimination and modeling scenarios. Attached Figure Description

[0015] Figure 1This application provides a scenario for a method for character voice differentiation and modeling based on multi-dimensional features. Figure 2 This application provides an embodiment of a method for character voice differentiation and modeling based on multi-dimensional features. Figure 3 This application provides an embodiment of a role-voice differentiation and modeling device based on multi-dimensional features. Figure 4 This is an electronic device according to an embodiment of the present application.

[0016] Figure 5 This is a computer-readable storage medium according to an embodiment of the present application.

[0017] Figure 6 This application provides an embodiment of a role-voice differentiation and modeling system based on multi-dimensional features. Detailed Implementation

[0018] like Figure 1 As shown, this is a scenario illustrating a method for character voice differentiation and modeling based on multi-dimensional features according to an embodiment of this application; as... Figure 2 The image shows an embodiment of this application of a method for character voice differentiation and modeling based on multi-dimensional features, comprising the following steps: Obtain source character voice segments from initial audiovisual data, extract acoustic multidimensional features from the source character voice segments, and bind the acoustic multidimensional features to a preset character tag used to identify the source character voice segment's affiliation, so as to generate a feature association sequence; The structural similarity of the acoustic multidimensional features within the feature association sequence is analyzed, and based on the structural similarity, the acoustic multidimensional features that reach a preset approximation threshold are subjected to feature clustering processing to generate a character-specific feature cluster. The acoustic multidimensional features within the character-specific feature cluster are imported into the initial acoustic topology network to update the node parameters of the initial acoustic topology network until the output distribution of the initial acoustic topology network approximates the feature distribution of the character-specific feature cluster, thereby generating the basic character acoustic network. A verification speech segment is obtained and imported into the basic character acoustic network to generate expected distribution features. The representation offset state between the expected distribution features and the original features of the verification speech segment is detected. Then, the node parameters of the basic character acoustic network are adjusted according to the representation offset state to generate the final character acoustic network.

[0019] Optionally, the extraction of acoustic multidimensional features from the source character's speech segment includes: Identify the fundamental frequency wave sequence of the pronunciation in the source character's speech segment, and extract the harmonic structure features of the fundamental frequency wave sequence to construct a timbre representation vector; The pitch curve trajectory and syllable span timestamp within the source character's speech segment are detected to generate intonation and speech rate representation vectors. The pronunciation intensity amplitude waveform of the source character's voice segment is extracted to generate an emotion representation vector; The acoustic multidimensional features are generated based on the timbre representation vector, the intonation and speech rate representation vector, and the emotion representation vector.

[0020] Preferably, the specific implementation process for identifying the fundamental frequency wave sequence in the source character's speech segment is as follows: After the source character's speech segment has been separated from the initial audiovisual data, sampling rate unification processing and channel normalization reading processing are first performed on the source character's speech segment to form the time sampling aperture corresponding to the source character's speech segment. The time sampling aperture is used to limit the division length and sampling reading order of subsequent continuous time windows, so that the source character's speech segment has a consistent time reference in the subsequent acoustic reading process. Subsequently, the source character's speech segment is divided into speech frames according to the continuous time windows configured by the time sampling aperture, and a speech frame carrying record is established for each speech frame. The speech frame carrying record includes the speech frame start time, speech frame end time, short-time energy distribution, zero-crossing change distribution, and spectral peak distribution. The short-time energy distribution is used to describe the energy concentration of the sound wave amplitude within the speech frame, the zero-crossing change distribution is used to describe the density of sound wave symbol flipping within the speech frame, and the spectral peak distribution is used to describe the peak arrangement of the frequency components within the speech frame. Based on the short-time energy distribution, zero-crossing change distribution, and spectral peak distribution recorded in the speech frame, each speech frame undergoes a spoken segment determination process to form a spoken segment marker. This marker is not a content labeling of the source character's speech segment, but rather a technical marker indicating the presence of periodic vocal components within the speech frame. Subsequently, only the speech frames with the spoken segment markers are read from the periodic correlation peaks and low-frequency peak candidate positions, and a fundamental frequency candidate trajectory is formed based on the correspondence between the periodic correlation peaks and the low-frequency peak candidate positions. The fundamental frequency candidate trajectory represents the source of the periodic vocalization changes in the source character's speech segment over time. However, the fundamental frequency candidate trajectory is still affected by short pauses, consonant plosives, and background noise. Therefore, a continuity correction process is performed on the fundamental frequency candidate trajectory according to the frequency jump amplitude between adjacent speech frames to remove isolated frequency points that do not satisfy the continuous change relationship between adjacent speech frames, while retaining frequency points with continuous vocalization relationships with preceding and following speech frames. Based on the candidate fundamental frequency trajectory after continuous correction, the frequency points are rearranged according to the start and end times of the speech frames to form the fundamental frequency wave sequence. The technical essence of the fundamental frequency wave sequence is an ordered data chain of the fundamental frequency of the vocalized portion of the source character's speech segment, which varies over time. This data chain participates in subsequent harmonic structure feature extraction, enabling the subsequent timbre representation vector to use the periodicity of the character's vocalization as its acoustic source, rather than relying solely on a single frequency point or average frequency value.

[0021] Preferably, in the specific technical implementation of extracting the harmonic structure features of the fundamental frequency wave sequence and constructing the timbre representation vector, the fundamental frequency position corresponding to each speech frame in the fundamental frequency wave sequence is first read, and adjacent spectral peaks are retrieved in the octave direction of each fundamental frequency position, thereby forming a harmonic peak arrangement corresponding to the fundamental frequency wave sequence. The harmonic peak arrangement is used to describe the energy distribution relationship between the fundamental frequency component and the octave component in the source character's speech segment. The harmonic peak arrangement is different from a simple spectral peak distribution; its arrangement order is defined by the fundamental frequency wave sequence, thus reflecting the stable timbre cues jointly generated by the character's vocal cavity and vocal cord vibration. Subsequently, peak energy comparison processing, inter-peak span reading processing, and high and low frequency energy attenuation reading processing are performed on the harmonic peak arrangement to form the harmonic structure features. The peak energy comparison processing is used to obtain the relative energy relationship between the fundamental frequency component and each harmonic component, and the relative energy relationship is used as the fundamental frequency energy description field and harmonic frequency energy description field in the harmonic structure feature; the peak span reading processing is used to obtain the frequency interval stability between adjacent harmonic peaks, and the frequency interval stability is used as the harmonic interval description field in the harmonic structure feature; the high and low frequency energy attenuation reading processing is used to obtain the energy decrease pattern from low frequency harmonic components to high frequency harmonic components, and the energy decrease pattern is used as the high and low frequency attenuation description field in the harmonic structure feature. The technical essence of the harmonic structure feature is a multi-stage timbre stability description field formed around the fundamental frequency wave sequence of the sound production. It can preserve the vocal texture of the same character in different sentences in a field-based manner and provide a structural source for the timbre dimension of subsequent acoustic multi-dimensional features. When constructing the timbre representation vector, the timbre vector field order is first established. This order is formed during the acoustic feature extraction initialization phase by configuring the timbre field name, timbre field arrangement, and timbre field value range. This timbre vector field order ensures that the timbre representation vectors obtained from voice segments of different source characters have the same field caliber. Subsequently, the fundamental frequency energy description field, the harmonic frequency energy description field, the harmonic interval description field, and the high and low frequency attenuation description field in the harmonic structure feature are aggregated and encapsulated according to the timbre vector field order to form the timbre representation vector. Each element in the timbre representation vector corresponds to an acoustic description field in the harmonic structure feature. The timbre representation vector serves as the input source for the timbre dimension when generating the acoustic multidimensional features, enabling the acoustic multidimensional features to retain the more stable structural parts of the differences in the character's voice.

[0022] Preferably, the specific implementation process for detecting the pitch shift trajectory and syllable span timestamp within the source character's speech segment is as follows: After the fundamental frequency wave sequence of pronunciation has been formed, the fundamental frequency wave sequence of pronunciation is first time-aligned with the start time and end time of the speech frame recorded in the speech frame to form a fundamental frequency time-aligned sequence. The fundamental frequency time-aligned sequence represents the actual time position of each fundamental frequency point in the source character's speech segment, and it is used to convert the changes in pronunciation pitch into temporal data that can participate in intonation analysis. Subsequently, the rising, falling, and stationary changes between adjacent fundamental frequency points are read from the fundamental frequency time-aligned sequence, and multiple adjacent fundamental frequency points are connected into modulation segments according to the direction of continuous change, thereby forming the pitch shift trajectory based on the arrangement order of multiple modulation segments. The technical essence of the pitch shift trajectory is the continuous change path of the pronunciation pitch in the source character's speech segment as it rises, falls, or remains constant over time. It is not equivalent to a single fundamental frequency value, but is formed by the directional change relationship of multiple fundamental frequency points on the time axis. The rise and fall modulation trajectory reflects the intonation trend of a character in dialogue, enabling subsequent intonation and speech rate representation vectors to preserve acoustic change patterns such as upward at the end of a sentence, downward in the middle of a sentence, and continuous smoothness. For the syllable span timestamp, the short-time energy distribution, the zero-crossing change distribution, and the spectral peak distribution are first read from the speech frame carrying record, and combined with the spoken segment markers to determine syllable boundary candidate positions; the syllable boundary candidate positions represent the time position at which the articulation energy, the density of sound wave symbol flips, or the spectral structure changes in the source character's speech segment. Subsequently, based on the spoken duration relationship between adjacent syllable boundary candidate positions, boundary merging and boundary removal processes are performed on the syllable boundary candidate positions to form syllable boundary confirmation positions. The syllable boundary confirmation positions are used to replace the syllable boundary candidate positions in the syllable duration range reading, ensuring that boundary candidate points caused by short-time noise do not directly enter the syllable span timestamp. Based on each set of adjacent syllable boundary confirmation positions, the start and end times of the corresponding syllable are read to form the syllable span timestamp. The technical essence of the syllable span timestamp is to structurally record the duration of each pronunciation unit in the source character's voice segment on the time axis. The syllable span timestamp participates in the generation of subsequent intonation and speech rate representation vectors, so that the rhythm of the character's voice is no longer roughly represented by the average duration of the whole segment, but is represented by the syllable-level time span.

[0023] Preferably, in the specific technical implementation of generating the intonation and speech rate representation vectors, the rising and falling tone trajectory and the syllable span timestamp are first read, and directional encoding, duration reading, and transition position reading are performed on each tone segment in the rising and falling tone trajectory to form an intonation description field. The directional encoding process converts rising, falling, and stationary changes in the tone segments into directional fields that can participate in vector arrangement. These directional fields are used as components of the intonation description field in subsequent field-level cascading configuration processing. The duration reading process records the time coverage of each tone segment in the source character's speech segment and uses this time coverage as a component of the intonation description field in subsequent field-level cascading configuration processing. The transition position reading process records the time position in the rising and falling tone trajectory where a change occurs from rising to falling, from falling to rising, or from stationary to non-stationary, and uses this time position as a component of the intonation description field in subsequent field-level cascading configuration processing. The resulting intonation description field reflects the pitch variation structure of the character's speech within a sentence. Subsequently, the start time, end time, and interval between adjacent syllables of each syllable in the syllable span timestamp are read, and span normalization processing is performed on the above time fields to form a speech rate description field. This span normalization processing does not change the time order of the syllable span timestamps, but rather maps the syllable duration range in speech segments of different source characters to a unified value caliber, enabling the speech rate description fields corresponding to speech segments of different source characters to be compared on the same dimension. The speech rate description field is used to represent syllable duration, syllable interval tightness, and pronunciation rhythm distribution. Next, the order of the intonation and speech rate vector fields is established. This order is formed by configuring the positions of the intonation field and the speech rate field during the acoustic feature extraction initialization stage. The order of the intonation and speech rate vector fields is used to limit the arrangement of the intonation description field and the speech rate description field in the same vector. Based on the order of the intonation and speech rate vector fields, the intonation description field and the speech rate description field are configured at the field level to generate the intonation and speech rate representation vectors. The technical essence of the intonation and speech rate representation vector is to incorporate the high and low undulation paths and syllable time spans in the source character's speech segment into the same ordered vector, so that subsequent acoustic multidimensional features can simultaneously preserve the intonation changes and pronunciation rhythm of the character's speech, rather than treating intonation and speech rate as two unrelated isolated data fields.

[0024] Preferably, the specific implementation process of extracting the pronunciation intensity amplitude waveform of the source character's speech segment and generating the emotion representation vector is as follows: After the source character's speech segment has completed speech frame division and formed the speech frame carrying record, the audible pronunciation time range is first located from the source character's speech segment based on the audible segment marker, and the original amplitude sequence is read within the audible pronunciation time range. The original amplitude sequence represents the change in sound wave intensity of the source character's speech segment at each sampling position. After absolute amplitude reading processing and local smoothing processing, the original amplitude sequence forms a candidate waveform for pronunciation intensity. The candidate waveform for pronunciation intensity is used to remove the influence of silent intervals and short-term spikes on the reading of intensity changes, but the candidate waveform for pronunciation intensity still retains the fluctuations in intensity caused by changes in vocal intensity in the source character's speech segment. Subsequently, the candidate waveform for pronunciation intensity is time-aligned with the syllable span timestamp, so that the pronunciation intensity change range corresponding to each syllable can be read in the candidate waveform for pronunciation intensity. Based on the time-aligned candidate waveform of vocal intensity, the intensity rise segment, intensity sustaining segment, and intensity fall segment within each syllable range are extracted. These segments are then connected in chronological order according to the timestamps of the syllable span to form the vocal intensity amplitude waveform. The technical essence of this vocal intensity amplitude waveform is the intensity envelope structure of the source character's speech segment over time within the vocal range, reflecting the acoustic performance of the character's speech in terms of vocal intensity, stress distribution, and dynamic fluctuations. Subsequently, the intensity peak position, intensity trough position, intensity rise duration, intensity fall duration, and syllable-level intensity distribution are read from the vocal intensity amplitude waveform. These positions are then encapsulated and processed according to the order of the emotion vector fields to generate the emotion representation vector. The order of the emotion vector fields is formed during the acoustic feature extraction initialization phase by configuring the positions of the intensity change field and the syllable-level intensity field. This order ensures that the emotion representation vectors of different source character speech segments have the same field arrangement. The emotion representation vectors then serve as the input source for the pronunciation intensity dimension when generating the subsequent acoustic multidimensional features, enabling the acoustic multidimensional features to contain information on the intensity changes of the character's speech under different emotional expression states.

[0025] Preferably, the specific implementation process for generating the acoustic multidimensional features based on the timbre representation vector, the intonation and speed representation vector, and the emotion representation vector is as follows: After the timbre representation vector, the intonation and speed representation vector, and the emotion representation vector have been formed respectively, field source indexing processing is first performed on the timbre representation vector, the intonation and speed representation vector, and the emotion representation vector to form vector source indexing records. The vector source indexing records the temporal source relationship, field source relationship, and vector segment boundary between each vector and the source character's speech segment, so that the timbre dimension, intonation and speed dimension, and emotion dimension can be distinguished when generating the acoustic multidimensional features subsequently. Subsequently, dimensional compatibility mapping processing is performed on the energy field in the timbre representation vector, the temporal field in the intonation and speed representation vector, and the intensity field in the emotion representation vector to form dimensional compatibility vectors corresponding to the timbre representation vector, the intonation and speed representation vector, and the emotion representation vector, respectively. The dimensional compatibility mapping process is used to map fields with different physical meanings to a unified numerical comparison caliber, avoiding the direct mixing of frequency, time, and amplitude as data with the same dimension for calculation. Based on the dimensional compatibility vectors corresponding to the timbre representation vector, the pitch and rate representation vectors, and the emotion representation vector, an acoustic multidimensional feature carrying record is established according to the order of acoustic multidimensional feature fields. The order of acoustic multidimensional feature fields is formed by configuring the timbre field segment, pitch and rate field segment, and emotion field segment during the acoustic feature extraction initialization stage. The order of acoustic multidimensional feature fields is used to define the arrangement position of each dimension field in the acoustic multidimensional feature carrying record. Subsequently, the dimensional compatibility vector corresponding to the timbre representation vector is written into the timbre field segment of the acoustic multidimensional feature carrying record, the dimensional compatibility vector corresponding to the pitch and rate representation vector is written into the pitch and rate field segment of the acoustic multidimensional feature carrying record, and the dimensional compatibility vector corresponding to the emotion representation vector is written into the emotion field segment of the acoustic multidimensional feature carrying record. Based on the recorded acoustic multidimensional features, field boundary encapsulation and source indexing association processes are then performed to form the acoustic multidimensional features. The technical essence of these acoustic multidimensional features is an ordered acoustic representation of the source character's speech fragment, simultaneously including timbre, intonation, speech rate, and emotional dimensions. After being bound to the preset character tag, these acoustic multidimensional features participate in the generation of the feature association sequence and serve as a multidimensional comparison object in the subsequent structural similarity analysis process. This allows character speech differentiation and modeling to no longer rely on a single voiceprint template but rather on a multidimensional acoustic structure for subsequent processing.

[0026] Optionally, the step of binding the acoustic multidimensional features to a preset role tag used to identify the source role's speech segment affixes to generate a feature association sequence includes: Assign a preset character tag with a unique identifier to the source character's voice clip; Construct a multidimensional data mapping table and write the preset role tag into the index field of the multidimensional data mapping table; The acoustic multidimensional features are stored in the numerical storage field of the multidimensional data mapping table; Establish a unique mapping link between the index field and the numerical storage field, and arrange the unique mapping links in chronological order to generate the feature association sequence.

[0027] Preferably, the specific implementation process of assigning the preset role tag with a unique identifier attribute to the source character voice segment is as follows: After the source character voice segment has completed the acoustic multidimensional feature generation, the segment source location field corresponding to the source character voice segment is first read. The segment source location field includes the source identifier of the initial audiovisual data, the segment start time, segment end time, and audio track position of the source character voice segment in the initial audiovisual data. The segment source location field is used to fix the source position of the source character voice segment in the initial audiovisual data, so that the assignment of the preset role tag is not an arbitrary labeling of isolated audio data, but a correspondence with the time position and audio track position of the source character voice segment in the initial audiovisual data. Subsequently, the role tag configuration record is read. The role tag configuration record is established before character voice processing of the initial audiovisual data. The role tag configuration record includes a candidate role tag field, the allocation status of the candidate role tag field, the segment source location field corresponding to the candidate role tag field, and the occupancy verification status of the candidate role tag field. The technical essence of the role tag configuration record is to provide a distinguishable data index source for different source role voice segments before the acoustic multidimensional features enter the subsequent structural similarity analysis. Specifically, the segment source location field is matched and verified with the segment source location field corresponding to the candidate role tag field in the role tag configuration record. When the occupancy verification status of the candidate role tag field indicates that the candidate role tag field is not occupied by other source role voice segments, the candidate role tag field is configured as the preset role tag corresponding to the source role voice segment. The unique identifier attribute indicates that the preset role tag corresponds to only one segment source location field within the processing range of the same initial audiovisual data, and the same segment source location field corresponds to only one preset role tag. Based on the unique identifier attribute, a stable binding relationship can be formed between the subsequent acoustic multidimensional features and the preset role tag, avoiding confusion in role attribution caused by multiple source role voice segments sharing the same data index source in the feature association sequence. The preset role tag is subsequently written into the index field of the multidimensional data mapping table, and points to the numerical storage field through the index field, enabling the role attribution source of the source role voice segment to enter the feature association sequence along with the acoustic multidimensional features.

[0028] Preferably, in the specific technical implementation of constructing the multidimensional data mapping table, the source character voice segment with the assigned preset role tag, the acoustic multidimensional features corresponding to the source character voice segment, and the segment source location field are read first. A mapping table field configuration record is then established based on the preset role tag, the acoustic multidimensional features, and the segment source location field. The mapping table field configuration record includes index field configuration, numerical storage field configuration, segment time field configuration, segment source location field configuration, and link order field configuration. The index field configuration is used to limit the writing position of the preset role tag in the multidimensional data mapping table; the numerical storage field configuration is used to limit the storage position of the acoustic multidimensional features in the multidimensional data mapping table; the segment time field configuration is used to limit the recording position of the segment start time and segment end time of the source character voice segment in the multidimensional data mapping table; the segment source location field configuration is used to limit the recording position of the source identifier and audio track position of the initial audiovisual data in the multidimensional data mapping table; and the link order field configuration is used to limit the reading field when subsequent unique mapping links are arranged in chronological order. Subsequently, the multidimensional data mapping table is established according to the field configuration records of the mapping table, and each source role's voice segment is used as a mapping record in the multidimensional data mapping table, so that each mapping record simultaneously possesses the index field, the numerical storage field, the segment time field, the segment source location field, and the link order field. The technical essence of the multidimensional data mapping table is to incorporate the preset role tag, the acoustic multidimensional feature, and the segment source location field into the same readable data structure, so that when establishing the unique mapping link later, the preset role tag, the acoustic multidimensional feature, and the segment source location field can be directly read from the same mapping record. Compared with the method of storing only the role tag and acoustic feature separately, the multidimensional data mapping table makes the preset role tag and the acoustic multidimensional feature form a table-based relationship at the field level. When analyzing the structural similarity of the acoustic multidimensional features in the feature association sequence later, the source role's voice segment corresponding to each acoustic multidimensional feature can be traced based on the multidimensional data mapping table, reducing the situation where the acoustic multidimensional feature enters the feature clustering process after being separated from the role's source.

[0029] Preferably, the specific implementation process of writing the preset role tag into the index field of the multidimensional data mapping table is as follows: After the multidimensional data mapping table has been established according to the mapping table field configuration record, the preset role tag corresponding to the source role voice segment is first read, and the mapping record corresponding to the source role voice segment in the multidimensional data mapping table is read. The mapping record forms a correspondence with the source role voice segment through the segment source positioning field. Therefore, before writing the preset role tag, the segment source positioning field in the mapping record is first checked for consistency with the segment source positioning field corresponding to the source role voice segment to determine that the segment start time, segment end time, and audio track position corresponding to the mapping record all come from the same source role voice segment. After completing the consistency check, the preset role tag is written into the index field of the mapping record, and the occupancy status of the index field is written into the role tag configuration record, so that the role tag configuration record can record that the preset role tag has established a correspondence with the mapping record. The technical essence of the index field is that it serves as the role affiliation index position in the multidimensional data mapping table, carrying the preset role marker. It does not directly store the numerical content of the acoustic multidimensional features, but rather points to the numerical storage field in subsequent unique mapping links. By storing the preset role marker in the index field, when forming the feature association sequence, the index field can be read first, and then the acoustic multidimensional features in the numerical storage field can be located through the unique mapping link, thus establishing a clear field-level pointing relationship between the role affiliation source and the acoustic multidimensional features. The index field also participates in the subsequent timeline ordering process together with the link sequence field. Specifically, the index field provides the role affiliation entry point for each unique mapping link, and the link sequence field provides the time arrangement basis for each unique mapping link in the initial audiovisual data. Together, they ensure that the feature association sequence retains both the preset role marker and the chronological relationship of the source role's voice segments on the timeline.

[0030] Preferably, in the specific implementation of storing the acoustic multidimensional features into the numerical storage field of the multidimensional data mapping table, the acoustic multidimensional feature carrying record corresponding to the acoustic multidimensional features is first read. The acoustic multidimensional feature carrying record includes a timbre field segment, a pitch and rate field segment, and an emotion field segment. The timbre field segment is used to carry a dimensionally compatible vector formed by the timbre representation vector, the pitch and rate field segment is used to carry a dimensionally compatible vector formed by the pitch and rate representation vector, and the emotion field segment is used to carry a dimensionally compatible vector formed by the emotion representation vector. Subsequently, the mapping record in the multidimensional data mapping table that has been written with the preset role tag is read, and the index field in the mapping record and the source index association information in the acoustic multidimensional feature carrying record are verified to ensure that the mapping record into which the acoustic multidimensional features are stored is consistent with the source role speech segment pointed to by the preset role tag. After completing the corresponding verification, the timbre field segment, the pitch and rate field segment, and the emotion field segment in the acoustic multidimensional feature-carrying record are written into the numerical storage field according to the numerical storage field configuration. The technical essence of the numerical storage field is that it is the ordered numerical field position used to store the acoustic multidimensional features in the multidimensional data mapping table. The numerical storage field distinguishes the timbre dimension, pitch and rate dimension, and emotion dimension through field boundaries, enabling the acoustic multidimensional features to be read according to the same field caliber when subsequently analyzing the structural similarity. The numerical storage field is not an independent result of renaming the acoustic multidimensional features, but rather the storage position of the acoustic multidimensional features in the multidimensional data mapping table; after the acoustic multidimensional features are stored in the numerical storage field, their field content still participates in the subsequent unique mapping link establishment and structural similarity analysis. With the above storage method, the acoustic multidimensional features are not stored separately from the preset role marker, but form a field-level correspondence with the index field in the same mapping record. The field-level correspondence provides a data basis for the index field to point to the numerical storage field in the future.

[0031] Preferably, the specific implementation process for establishing a unique mapping link between the index field and the numerical storage field is as follows: After the preset role marker has been written into the index field and the acoustic multidimensional feature has been stored in the numerical storage field, the index field, the numerical storage field, the segment time field, and the segment source location field in the same mapping record are read first, and a field pointing relationship is established based on the index field and the numerical storage field. The field pointing relationship indicates that the preset role marker in the index field can locate the acoustic multidimensional feature in the same mapping record. The field pointing relationship also records the segment time field and the segment source location field, so that the field pointing relationship not only has a role affiliation entry, but also a time location source and an initial audiovisual data source. Subsequently, a uniqueness verification process is performed on the field pointing relationship. The uniqueness verification process includes index field duplication verification, numerical storage field occupancy verification, and segment source location field consistency verification. The index field duplication check is used to determine whether the same preset role marker is written into a mapping record that does not belong to the same source role voice segment. The numerical storage field occupancy check is used to determine whether the same acoustic multidimensional feature is repeatedly stored in multiple non-corresponding mapping records. The segment source location field consistency check is used to determine whether the index field and the numerical storage field correspond to the same segment source location field. After the uniqueness check, the field pointing relationship is encapsulated into the unique mapping link. The technical essence of the unique mapping link is to establish a deterministic reading path from the preset role marker to the acoustic multidimensional feature within the multidimensional data mapping table. The unique mapping link enables subsequent reading of the feature association sequence to first locate the index field from the preset role marker, and then enter the numerical storage field from the index field to read the acoustic multidimensional feature. The unique mapping link also retains the segment time field and the segment source location field. Therefore, when arranging the unique mapping link in chronological order, the segment time field can be directly read to complete the sequential arrangement without having to return to the initial audiovisual data to find the time position of the source role voice segment.

[0032] Preferably, the specific implementation process of arranging the unique mapping links in chronological order to generate the feature association sequence is as follows: After multiple unique mapping links have been formed in the multidimensional data mapping table, the segment time field retained in each unique mapping link is read first, and the time position of each unique mapping link in the initial audiovisual data is determined according to the segment start time and segment end time in the segment time field. Subsequently, chronological order arrangement processing is performed on each unique mapping link, which arranges each unique mapping link according to the order of segment start times; when the voice segments of different source characters overlap in time range, the audio track position in the segment source positioning field is read again, and the unique mapping links with overlapping time are arranged in parallel segments according to the audio track position. The parallel segment arrangement is used to retain the time overlap relationship at different audio track positions, so that the unique mapping links with overlapping time are not mistakenly merged into the same unique mapping link. After the chronological order arrangement processing is completed, the arranged unique mapping links are subjected to sequence encapsulation processing to generate the feature association sequence. The technical essence of the feature association sequence is an association data sequence formed by arranging the source character's voice segments in the initial audiovisual data according to the timeline order of the unique mapping link as the basic sequence unit. Each unique mapping link retains the index field, the numerical storage field, the segment time field, and the segment source location field in the feature association sequence. Therefore, the feature association sequence can read the preset character tag through the index field, read the acoustic multidimensional features through the numerical storage field, and trace the position of the source character's voice segment in the initial audiovisual data through the segment time field and the segment source location field. The feature association sequence serves as the source for reading the acoustic multidimensional features in the subsequent structural similarity analysis process, ensuring that any two acoustic multidimensional features maintain their binding relationship with the corresponding preset character tag before being extracted and spatially projected. Thus, the subsequent feature clustering process can compare the structural similarity of the acoustic multidimensional features while retaining the character's attribution source, enabling the generation of character-specific feature clusters to inherit the character attribution relationship and timeline order relationship already established in the feature association sequence.

[0033] Optionally, the step of parsing the structural similarity of the acoustic multidimensional features within the feature association sequence includes: Extract any two of the acoustic multidimensional features from the feature association sequence; Any two of the acoustic multidimensional features are projected onto a multidimensional vector space that carries the acoustic multidimensional features to generate two corresponding spatial projection points. Measure the spatial Euclidean span of the two spatial projection points in the multidimensional vector space; The structural similarity is generated based on the spatial Euclidean span, wherein the value of the structural similarity increases proportionally as the spatial Euclidean span decreases.

[0034] Preferably, the specific implementation process for extracting any two acoustic multidimensional features from the feature association sequence is as follows: After the feature association sequence has been formed by arranging multiple unique mapping links in chronological order, the sequence units in the feature association sequence are first read. Each sequence unit retains the index field, the numerical storage field, the segment time field, and the segment source location field. The technical essence of the sequence unit is the serialized carrier form of the unique mapping link in the feature association sequence. The sequence unit can read the preset role marker through the index field and read the acoustic multidimensional features through the numerical storage field. Subsequently, any two sequence units are selected from the feature association sequence according to the comparison window configuration record. The comparison window configuration record is established before structural similarity analysis. The comparison window configuration record includes the time interval reading caliber, the role marker reading caliber, the field integrity reading caliber, and the comparison pair arrangement caliber. The time interval reading caliber is used to determine whether two sequence units need to be read adjacently or across segments according to the segment time field. The role marker reading caliber is used to determine whether the preset role markers of two sequence units need to enter the same acoustic feature comparison pair record. The field integrity reading caliber is used to determine whether the numerical storage fields in the two sequence units both contain timbre dimension, intonation and speech rate dimension, and emotion dimension. The comparison pair arrangement caliber is used to determine the reading order of the two sequence units in the acoustic feature comparison pair record. After selecting two sequence units based on the comparison window configuration record, the corresponding acoustic multidimensional features are read from the numerical storage fields of each of the two sequence units, and the acoustic feature comparison pair record is established. The acoustic feature comparison pair record includes two acoustic multidimensional features, preset role markers corresponding to the two acoustic multidimensional features, segment time fields corresponding to the two acoustic multidimensional features, and segment source location fields corresponding to the two acoustic multidimensional features. The technical essence of any two of the aforementioned acoustic multidimensional features is that they are two acoustic multidimensional features to be compared extracted from the feature association sequence. These are not isolated features detached from the source of the role attribution, but rather, through the acoustic feature comparison pair, the correspondence between the feature features and the preset role marker and the source role's speech segment is preserved. Therefore, when any two of the aforementioned acoustic multidimensional features are subsequently projected onto the multidimensional vector space, two acoustic multidimensional features with complete fields and traceable sources in the acoustic feature comparison pair record can be used as input, allowing the structural similarity to reflect the degree of closeness between the two source role's speech segments in the multidimensional acoustic structure.

[0035] Preferably, before projecting any two of the acoustic multidimensional features onto the multidimensional vector space used to carry the acoustic multidimensional features, the multidimensional vector space is first constructed. Specifically, the timbre field segment, intonation and speech rate field segment, and emotion field segment from the acoustic multidimensional features are read, and a vector space field configuration record is established based on the timbre field segment, the intonation and speech rate field segment, and the emotion field segment. The vector space field configuration record includes timbre dimension coordinate configuration, intonation and speech rate dimension coordinate configuration, emotion dimension coordinate configuration, and field dimension compatibility configuration. The timbre dimension coordinate configuration is used to arrange the various fields in the timbre field segment as timbre coordinate positions in the multidimensional vector space. The intonation and speech rate dimension coordinate configuration is used to arrange the various fields in the intonation and speech rate field segment as intonation and speech rate coordinate positions in the multidimensional vector space. The emotion dimension coordinate configuration is used to arrange the various fields in the emotion field segment as emotion coordinate positions in the multidimensional vector space. The field dimension compatibility configuration is used to continue the unified numerical comparison caliber formed by the previous dimension compatibility mapping process, avoiding frequency, time, and intensity fields from directly participating in span measurement without undergoing the same caliber mapping. Subsequently, the multidimensional vector space is established based on the vector space field configuration record. The technical essence of the multidimensional vector space is to carry a unified coordinate carrying structure for each field segment in the acoustic multidimensional feature. Each coordinate position corresponds to a field position in the acoustic multidimensional feature, and the value of each coordinate position comes from the field content in the acoustic multidimensional feature that has undergone dimension compatibility mapping processing. The multidimensional vector space is not a result of renaming acoustic data, but rather a unified coordinate structure that transforms the timbre, intonation, speech rate, and emotion dimensions of the acoustic multidimensional features into a spatially measurable structure. Based on this multidimensional vector space, two subsequent acoustic multidimensional features can be projected into two spatial projection points under the same field order and value caliber, allowing acoustic differences between speech segments from different source characters to be read within the multidimensional vector space. The multidimensional vector space also maintains a connection with the acoustic feature comparison pair records, ensuring that after projection, two acoustic multidimensional features entering the multidimensional vector space can still be traced back to the corresponding preset character marker, segment time field, and segment source location field through the acoustic feature comparison pair records.

[0036] Preferably, in the specific technical implementation of generating the two corresponding spatial projection points, the first and second acoustic multidimensional features in the acoustic feature comparison record are read first. Then, according to the field arrangement order configured in the vector space field configuration record, the timbre field segment, intonation and speech rate field segment, and emotion field segment of the first acoustic multidimensional feature, as well as the timbre field segment, intonation and speech rate field segment, and emotion field segment of the second acoustic multidimensional feature, are read respectively. Subsequently, according to the timbre dimension coordinate configuration, the timbre field segments of the two acoustic multidimensional features are written to the timbre coordinate positions in the multidimensional vector space; according to the intonation and speech rate dimension coordinate configuration, the intonation and speech rate field segments of the two acoustic multidimensional features are written to the intonation and speech rate coordinate positions in the multidimensional vector space; and according to the emotion dimension coordinate configuration, the emotion field segments of the two acoustic multidimensional features are written to the emotion coordinate positions in the multidimensional vector space. The aforementioned projection process creates a spatial projection point for the first acoustic multidimensional feature in the multidimensional vector space, and another spatial projection point for the second acoustic multidimensional feature in the same space. The technical essence of this spatial projection point is the coordinate representation of the acoustic multidimensional feature in the multidimensional vector space. Each coordinate component originates from the field content of the acoustic multidimensional feature, and each coordinate component follows the field arrangement criteria defined by the vector space field configuration record. To avoid field misalignment in the span measurement between the two spatial projection points due to missing fields, after forming the spatial projection points, a coordinate integrity check is performed on both spatial projection points. This check verifies whether the coordinate components of the two spatial projection points at the timbre, pitch, and rate coordinates, as well as the emotion coordinate, all have readable values. If a readable value is missing at a certain coordinate position, the process returns to the acoustic feature comparison record to reread the corresponding acoustic multidimensional feature, thus preventing missing fields from directly entering the spatial Euclidean span measurement. Through the above projection processing, the two acoustic multidimensional features are transformed from field-based acoustic representations into two spatial projection points. These two spatial projection points are used as direct measurement objects in subsequent spatial Euclidean span measurements, enabling structural similarity analysis to unfold around coordinate differences in the same multidimensional vector space.

[0037] Preferably, the specific implementation process for measuring the spatial Euclidean span of the two spatial projection points in the multidimensional vector space is as follows: After the two spatial projection points have passed the coordinate integrity verification process, the coordinate components with the same coordinate position in the two spatial projection points are first read according to the vector space field configuration record, and a coordinate difference reading process is performed on each coordinate component with the same coordinate position to form a coordinate difference record. The coordinate difference record includes a timbre coordinate difference field, a pitch and speed coordinate difference field, and an emotion coordinate difference field; the timbre coordinate difference field comes from the component difference of the two spatial projection points at the timbre coordinate position, the pitch and speed coordinate difference field comes from the component difference of the two spatial projection points at the pitch and speed coordinate positions, and the emotion coordinate difference field comes from the component difference of the two spatial projection points at the emotion coordinate position. Subsequently, a square accumulation reading process is performed on the coordinate difference record to form a span accumulation field. The technical meaning of the square accumulation reading process is to first convert the positive and negative differences in the timbre coordinate difference field, the pitch and speed coordinate difference field, and the emotion coordinate difference field into non-negative differences, and then accumulate the non-negative differences in the timbre dimension, pitch and speed dimension, and emotion dimension to the same span reading caliber, avoiding the mutual cancellation of coordinate differences in opposite directions when directly added. Based on the span accumulation field, square root mapping processing is then performed to form the spatial Euclidean span. The technical essence of the spatial Euclidean span is the comprehensive distance reading result formed by the two spatial projection points along each coordinate position in the multidimensional vector space, and its value comes from the overall difference between the two acoustic multidimensional features in the timbre dimension, pitch and speed dimension, and emotion dimension. The spatial Euclidean span does not directly represent the role assignment result, but rather serves as an intermediate measurement result for the subsequent generation of the structural similarity. A smaller spatial Euclidean span indicates a higher degree of closeness in the coordinates of the two acoustic multidimensional features within the multidimensional vector space; conversely, a larger spatial Euclidean span indicates a lower degree of closeness in the coordinates of the two acoustic multidimensional features within the multidimensional vector space. Through this measurement method, the spatial Euclidean span can transform the differences scattered across different acoustic field segments into a comprehensive span result that can be used for subsequent structural similarity generation.

[0038] Preferably, the specific implementation process for generating the corresponding structural similarity based on the spatial Euclidean span is as follows: After obtaining the spatial Euclidean span, the similarity conversion configuration record is first read. The similarity conversion configuration record is established before structural similarity analysis. The similarity conversion configuration record includes the span reading range, the similarity output range, and the conversion caliber from span to similarity. The span reading range is used to limit the comparable range of the spatial Euclidean span in the same batch of source character speech segments. The similarity output range is used to limit the output value range of the structural similarity. The conversion caliber from span to similarity is used to convert the spatial Euclidean span into the structural similarity. In specific implementation, the spatial Euclidean span is first written into the similarity conversion carrying record, and the two preset character tags, the two segment time fields, and the two segment source positioning fields in the acoustic feature comparison pair record are simultaneously written into the similarity conversion carrying record, so that the structural similarity obtained subsequently can still be traced back to the two sequence units in the feature association sequence through the similarity conversion carrying record. Subsequently, based on the conversion caliber from span to similarity, a reverse proportional mapping process is performed on the spatial Euclidean span to generate the structural similarity. The technical meaning of this reverse proportional mapping process is to convert the spatial Euclidean span from a distance-based measurement result into a similarity-based judgment result, so that a smaller spatial Euclidean span corresponds to a higher structural similarity, and a larger spatial Euclidean span corresponds to a lower structural similarity. The technical essence of structural similarity is a numerical representation of the closeness of two acoustic multidimensional features at the multidimensional acoustic structure level. It originates from the spatial Euclidean span and still retains the role attribution and temporal origin of the two acoustic multidimensional features. The structural similarity is subsequently compared with a preset approximation threshold to determine whether the two acoustic multidimensional features meet the conditions for feature clustering processing. Therefore, the structural similarity is not an isolated comparison value, but rather a criterion connecting the feature association sequence and the subsequent generation process of role-specific feature clusters.

[0039] Preferably, the technical implementation process of the structural similarity value increasing proportionally based on the reduction of the spatial Euclidean span is as follows: When configuring the conversion caliber from span to similarity in the similarity conversion configuration record, the inverse relationship between the spatial Euclidean span and the structural similarity is first defined, so that when the spatial Euclidean span decreases, the structural similarity increases according to the same conversion caliber; when the spatial Euclidean span increases, the structural similarity decreases according to the same conversion caliber. This proportional increase does not mean that the two values ​​have the same physical dimensions, but rather that within the same output range defined by the similarity conversion configuration record, the reduction in the spatial Euclidean span is mapped to the increase in the structural similarity according to a uniform proportional relationship. Since the spatial Euclidean span originates from the coordinate difference between the two spatial projection points in the multidimensional vector space, a reduction in the spatial Euclidean span indicates a decrease in the comprehensive differences between the two acoustic multidimensional features in the timbre, intonation, speech rate, and emotion dimensions. Therefore, the structural similarity increases as the spatial Euclidean span decreases, enabling subsequent preset approximation threshold comparisons to better meet the processing requirements of character voice differentiation and modeling. In specific implementation, the structural similarity after inverse proportional mapping processing is written into a structural similarity record. The structural similarity record includes the structural similarity, the spatial Euclidean span, the two spatial projection points, the two acoustic multidimensional features, the two preset character markers, the two segment time fields, and the two segment source location fields. The structural similarity record is used to store the structural similarity along with the spatial Euclidean span, two spatial projection points, two acoustic multidimensional features, two preset role markers, two segment time fields, and two segment source location fields. This allows for direct reading of the structural similarity and the corresponding two acoustic multidimensional features when determining whether the preset approximation threshold has been reached based on the structural similarity. Thus, the generation process of the structural similarity establishes a clear technical connection between data source, projection process, span measurement, and similarity conversion. Subsequent feature aggregation processing can continue to read the acoustic multidimensional features that have reached the preset approximation threshold based on the structural similarity record and incorporate these features into the generation process of the role-specific feature cluster.

[0040] Optionally, the step of performing feature clustering processing on the acoustic multidimensional features that reach a preset approximation threshold based on the structural similarity to generate a character-specific feature cluster includes: When the structural similarity exceeds the preset approximation threshold used to determine whether the acoustic multidimensional features have undergone feature clustering processing, it is confirmed that any two acoustic multidimensional features have a consistent attribution attribute. Establish a convergence boundary for the acoustic multidimensional features that possess the aforementioned belonging consistency attribute, and encapsulate all the acoustic multidimensional features located inside the convergence boundary into a data array; Read the preset role marker pointed to by the acoustic multidimensional features in the data array within the feature association sequence; Assign the preset role tag to the data array, and generate the role-specific feature cluster based on the data array with the preset role tag.

[0041] Preferably, when the structural similarity exceeds the preset approximation threshold used to determine whether the acoustic multidimensional features have undergone feature clustering processing, the specific implementation process for confirming that any two acoustic multidimensional features have a consistent attribution attribute is as follows: After the structural similarity record has been formed by the preceding structural similarity analysis, the structural similarity, spatial Euclidean span, two spatial projection points, two acoustic multidimensional features, two preset role markers, two segment time fields, and two segment source location fields in the structural similarity record are read first, and the clustering determination configuration record is read simultaneously. The clustering determination configuration record is configured and formed before the feature clustering processing is performed, and the clustering determination configuration record includes the preset approximation threshold, the threshold applicable field range, the role marker comparison caliber, and the clustering determination output caliber. The essence of the preset approximation threshold is to determine whether two acoustic multidimensional features reach the similarity boundary of convergence conditions at the multidimensional acoustic structure level. The preset approximation threshold is not an arbitrary empirical value, but is configured based on the distribution range of structural similarity corresponding to the source character speech segments in the same batch, the field completeness of the acoustic multidimensional features, and the processing granularity of character speech differentiation and modeling. Specifically, the threshold application field range is used to limit the preset approximation threshold to structural similarity after the timbre dimension, intonation and speech rate dimension, and emotion dimension are all involved. The character tag comparison caliber is used to continue reading two preset character tags after the structural similarity exceeds the preset approximation threshold. The convergence determination output caliber is used to form a consistent attribution attribute record. Subsequently, the structural similarity is numerically compared with the preset approximation threshold. When the structural similarity exceeds the preset approximation threshold, it indicates that the two spatial projection points corresponding to the two acoustic multidimensional features have a relatively close coordinate proximity in the multidimensional vector space, and thus the two acoustic multidimensional features are written into the consistent attribution attribute record. The technical essence of the so-called belonging consistency attribute is that the two acoustic multidimensional features satisfy the same role acoustic expression convergence condition at the structural similarity level. The so-called belonging consistency attribute does not directly replace the preset role label, but provides an acoustic structural basis for the subsequent establishment of convergence boundary, so that the subsequent feature convergence processing can continue to read the acoustic multidimensional features with the so-called belonging consistency attribute based on the structural similarity record.

[0042] Preferably, in the specific technical implementation of establishing the convergence boundary of the acoustic multidimensional features with the aforementioned belonging-consistency attribute, the two acoustic multidimensional features written in the belonging-consistency attribute record are first read, and the structural similarity record, spatial Euclidean span, and two spatial projection points corresponding to the belonging-consistency attribute record are also read. Since the two spatial projection points have been formed by projecting the two acoustic multidimensional features onto the multidimensional vector space, the two spatial projection points can directly represent the coordinate positions of the two acoustic multidimensional features in the timbre dimension, pitch and speech rate dimension, and emotion dimension. Subsequently, the boundary expansion aperture corresponding to the preset approximation threshold is read according to the boundary generation configuration record. The boundary generation configuration record is configured and formed before performing feature convergence processing, and the boundary generation configuration record includes the boundary center reading aperture, the boundary span reading aperture, the boundary field coverage range, and the boundary member acceptance aperture. The boundary center reading caliber is used to determine the convergence boundary center position based on the coordinate positions of the two spatial projection points. The boundary span reading caliber is used to determine the convergence boundary span range based on the spatial Euclidean span and the preset approximation threshold. The boundary field coverage range is used to limit the convergence boundary to simultaneously cover the timbre dimension, intonation and speech rate dimension, and emotion dimension. The boundary member acceptance caliber is used to determine whether the acoustic multidimensional features read from the feature association sequence can be included in the convergence boundary. Based on the boundary configuration record, boundary center reading processing and boundary span reading processing are performed on the two acoustic multidimensional features with the belonging consistency attribute to form a convergence boundary carrying record. The convergence boundary carrying record includes the convergence boundary center position, the convergence boundary span range, the preset approximation threshold, the two acoustic multidimensional features, and the two preset role markers. The technical essence of the convergence boundary is that it forms a readable coordinate range in the multidimensional vector space around the acoustic multidimensional features that possess the belonging consistency attribute. The convergence boundary is used to define which acoustic multidimensional features can be included in the same convergence range in the multidimensional acoustic structure. Through the record carried by the convergence boundary, the acoustic multidimensional features used for boundary membership judgment can be read again in the feature association sequence, and it can be determined whether the candidate spatial projection point corresponding to the acoustic multidimensional features used for boundary membership judgment is located inside the convergence boundary. Thus, the convergence boundary becomes the technical connecting object between the belonging consistency attribute and the data array.

[0043] Preferably, the specific implementation process of encapsulating all the acoustic multidimensional features located inside the convergence boundary into a data array is as follows: After the convergence boundary carrying record is formed, the acoustic multidimensional features in each sequence unit are read from the feature association sequence, and the acoustic multidimensional features in each sequence unit are projected into the multidimensional vector space according to the vector space field configuration record to form candidate spatial projection points. The candidate spatial projection points use the same timbre coordinate position, pitch and speed coordinate position, and emotion coordinate position as the two preceding spatial projection points, so the candidate spatial projection points can be used to determine their positional relationship with the convergence boundary under the same coordinate caliber. Subsequently, each candidate spatial projection point is processed to determine whether it is inside or outside the boundary with the center position and span range of the convergence boundary in the convergence boundary carrying record to form a boundary member record. The boundary member record includes candidate spatial projection points located inside the convergence boundary, the acoustic multidimensional features corresponding to the candidate spatial projection points located inside the convergence boundary, the preset role markers corresponding to the acoustic multidimensional features, the segment time field corresponding to the acoustic multidimensional features, and the segment source location field corresponding to the acoustic multidimensional features. The function of the boundary member record is to reconnect the acoustic multidimensional features that satisfy the convergence boundary with the preset role markers, the segment time field, and the segment source location field in the feature association sequence, so that the role attribution source and time source of the acoustic multidimensional features will not be lost when the data array is subsequently encapsulated. Subsequently, the boundary member record is processed by field sorting and array position writing according to the array encapsulation configuration record, which includes the array row arrangement size, the array column arrangement size, the array source field writing size, and the array boundary field writing size. The array row arrangement is used to arrange the acoustic multidimensional features located within the convergence boundary according to the segment time field. The array column arrangement is used to arrange the field content of the acoustic multidimensional features according to the timbre dimension, tone and speech rate dimension, and emotion dimension. The array source field writing entry is used to write the preset role marker, the segment time field, and the segment source location field along with the acoustic multidimensional features into the same array row. The array boundary field writing entry is used to write the convergence boundary carrying record into the boundary description field of the data array. The resulting data array is not a simple stack of acoustic data, but an ordered data array based on the convergence boundary as the encapsulation basis, the acoustic multidimensional features as the array body, and retaining the preset role marker, the segment time field, the segment source location field, and the convergence boundary carrying record corresponding to the array source field writing entry. The data array provides a traceable data foundation for subsequent reading of the preset role marker and generation of the role-specific feature cluster.

[0044] Preferably, the specific implementation process of reading the preset role marker pointed to by the acoustic multidimensional features in the data array within the feature association sequence is as follows: After the data array has been formed according to the array encapsulation configuration record, the acoustic multidimensional features corresponding to each array row in the data array are first read, and the preset role marker, the segment time field, and the segment source location field corresponding to each acoustic multidimensional feature are read according to the array source field write entry. Since the acoustic multidimensional features in the data array originate from the sequence units in the feature association sequence, each acoustic multidimensional feature in the data array can still be traced back to the index field and the numerical storage field through the unique mapping link. Subsequently, according to the segment time field and the segment source location field corresponding to each acoustic multidimensional feature in the data array, the sequence unit corresponding to the acoustic multidimensional feature is read from the feature association sequence, and the preset role marker is read from the index field of the corresponding sequence unit. The above reading process performs a consistency check between the preset role markers recorded in the data array and the preset role markers in the feature association sequence. This avoids misalignment of role attribution sources caused by array row arrangement processing after the acoustic multidimensional features enter the data array. The technical essence of the preset role markers here is that the acoustic multidimensional features in the data array trace back to the role attribution index of the source character's speech segment. The preset role markers are not newly generated markers, but are obtained by reading the index field in the feature association sequence. After the consistency check is completed, the preset role marker corresponding to each acoustic multidimensional feature is written into a role marker reading record. The role marker reading record includes the acoustic multidimensional feature in the data array, the preset role marker pointed to by the acoustic multidimensional feature, the segment time field, and the segment source location field. The role marker reading record provides a basis for subsequently assigning the preset role markers corresponding to the data array, ensuring that the role attribution source of the data array comes from the feature association sequence, rather than being determined separately after feature aggregation processing. This maintains the role attribution continuity between the feature association sequence, the data array, and the character-specific feature cluster.

[0045] Preferably, in the specific technical implementation of assigning the preset role tag to the data array, the role tag reading record is first read, and the tag distribution state of the preset role tags pointed to by each of the acoustic multidimensional features in the data array is statistically analyzed. The tag distribution state is used to indicate whether the preset role tags corresponding to each array row in the same data array are concentrated in pointing to the same role affiliation index. Subsequently, the tag distribution state is processed for tag consistency determination according to the tag assignment configuration record. The tag assignment configuration record is configured and formed before feature aggregation processing. The tag assignment configuration record includes a tag consistency reading calibrator, an array tag writing calibrator, and a conflict tag retention calibrator. The tag consistency reading calibrator is used to determine whether most array rows in the data array point to the same preset role tag. The array tag writing calibrator is used to write the preset role tags that satisfy the tag consistency reading calibrator into the array affiliation field of the data array. The conflict tag retention calibrator is used to retain the preset role tag, the segment time field, and the segment source location field of the corresponding array row when a small number of array rows point to different preset role tags, so as to avoid the data array losing local source differences after assigning the preset role tag. Based on the assigned configuration record, when the label distribution state meets the label consistency reading caliber, the corresponding preset role label is written into the array affiliation field of the data array to form the data array with the preset role label. The technical essence of assigning the preset role label to the data array is to write the role affiliation index, which is the common point of multiple acoustic multidimensional features within the data array, into an array-level field, enabling the data array to enter the subsequent role-specific feature cluster generation process as a candidate carrier of the same role's acoustic structure. This process does not change the original preset role label corresponding to each acoustic multidimensional feature, but rather adds an array-level role affiliation entry to the data array while retaining the preset role label, the segment time field, and the segment source location field corresponding to each array row. Therefore, when generating the role-specific feature cluster based on the data array with the preset role label, both the array-level preset role label and the array row-level preset role label can be read simultaneously, ensuring that the role-specific feature cluster has both the overall role affiliation after aggregation and retains the traceability of the source of each acoustic multidimensional feature.

[0046] Preferably, the specific implementation process for generating the role-specific feature cluster based on the data array with the preset role marker is as follows: After the preset role marker has been written to the data array at the array level, the array attribution field, array row field, array boundary description field, and the acoustic multidimensional features corresponding to each array row in the data array are read first. The array attribution field is used to read the preset role marker corresponding to the data array, the array row field is used to read each acoustic multidimensional feature located inside the convergence boundary, and the array boundary description field is used to read the convergence boundary carrying record used when generating the data array. Subsequently, feature cluster encapsulation processing is performed on the data array, which includes array attribution field encapsulation, acoustic multidimensional feature encapsulation, convergence boundary carrying record encapsulation, and source field encapsulation. The array attribution field encapsulation is used to write the preset role marker corresponding to the data array into the cluster attribution field of the role-specific feature cluster; the acoustic multidimensional feature encapsulation is used to write all the acoustic multidimensional features within the data array into the intra-cluster feature field of the role-specific feature cluster; the convergence boundary carrying record encapsulation is used to write the convergence boundary carrying record into the cluster boundary field of the role-specific feature cluster; the source field encapsulation is used to write the segment time field and the segment source location field corresponding to each acoustic multidimensional feature into the cluster source field of the role-specific feature cluster. Based on the above encapsulation processing, the role-specific feature cluster is formed. The technical essence of the role-specific feature cluster is to use the preset role marker as the role attribution entry, the acoustic multidimensional features located inside the convergence boundary as the intra-cluster acoustic content, and retain the role acoustic feature carrying results of the convergence boundary carrying record and the segment source location field. When the character-specific feature cluster is subsequently imported into the initial acoustic topology network, it can provide the acoustic multidimensional features required for training through the cluster-intra-cluster feature fields, the character affiliation source through the cluster affiliation field, the cluster boundary source through the cluster boundary field, and the time and trajectory source of the source character's speech fragment in the initial audiovisual data through the cluster source field. Therefore, the character-specific feature cluster is not simply a stack of acoustic multidimensional features, but rather an ordered character acoustic carrying result formed by structural similarity, the preset approximation threshold, the cluster boundary, the data array, and the preset character label, enabling the subsequent generation of the basic character acoustic network to continue based on the multidimensional acoustic structure under the same character affiliation.

[0047] Optionally, the initial acoustic topology network includes an input mapping layer, a multi-level implicit resolution layer, and a reconstructed output layer; the step of importing the acoustic multidimensional features within the character-specific feature cluster into the initial acoustic topology network to update the node parameters of the initial acoustic topology network until the output distribution of the initial acoustic topology network approximates the feature distribution of the character-specific feature cluster to generate a basic character acoustic network includes: The acoustic multidimensional features within the character-specific feature cluster are input into the input mapping layer to perform dimensionality reduction, and the dimensionality-reduced features are passed to the multi-level implicit parsing layer to extract deep vocal node features. The reconstructed output layer generates a simulated speech distribution map based on the deep vocal node features; Extract the distribution difference gradient of the feature distribution of the simulated speech distribution map for the character-specific feature cluster, and adjust the node activation weights of the multi-level implicit parsing layer layer by layer along the inverse feature propagation path of the distribution difference gradient. The node activation weights are continuously adjusted until the distribution difference gradient converges to a floating range that limits the output distribution to approximate the feature distribution, thereby locking the current node activation weights to generate the basic character acoustic network.

[0048] Preferably, the specific implementation process of inputting the acoustic multidimensional features within the character-specific feature cluster into the input mapping layer for dimensionality reduction is as follows: After the character-specific feature cluster has been encapsulated by the data array with the preset character tag, the cluster affiliation field, intra-cluster feature field, cluster boundary field, and cluster source field in the character-specific feature cluster are read first, and each acoustic multidimensional feature is read from the intra-cluster feature field in array row order. The cluster affiliation field is used to identify the preset character tag corresponding to the character-specific feature cluster, the cluster boundary field is used to record the cluster boundary carrying record, the cluster source field is used to record the segment time field and segment source location field corresponding to each acoustic multidimensional feature, and the intra-cluster feature field is used to provide the training input source for this import into the initial acoustic topology network. Subsequently, input field alignment processing is performed on each acoustic multidimensional feature according to the input field configuration record. The input field configuration record is configured and formed when the initial acoustic topology network is established. The input field configuration record includes the timbre field segment reading caliber, the intonation and speech rate field segment reading caliber, the emotion field segment reading caliber, and the input field arrangement caliber. Through the input field alignment process, the timbre field segment, the intonation and rate field segment, and the emotion field segment are arranged into an initial input vector that the input mapping layer can read. The technical essence of the input mapping layer is a field compression and transformation layer set at the front end of the initial acoustic topology network. It is used to convert the numerous acoustic multidimensional features from different sources into dimensionality-reduced features with fewer fields and a unified arrangement. This dimensionality reduction does not delete any acoustic dimension, but rather compresses and expresses fields with strong covariance relationships among the acoustic multidimensional features, while preserving the main directions of change for the timbre, intonation and rate, and emotion dimensions within the same character's acoustic structure. In specific implementation, the input mapping layer first reads similar fields from the initial input vector, then performs scale-consistent mapping on these similar fields to form a field-consistent input result; subsequently, it performs linear compression and nonlinear activation processing on the field-consistent input result to form the dimensionality-reduced features. The reduced-dimensional features retain the attribution relationship between each acoustic multidimensional feature and the character-specific feature cluster, and serve as the input source for the subsequent multi-level implicit parsing layer to extract the deep vocal node features, enabling the initial acoustic topology network to extract the stable acoustic structure of the same character from the acoustic multidimensional features that have already been aggregated.

[0049] Preferably, in the specific technical implementation of passing the dimensionality-reduced features to the multi-level implicit parsing layer to extract deep vocal node features, the dimensionality-reduced features formed by the input mapping layer are first written into the implicit parsing input record. The implicit parsing input record includes the dimensionality-reduced features, the cluster affiliation field, the cluster boundary field, and the cluster source field. The implicit parsing input record is used to ensure that the dimensionality-reduced features, after entering the multi-level implicit parsing layer, can still be traced back to the source of the acoustic multi-dimensional features in the character-specific feature cluster. The technical essence of the multi-level implicit parsing layer is a hierarchical parsing structure composed of multiple sequentially connected implicit feature parsing layers. Each implicit feature parsing layer reads the intermediate vocal features output by the previous layer and performs node weighting processing, node activation processing, and adjacent node propagation processing on the intermediate vocal features to form intermediate vocal features that can be read by the next layer. The "deep" in the deep vocal node features does not refer to a generalized expression with an uncertain level of abstraction, but rather to the hierarchical acoustic representation result formed after the dimensionality-reduced features have been sequentially processed by multiple implicit feature parsing layers in the multi-level implicit parsing layer. Compared with the dimensionality-reduced features, the deep vocal node features have integrated the interactive changes between the timbre dimension, intonation and speech rate dimension, and emotion dimension. In specific implementation, the front implicit feature parsing layer in the multi-level implicit parsing layer mainly reads the local field relationships in the dimensionality-reduced features to form local vocal node features; the middle implicit feature parsing layer continues to read the local vocal node features and performs inter-node transfer processing on the correlation changes between the timbre dimension and the intonation and speech rate dimensions to form combined vocal node features; the rear implicit feature parsing layer reads the combined vocal node features and performs node activation processing in conjunction with the intensity change information corresponding to the emotion dimension to form the deep vocal node features. The technical essence of the deep vocal node features is that they are a representation of the character's vocal structure formed by hierarchical parsing of multiple acoustic multidimensional features within the character-specific feature cluster. This representation describes the comprehensive acoustic relationship between the same character's timbre stability, intonation variation path, speech rate rhythm, and vocal intensity variations. These deep vocal node features are then passed to the reconstruction output layer, enabling it to generate the simulated speech distribution map based on the parsed character vocal structure, rather than generating an output distribution based solely on a single acoustic field.

[0050] Preferably, the specific implementation process of the reconstruction output layer generating an analog speech distribution map based on the deep vocal node features is as follows: After the deep vocal node features are formed by the multi-level implicit parsing layer, a reconstruction output carrying record is first established, and the deep vocal node features, the cluster affiliation field, the cluster boundary field, and the cluster source field are written into the reconstruction output carrying record. The reconstruction output carrying record is used to correspond the deep vocal node features with the role affiliation source and clustering range source of the role-specific feature cluster, so that the subsequently generated analog speech distribution map can be compared with the feature distribution of the role-specific feature cluster. The technical essence of the reconstruction output layer is a distribution reconstruction layer set at the end of the initial acoustic topology network, which reads the deep vocal node features and converts the deep vocal node features into a spectroscopic acoustic result that can characterize the output distribution. In specific implementation, the reconstruction output layer first performs node expansion processing on the deep vocal node features to form a reconstructed node expansion field. This field is used to re-expand the vocal node information at the end of the layer to the output field positions corresponding to the timbre, intonation, speech rate, and emotion dimensions. Subsequently, the reconstruction output layer performs distribution value mapping processing on the reconstructed node expansion field to form timbre distribution fields, intonation and speech rate distribution fields, and emotion distribution fields. These fields are then arranged into the simulated speech distribution map according to the output spectrogram field configuration record. The output spectrogram field configuration record is configured during the initial acoustic topology network establishment and includes the output field arrangement aperture, output distribution reading aperture, and output spectrogram boundary aperture. The technical essence of the simulated speech distribution map is the character acoustic output distribution carrying result obtained from the reconstruction of the deep vocal node features, which includes timbre distribution fields, intonation and speech rate distribution fields, and emotion fields corresponding to the feature distribution of the character-specific feature clusters. The simulated speech distribution map does not represent the actual played speech data, but rather the acoustic distribution reconstruction result of the initial acoustic topology network for the character-specific feature clusters under the current node parameters. Using this simulated speech distribution map, the output distribution of the initial acoustic topology network can be compared with the feature distribution of the character-specific feature clusters using the same field caliber to extract the distribution difference gradient, and the node parameters of the initial acoustic topology network can be updated accordingly.

[0051] Preferably, the specific implementation process of extracting the distribution difference gradient of the simulated speech distribution map for the feature distribution of the character-specific feature cluster and determining the inverse feature propagation path of the distribution difference gradient is as follows: After the simulated speech distribution map has been formed, the cluster feature fields in the character-specific feature cluster are first read, and the feature distribution of the character-specific feature cluster is formed based on the timbre field segment, intonation and speech rate field segment, and emotion field segment of each acoustic multidimensional feature in the cluster feature field. The technical essence of the feature distribution is the field distribution result of all acoustic multidimensional features in the character-specific feature cluster in the timbre dimension, intonation and speech rate dimension, and emotion dimension, which serves as the comparison object for approximating the output distribution of the initial acoustic topology network. Subsequently, the simulated speech distribution map and the feature distribution are subjected to the same field difference reading processing according to the distribution difference reading configuration record, which includes the timbre distribution difference reading caliber, intonation and speech rate distribution difference reading caliber, emotion distribution difference reading caliber, and difference propagation direction configuration. Through the aforementioned field difference reading processing, timbre distribution difference fields, intonation and speech rate distribution difference fields, and emotion distribution difference fields are obtained respectively. Based on these fields, a distribution difference gradient is formed. The technical essence of this distribution difference gradient is to represent the gradient-type difference result of the deviation direction and magnitude of the simulated speech distribution map relative to the feature distribution at each output field position. This distribution difference gradient is not merely an error value, but rather records the directional contribution of each output field position to the node parameter update. The technical essence of the reverse feature propagation path is the node parameter update path from the reconstructed output layer back to the input mapping layer along the multi-level implicit parsing layer. It is used to propagate the distribution difference gradient backward from the output distribution position to the node activation weights that generate the corresponding output distribution. In specific implementation, firstly, the field source relationship between the analog speech distribution map and the deep vocal node features is determined based on the reconstructed output carrying record. Then, the node connection relationship between the deep vocal node features and each level of implicit feature parsing layer is determined based on the implicit parsing input record. Finally, the field source relationship and the node connection relationship are arranged into the reverse feature propagation path. Through this reverse feature propagation path, when adjusting the node activation weights of the multi-level implicit parsing layers layer by layer, the distribution difference gradient can be passed to the node position corresponding to the difference field, giving the node parameter update a clear propagation source.

[0052] Preferably, in the specific technical implementation of adjusting the node activation weights of the multi-level implicit parsing layer layer by layer along the reverse feature propagation path of the distribution difference gradient, the output layer connection segment, implicit layer connection segment, and input mapping connection segment in the reverse feature propagation path are read first, and the distribution difference gradient is read layer by layer in the order from the reconstructed output layer to the front end of the multi-level implicit parsing layer. The technical essence of the node activation weights is that each implicit feature parsing layer in the multi-level implicit parsing layer is used to adjust the node input contribution and node output contribution, which determines the numerical contribution of the dimensionality-reduced features to the deep vocal node features during the hierarchical parsing process. In specific implementation, the node activation weights corresponding to the distribution difference gradient are first read in the implicit feature parsing layer near the reconstructed output layer, and the weights of the node activation weights are increased or decreased according to the deviation direction of the distribution difference gradient to form a node activation weight update record. The updated records of the activation weights of the rear nodes continue to serve as the basis for updating the middle implicit feature parsing layer. This allows the middle implicit feature parsing layer to read the corresponding activation weights of the middle nodes based on the updated records of the rear nodes and to perform middle node activation weight adjustment processing to form the updated records of the middle nodes. Subsequently, the updated records of the middle nodes are passed to the front implicit feature parsing layer, allowing the front implicit feature parsing layer to read the activation weights of the front nodes related to the dimensionality-reduced features and to perform front node activation weight adjustment processing to form the updated records of the front nodes. This layer-by-layer adjustment process allows the distribution difference gradient to be passed from the output distribution position to each level of the implicit feature parsing layer along the reverse feature propagation path, avoiding the situation where only the output field is modified at the end of the reconstructed output layer without changing the formation process of the deep vocal node features. The adjustment results of the node activation weights are then written into the node parameter update record, which includes the updated records of the activation weights of the rear nodes, the middle nodes, the front nodes, and the corresponding distribution difference gradient. By updating the node parameters, when importing the acoustic multidimensional features within the character-specific feature cluster in the next round, the multi-level implicit parsing layer can reconstruct the deep vocal node features based on the updated node activation weights, so that the simulated speech distribution map gradually approaches the feature distribution of the character-specific feature cluster.

[0053] Preferably, the node activation weights are continuously adjusted until the distribution difference gradient converges to a floating range used to limit the output distribution to approximate the feature distribution, thereby locking the current node activation weights. The specific implementation process for generating the basic character acoustic network is as follows: After each round of node activation weight adjustment, the acoustic multidimensional features within the character-specific feature cluster are reread based on the node parameter update record, and then re-passed through the input mapping layer, the multi-level implicit parsing layer, and the reconstructed output layer to form a new round of the analog speech distribution map. Subsequently, the new round of the analog speech distribution map and the feature distribution of the character-specific feature cluster are again subjected to the same field difference reading process to obtain a new round of the distribution difference gradient. The technical essence of continuously triggering the adjustment of node activation weights is to repeatedly perform input mapping, implicit parsing, reconstructed output, distribution difference gradient reading, and node activation weight adjustment within the same acoustic distribution range of the character-specific feature cluster, so that the output distribution of the initial acoustic topology network gradually approaches the feature distribution of the character-specific feature cluster. The floating range is configured by a convergence determination configuration record before training begins. This record includes the floating ranges for timbre distribution, pitch and rate distribution, emotion distribution, and overall distribution. The technical essence of these floating ranges is to limit the acceptable range of difference between the output distribution and the feature distribution. They originate from the natural fluctuations between the various acoustic multidimensional features within the character-specific feature cluster, rather than arbitrarily set fixed termination conditions. When the distribution difference gradient falls within the corresponding floating range in the timbre, pitch and rate, and emotion dimensions, and the overall distribution difference also falls within the overall distribution floating range, the distribution difference gradient is determined to have converged to the floating range. Subsequently, the current node activation weights are read and written into the basic network parameter carrying record. This record also includes the field configuration of the input mapping layer, the hierarchical connection relationships of the multi-level implicit resolution layer, the output spectrogram field configuration of the reconstructed output layer, and the cluster affiliation field. Based on the basic network parameter carrying record, the current node activation weights are locked to generate the basic character acoustic network. The technical essence of the basic character acoustic network is that the character acoustic network is driven by the character-specific feature cluster to complete the node parameter update. It can use the multi-dimensional acoustic structure of the character-specific feature cluster as the source of network parameters, and provide a basic network object for subsequent verification of speech segment import and representation of offset state detection.

[0054] Preferably, the criteria used to end training can be further explained through the convergence determination configuration record. Specifically, before the character-specific feature cluster enters the initial acoustic topology network, the field fluctuations between the acoustic multidimensional features within each character-specific feature cluster are read, and the timbre distribution fluctuation range, the pitch and speed distribution fluctuation range, the emotion distribution fluctuation range, and the overall distribution fluctuation range are configured based on these field fluctuations. The field fluctuations include the dispersion of timbre field segments, pitch and speed field segments, and emotion field segments within the same character-specific feature cluster. For ease of understanding, the timbre distribution fluctuation range can be configured such that the difference between the timbre distribution field in the simulated speech distribution map and the timbre field segment in the feature distribution does not exceed a certain percentage (e.g., 3%) of the average timbre field segment; the intonation and speech rate distribution fluctuation range can be configured such that the difference between the intonation and speech rate distribution fields in the simulated speech distribution map and the intonation and speech rate segment in the feature distribution does not exceed a certain percentage (e.g., 5%) of the average intonation and speech rate segment; and the emotion distribution fluctuation range can be configured such that the difference between the emotion distribution field in the simulated speech distribution map and the emotion field segment in the feature distribution does not exceed a certain percentage (e.g., 4%) of the average emotion field segment. The specific values ​​mentioned above are for illustrative purposes only; in actual configuration, adjustments should be made based on the field fluctuations within each of the acoustic multidimensional features within the character-specific feature cluster. In this way, the training termination criterion is not a fixed number of iterations, but rather whether the difference in the same field between the simulated speech distribution map and the feature distribution falls within the fluctuation range, ensuring that the formation of the basic character acoustic network corresponds to the actual distribution fluctuations of the character-specific feature cluster.

[0055] Preferably, the output distribution approximating the fluctuation range of the feature distribution means that the output distribution in the analog speech distribution map falls within the allowable difference range of the feature distribution in the timbre, intonation and speed, and emotion dimensions, rather than only a single field meeting the difference requirement. Specifically, the timbre distribution field in the analog speech distribution map and the timbre field segment in the feature distribution are first processed for same-field difference reading. Then, the intonation and speed distribution fields in the analog speech distribution map and the intonation and speed field segments in the feature distribution are processed for same-field difference reading. Finally, the emotion distribution field in the analog speech distribution map and the emotion field segment in the feature distribution are processed for same-field difference reading. If the field differences in the above three dimensions all fall within the corresponding timbre distribution fluctuation range, intonation and speed distribution fluctuation range, and emotion distribution fluctuation range, and the overall distribution difference formed after field aggregation and encapsulation of the three dimensions falls within the overall distribution fluctuation range, then it is determined that the output distribution approximates the fluctuation range of the feature distribution. For ease of understanding, when the character-specific feature cluster includes multiple acoustic multidimensional features (e.g., one hundred acoustic multidimensional features), if the differences in the timbre distribution field, the differences in the pitch and rate distribution fields, and the differences in the emotion distribution field in the new round of the simulated speech distribution map are all within the fluctuation range of the timbre distribution, the fluctuation range of the pitch and rate distribution fields, and the fluctuation range of the emotion distribution fields, and the magnitude of the distribution difference gradient changes obtained in multiple consecutive training rounds (e.g., three training rounds) are all within the fluctuation range of the overall distribution, then it can be determined that the distribution difference gradient converges to the fluctuation range used to limit the output distribution to approximate the feature distribution. This determination method can avoid premature termination of node activation weight adjustment due to accidental fluctuations in a single round, and make the node parameter update of the basic character acoustic network have a more stable termination condition.

[0056] Preferably, locking the current node activation weights means that after the distribution difference gradient converges to the floating range, the node activation weights of each implicit feature parsing layer within the multi-level implicit parsing layer in the current training round are written into the basic network parameter carrying record, and the weight increase / decrease processing of the aforementioned node activation weights is stopped. Specifically, the current node activation weights include the front node activation weights in the front implicit feature parsing layer, the middle node activation weights in the middle implicit feature parsing layer, and the rear node activation weights in the rear implicit feature parsing layer; the front node activation weights are used to adjust the numerical contribution of the dimensionality-reduced features to the local vocal node features, the middle node activation weights are used to adjust the numerical contribution of the local vocal node features to the combined vocal node features, and the rear node activation weights are used to adjust the numerical contribution of the combined vocal node features to the deep vocal node features. When locking the current node activation weights, the activation weights of the front nodes, the middle nodes, and the rear nodes are first written into the corresponding weight fields of the basic network parameter carrying record. Then, the basic network parameter carrying record is associated and encapsulated with the field configuration of the input mapping layer, the hierarchical connection relationship of the multi-level implicit resolution layer, the output spectrogram field configuration of the reconstructed output layer, and the cluster affiliation field. The resulting basic role acoustic network, when subsequently receiving the native features corresponding to the verification speech segment, can perform input mapping, implicit resolution, and reconstructed output processing according to the locked node activation weights. If the subsequent representation offset state triggers node parameter update iteration, the locked node activation weights in the basic network parameter carrying record serve as the starting point for subsequent parameter adjustments, rather than being reconfigured without a source.

[0057] Preferably, the initial acoustic topology network is not composed of a single convolutional structure, but is constructed based on one-dimensional temporal convolution, gated residual propagation, and field distribution reconstruction. The essence of the one-dimensional temporal convolution is to locally read timbre, intonation, speech rate, and emotion segments at adjacent time positions along the temporal arrangement direction of the acoustic multidimensional features within the character-specific feature cluster, allowing the continuous acoustic changes between adjacent source character speech segments to enter the initial acoustic topology network. Since the acoustic multidimensional features originate from the source character speech segments, and the character-specific feature cluster retains the segment time field and the segment source location field, the one-dimensional temporal convolution does not process according to the two-dimensional pixel neighborhood of an image, but reads adjacent acoustic fields according to the temporal sequence of speech segments. The essence of the gated residual propagation is to configure the propagation ratio for intermediate vocal features at different levels in the multi-level implicit resolution layers, allowing intermediate vocal features from the previous level to enter the next level in a controlled manner, preventing the dimensionality-reduced features from losing their original acoustic dimension source during multi-level propagation. The technical essence of the field distribution reconstruction is to re-expand the deep vocal node features in the reconstruction output layer into output fields comparable to the feature distribution of the character-specific feature cluster, thereby forming the simulated speech distribution map. Thus, the initial acoustic topology network reads adjacent acoustic changes through one-dimensional temporal convolution, preserves hierarchical vocal relationships through gated residual propagation, and forms the output distribution through field distribution reconstruction. This enables the initial acoustic topology network to update node parameters based on the temporal continuity of the character's speech, multi-dimensional acoustic correlation, and the comparability of the output distribution.

[0058] Preferably, the input mapping layer can be constructed based on the principle of field-consistent mapping and one-dimensional temporal convolution pre-compression. Specifically, the input mapping layer first reads the intra-cluster feature fields in the character-specific feature cluster, and extracts the timbre field segment, intonation and speech rate field segment, and emotion field segment from each of the acoustic multidimensional features according to the input field configuration. Since the above field segments originate from different physical meanings such as frequency, time, and amplitude, the input mapping layer first performs scale-consistent mapping processing on the timbre field segment, the intonation and speech rate field segment, and the emotion field segment, so that different field segments can enter the same numerical reading caliber. Subsequently, the input mapping layer arranges the acoustic multidimensional features according to the segment time field, and performs one-dimensional temporal convolution reading processing between the acoustic multidimensional features at adjacent time positions. The one-dimensional temporal convolution reading processing is used to read the continuous changes in timbre, intonation and speech rate, and emotion intensity between adjacent source character speech segments, rather than just reading the static fields of a single acoustic multidimensional feature. After completing the one-dimensional temporal convolution reading process, the input mapping layer performs field compression mapping on the convolution reading result to form the dimensionality-reduced features. The technical essence of these dimensionality-reduced features is to reduce field redundancy and standardize the field arrangement of the network input result while retaining the main sources of acoustic dimensionality variation in the multi-dimensional acoustic features. Through this method, the input mapping layer not only performs the function of dimensionality-compatible input organization but also the pre-extraction of temporally adjacent acoustic variations, making the dimensionality-reduced features more suitable for entering the multi-level implicit resolution layer.

[0059] Preferably, the multi-level implicit parsing layer can be constructed based on the principle of multi-layer one-dimensional temporal convolution and gated residual propagation. Specifically, the multi-level implicit parsing layer includes a front implicit feature parsing layer, a middle implicit feature parsing layer, and a rear implicit feature parsing layer. The front implicit feature parsing layer reads the dimensionality-reduced features and extracts local vocal node features at adjacent time positions through one-dimensional temporal convolution reading processing; the local vocal node features are used to represent short-term timbre changes, short-term intonation changes, and short-term pronunciation intensity changes of the same character between adjacent source character speech segments. The middle implicit feature parsing layer continues to read the local vocal node features and forms combined vocal node features through one-dimensional temporal convolution reading processing that expands the time reading range; the combined vocal node features are used to represent the correlation between intonation trends, speech rate rhythms, and pronunciation intensity changes among multiple adjacent source character speech segments. The rear implicit feature parsing layer reads the combined vocal node features and forms the deep vocal node features through gated residual propagation processing. The essence of the gated residual propagation processing technique is to determine the proportion of intermediate vocal features from the previous level to the next level based on the node activation weights of the current level. This allows the local vocal node features and the combined vocal node features to participate in the formation of the deep vocal node features in a controlled manner. The "deep" in the deep vocal node features refers to the hierarchical acoustic representation result formed after the dimensionality-reduced features have undergone continuous analysis by the front implicit feature analysis layer, the middle implicit feature analysis layer, and the rear implicit feature analysis layer. This hierarchical acoustic representation result is no longer a single timbre field, a single tone and rate field, or a single emotion field, but rather integrates the hierarchical relationships between the timbre dimension, tone and rate dimension, and emotion dimension.

[0060] Preferably, the reconstructed output layer can be constructed based on the principles of transposed temporal mapping and field distribution reconstruction. Specifically, the reconstructed output layer reads the deep vocal node features and first performs node expansion processing on the deep vocal node features to form reconstructed node expansion fields. The node expansion processing is used to re-expand the node information in the deep vocal node features, which has been compressed and fused through multiple levels of implicit parsing layers, to the output field positions corresponding to the timbre dimension, intonation and speech rate dimension, and emotion dimension. Subsequently, the reconstructed output layer performs transposed temporal mapping processing on the reconstructed node expansion fields according to the output spectrogram field configuration record, so that the reconstructed node expansion fields are restored to readable output distribution fields according to the temporal arrangement relationship of the acoustic multidimensional features in the character-specific feature cluster. The transposed temporal mapping does not restore the original speech waveform, but rather remaps the deep vocal node features according to the time field and acoustic field into timbre distribution fields, intonation and speech rate distribution fields, and emotion distribution fields. Subsequently, the reconstructed output layer performs field distribution reconstruction processing on the timbre distribution field, the intonation and rate distribution field, and the emotion distribution field to form the simulated speech distribution map. The technical essence of the simulated speech distribution map is that the output distribution carried by the initial acoustic topology network based on the character-specific feature clusters is used to perform field difference reading processing with the feature distribution of the character-specific feature clusters. Through the above transposed temporal mapping and field distribution reconstruction, the reconstructed output layer can transform the deep vocal node features into a comparable output distribution, enabling the subsequent distribution difference gradient to return to the multi-level implicit parsing layer along the reverse feature propagation path and adjust the node activation weights.

[0061] Preferably, from the perspective of the overall training principle, the initial acoustic topology network uses the input mapping layer to complete field compression and temporal adjacency reading, the multi-level implicit resolution layer to complete the hierarchical extraction of the character's vocal structure, and the reconstruction output layer to complete the reconstruction of the output distribution. The one-dimensional temporal convolution in the input mapping layer is used to read the short-term change relationships between adjacent acoustic multi-dimensional features; the multi-level one-dimensional temporal convolution in the multi-level implicit resolution layer is used to expand the temporal reading range and progressively resolve local acoustic changes into the character's vocal structure; the gated residual propagation in the multi-level implicit resolution layer is used to retain intermediate vocal features at different levels, so that the deep vocal node features include both the acoustic representation at the end of the hierarchy and the source of preceding local vocal node features and combined vocal node features; the transposed temporal mapping in the reconstruction output layer is used to restore the deep vocal node features to the simulated speech distribution map that can be aligned and compared with the feature distribution. Therefore, the initial acoustic topology network can be understood as a temporal acoustic topology network for role-based speech modeling. Its basic construction principles include one-dimensional temporal convolution, gated residual propagation, and field distribution reconstruction, rather than simply relying on traditional static voiceprint template matching, or simply relying on two-dimensional convolution to read image-based spectral textures.

[0062] Preferably, the initial acoustic topology network uses one-dimensional temporal convolution instead of two-dimensional convolution because the acoustic multidimensional features in the character-specific feature cluster have already been encapsulated into timbre, intonation, and speech rate segments, as well as emotion segments. Subsequent training needs to focus on reading the continuous changes of multiple acoustic multidimensional features over time. Two-dimensional convolution is more suitable for reading the pixel relationships of horizontal and vertical neighborhoods in a planar image, while the training input in the character-specific feature cluster is not two-dimensional image pixels, but a sequence of acoustic fields arranged according to the segment time field. Therefore, the input mapping layer and the multi-level implicit parsing layer use one-dimensional temporal convolution for reading and processing, which can read the changes between adjacent acoustic fields along the time direction defined by the segment time field, so that the timbre continuity, intonation trend, and pronunciation intensity changes of the same character in different dialogue segments can be incorporated into the deep vocal node features. For scenarios requiring enhanced long-distance segment association reading, gated residual propagation processing can be added to the multi-level implicit resolution layer. This allows local vocal node features from earlier time positions to be retained across layers to subsequent levels. However, this gated residual propagation processing still revolves around the temporal field of the acoustic multidimensional features, rather than introducing image processing structures unrelated to the character-specific feature clusters. Through this construction method, each layer of the initial acoustic topology network establishes a clear technical connection with the character-specific feature clusters, the acoustic multidimensional features, the feature distribution, and the simulated speech distribution map.

[0063] Optionally, the step of obtaining the verification speech segment and importing the verification speech segment into the basic role acoustic network to generate the expected distribution features includes: Audio segments are extracted from independent audio-visual data sources other than the initial audio-visual data as the verification speech segments, and the native features of the verification speech segments are extracted. The native features are passed as forward propagation inputs to the input endpoint of the basic character acoustic network; The original features are processed by the basic character acoustic network via the internal node connection path, and the output vector array is extracted from the end output endpoint of the basic character acoustic network; The output vector array is reshaped to generate the expected distribution features.

[0064] Preferably, the specific implementation process of extracting audio segments from independent audiovisual data sources other than the initial audiovisual data as the verification speech segments and extracting the native features of the verification speech segments is as follows: After the basic character acoustic network has completed the node parameter update by the character-specific feature cluster, the verification source configuration record is read first. The verification source configuration record is configured and formed before the verification process is executed. The verification source configuration record includes the source identifier of the independent audiovisual data source, the audio track reading aperture, the segment time reading aperture, the character tag reading aperture, and the acoustic field reading aperture. The technical essence of the independent audiovisual data source is that it is an audiovisual data source that has not participated in the generation of the character-specific feature cluster and has not participated in the update of the basic character acoustic network. It has a distinguishable configuration record from the initial audiovisual data in terms of source identifier, segment time reading aperture, and audio track reading aperture, so that the verification speech segment can be used as external acoustic input to reread the basic character acoustic network, instead of repeatedly reading the source character speech segments in the training source. In specific implementation, candidate audio tracks are first read from the independent audiovisual data source according to the audio track reading aperture, and voice activity localization processing is performed on the candidate audio tracks according to the segment time reading aperture to form a verification candidate segment record. The verification candidate segment record includes the candidate segment start time, candidate segment end time, candidate segment energy distribution, candidate segment spectral distribution, and source identifier of the independent audiovisual data source. The candidate segment start time and candidate segment end time are used to limit the truncation range of the candidate audio segments in the independent audiovisual data source. The candidate segment energy distribution and candidate segment spectral distribution are used to determine whether the candidate audio segments have readable character acoustic performance during subsequent character source matching processing. Subsequently, character source matching processing is performed on the verification candidate segment record according to the character mark reading aperture, and candidate audio segments that correspond to the preset character mark corresponding to the basic character acoustic network are selected, and the selected candidate audio segments are truncated as the verification speech segments. The technical essence of the verification speech segment is an external speech segment from the independent audiovisual data source used to verify the output distribution of the basic character acoustic network. Subsequently, following the acoustic field reading caliber used in the previous extraction of the acoustic multidimensional features, the verification speech segment is subjected to fundamental frequency reading, harmonic structure reading, pitch shift reading, syllable span reading, and pronunciation intensity reading to form the original features. The technical essence of these original features is the verification input representation formed by the field-based reading of the original acoustic characteristics carried by the verification speech segment itself. Since it is not reconstructed by the basic role acoustic network, it can serve as a reference for the subsequent expected distribution features.

[0065] Preferably, in the specific technical implementation of passing the native features as forward propagation input to the input endpoint of the basic role acoustic network, a verification input carrying record is first established, and the verification speech segment, the native features, the source identifier of the independent audiovisual data source, and the preset role tag are written into the verification input carrying record. The verification input carrying record is used to map the source of the native features, the role affiliation entry, and the subsequent forward propagation reading entry, so that the native features can still be traced back to the verification speech segment after entering the basic role acoustic network. Subsequently, the native features are processed by field arrangement according to the input endpoint configuration record. The input endpoint configuration record is saved along with the basic network parameter carrying record when the basic role acoustic network is generated. The input endpoint configuration record includes the input position of the timbre field, the input position of the pitch and speed field, the input position of the emotion field, the dimension mapping caliber of the input field, and the verification caliber for missing input fields. The technical essence of the input endpoint is that it is the field entry point for the basic role acoustic network to receive external acoustic input, and its field arrangement is consistent with the field arrangement when the input mapping layer reads the acoustic multidimensional features. In specific implementation, firstly, the frequency, time, and intensity fields in the original features are processed for dimensional compatibility according to the input field dimension mapping caliber to form dimensional compatible original features. Then, the timbre, intonation, and speech rate segments, as well as the emotion segment, in the dimensional compatible original features are checked for readable values ​​according to the input field missing verification caliber. After the check is completed, the dimensional compatible original features are encapsulated as the forward propagation input and transmitted to the input endpoint of the basic character acoustic network. The technical essence of the forward propagation input is that the original features, after field arrangement and dimensional compatibility processing, form a network input object that has the same field reading caliber as the acoustic multidimensional features input during the training phase. Through the above processing, the basic character acoustic network does not read arbitrary format verification audio data, but rather the forward propagation input that is adapted to the field structure corresponding to the character-specific feature cluster. The subsequently generated expected distribution features can form a comparison basis with the original features that share the same source and field caliber.

[0066] Preferably, the specific implementation process of the basic character acoustic network processing the forward propagation input formed by the native features via the internal node connection path is as follows: After the forward propagation input is transmitted to the input endpoint, the field configuration of the input mapping layer, the hierarchical connection relationship of the multi-level implicit resolution layer, the output spectrogram field configuration of the reconstructed output layer, and the locked node activation weights in the basic network parameter carrying record are read first, and an internal node connection path is established based on the above records. The technical essence of the internal node connection path is the node connection order from the input endpoint through the input mapping layer, the multi-level implicit resolution layer, and the reconstructed output layer to the final output endpoint when the basic character acoustic network reads the forward propagation input. The internal node connectivity path includes an input mapping node, a front implicit parsing node, a middle implicit parsing node, a rear implicit parsing node, and a reconstructed output node. The input mapping node reads the forward propagation input and performs field compression and transformation processing according to the field configuration of the input mapping layer to form a verification dimensionality reduction feature. The front implicit parsing node reads the verification dimensionality reduction feature and performs local vocalization relationship reading processing according to the node activation weights to form verification local vocalization node features. The middle implicit parsing node reads the verification local vocalization node features and performs combined vocalization relationship reading processing according to the node activation weights to form verification combined vocalization node features. The rear implicit parsing node reads the verification combined vocalization node features and performs hierarchical end acoustic representation processing according to the node activation weights to form verification deep vocalization node features. The reconstructed output node reads the verification deep vocalization node features and performs output field mapping processing according to the output spectrogram field configuration of the reconstructed output layer to form a verification output field. The internal node connectivity path ensures that the forward propagation input does not directly jump to the expected distribution features. Instead, it sequentially undergoes field compression and transformation, local vocal relationship reading, combined vocal relationship reading, hierarchical end acoustic representation processing, and output field mapping. This allows the basic role acoustic network to perform forward reading of the acoustic performance of the verification speech segment under the current node parameters. Therefore, the subsequently extracted output vector array reflects the current output distribution of the basic role acoustic network for the forward propagation input, rather than a direct copy of the original features themselves.

[0067] Preferably, in the specific implementation of extracting the output vector array from the end output endpoint of the basic role acoustic network, after the reconstructed output node has formed the verification output field, the end output endpoint configuration record is first read. The end output endpoint configuration record is stored corresponding to the output spectrogram field configuration of the reconstructed output layer. The end output endpoint configuration record includes the output row arrangement, output column arrangement, output field boundary, and output source indexing. The technical essence of the end output endpoint is that after the basic role acoustic network completes the forward propagation processing, it outputs a field with a readable acoustic distribution result. It is used to organize the verification output field into a subsequently reconfigurable array structure. In specific implementation, according to the output source indexing, the verification output field is first established with the verification speech segment, the original feature, and the preset role marker to form a verification output source record. The verification output source record then participates in the array row indexing of the output vector array, so that the output vector array can trace back to the corresponding verification speech segment. Subsequently, the verification output fields are arranged into multiple array rows according to the time segment order within the verification speech segment, following the output row arrangement method. Then, the fields in each array row are arranged into multiple array columns according to the output column arrangement method. These array columns include timbre output columns, intonation and speed output columns, and emotion output columns. The technical essence of the output vector array is that it represents the two-dimensional fielded output result of the basic character acoustic network under the current node parameters for the forward propagation input. Each array row of the output vector array represents the output position corresponding to a time segment in the verification speech segment, and each array column represents the output field corresponding to an acoustic dimension in the timbre, intonation and speed, or emotion output columns. The elements at the intersection of the array row and array column represent the distribution values ​​of the basic character acoustic network output in the corresponding time segment and corresponding acoustic dimension. The output vector array is not directly used as the final verification conclusion, but rather as an intermediate output result for subsequent reshaping processing. It retains the time order, acoustic dimension, and source index, enabling the expected distribution features to be generated while maintaining field boundaries.

[0068] Preferably, in the specific technical implementation of reshaping the output vector array to generate the expected distribution features, the array rows, array columns, and elements at the intersections of array rows and array columns in the output vector array are first read, and the expected distribution reshaping configuration record is read. The expected distribution reshaping configuration record is established based on the output spectrogram field configuration of the reconstructed output layer before the verification process is performed. The expected distribution reshaping configuration record includes time segment merging caliber, acoustic dimension realignment caliber, output field smoothing caliber, and distribution field encapsulation caliber. The technical essence of the reshaping process is to convert the output results arranged by array rows and array columns in the output vector array into a distributed acoustic representation that can be used for subsequent characterization offset state detection with the original features, rather than changing the source fields of the output vector array. In specific implementation, firstly, the temporal continuity relationship between adjacent array rows is read according to the time segment merging caliber, and array rows belonging to the same speech continuity range are sequentially concatenated to form a temporally continuous output field. Then, according to the acoustic dimension repositioning caliber, elements in the timbre output column are repositioned as the expected timbre distribution field, elements in the pitch and speed output column are repositioned as the expected pitch and speed distribution field, and elements in the emotion output column are repositioned as the expected emotion distribution field. Subsequently, according to the output field smoothing caliber, the expected timbre distribution field, the expected pitch and speed distribution field, and the expected emotion distribution field undergo field continuity correction processing to reduce abrupt field jumps caused by discrete node outputs between adjacent time segments. After completing the field continuity correction processing, according to the distribution field encapsulation caliber, the expected timbre distribution field, the expected pitch and speed distribution field, the expected emotion distribution field, the verification output source record, and the preset role marker are written into the expected distribution carrying record to form the expected distribution feature. The technical essence of the expected distribution feature is that the basic character acoustic network derives the character acoustic output distribution based on the original features. This distribution represents the timbre, intonation, speech rate, and emotional distribution that the verified speech segment should exhibit under the current node parameters of the basic character acoustic network. The expected distribution feature is then used to perform representation offset detection with the original features, enabling the basic character acoustic network to examine the deviation between its output distribution and the original acoustic performance through independent speech sources.

[0069] Preferably, the verification speech segment, the original features, the forward propagation input, the internal node connectivity path, the output vector array, and the expected distribution features have a continuous technical connection. Specifically, the verification speech segment is extracted from an independent audiovisual data source other than the initial audiovisual data, distinguishing the verification input source from the source character speech segment used when generating the character-specific feature cluster; the original features are formed by reading the verification speech segment through acoustic fields, transforming the original acoustic representation of the verification speech segment into a field-based representation that can be input into the basic character acoustic network; the forward propagation input is formed by processing the original features through field arrangement and field dimension compatibility, enabling the original features to enter the basic character according to the reading caliber of the input endpoint. An acoustic network is used; the internal node connectivity paths are established based on the basic network parameter carrying records, enabling the forward propagation input to be sequentially transmitted along the input mapping layer, the multi-level implicit resolution layer, and the reconstructed output layer; the output vector array is extracted from the terminal output endpoints, allowing the output results of the basic role acoustic network on the forward propagation input to be arrayed according to time segments and acoustic dimensions; the expected distribution features are formed by reshaping the output vector array, transforming it into a distributed acoustic representation that can be compared with the original features. This connection allows the source of the expected distribution features to be traced layer by layer to the verification speech segment, and also allows the field structure of the expected distribution features to be connected with the representational offset state of the expected distribution features and the original features of the verification speech segment in subsequent detection. Through this processing chain, the verification process of the basic character acoustic network no longer stops at a single output read, but can form a verifiable data processing process between the independent audiovisual data source, the original features, the internal node connection paths, the output vector array, and the expected distribution features, thereby providing a clear data source for subsequent adjustment of the node parameters of the basic character acoustic network according to the representation offset state.

[0070] Preferably, the training sample construction process of the basic character acoustic network is as follows: After the character-specific feature clusters have been generated, the cluster affiliation field, intra-cluster feature field, cluster boundary field, and cluster source field in the character-specific feature clusters are read first, and each of the acoustic multidimensional features in the intra-cluster feature field is used as the source of training samples. In specific implementation, the acoustic multidimensional features are first arranged in chronological order according to the segment time field recorded in the cluster source field, so that the acoustic multidimensional features corresponding to consecutive source character speech segments from the same character can maintain the sequential relationship in the initial audiovisual data; then, window truncation processing is performed on the arranged acoustic multidimensional features according to the training sample window configuration record to form multiple character acoustic training samples. The training sample window configuration record is configured and formed before the start of training, and the training sample window configuration record includes the window length reading caliber, window sliding interval reading caliber, field integrity verification caliber, sample affiliation writing caliber, and sample boundary writing caliber. The window length reading caliber is used to limit the number of acoustic multidimensional features read continuously within a character's acoustic training sample. The window sliding interval reading caliber is used to limit the degree of temporal overlap between adjacent character acoustic training samples. The field integrity verification caliber is used to check whether the timbre field segment, intonation and speech rate field segment, and emotion field segment in the acoustic multidimensional features all have readable values. The sample affiliation writing caliber is used to write the preset character tag in the cluster affiliation field into each character acoustic training sample. The sample boundary writing caliber is used to write the cluster boundary carrying record in the cluster boundary field into each character acoustic training sample. For example, the window length reading caliber can be configured to be multiple consecutive time segments, and the window sliding interval reading caliber can be configured to be a time segment shorter than the window length, so that some continuous speech relationship is preserved between adjacent character acoustic training samples. In a specific implementation, eight consecutive time segments can be used as one character acoustic training sample, and four time segments can be used as the window sliding interval, so that the character acoustic training sample can retain a longer speech change process without completely repeating the same segment of acoustic multidimensional features. After completing the window capture process, each character's acoustic training sample is encapsulated into a training sample carrying record. The training sample carrying record includes a sample input field, a sample attribution field, a sample boundary field, and a sample source field. The sample input field is written with the acoustic multidimensional features within the window, the sample attribution field is written with the preset character tag, the sample boundary field is written with the convergence boundary carrying record, and the sample source field is written with the corresponding segment time field and segment source location field.The technical essence of the character acoustic training sample is that it is a training input object formed by encapsulating the acoustic multidimensional features within the character-specific feature cluster according to a time window. It carries the field content of the same character in the dimensions of timbre, intonation and speech rate, and emotion, as well as the preset character tag and the clustering boundary carrying record, so that when training the initial acoustic topology network, it can simultaneously read the source of character affiliation, the source of clustering range, and the source of multidimensional acoustic fields.

[0071] Preferably, the structure of the basic character acoustic network is constructed based on the principles of one-dimensional temporal convolution and gated node parsing, and uses the input mapping layer, the multi-level implicit parsing layer, and the reconstructed output layer as a sequentially connected training structure. The technical essence of the one-dimensional temporal convolution principle is to read the field change relationships between adjacent acoustic multi-dimensional features in the temporal segment direction, enabling the initial acoustic topology network to recognize the timbre continuity, intonation shifts, speech rate changes, and vocal intensity fluctuations of the same character's speech in continuous time segments, rather than treating each acoustic multi-dimensional feature as an independent, unrelated point. The technical essence of the gated node parsing principle is to configure adjustable node activation weights for each level of implicit feature parsing, so that different acoustic dimensions have updatable numerical contributions when generating the deep vocal node features. In specific implementation, the input mapping layer includes a field alignment unit, a scale mapping unit, and a compression projection unit. The field alignment unit reads the sample input fields from the character acoustic training samples according to the timbre field segment, the intonation and speed field segment, and the emotion field segment. The scale mapping unit performs dimensionality-compatible mapping processing on the frequency field, time field, and intensity field. The compression projection unit compresses the sample input fields after the dimensionality-compatible mapping processing into dimensionality-reduced features. For example, the input mapping layer may include three field projection channels, corresponding to the timbre field segment, the intonation and speed field segment, and the emotion field segment, respectively. Each field projection channel can be configured with multiple compression nodes, forming the dimensionality-reduced features of uniform length after the compression projection unit completes the compression projection. The multi-level implicit resolution layer includes a front-end temporal convolutional resolution layer, a middle-end gated fusion resolution layer, and a rear-end acoustic structure resolution layer. The front-end temporal convolutional resolution layer reads the dimensionality-reduced features along the time segment direction and uses a one-dimensional convolutional kernel to read the local acoustic changes between adjacent time segments to form local vocal node features. The middle-end gated fusion resolution layer reads the local vocal node features and performs gated fusion processing on the common changes between the timbre dimension, intonation and speech rate dimension, and emotion dimension according to the node activation weights to form combined vocal node features. The rear-end acoustic structure resolution layer reads the combined vocal node features and performs hierarchical end acoustic representation processing on the stable structure of character vocalization across time segments to form the deep vocal node features. For example, the multi-level implicit parsing layer can be configured with 6 levels of implicit feature parsing layers, wherein the first 2 levels use one-dimensional temporal convolution to read local temporal changes, the middle 2 levels use gated nodes to read the combination relationship between acoustic dimensions, and the last 2 levels use hierarchical end acoustic representation processing to generate the deep speech node features; the one-dimensional convolution kernel can cover multiple adjacent time segments, such as covering 3 or 5 consecutive time segments, in order to read local continuous changes in speech segments.The reconstructed output layer includes a node unrolling unit, a distribution mapping unit, and a graph encapsulation unit. The node unrolling unit unrolls the deep vocal node features to timbre output positions, intonation and speech rate output positions, and emotion output positions. The distribution mapping unit converts the fields expanded to the timbre output positions, intonation and speech rate output positions, and emotion output positions into analog speech distribution fields. The graph encapsulation unit encapsulates the analog speech distribution fields into an analog speech distribution graph. Therefore, the basic character acoustic network does not simply rely on convolution to read local acoustic features, nor does it rely solely on fully connected numerical mapping structures for numerical mapping. Instead, it first uses one-dimensional temporal convolution to read the temporal continuity, then uses gated nodes to parse and read the combination relationships between multi-dimensional acoustic fields, and finally, the reconstructed output layer generates an analog speech distribution graph that can be compared with the feature distribution of the character-specific feature cluster.

[0072] Preferably, when training the basic character acoustic network, a loss function is constructed based on the difference between the simulated speech distribution map and the feature distribution of the character-specific feature cluster, and the training is terminated based on the change in the loss value calculated by the loss function. Specifically, in each training round, the character acoustic training samples are input into the input mapping layer to form the dimensionality-reduced features; the dimensionality-reduced features are passed to the multi-level implicit parsing layer to form the deep vocal node features; and the reconstruction output layer generates the simulated speech distribution map based on the deep vocal node features. Subsequently, the feature distribution corresponding to the character acoustic training samples is read, and the simulated speech distribution map and the feature distribution are processed according to the timbre field segment, intonation and speech rate field segment, and emotion field segment to form a distribution reconstruction loss function, and the distribution reconstruction loss value is calculated based on the distribution reconstruction loss function. The technical essence of the distribution reconstruction loss function is to measure whether the simulated speech distribution map is close to the feature distribution of the character-specific feature cluster during training. The distribution reconstruction loss value reads the differences in timbre distribution, intonation and speech rate distribution, and emotion distribution, so that the training process does not converge only around a single voiceprint template, but converges around a multi-dimensional acoustic structure. In order to ensure that the basic character acoustic network retains the character affiliation relationship corresponding to the preset character label after training, the consistency relationship between the sample affiliation field and the cluster affiliation field is also read to form a character affiliation consistency loss function, and the character affiliation consistency loss value is calculated based on the character affiliation consistency loss function. The character affiliation consistency loss function is used to constrain the character acoustic training samples within the same character-specific feature cluster to not deviate from the character affiliation entry corresponding to the preset character label during training. Further, the clustering boundary carrying record in the sample boundary field is read, and the output distribution corresponding to the simulated speech distribution map is compared to see if it is still within the clustering range defined by the clustering boundary carrying record, to form a boundary preservation loss function, and the boundary preservation loss value is calculated based on the boundary preservation loss function; the boundary preservation loss function is used to reduce the situation where the output distribution deviates from the original clustering range of the role-specific feature cluster during training. The above multiple loss functions are weighted according to the training loss configuration record to form a comprehensive training loss function, and the comprehensive training loss value is calculated based on the comprehensive training loss function. The training loss configuration record is configured before training begins, and the training loss configuration record includes distribution reconstruction loss weight, role affiliation consistency loss weight, boundary preservation loss weight, and training end determination criteria.For example, the distribution reconstruction loss weight can be set to a higher weight, and the role affiliation consistency loss weight and the boundary preservation loss weight can be set as auxiliary weights. In a specific implementation, a configuration of 60% for the distribution reconstruction loss weight, 20% for the role affiliation consistency loss weight, and 20% for the boundary preservation loss weight can be adopted. The training end determination criteria include loss decrease magnitude determination, distribution difference gradient determination, and continuous round stability determination. The loss decrease magnitude determination is used to read the decrease magnitude of the comprehensive training loss value between adjacent training rounds. The distribution difference gradient determination is used to read whether the distribution difference gradient enters the floating range. The continuous round stability determination is used to read whether the decrease magnitude of the comprehensive training loss value and the distribution difference gradient remain within the corresponding determination range in multiple consecutive training rounds. When the decrease magnitude of the comprehensive training loss value in multiple consecutive training rounds is lower than the training floating threshold, and the distribution difference gradient converges to the floating range in the timbre dimension, tone and speed dimension, and emotion dimension, the adjustment of the node activation weight is stopped. For example, the decrease in the comprehensive training loss value can be detected over five consecutive training epochs, and the training fluctuation threshold can be configured to a small numerical range. This value is only used to illustrate the training termination criteria and does not limit the protection scope of the basic character acoustic network. Locking the current node activation weights means locking the node activation weights that have already been updated in the front temporal convolutional parsing layer, the middle gate-controlled fusion parsing layer, and the rear acoustic structure parsing layer of the multi-level implicit parsing layers, and writing the field configuration of the input mapping layer, the hierarchical connection relationship of the multi-level implicit parsing layers, the output spectrogram field configuration of the reconstructed output layer, and the current node activation weights into the basic network parameter carrying record. The basic character acoustic network formed based on the basic network parameter carrying record can generate the expected distribution features along the same input mapping, implicit parsing, and reconstructed output path when subsequently reading the original features of the verified speech segment.

[0073] Optionally, detecting the representational shift state between the expected distribution features and the original features of the verified speech segment includes: The reconstructed spectral envelope curve, which contains the expected distribution features, is extracted from the multidimensional spectral map used to carry the spectral envelope data. Obtain the true spectral envelope curve corresponding to the original feature; The frequency deviation and amplitude attenuation of the peak points of the reconstructed spectral envelope curve relative to the corresponding peak points of the real spectral envelope curve are compared. Based on the frequency deviation and the amplitude attenuation, an offset feature matrix is ​​constructed, and the offset state is determined based on the offset feature matrix.

[0074] Preferably, the specific implementation process of extracting the reconstructed spectral envelope curve of the expected distribution features from the multidimensional spectrogram used to carry the spectral envelope data is as follows: After the expected distribution features have been formed by the reshaping process of the output vector array, the expected timbre distribution field, expected pitch and speed distribution field, expected emotion distribution field, time continuous output field, verification output source record, and the preset role label in the expected distribution features are read first, and the multidimensional spectrogram is established according to the spectral envelope spectrum configuration record. The time continuous output field comes from the time continuous output result formed when the output vector array is reshaped, and the time continuous output field is used to provide the arrangement basis of each time segment in the verification speech segment for the multidimensional spectrogram. The spectral envelope spectrum configuration record is configured and formed before performing the representation offset state detection. The spectral envelope spectrum configuration record includes the time segment arrangement aperture, the frequency sampling position arrangement aperture, the envelope amplitude writing aperture, the acoustic dimension source writing aperture, and the spectrum source indexing aperture. The time segment arrangement caliber is used to arrange the continuous time segments corresponding to the continuous time output field into the time reading direction of the multidimensional spectrogram. The frequency sampling position arrangement caliber is used to arrange multiple frequency sampling positions formed during the spectral envelope reading process into the frequency reading direction of the multidimensional spectrogram. The envelope amplitude writing caliber is used to write the envelope amplitude values ​​at the corresponding time segments and corresponding frequency sampling positions into the multidimensional spectrogram. The acoustic dimension source writing caliber is used to record that the envelope amplitude value originates from the expected timbre distribution field, the expected pitch and speed distribution field, or the expected emotion distribution field. The spectrogram source indexing caliber is used to establish a correspondence between the multidimensional spectrogram and the expected distribution features, the verified speech segments, and the preset role markers. The technical essence of the multidimensional spectrogram is to convert the distributed acoustic representation in the expected distribution features into a spectrogram carrying structure with time segments, frequency sampling positions, envelope amplitude, and acoustic dimension sources arranged together, enabling the expected distribution features to extract spectral envelope data under a unified reading caliber. When constructing the multidimensional spectrogram, firstly, the continuous time output field in the expected distribution features is read according to the time segment arrangement, and the continuous time output field is written into the time reading direction of the multidimensional spectrogram. Then, the distribution values ​​related to the spectral envelope in the expected timbre distribution field are read according to the frequency sampling position arrangement, and combined with the time position relationship in the expected pitch and speed distribution fields and the intensity change relationship in the expected emotion distribution field, the envelope amplitude is written into the multidimensional spectrogram. After completing the above writing, envelope outer edge reading processing is performed on the multidimensional spectrogram to read the outer edge change of the envelope amplitude along the frequency sampling position within the same time segment, and the outer edge change of the envelope amplitude is continuously connected to form the reconstructed spectral envelope curve.The essence of the reconstructed spectral envelope curve is that it is a curve representation of the expected distribution features derived by the basic character acoustic network based on the original features at the spectral envelope level. It originates from the expected distribution features, rather than from the direct sound wave readings of the verified speech segment. Through this reconstructed spectral envelope curve, the spectral envelope shape derived by the basic character acoustic network can be compared with the actual spectral envelope curve corresponding to the original features at the same frequency sampling position and the same time segment aperture, thereby providing a spectral envelope source from the network output side for the detection of the characterization of the offset state.

[0075] Preferably, in the specific technical implementation of obtaining the true spectral envelope curve corresponding to the original feature, the verification speech segment, the original feature, the source identifier of the independent audiovisual data source, and the preset role marker in the verification input carrying record are read first. Then, a true spectral envelope reading record is established based on the spectral envelope map configuration record, which is identical to the multidimensional spectral map. The true spectral envelope reading record includes an original time segment field, an original frequency sampling position field, an original envelope amplitude field, an original acoustic dimension source field, and an original source indexing field. The original time segment field originates from the segment time reading result of the verification speech segment; the original frequency sampling position field originates from the fundamental frequency reading result and harmonic structure reading result in the original feature; the original envelope amplitude field originates from the spectral energy edge reading result and vocal intensity reading result in the original feature; the original acoustic dimension source field records the source relationship between the original envelope amplitude field and the timbre dimension, pitch and speed dimension, and emotion dimension; and the original source indexing field maintains the correspondence between the true spectral envelope reading record and the source identifier of the verification speech segment and the independent audiovisual data source. In specific implementation, the original spectral distribution field corresponding to the verification speech segment is first read from the original features, and the outer edge position of the spectral energy within each time segment is read according to the original spectral distribution field. Then, the outer edge position of the spectral energy is time-aligned with the fundamental frequency sequence, harmonic structure features, and amplitude waveform of the speech intensity in the original features to form a candidate curve for the true spectral envelope. The candidate curve for the true spectral envelope represents the spectral outer edge change of the verification speech segment in its original acoustic performance. However, the candidate curve for the true spectral envelope still contains short-term noise spikes, residual silence boundaries, and local jump fields at syllable transitions. Therefore, based on the syllable span timestamps in the original features, syllable boundary alignment and local continuity correction processing are performed on the candidate curve for the true spectral envelope to correct the local jump fields and form the true spectral envelope. The technical essence of the true spectral envelope is that it is a spectral envelope comparison curve formed after the original acoustic performance carried by the verification speech segment itself is read in a field-based manner and the outer edge of the envelope is processed. It serves as a comparison benchmark for the reconstructed spectral envelope curve. Both the real spectral envelope curve and the reconstructed spectral envelope curve are arranged in time segments and frequency sampling positions according to the spectral envelope map configuration record. Therefore, when comparing the frequency deviation and amplitude attenuation of the peak points of the reconstructed spectral envelope curve relative to the corresponding peak points of the real spectral envelope curve, it is not necessary to perform temporary conversion between different field calibers. The peak correspondence can be determined directly under the same multidimensional spectral map reading caliber.

[0076] Preferably, before comparing the peak points of the reconstructed spectral envelope curve with the corresponding peak points of the real spectral envelope curve, the peak points of the reconstructed spectral envelope curve and the corresponding peak points of the real spectral envelope curve are first determined. Specifically, according to the peak reading configuration record, the local amplitude rising segment, local amplitude peak position, and local amplitude falling segment of the reconstructed spectral envelope curve within each time segment are first read, and the local amplitude peak position between the local amplitude rising segment and the local amplitude falling segment is marked as the reconstructed peak point. Subsequently, according to the same peak reading configuration record, the local amplitude rising segment, local amplitude peak position, and local amplitude falling segment of the real spectral envelope curve within the same time segment are read, and the local amplitude peak position between the local amplitude rising segment and the local amplitude falling segment is marked as the real peak point. The peak reading configuration record is configured and formed before performing the representation offset state detection, and the peak reading configuration record includes time segment matching caliber, frequency neighborhood matching caliber, peak sequence matching caliber, and amplitude stability reading caliber. The time segment matching criterion is used to limit the reconstructed peak point and the actual peak point to originating from the same or adjacent verification time range. The frequency neighborhood matching criterion is used to limit the reconstructed peak point and the actual peak point to have a comparable proximity relationship in frequency sampling position. The peak sequence matching criterion is used to establish a peak correspondence relationship according to the arrangement order from low frequency to high frequency when multiple peak points exist in the same time segment. The amplitude stability reading criterion is used to remove invalid peak points formed by local noise spikes. Based on the peak reading configuration record, a one-to-one correspondence relationship is established between each reconstructed peak point and the actual peak point that satisfies the time segment matching criterion, the frequency neighborhood matching criterion, and the peak sequence matching criterion to form a peak correspondence relationship record. The peak correspondence relationship record includes the reconstructed peak point, the actual peak point, the frequency sampling position of the reconstructed peak point, the frequency sampling position of the actual peak point, the envelope amplitude value of the reconstructed peak point, the envelope amplitude value of the actual peak point, and the corresponding time segment field. The technical essence of the corresponding peak point is that it is the real peak point that has a comparable relationship with the reconstructed peak point within the same time segment and similar frequency reading range. By recording the peak correspondence, when calculating the frequency deviation and amplitude attenuation, a set of frequency sampling positions and envelope amplitude values ​​of the reconstructed peak point and the real peak point can be directly read, avoiding the incorrect use of peak points from different time segments or different frequency ranges for offset detection.

[0077] Preferably, the specific implementation process for comparing the frequency deviation and amplitude attenuation of the peak points of the reconstructed spectral envelope curve relative to the corresponding peak points of the real spectral envelope curve is as follows: After the peak correspondence record has been formed, the frequency sampling positions corresponding to each set of reconstructed peak points and real peak points are first read, and the difference reading processing is performed on the frequency sampling positions of the reconstructed peak points and the real peak points to form the frequency deviation. The technical essence of the frequency deviation is the offset amplitude of the reconstructed spectral envelope curve derived by the basic role acoustic network at the peak frequency position relative to the original acoustic performance carried by the verification speech segment itself, which is used to describe the deviation state of the output distribution on the frequency structure. The frequency deviation is not a single result obtained by coarsely subtracting the entire curve, but a peak-level offset result formed after reading the difference in frequency sampling positions for each set of reconstructed peak points and real peak points with corresponding relationships. Subsequently, the envelope amplitude values ​​corresponding to each set of reconstructed peak points and true peak points are read. Using the envelope amplitude value of the true peak point as a reference, the degree of decrease in the envelope amplitude value of the reconstructed peak point relative to the envelope amplitude value of the true peak point is read to form the amplitude attenuation. The technical essence of the amplitude attenuation is the degree of amplitude weakening of the reconstructed spectral envelope curve derived from the basic acoustic network at the peak intensity position relative to the true spectral envelope curve. It is used to describe the deviation of the output distribution on the acoustic intensity structure. When the envelope amplitude value of the reconstructed peak point is higher than that of the true peak point, the amplitude attenuation is recorded as an amplitude enhancement field according to the amplitude difference direction; when the envelope amplitude value of the reconstructed peak point is lower than that of the true peak point, the amplitude attenuation is recorded as an amplitude attenuation field according to the amplitude difference direction. Both the amplitude enhancement field and the amplitude attenuation field are written into the amplitude offset reading record. The amplitude offset reading record includes the amplitude attenuation amount, the amplitude enhancement field, the amplitude attenuation field, the reconstructed peak point, the true peak point, and the corresponding time segment field. This amplitude offset reading record provides a basis for reading the peak amplitude offset direction when subsequently constructing the offset feature matrix. The frequency deviation amount and the amplitude attenuation amount jointly participate in the representation of the offset state detection because the deviation of the character's voice is not only manifested as a shift in the spectral peak position but also as a change in the envelope peak intensity; reading only the frequency deviation amount would miss the intensity shift, and reading only the amplitude attenuation amount would miss the frequency structure shift. By simultaneously reading the frequency deviation amount and the amplitude attenuation amount, the subsequent offset feature matrix can express the offset relationship between the expected distribution feature and the original feature from both the frequency position and envelope amplitude levels.

[0078] Preferably, in the specific technical implementation of constructing the offset feature matrix based on the frequency deviation and the amplitude attenuation, an offset matrix configuration record is first established. The offset matrix configuration record is configured and formed before performing the offset state detection. The offset matrix configuration record includes the matrix row arrangement aperture, the matrix column arrangement aperture, the offset element writing aperture, the peak source writing aperture, and the offset direction writing aperture. The matrix row arrangement is used to arrange each set of reconstructed peak points and true peak points in the peak correspondence record into a matrix row of the offset feature matrix. The matrix column arrangement is used to arrange the frequency deviation column, amplitude attenuation column, time segment column, acoustic dimension source column, and peak sequence column into multiple matrix columns of the offset feature matrix. The offset element writing entry is used to write the frequency deviation and amplitude attenuation corresponding to each set of reconstructed peak points and true peak points into the corresponding matrix column of the corresponding matrix row. The peak source writing entry is used to write the time segment field, frequency sampling position, and peak sequence corresponding to the matrix row into the corresponding matrix column. The offset direction writing entry is used to record the offset direction of the frequency deviation and the amplitude offset direction of the amplitude attenuation. Subsequently, the frequency deviation, amplitude attenuation, time segment field, acoustic dimension source field, peak correspondence record, and amplitude offset reading record are read according to the offset matrix configuration, and the above fields are written into the offset feature matrix row by row. The technical essence of the offset feature matrix is ​​a matrix-based carrying result that structurally arranges the peak-level offset results between the reconstructed spectral envelope curve and the real spectral envelope curve. Each matrix row of the offset feature matrix corresponds to a set of reconstructed peak points and real peak points, and each matrix column of the offset feature matrix corresponds to an offset reading field or a source reading field. The matrix element value at the intersection of the matrix row and matrix column in the offset feature matrix represents the specific value of the peak correspondence in the offset reading field or source reading field. The matrix element values ​​in the frequency deviation column represent the frequency position offset of the corresponding set of reconstructed peak points and the true peak points; the matrix element values ​​in the amplitude attenuation column represent the envelope amplitude offset of the corresponding set of reconstructed peak points and the true peak points; the matrix element values ​​in the time segment column represent the time position of the corresponding set of reconstructed peak points and the true peak points in the verified speech segment; the matrix element values ​​in the acoustic dimension source column represent the main sources of the corresponding set of reconstructed peak points and the true peak points from the timbre dimension, intonation and speech rate dimension, or emotion dimension; and the matrix element values ​​in the peak order column represent the arrangement position of the corresponding set of reconstructed peak points and the true peak points within the same time segment.Through the offset feature matrix, the frequency deviation and amplitude attenuation are no longer stored as scattered peak offset fields, but are formed into a readable offset structure according to time segment, acoustic dimension source and peak sequence, providing a data basis for subsequent determination of the characterization offset state based on the offset feature matrix.

[0079] Preferably, the specific implementation process for determining the characterization offset state based on the offset feature matrix is ​​as follows: After the offset feature matrix has been formed, the offset state determination configuration record is read first. The offset state determination configuration record is configured and formed before performing the characterization offset state detection. The offset state determination configuration record includes frequency offset determination caliber, amplitude offset determination caliber, time continuous offset determination caliber, acoustic dimension offset determination caliber, frequency allowable range, amplitude allowable range, and offset state output caliber. The frequency allowable range is configured based on the original frequency sampling position field corresponding to the verified speech segment, the frequency sampling position distribution of the real spectrum envelope curve, and the frequency fluctuation range of the same acoustic dimension within the character-specific feature cluster. The frequency allowable range is used to provide determination boundaries for the matrix element values ​​of the frequency deviation column. The amplitude allowable range is configured based on the original envelope amplitude field corresponding to the verified speech segment, the envelope amplitude distribution of the real spectrum envelope curve, and the amplitude fluctuation range of the same acoustic dimension within the character-specific feature cluster. The amplitude allowable range is used to provide determination boundaries for the matrix element values ​​of the amplitude attenuation column. The frequency offset determination caliber is used to read the matrix element values ​​of the frequency deviation column in the offset feature matrix and determine whether the frequency position offset exceeds the allowed frequency range; the amplitude offset determination caliber is used to read the matrix element values ​​of the amplitude attenuation column in the offset feature matrix and determine whether the envelope amplitude offset exceeds the allowed amplitude range; the time continuity offset determination caliber is used to read the time segment column in adjacent matrix rows and determine whether the offset is concentrated in continuous time segments; the acoustic dimension offset determination caliber is used to read the acoustic dimension source column and determine whether the offset is mainly concentrated in the timbre dimension, intonation and speech rate dimension, or emotion dimension; the offset state output caliber is used to encapsulate the above determination results into the characterization offset state. In specific implementation, firstly, frequency offset aggregation reading processing is performed on the frequency offset column in the offset feature matrix to form a frequency offset status field; then, amplitude offset aggregation reading processing is performed on the amplitude attenuation column in the offset feature matrix to form an amplitude offset status field; subsequently, time continuity reading processing is performed on the frequency offset status field and the amplitude offset status field in combination with the time segment column to form a time continuous offset status field; further, acoustic dimension attribution reading processing is performed on the frequency offset status field, the amplitude offset status field, and the time continuous offset status field in combination with the acoustic dimension source column to form an acoustic dimension offset status field. The technical essence of the offset status representation is the structured state result of the deviation of the expected distribution feature from the original feature at the spectral envelope level. It is not a single error value, but a comprehensive judgment result including the frequency offset status field, the amplitude offset status field, the time continuous offset status field, and the acoustic dimension offset status field.The representational offset state is directly related to the adjustment of node parameters in the basic character acoustic network: when the representational offset state shows that the offset is concentrated in the frequency offset state field, the target spectral band related to the frequency structure can be located subsequently; when the representational offset state shows that the offset is concentrated in the amplitude offset state field, the target spectral band related to the envelope strength can be located subsequently; when the representational offset state shows that the offset has temporal continuity, the source of the output distribution within the corresponding time segment can be read subsequently. Thus, the representational offset state converts the offset relationship between the reconstructed spectral envelope curve and the real spectral envelope curve into a readable state basis required for subsequent adjustment of the node parameters in the basic character acoustic network, enabling the generation of the final character acoustic network to inherit the original acoustic performance of the verified speech segment.

[0080] Optionally, adjusting the node parameters of the basic character acoustic network according to the representation offset state to generate the final character acoustic network includes: Analyze the matrix element values ​​in the offset feature matrix corresponding to the offset state; If it is determined that the value of a matrix element exceeds the extreme value of the tolerance limit used to determine whether the characterization of the offset state needs to trigger the node parameter update iteration, then the target frequency band pointed to by the value of the matrix element is extracted. Locate the dedicated node sub-cluster in the basic role acoustic network that processes the target frequency band, reset the error update step size of the dedicated node sub-cluster in the reverse feature propagation process to a preset amplification step size, and trigger a new round of node parameter update iteration based on the preset amplification step size; If it is determined that all the values ​​of the matrix elements in the offset feature matrix are within the extreme values ​​of the tolerance limit, then all node parameters of the basic character acoustic network are frozen, and the basic character acoustic network currently in a frozen state is output as the final character acoustic network.

[0081] Preferably, the specific implementation process of parsing the matrix element values ​​in the offset feature matrix corresponding to the offset state is as follows: After the offset state has been determined based on the offset feature matrix, the frequency offset state field, amplitude offset state field, time continuity offset state field, and acoustic dimension offset state field in the offset state are read first, and the matrix row arrangement, matrix column arrangement, and matrix element values ​​in the offset feature matrix are read simultaneously. Each matrix row of the offset feature matrix corresponds to a set of reconstructed peak points and the true peak points, and each matrix column of the offset feature matrix corresponds to a frequency deviation column, amplitude attenuation column, time segment column, acoustic dimension source column, or peak sequence column. The matrix element values ​​represent the reading results of the set of reconstructed peak points and the true peak points in the corresponding matrix column. In specific implementation, firstly, the time segment field to which the matrix element value belongs is read according to the time segment column. Then, the acoustic dimension source field to which the matrix element value belongs is read according to the acoustic dimension source column. The acoustic dimension source field is used to point to the timbre dimension, intonation and speech rate dimension, or emotion dimension. Subsequently, the frequency deviation and amplitude attenuation in the matrix row are read according to the frequency deviation column and the amplitude attenuation column, respectively, to form an offset element parsing record. The offset element parsing record includes the matrix element value, the matrix row to which the matrix element value belongs, the matrix column to which the matrix element value belongs, the reconstructed peak point corresponding to the matrix element value, the true peak point corresponding to the matrix element value, the time segment field corresponding to the matrix element value, and the acoustic dimension source field corresponding to the matrix element value. The technical essence of parsing the matrix element value is to restore the offset judgment result already formed in the offset state to the field position of the executable node parameter adjustment, so that subsequently, it is not only known that there is an offset between the expected distribution feature and the original feature, but also that the offset occurs in which acoustic dimension, which time segment, and which peak sequence. The offset element parsing record then serves as the basis for determining whether the matrix element value exceeds the tolerance limit extreme value, and also as the data source for extracting the target spectrum band and locating the dedicated node subcluster. Through the above parsing process, a sequential relationship is formed between the characterization offset state, the offset feature matrix, the matrix element value, the offset element parsing record, the target spectrum band, and the node parameter update iteration, avoiding the characterization offset state being treated as an isolated verification result and thus unable to enter the node parameter adjustment process of the basic role acoustic network.

[0082] Preferably, if it is determined that the value of a matrix element exceeds the tolerance limit extreme value used to determine whether the representation offset state needs to trigger node parameter update iteration, then in the specific technical implementation of extracting the target spectrum band pointed to by the matrix element value, the offset adjustment determination configuration record is first read. The offset adjustment determination configuration record is configured and formed before the representation offset state detection or before the node parameter adjustment. The offset adjustment determination configuration record includes the frequency tolerance limit extreme value, the amplitude tolerance limit extreme value, the time continuity tolerance limit extreme value, the acoustic dimension tolerance limit extreme value, and the frequency band extraction aperture. The technical essence of the tolerance limit extreme value is to determine whether the representation offset state has exceeded the acceptable output offset range of the basic character acoustic network. The tolerance limit extreme value is not an arbitrary fixed value, but is configured based on the cluster feature fields in the character-specific feature cluster, the cluster boundary carrying record, the natural fluctuation range of the real spectrum envelope curve, and the floating range formed during the training phase of the basic character acoustic network. Specifically, the frequency tolerance limit extreme value is configured based on the frequency fluctuation range of the same acoustic dimension within the character-specific feature cluster; the amplitude tolerance limit extreme value is configured based on the envelope amplitude fluctuation range of the same acoustic dimension within the character-specific feature cluster; the time continuity tolerance limit extreme value is configured based on the allowable offset duration range within a continuous time segment; and the acoustic dimension tolerance limit extreme value is configured based on the field contribution differences of the timbre dimension, intonation and speech rate dimension, and emotion dimension. Subsequently, the matrix element values ​​in the offset element parsing record are compared with the corresponding tolerance limit extreme values. When the matrix element value in the frequency deviation column exceeds the frequency tolerance limit extreme value, or the matrix element value in the amplitude attenuation column exceeds the amplitude tolerance limit extreme value, or the time continuity offset state field exceeds the time continuity tolerance limit extreme value, or the acoustic dimension offset state field exceeds the acoustic dimension tolerance limit extreme value, it is determined that a matrix element value exceeds the tolerance limit extreme value. The technical meaning of the matrix element values ​​exceeding the tolerance limit extreme value is that the offset between the expected distribution features currently output by the basic role acoustic network and the original features of the verified speech segment can no longer be retained as acceptable natural fluctuations, and the node parameter update iteration needs to be triggered. After completing the above determination, according to the frequency band extraction aperture, the reconstructed peak point, the true peak point, the frequency sampling position, the time segment field, and the acoustic dimension source field corresponding to the matrix element values ​​exceeding the tolerance limit extreme value are read, and the frequency reading range is extracted around the corresponding frequency sampling position to form the target spectrum band.The technical essence of the target spectrum band is the local range of the spectrum envelope pointed to by the matrix element values ​​that exceed the extreme value of the tolerance limit, which requires node parameter update and iteration. It is used to convert the offset detection result into the frequency band entry for the subsequent positioning of the dedicated node sub-cluster.

[0083] Preferably, when extracting the target frequency band pointed to by the matrix element value, to avoid the target frequency band containing only a single frequency sampling position and failing to reflect the spectral envelope changes around the peak, the frequency band extension configuration record is further read. The frequency band extension configuration record includes the center frequency reading aperture, the adjacent frequency sampling position reading aperture, the time segment coverage aperture, the acoustic dimension source preservation aperture, and the target frequency band encapsulation aperture. The center frequency reading aperture is used to determine the frequency sampling position corresponding to the matrix element value that exceeds the tolerance limit extreme value as the center reading position of the target frequency band. The adjacent frequency sampling position reading aperture is used to read the frequency sampling positions on both sides of the center reading position that are continuously related to the reconstructed spectral envelope curve and the real spectral envelope curve. The time segment coverage aperture is used to include the time segment that has shifted and the adjacent time segments that are continuously associated with it into the time reading range of the target frequency band. The acoustic dimension source preservation aperture is used to retain the timbre dimension, pitch and speech rate dimension or emotion dimension to which the matrix element value belongs. The target frequency band encapsulation aperture is used to write the above fields into the target frequency band record. In specific implementation, firstly, the frequency sampling positions corresponding to the matrix element values ​​exceeding the tolerance limit are written into the center frequency field of the target spectrum band record. Then, based on the adjacent frequency sampling position reading aperture, the frequency sampling positions adjacent to the center frequency field and whose envelope amplitudes change continuously are read and written into the frequency range field of the target spectrum band record. Subsequently, based on the time segment coverage aperture, the time segment field and its continuously associated time segments are read and written into the time range field of the target spectrum band record. Finally, based on the acoustic dimension source retention aperture, the acoustic dimension source field is read and written into the acoustic dimension source field of the target spectrum band record. The target spectrum band record also includes the matrix element values, the frequency deviation, the amplitude attenuation, the peak correspondence record, and the offset element resolution record, so that the target spectrum band not only represents a frequency range but also retains the offset source, time source, and acoustic dimension source. The target spectrum band record is then used to establish a band-to-node mapping record, and the dedicated node sub-cluster in the basic role acoustic network that processes the target spectrum band is located through the band-to-node mapping record, so that the node parameter update iteration can be carried out around the local range of the offset spectrum envelope, rather than performing indiscriminate updates on all node parameters of the basic role acoustic network.

[0084] Preferably, the specific implementation process for locating the dedicated node subcluster for processing the target spectrum band in the basic role acoustic network is as follows: After the target spectrum band record is formed, the field configuration of the input mapping layer, the hierarchical connection relationship of the multi-level implicit resolution layer, the output spectrum field configuration of the reconstructed output layer, and the currently locked or iterated node activation weights are read from the basic network parameter carrying record. A frequency band to node mapping record is then established based on the frequency range field, time range field, and acoustic dimension source field in the target spectrum band record. The frequency band to node mapping record includes the target spectrum band record, the output field position of the reconstructed output layer, the node connection relationship of the multi-level implicit resolution layer, and the input field source relationship of the input mapping layer. The frequency band to node mapping record is used to establish a readable correspondence between the target spectrum band and the updatable nodes in the basic role acoustic network. The basic character acoustic network is constructed during the training phase based on the principles of one-dimensional temporal convolution and gated node resolution. The input mapping layer includes field alignment units, scale mapping units, and compression projection units. The multi-level implicit resolution layer includes a front temporal convolution resolution layer, a middle gated fusion resolution layer, and a rear acoustic structure resolution layer. The reconstructed output layer includes node unrolling units, distribution mapping units, and spectrogram encapsulation units. The technical essence of the dedicated node sub-clusters is that they are combinations of nodes in the basic character acoustic network that have direct or indirect numerical contributions to the output distribution of the target spectral band. They are not arbitrarily selected sets of nodes, but are formed by reverse positioning of the target spectral band along the connectivity relationships of the reconstructed output layer, the multi-level implicit resolution layer, and the input mapping layer. In specific implementation, firstly, based on the frequency band to node mapping record and the output spectrogram field configuration of the reconstructed output layer, the output field position corresponding to the target frequency band in the analog speech distribution spectrum or the expected distribution feature is read, and the output field position is mapped to the target output node in the spectrogram encapsulation unit and the distribution mapping unit; then, from the target output node, backtracking along the node unrolling unit to the rear acoustic structure parsing layer, the rear acoustic structure nodes that have node connection relationships with the target output node are read; then, from the rear acoustic structure nodes, backtracking to the middle gate control fusion parsing layer, the middle gate control fusion nodes that have gate contribution relationships to the acoustic dimension source field are read; further, from the middle gate control fusion nodes, backtracking to the front temporal convolution parsing layer, the front temporal convolution parsing nodes covering the time range field and the frequency range field are read; finally, according to the field configuration of the input mapping layer, the input mapping nodes that provide the dimensionality-reduced features to the front temporal convolution parsing nodes are read.The aforementioned target output node, rear acoustic structure node, middle gate control fusion node, front temporal convolution parsing node, and input mapping node are collectively encapsulated into the dedicated node sub-cluster, and the dedicated node sub-cluster is written into a dedicated node sub-cluster record. This dedicated node sub-cluster record then serves as a limiting range for the error update step size reset and the node parameter update iteration, ensuring that subsequent updates focus on the node connection paths processing the target frequency band.

[0085] Preferably, in the specific technical implementation of resetting the error update step size of the dedicated node subcluster during the reverse feature propagation process to a preset amplification step size, and triggering a new round of node parameter update iterations based on the preset amplification step size, the step size adjustment configuration record is first read. This step size adjustment configuration record is configured before the start of the basic character acoustic network training or before the start of the verification process. The step size adjustment configuration record includes the basic error update step size, the preset amplification step size, the amplification applicable field range, the maximum step size limit, and the iteration stop readback limit. The technical essence of the error update step size is an adjustment parameter used in the reverse feature propagation path to control the single-round update amplitude of the node activation weights. It determines the adjustment amplitude when the distribution difference gradient or the offset corresponding to the representation offset state is transmitted to the node activation weights. The essence of the preset amplification step size is to temporarily configure a larger update step size for the dedicated node sub-cluster after the matrix element value exceeds the tolerance limit extreme value. The preset amplification step size is formed based on the tolerance limit extreme value, the extent to which the matrix element value exceeds the tolerance limit extreme value, the acoustic dimension source field of the target spectrum band, and the floating range configuration during the training phase of the basic role acoustic network, and its update range is limited by the maximum step size limit caliber. In specific implementation, the dedicated node sub-cluster record is first read, and the target output node, the rear acoustic structure node, the middle gate control fusion node, the front temporal convolution parsing node, and the input mapping node in the dedicated node sub-cluster record are written into the step size reset bearer record. The step size reset bearer record is used to record the applicable node range, applicable acoustic dimension source field, and applicable target spectrum band of the preset amplification step size, to prevent the preset amplification step size from being written into other nodes that have not processed the target spectrum band. Subsequently, the basic error update step size is replaced with the preset amplification step size, and the preset amplification step size is bound to the node activation weight corresponding to each node in the dedicated node sub-cluster. Then, based on the target spectrum band record, the native features and expected distribution features corresponding to the target spectrum band are reread, and the frequency offset state field, amplitude offset state field, time continuity offset state field, and acoustic dimension offset state field in the characterization offset state are used as reverse update sources to trigger a new round of node parameter update iterations.The execution order of the node parameter update iteration is as follows: First, the offset direction is read from the target output node of the reconstructed output layer, and then the offset direction is passed to the rear acoustic structure node to adjust the node activation weight corresponding to the rear acoustic structure node; then, the update result of the rear acoustic structure node is passed to the middle gate control fusion node to adjust the node activation weight corresponding to the middle gate control fusion node; then, the update result of the middle gate control fusion node is passed to the front temporal convolution parsing node to adjust the node activation weight corresponding to the front temporal convolution parsing node; finally, the field contribution relationship of the input mapping node is read back according to the update result of the front temporal convolution parsing node. After completing one round of the node parameter update iteration, the expected distribution feature is regenerated, and the representation offset state is re-detected so that the update effect of the preset amplification step size can be read again through the offset feature matrix.

[0086] Preferably, if it is determined that all the matrix element values ​​in the offset feature matrix are within the tolerance limit extreme value, then all node parameters of the basic character acoustic network are frozen, and the basic character acoustic network currently in a frozen state is output as the final character acoustic network. The specific implementation process is as follows: After completing the parsing of the offset feature matrix and the comparison of the tolerance limit extreme value, if all the matrix element values ​​in the frequency deviation column are within the frequency tolerance limit extreme value, all the matrix element values ​​in the amplitude attenuation column are within the amplitude tolerance limit extreme value, the time continuous offset state field is within the time continuous tolerance limit extreme value, and the acoustic dimension offset state field is within the acoustic dimension tolerance limit extreme value, then it is determined that all the matrix element values ​​in the offset feature matrix are within the tolerance limit extreme value. The technical meaning of this determination is that the offset of the expected distribution feature relative to the original feature of the verified speech segment is still within the output fluctuation range allowed to be retained by the basic character acoustic network, and it is not necessary to continue to trigger the node parameter update iteration. Subsequently, the parameter freeze configuration record is read. This record includes the freeze parameters for the input mapping layer, the multi-level implicit resolution layer, the reconstructed output layer, the node activation weights, the output graph field configuration, and the freeze state writing entry. The technical essence of freezing all node parameters of the basic character acoustic network is to mark the updatable parameters that have already been trained and validated in the basic character acoustic network as non-updatable, so that the node activation weights and related field configurations will not be changed when the basic character acoustic network is subsequently read. In specific implementation, firstly, the field configuration parameters in the field alignment unit, the scale mapping unit, and the compression projection unit are frozen according to the input mapping layer freezing caliber. Then, the node activation weights, temporal convolution readout parameters, gated fusion parameters, and hierarchical end acoustic representation parameters in the front temporal convolution parsing layer, the middle gated fusion parsing layer, and the rear acoustic structure parsing layer are frozen according to the multi-level implicit parsing layer freezing caliber. Subsequently, the output field configuration parameters in the node unrolling unit, the distribution mapping unit, and the graph encapsulation unit are frozen according to the reconstruction output layer freezing caliber. After completing the above freezing process, all node parameters of the basic character acoustic network, the preset character label, the basic network parameter carrying record, the offset feature matrix, the representation offset state, and the frozen state are written into the final network carrying record. The final network carrying record is used to store the network parameter source, verification result source, and frozen state source of the final character acoustic network. Then, the basic character acoustic network in the frozen state corresponding to the final network carrying record is output as the final character acoustic network.The technical essence of the final character acoustic network is that it is a character acoustic network after the original features of the verified speech segment have been verified and the necessary node parameters have been adjusted. It can take on the training source of the character-specific feature cluster and the verification source of the representation offset state, and is used for subsequent multi-character speech differentiation and modeling processing.

[0087] like Figure 3 As shown, this is an embodiment of a character voice differentiation and modeling device based on multi-dimensional features, which includes: The feature association sequence generation module is used to obtain source character voice segments in the initial audiovisual data, extract the acoustic multidimensional features of the source character voice segments, and bind the acoustic multidimensional features to a preset character tag used to identify the source character voice segment to generate a feature association sequence. The character-specific feature cluster generation module is used to analyze the structural similarity of the acoustic multidimensional features within the feature association sequence, and based on the structural similarity, to perform feature clustering processing on the acoustic multidimensional features that reach a preset approximation threshold to generate character-specific feature clusters. The basic character acoustic network generation module is used to import the acoustic multidimensional features within the character-specific feature cluster into the initial acoustic topology network to update the node parameters of the initial acoustic topology network until the output distribution of the initial acoustic topology network approximates the feature distribution of the character-specific feature cluster, thereby generating the basic character acoustic network. The final character acoustic network generation module is used to acquire a verification speech segment, import the verification speech segment into the basic character acoustic network to generate expected distribution features, detect the representation offset state between the expected distribution features and the original features of the verification speech segment, and then adjust the node parameters of the basic character acoustic network according to the representation offset state to generate the final character acoustic network.

[0088] like Figure 4 As shown, an electronic device according to an embodiment of this application includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, it implements the role voice differentiation and modeling method based on multi-dimensional features as described in any one of this application.

[0089] like Figure 5 As shown, a computer-readable storage medium is provided in an embodiment of this application. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the role voice differentiation and modeling method based on multi-dimensional features as described in any one of this application.

[0090] like Figure 6As shown, this is an embodiment of a role-voice differentiation and modeling system based on multi-dimensional features, which includes a feature association processing end, a feature aggregation processing end, an acoustic network training end, and an acoustic network verification end. The feature association processing end is used to obtain source character voice segments in the initial audiovisual data, extract the acoustic multidimensional features of the source character voice segments, and bind the acoustic multidimensional features to a preset character tag used to identify the source character voice segment to generate a feature association sequence. The feature clustering processing end is used to analyze the structural similarity of the acoustic multidimensional features in the feature association sequence, and based on the structural similarity, to perform feature clustering processing on the acoustic multidimensional features that reach a preset approximation threshold to generate a character-specific feature cluster. The acoustic network training terminal is used to import the acoustic multidimensional features within the character-specific feature cluster into the initial acoustic topology network to update the node parameters of the initial acoustic topology network until the output distribution of the initial acoustic topology network approximates the feature distribution of the character-specific feature cluster, so as to generate a basic character acoustic network. The acoustic network verification terminal is used to acquire verification speech segments, import the verification speech segments into the basic character acoustic network to generate expected distribution features, detect the representation offset state between the expected distribution features and the original features of the verification speech segments, and then adjust the node parameters of the basic character acoustic network according to the representation offset state to generate the final character acoustic network.

[0091] The above Figures 3-6 For an exemplary description, please refer to the above. Figure 1 This will not be elaborated upon here.

Claims

1. A method for character voice differentiation and modeling based on multi-dimensional features, characterized in that, include: Obtain source character voice segments from initial audiovisual data, extract acoustic multidimensional features from the source character voice segments, and bind the acoustic multidimensional features to a preset character tag used to identify the source character voice segment's affiliation, so as to generate a feature association sequence; The structural similarity of the acoustic multidimensional features within the feature association sequence is analyzed, and based on the structural similarity, the acoustic multidimensional features that reach a preset approximation threshold are subjected to feature clustering processing to generate a character-specific feature cluster. The acoustic multidimensional features within the character-specific feature cluster are imported into the initial acoustic topology network to update the node parameters of the initial acoustic topology network until the output distribution of the initial acoustic topology network approximates the feature distribution of the character-specific feature cluster, thereby generating the basic character acoustic network. A verification speech segment is obtained and imported into the basic character acoustic network to generate expected distribution features. The representation offset state between the expected distribution features and the original features of the verification speech segment is detected. Then, the node parameters of the basic character acoustic network are adjusted according to the representation offset state to generate the final character acoustic network.

2. The method for character voice differentiation and modeling based on multi-dimensional features according to claim 1, characterized in that, The extraction of acoustic multidimensional features from the source character's speech segment includes: Identify the fundamental frequency wave sequence of the pronunciation in the source character's speech segment, and extract the harmonic structure features of the fundamental frequency wave sequence to construct a timbre representation vector; The pitch curve trajectory and syllable span timestamp within the source character's speech segment are detected to generate intonation and speech rate representation vectors. The pronunciation intensity amplitude waveform of the source character's voice segment is extracted to generate an emotion representation vector; The acoustic multidimensional features are generated based on the timbre representation vector, the intonation and speech rate representation vector, and the emotion representation vector.

3. The method for character voice differentiation and modeling based on multi-dimensional features according to claim 2, characterized in that, The step of binding the acoustic multidimensional features to a preset role tag used to identify the source role's speech segment affixes, in order to generate a feature association sequence, includes: Assign a preset character tag with a unique identifier to the source character's voice clip; Construct a multidimensional data mapping table and write the preset role tag into the index field of the multidimensional data mapping table; The acoustic multidimensional features are stored in the numerical storage field of the multidimensional data mapping table; Establish a unique mapping link between the index field and the numerical storage field, and arrange the unique mapping links in chronological order to generate the feature association sequence.

4. The method for character voice differentiation and modeling based on multi-dimensional features according to claim 1, characterized in that, The process of analyzing the structural similarity of the acoustic multidimensional features within the feature association sequence includes: Extract any two of the acoustic multidimensional features from the feature association sequence; Any two of the acoustic multidimensional features are projected onto a multidimensional vector space that carries the acoustic multidimensional features to generate two corresponding spatial projection points. Measure the spatial Euclidean span of the two spatial projection points in the multidimensional vector space; The structural similarity is generated based on the spatial Euclidean span, wherein the value of the structural similarity increases proportionally as the spatial Euclidean span decreases.

5. The method for character voice differentiation and modeling based on multi-dimensional features according to claim 4, characterized in that, Based on the structural similarity, the acoustic multidimensional features that reach a preset approximation threshold are subjected to feature clustering processing to generate a character-specific feature cluster, including: When the structural similarity exceeds the preset approximation threshold used to determine whether the acoustic multidimensional features have undergone feature clustering processing, it is confirmed that any two acoustic multidimensional features have a consistent attribution attribute. Establish a convergence boundary for the acoustic multidimensional features that possess the aforementioned belonging consistency attribute, and encapsulate all the acoustic multidimensional features located inside the convergence boundary into a data array; Read the preset role marker pointed to by the acoustic multidimensional features in the data array within the feature association sequence; Assign the preset role tag to the data array, and generate the role-specific feature cluster based on the data array with the preset role tag.

6. The method for character voice differentiation and modeling based on multi-dimensional features according to claim 1, characterized in that, The initial acoustic topology network comprises an input mapping layer, multi-level implicit resolution layers, and a reconstructed output layer; the step of importing the acoustic multidimensional features within the character-specific feature cluster into the initial acoustic topology network to update the node parameters of the initial acoustic topology network until the output distribution of the initial acoustic topology network approximates the feature distribution of the character-specific feature cluster, thereby generating a basic character acoustic network, includes: The acoustic multidimensional features within the character-specific feature cluster are input into the input mapping layer to perform dimensionality reduction, and the dimensionality-reduced features are passed to the multi-level implicit parsing layer to extract deep vocal node features. The reconstructed output layer generates a simulated speech distribution map based on the deep vocal node features; Extract the distribution difference gradient of the feature distribution of the simulated speech distribution map for the character-specific feature cluster, and adjust the node activation weights of the multi-level implicit parsing layer layer by layer along the inverse feature propagation path of the distribution difference gradient. The node activation weights are continuously adjusted until the distribution difference gradient converges to a floating range that limits the output distribution to approximate the feature distribution, thereby locking the current node activation weights to generate the basic character acoustic network.

7. A device for character voice differentiation and modeling based on multi-dimensional features, characterized in that, include: The feature association sequence generation module is used to obtain source character voice segments in the initial audiovisual data, extract the acoustic multidimensional features of the source character voice segments, and bind the acoustic multidimensional features to a preset character tag used to identify the source character voice segment to generate a feature association sequence. The character-specific feature cluster generation module is used to analyze the structural similarity of the acoustic multidimensional features within the feature association sequence, and based on the structural similarity, to perform feature clustering processing on the acoustic multidimensional features that reach a preset approximation threshold to generate character-specific feature clusters. The basic character acoustic network generation module is used to import the acoustic multidimensional features within the character-specific feature cluster into the initial acoustic topology network to update the node parameters of the initial acoustic topology network until the output distribution of the initial acoustic topology network approximates the feature distribution of the character-specific feature cluster, thereby generating the basic character acoustic network. The final character acoustic network generation module is used to acquire a verification speech segment, import the verification speech segment into the basic character acoustic network to generate expected distribution features, detect the representation offset state between the expected distribution features and the original features of the verification speech segment, and then adjust the node parameters of the basic character acoustic network according to the representation offset state to generate the final character acoustic network.

8. An electronic device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the character voice differentiation and modeling method based on multi-dimensional features as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the role-voice differentiation and modeling method based on multi-dimensional features as described in any one of claims 1 to 6.

10. A character voice differentiation and modeling system based on multi-dimensional features, characterized in that, It includes a feature association processing end, a feature aggregation processing end, an acoustic network training end, and an acoustic network verification end; The feature association processing end is used to obtain source character voice segments in the initial audiovisual data, extract the acoustic multidimensional features of the source character voice segments, and bind the acoustic multidimensional features to a preset character tag used to identify the source character voice segment to generate a feature association sequence. The feature clustering processing end is used to analyze the structural similarity of the acoustic multidimensional features in the feature association sequence, and based on the structural similarity, to perform feature clustering processing on the acoustic multidimensional features that reach a preset approximation threshold to generate a character-specific feature cluster. The acoustic network training terminal is used to import the acoustic multidimensional features within the character-specific feature cluster into the initial acoustic topology network to update the node parameters of the initial acoustic topology network until the output distribution of the initial acoustic topology network approximates the feature distribution of the character-specific feature cluster, so as to generate a basic character acoustic network. The acoustic network verification terminal is used to acquire verification speech segments, import the verification speech segments into the basic character acoustic network to generate expected distribution features, detect the representation offset state between the expected distribution features and the original features of the verification speech segments, and then adjust the node parameters of the basic character acoustic network according to the representation offset state to generate the final character acoustic network.

Images