An AI multi-modal audio collection and news script automatic generation method

Abstract

The application relates to the technical field of text generation, and specifically discloses an AI multi-modal audio collection and news script automatic generation method, which comprises the following steps: decomposing original audio into a close-up semantic layer, a medium-shot event layer and a long-shot base layer through soundscape topology analysis, constructing an auditory suspense diagram to identify suspense modes such as accumulation, release and suspension, and outputting a topology vector and a suspense matrix; combining the topology vector to complete spatial reference loss through cross-modal reference resolution, analyzing rhythm characteristics to construct a rhetoric template, and reconstructing the suspense matrix into a text skeleton with a laying-out cluster, a climax node and a blank position; inferring the meaning of an exclamation based on micro-rhythm and converting the exclamation into a paralanguage mark, semanticizing medium and long-shot layer semantics into atmosphere text to fill the blank position, and generating a news script with a soundscape intertext index. The application realizes paradigm conversion from an auditory timeline to a text spatial hierarchy, solves the problems of spoken language reference fracture and acoustic evidence tracing, and is suitable for an automatic news production system.

Description

Technical Field

[0001] This invention relates to the field of text generation technology, specifically to an AI-based multimodal audio acquisition and automatic news manuscript generation method. Background Technology

[0002] Current audio news generation technologies primarily rely on speech recognition engines to convert live recordings into text, and then generate news articles through text summarization or template filling methods. These technologies typically use single-channel or simple array microphones to capture audio, employ noise suppression algorithms to filter out background ambient noise, retaining only the transcribed target speech content, and treating the audio signal as a pure one-dimensional speech stream.

[0003] However, news scene audio is essentially a three-dimensional soundscape with spatial distribution characteristics, comprising a near-field semantic layer—the highly directional speech of reporters and interviewees, a mid-field event layer—diffuse environmental bursts, and a far-field basal layer—continuous environmental noise. Existing technologies ignore this spatial topology of the soundscape, resulting in generated text that cannot reproduce the information about the development of events conveyed by distant alarms gradually increasing or sudden noises in the background, making news reports lack a sense of spatial presence.

[0004] More critically, existing technologies filter out silent segments and abrupt changes in ambient sound as noise, disrupting the auditory suspense mechanism carefully constructed through acoustic patterns such as build-up, release, and suspension in news audio. This results in flat, monotonous text lacking the narrative rhythm and dramatic tension controlled by professional journalists. Furthermore, spoken language contains numerous cross-modal references that rely on shared acoustic context, such as "this direction" or "that group of people." Current technologies lack the ability to combine acoustic location information with textual reference resolution, leading to referential ambiguity. At the same time, the implied meanings carried by the microprosodic features of speech are completely ignored, resulting in automated news lacking critical depth and credibility assessment capabilities.

[0005] Therefore, there is an urgent need for an automatic news article generation method that can analyze soundscape topology, identify auditory suspense, resolve cross-modal references, and infer implied meanings, so as to achieve a professional paradigm shift from auditory timeline logic to textual spatial hierarchy logic. Summary of the Invention

[0006] The purpose of this invention is to provide an AI multimodal audio acquisition and automatic news text generation method to solve the technical problems of existing audio news generation technology, which ignores the spatial dimension of audio and auditory narrative logic, resulting in the lack of immediacy in news texts, ambiguity of spoken language references, and inability to capture implied meanings.

[0007] To solve the above-mentioned technical problems, the present invention specifically provides the following technical solution: A method for AI-powered multimodal audio acquisition and automatic news transcript generation includes the following steps: S1. Acquire audio stream and parse it into three-layer soundscape, extract features to construct auditory suspense map to identify suspense patterns, and output topological vector and suspense matrix; S2. Recognize near-field speech to generate original text, combine the topological vector to resolve the reference to generate semantic text, analyze the rhythm to construct a rhetorical template, and reconstruct a text skeleton with a prelude cluster, climax node, and blank space according to the suspense matrix; S3. Based on micro-prosody, infer the implied meaning, semanticize the mid-to-far view layer into atmospheric text to fill the blank space, convert the implied meaning into paralinguistic markers to insert climax nodes, optimize the rhythm of the prelude cluster sentences, and generate news articles with sound and scene intertextuality indexes.

[0008] As a preferred embodiment of the present invention, S1 specifically includes: S11. Acquire at least four channels of raw audio stream using a distributed microphone array, and perform timestamp synchronization and acoustic calibration on the audio signal of each channel; S12. Input the synchronized calibrated multi-channel audio stream into the pre-trained sound scene topology decomposition network, and decompose it into a near-field semantic layer, a mid-field event layer and a far-field base layer based on independent component analysis; wherein, the near-field semantic layer is a highly directional target sound source, the mid-field event layer is a diffuse environmental burst sound, and the far-field base layer is a continuous environmental noise base. S13. Perform beamforming and speech activity detection on the near-field semantic layer, extract the speech content stream, and use the time difference of arrival algorithm to calculate the spatial azimuth and distance estimates of each speech segment as acoustic positioning coordinates; perform event boundary detection and type classification on the mid-field event layer, extract the event start and end time, peak frequency and energy envelope of the burst sound as environmental burst sound type and time-frequency features; perform power spectral density estimation on the far-field base layer, and extract the power change curves of the low-frequency band, mid-frequency band and high-frequency band as the noise power spectrum evolution trend; S14. Based on the energy gradient slope of the mid-range event layer, the near-range speech response delay, and the silence gap between sentences, identify the building-up, release, and suspension patterns, and generate an auditory suspense dependency matrix. S15. Combine the spatial orientation labels, reverberation time estimates, and interlayer energy coupling coefficients of the near-field semantic layer, mid-field event layer, and far-field base layer into a soundscape topology description vector, and output the soundscape topology description vector, acoustic positioning coordinates, and auditory suspense dependency matrix.

[0009] As a preferred embodiment of the present invention, S12 specifically includes: S121. Perform a short-time Fourier transform on the synchronized calibrated multi-channel audio stream to convert it to the time-frequency domain and obtain the multi-channel complex spectrum matrix; S122. Input the complex spectrum matrix into the sound source separation network based on deep independent component analysis. The sound source separation network estimates the unmixing matrix iteratively by minimizing the mutual information loss between each output channel and outputs multiple independent sound source components. S123. Extract spatial localization features for each independent sound source component, calculate the phase difference and amplitude difference of each component between different microphone channels, and obtain the estimated spatial azimuth angle of each component. S124. Based on the temporal stability of the estimated spatial azimuth angle and the time-frequency sparsity of the sound source components, the components are classified into a near-field semantic layer, a mid-field event layer, and a far-field base layer. Sound source components whose spatial azimuth angle changes continuously on the time axis and have a clear direction are classified into the near-field semantic layer. Sound source components that appear suddenly, have concentrated time-frequency energy, and have a spatial diffusion angle greater than a preset angle threshold are classified into the mid-field event layer. Components that exist for a long time, have stable spectral energy, and have no obvious spatial orientation are classified into the far-field base layer. The three-layer sound scene signals are output after decomposition, and a corresponding spatial azimuth label and inter-layer energy coupling coefficient are added to each layer signal.

[0010] As a preferred embodiment of the present invention, S14 specifically includes: S141. Perform time-series smoothing on the energy envelope of the mid-scene event layer, calculate the energy slope within the sliding window, and mark the energy slope as a potential accumulation candidate interval when the energy slope continuously exceeds a preset rise threshold and the duration is greater than a first duration threshold. S142. Detect the start time of the first speech segment in the near-field semantic layer after the end of the potential candidate interval, calculate the time difference between the end time of the potential candidate interval and the start time of the speech segment, and if the time difference is less than the second duration threshold, then mark the potential candidate interval as the potential mode. S143. Perform burst pulse detection on the mid-range event layer. When an event with an energy peak exceeding a preset peak threshold is detected, record the peak time of the event. Calculate the response delay between the peak time and the start time of the next adjacent speech segment in the near-range semantic layer. If the response delay is less than a third duration threshold, mark it as release mode. S144. Detect the length of the silent segment between adjacent statements in the near-field semantic layer. When the length of the silent segment exceeds the fourth duration threshold and the average energy of the mid-field event layer in the corresponding time period is lower than the silent energy threshold, mark it as a suspended mode. S145. Record the start timestamp, end timestamp, and associated soundscape layer identifier for each mode, and calculate the temporal dependency weights between the building-up mode and the release mode, and between the release mode and the suspension mode, to generate an auditory suspense dependency matrix.

[0011] As a preferred embodiment of the present invention, S2 specifically includes: S21. Perform speech recognition based on an end-to-end deep neural network on the speech content stream of the near-field semantic layer to generate an original text sequence with timestamps; S22. Extract indicator pronouns and spatially ambiguous references from the original text sequence, align them with spatial orientation labels and acoustic positioning coordinates in the soundscape topology description vector, calculate the spatial orientation angles they point to, and generate a semantic text sequence after referential resolution. S23. Extract the prosodic features of the near-field semantic layer, including the pitch curve extracted from the fundamental frequency curve, the pause duration distribution of adjacent word boundary detection, the speech rate change gradient of syllable duration statistics, and the breathing noise intensity of energy envelope analysis as breath sound features; construct a rhetorical prosodic template based on the numerical range of the prosodic features, wherein a rapid pitch rise and short pause duration are mapped to a tension-short sentence mapping pattern, a gentle pitch and moderate pause duration are mapped to a soothing-long sentence mapping pattern, and a sudden drop in pitch and abrupt change in pause duration are mapped to a suspense-sentence-break mapping pattern. S24. Based on the auditory suspense dependency matrix, the semantic units corresponding to the building-up mode timestamp in the semantic text sequence after the referential resolution are aggregated into a prelude paragraph cluster, the semantic units corresponding to the release mode timestamp are marked as climax statement nodes, and the semantic units corresponding to the suspension mode timestamp are marked as blank description positions. The logical relationship between the prelude paragraph cluster, the climax statement node and the blank description position is established using the dependency edges in the auditory suspense dependency matrix, and the original time-linear semantic text sequence is reconstructed into a news text skeleton with suspense levels.

[0012] As a preferred embodiment of the present invention, S22 specifically includes: S221. Perform part-of-speech tagging and dependency parsing on the original text sequence to identify the set of demonstrative pronouns and the set of spatial fuzzy references. The demonstrative pronouns include personal demonstrative pronouns and location demonstrative pronouns, and the spatial fuzzy references include directional adverbs and relative position phrases. For each identified demonstrative pronoun, extract its position index and grammatical role in the text. At the same time, based on the timestamp of the sentence in which the demonstrative pronoun is located, retrieve the azimuth and distance estimates of each sound source within the corresponding time window from the acoustic positioning coordinates. S222. Divide the spatial orientation labels in the soundscape topology description vector into several sector regions, each sector region is associated with a predefined orientation description phrase; calculate the sector region to which the sound source azimuth angle corresponding to the pronoun belongs, and use the orientation description phrase associated with the sector region as the spatial mapping result of the pronoun; S223. When the pronoun is a personal demonstrative pronoun and the azimuth of the sound source points to a specific speaker, the pronoun is replaced with the corresponding speaker's identity identifier or role description, and entity completion information is generated; the mapped spatial orientation description or entity completion information is backfilled into the position of the corresponding pronoun in the original text sequence, the original pronoun is deleted, and a semantic text sequence after dereference resolution is generated.

[0013] As a preferred embodiment of the present invention, S23 specifically includes: S231. Extract the fundamental frequency trajectory from the speech content stream of the near-field semantic layer, obtain the frame-by-frame pitch value using the cepstral method, generate the pitch curve after median filtering and smoothing, and calculate the global mean, standard deviation and maximum slope of the rising segment of the pitch curve. S232. Detect the length of the silent gap between adjacent words, statistically analyze the pause duration distribution within each sentence, calculate the ratio of short pauses to long pauses and the duration of the longest pause; calculate the ratio of the number of syllables to the window duration according to a fixed time window to obtain the speech rate change gradient sequence, and record the local peak and valley values ​​of the speech rate; perform energy envelope analysis on the high-frequency band of the speech signal, and extract the average intensity and peak intensity of breathing noise as breath sound features; S233. When the maximum slope of the rising segment of the pitch curve exceeds the first threshold, the proportion of short pauses in the pause duration distribution exceeds the second threshold, and the speech rate change gradient is greater than the third threshold, a tension-short sentence mapping pattern is constructed to indicate that the corresponding semantic unit is converted into a short sentence. S234. When the standard deviation of the pitch curve is lower than the fourth threshold, the longest pause duration is between the fifth and sixth thresholds, and the breath sound intensity is lower than the seventh threshold, a soothing-long sentence mapping pattern is constructed to indicate conversion to a compound long sentence. S235. When the pitch curve drops sharply and the drop exceeds the eighth threshold, the longest pause duration exceeds the ninth threshold, and the breath sound intensity increases suddenly, a suspense-sentence-break mapping mode is constructed to indicate the insertion of a sentence break mark or ellipsis at the pause position.

[0014] As a preferred embodiment of the present invention, S3 specifically includes: S31. Extract the micro-pauses, pitch change points and breath sound intensity of the near-field semantic layer, and infer from them whether they belong to a definite statement, hesitant speculation or ironic suggestion; S32. Perform emotional semantic mapping on the environmental sudden sound types of the mid-field event layer, mapping plosive sounds to near-field tension and sudden noise from crowds to mid-field chaos. Perform spatial distance encoding on the noise power spectrum evolution trend of the far-field base layer, mapping the rise of low-frequency power to far-field oppression, and combine to generate spatial atmosphere description text. S33. The implied meaning type is converted into a paralinguistic marker and inserted into the corresponding statement of the climax statement node. The spatial atmosphere description text is filled into the blank description space. The sentence rhythm is optimized for each sentence in the prelude paragraph cluster according to the rhetorical rhythm template, including adjusting the density of short sentences and the position of sentence breaks. S34. Establish a soundscape intertextual index, i.e., a two-way mapping table, for each spatial atmosphere description text and sub-language mark, record the start and end positions of the text and the corresponding soundscape layer identifiers and timestamp ranges in the original audio stream, and output a structured news article.

[0015] As a preferred embodiment of the present invention, S31 specifically includes: S311. Extract the fundamental frequency curve from the speech content stream of the near-field semantic layer, calculate the fundamental frequency value corresponding to each syllable, detect the points where the fundamental frequency change amplitude between adjacent syllables exceeds the preset abrupt change threshold, and mark them as pitch abrupt change points; perform high-frequency energy analysis on the speech signal, calculate the energy proportion in the 3kHz to 8kHz frequency band, and obtain the breath sound intensity time sequence curve after smoothing; detect the silent gaps between adjacent syllables, mark the gaps with a duration between 50 milliseconds and 200 milliseconds as micro-pauses, and count the frequency and distribution pattern of micro-pauses within each sentence; S312. When the fundamental frequency curve is generally flat, the global fundamental frequency standard deviation is lower than the first threshold, and the mean of the breath sound intensity time series curve is lower than the second threshold, the implied meaning type of the corresponding statement is inferred to be a deterministic statement. S313. When the frequency of the micro-pauses exceeds the third threshold, and the micro-pauses are mostly distributed at the beginning or turning point of the statement, and the fundamental frequency curve shows a gradual upward trend, the implied meaning of the corresponding statement is inferred to be a hesitant speculation. S314. When the pitch change points appear densely and the average distance between adjacent change points is less than the fourth threshold, and the breath sound intensity time sequence curve shows an instantaneous peak value exceeding the fifth threshold near the pitch change point, the implied meaning of the corresponding statement is inferred to be an ironic suggestion. S315. Output the implied meaning type label and corresponding confidence score for each statement, for subsequent use in inserting sub-language tags.

[0016] As a preferred embodiment of the present invention, S34 specifically includes: S341. Traverse each spatial atmosphere description text and each secondary language mark in the structured news article to obtain the start character position and end character position of the text or mark in the article; S342. Based on the source of the spatial atmosphere description text or the secondary language markup, trace back to the soundscape layer identifier and timestamp range in the corresponding original audio stream; wherein, the spatial atmosphere description text originates from the mid-range event layer or the far-range base layer, and the secondary language markup originates from the implied meaning inference result of the near-range semantic layer; S343. Establish a first bidirectional mapping entry for each spatial atmosphere description text, and record the starting character position, ending character position, corresponding sound scene layer identifier, and the start and end time of the environmental sudden sound in the mid-scene event layer or the time window of the noise power spectrum evolution trend in the far-scene base layer. S344. Establish a second bidirectional mapping entry for each sub-language tag, recording the starting character position, the ending character position, the corresponding near-field semantic layer identifier, and the start and end timestamps of the corresponding statement in the near-field semantic layer; S345. Organize all bidirectional mapping entries into the soundscape intertextual index and store it in a metadata file associated with the structured news article, so that when a user clicks on the atmosphere description text or sub-language marker in the article, they can jump to the corresponding soundscape layer and time segment in the original audio stream for playback verification.

[0017] Compared with the prior art, the present invention has the following advantages: 1. By analyzing the topological layers of soundscape and constructing auditory suspense maps, the acoustic features of near, mid and far views are mapped into textual narrative structures. Acoustic suspense patterns such as building momentum, releasing tension, and suspending tension are converted into textual rhythms such as foreshadowing, climax, and blank space, restoring the spatial dimension and auditory narrative tension of the news scene.

[0018] 2. Based on soundscape topology vectors, cross-modal reference resolution is performed to complete spoken references into spatial location descriptions. Combined with prosodic features such as pitch and pauses, a rhetorical template is constructed to achieve a paradigm shift from auditory timeline to textual spatial hierarchy.

[0019] 3. By utilizing microprosodic features to infer implied meanings and transform them into paralinguistic markers, the text gains critical depth; by establishing a two-way link between the text and the original audio through soundscape intertextuality indexing, the acoustic evidence for the content can be traced and verified. Attached Figure Description

[0020] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0021] Figure 1This is a flowchart illustrating the method described in Embodiment 1 of the present invention.

[0022] Figure 2 This is a framework diagram of the system described in Embodiment 2 of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] The concepts involved in this application will first be described with reference to the accompanying drawings. It should be noted that the following descriptions of various concepts are only for the purpose of making the content of this application easier to understand and do not constitute a limitation on the scope of protection of this application; furthermore, the embodiments and features in the embodiments of this application can be combined with each other unless otherwise specified. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] Example 1 like Figure 1 As shown, this invention provides an AI-based multimodal audio acquisition and automatic news text generation method, comprising the following steps: S1. Acquire audio streams, parse them into three-layer soundscapes, extract features to construct an auditory suspense map, identify suspense patterns, and output topological vectors and suspense matrices; specifically including: S11. Multi-channel audio synchronous acquisition and acoustic calibration, specifically: A distributed microphone array was deployed on-site, consisting of at least four microphone units arranged in a pre-defined geometric topology at key locations within the news scene. This ensured spatial sampling coverage of the near, mid, and far acoustic areas. Each microphone unit had a built-in analog-to-digital converter that converted acoustic pressure waves into a digital sampling sequence, forming an independent channel of raw audio stream. The sampling rate of each channel was uniformly set to a professional audio standard, and the quantization accuracy ensured that the dynamic range covered changes in sound pressure level on-site.

[0026] Timestamp synchronization is performed on the audio signal of each channel. A hardware synchronization triggering mechanism is used to distribute synchronization pulse signals to each microphone unit through a dedicated synchronization clock source, or a software time alignment algorithm is used to calculate the relative time delay deviation of each channel's data stream based on the cross-correlation function. The time axis is precisely aligned by sampling point interpolation or buffer adjustment to ensure the time consistency of multi-channel data.

[0027] Then, acoustic calibration is performed, including gain calibration, which uses a standard sound source to excite and measure the sensitivity differences of each microphone, calculates compensation coefficients to unify the output amplitude; phase calibration, which corrects the phase response deviation between channels; frequency response calibration, which compensates for the differences in the spectral characteristics of different microphones, eliminates acoustic measurement errors caused by equipment differences, and outputs a multi-channel audio stream that is timestamped and acoustically calibrated.

[0028] S12. Soundscape topological decomposition based on depth independent component analysis, specifically: S121. Multi-channel audio time-frequency transformation and complex spectrum matrix construction, specifically: Short-time Fourier transform (SFT) is performed on the multi-channel audio stream that has been time-stamped and acoustically aligned. The sliding analysis window length and frame shift step size are set, and either a Hanning or Hamming window is used to window and frame the time-domain signal. The time-domain sampling sequence of each channel is converted to a time-frequency domain representation using a fast Fourier transform (FFT) algorithm. The real and imaginary components of the corresponding frequency points for each time frame are calculated, constructing a multi-channel complex spectrum matrix containing amplitude and phase information. The dimensions of this matrix correspond to the number of microphone channels, the number of time frames, and the number of frequency bins, fully representing the energy distribution and phase relationship of the original sound field in the time-frequency domain, providing the input data structure for subsequent deep independent component analysis.

[0029] S122. Demixing and Source Separation using Deep Independent Component Analysis Networks A sound source separation network based on deep independent component analysis (DIA) is constructed, comprising a convolutional feature encoder module, an adaptive DIA module, and a deconvolutional signal reconstructor module. The complex spectral matrix is ​​first input to the convolutional feature encoder, which extracts high-order time-frequency features through multiple two-dimensional convolutional layers and batch normalization layers. Subsequently, the signal enters the adaptive DIA module, which constructs a nonlinear mapping function based on the InfoMax principle and estimates the unmixing matrix using a multilayer perceptron. During network training, a mutual information minimization loss function is employed, and variational approximation or kernel density estimation methods are used to calculate the statistical dependencies between output channels. The network parameters are iteratively optimized using a stochastic gradient descent algorithm to minimize the mutual information values ​​between channels, thereby maximizing statistical independence. Finally, the deconvolutional signal reconstructor maps the separated independent components back to the time-frequency domain, outputting multiple independent sound source components, each corresponding to a statistically independent acoustic source signal.

[0030] S123. Spatial orientation feature extraction and sound source localization calculation, specifically: Spatial localization feature extraction is performed on each independent sound source component. The phase difference of the sound source component at corresponding frequency points in different microphone channels is calculated. Based on the generalized cross-correlation phase transform algorithm or the rotation-invariant technique based on the subspace method, the time delay difference of the sound wave arriving at different microphones is estimated. At the same time, the amplitude ratio between each channel is calculated. Combined with the geometric topology parameters of the microphone array, a system of equations for phase difference and amplitude difference is established. Through geometric algebraic solution, the spatial azimuth angle of the sound source component relative to the microphone array reference point is calculated, including the horizontal azimuth angle and the vertical elevation angle. The estimated spatial azimuth angle values ​​of each sound source component are obtained, and the mapping relationship between the sound source position and signal characteristics is established.

[0031] S124. Soundscape topology layer classification and spatial attribute annotation, specifically: Three-level classification is performed based on the time stability of the spatial azimuth estimate and the time-frequency sparsity of the sound source components: The continuous variation characteristics of the spatial azimuth angle estimate in the time series are analyzed, and the angular change rate between adjacent time frames is calculated. When the change rate is consistently less than 5 degrees per second and the time azimuth angle variance is less than 15 degrees, the sound source component is determined to have time stability and is classified as a near-field semantic layer.

[0032] The time-frequency sparsity metric is calculated by determining the proportion of effective time-frequency units occupied by the energy of the sound source component in the time-frequency representation. When the energy concentration exceeds 0.7, i.e., the energy is concentrated in less than 30% of the time-frequency units, it is determined to be a sparse representation. For sound source components that simultaneously satisfy the time-frequency sparsity and whose spatial spread angle is greater than 60 degrees as estimated by the microphone array beamwidth, they are classified as mid-range event layers.

[0033] Steady-state components that persist for a long time, have an energy concentration of less than 0.3, exhibit a uniform distribution or no significant peak in spatial azimuth angle within the range of 0 to 360 degrees, and have a spatial diffusion angle covering the entire direction are classified as the prospective basement layer.

[0034] The interlayer energy coupling coefficient is calculated by statistically analyzing the energy cross-correlation values ​​of each layer within the overlapping time-frequency unit, and normalizing them to obtain a coupling coefficient between 0 and 1, which characterizes the degree of acoustic energy penetration between foreground, midground, and background scenes. Three layers of sound scene signals are output, and each layer is appended with a corresponding spatial orientation label and interlayer energy coupling coefficient.

[0035] S13. Three-layer soundscape feature extraction and acoustic parameter calculation, specifically: S131. Beamforming processing is performed on the near-field semantic layer obtained from S12 decomposition. A delay-summing beamformer or a minimum variance distortionless response beamformer is used. Utilizing the phase consistency of the received signals from each channel of the microphone array, the delay compensation amount of each channel relative to the target sound source direction is calculated. Time alignment and weighted summation are then performed on the multi-channel signals to enhance the signal-to-noise ratio of the sound source in the target direction and suppress interference and environmental noise from non-target directions, outputting an enhanced near-field speech signal. Speech activity detection is then performed on this enhanced signal, employing a dual-threshold detection algorithm based on short-time energy and zero-crossing rate, or an endpoint detection model based on a deep neural network. Energy and zero-crossing rate thresholds are set to identify the start and end frames of valid speech segments and extract the clean speech content stream.

[0036] The time difference of arrival (TDOA) algorithm is used to calculate the spatial azimuth and distance estimates of each speech segment as acoustic positioning coordinates. Specifically, the algorithm calculates the generalized cross-correlation function of the near-field semantic layer between each microphone channel, obtains accurate TDOA estimates through parabolic interpolation or phase transformation weighting, and establishes a nonlinear equation system by combining the geometric topology of the microphone array with the known array spacing parameters. The spatial azimuth and radial distance of the sound source relative to the array reference point are solved by closed-form algorithm or iterative least squares optimization to generate three-dimensional acoustic positioning coordinates that include horizontal angle, pitch angle and distance.

[0037] S132. Perform event boundary detection on the mid-range event layer obtained from S12 decomposition. Employ an endpoint detection algorithm based on the energy envelope change rate to calculate the root mean square energy of the mid-range event layer signal within a short-time sliding window, forming an energy envelope curve. First, estimate the ambient background noise floor energy. Set the rising edge trigger threshold to 10dB to 20dB of the average background noise energy, corresponding to a normalized energy value of 0.3 to 0.5. Set the falling edge release threshold to be 3dB to 6dB lower than the trigger threshold, corresponding to a normalized energy value of 0.2 to 0.4, forming a hysteresis comparison mechanism to avoid threshold crossing jitter. Mark the event start time when the energy envelope rises from below the rising edge trigger threshold to above the threshold, and mark the event end time when the energy envelope falls from above the falling edge release threshold to below the threshold, extracting the event start and end times of the burst sound.

[0038] The detected burst sounds are classified by type, and the peak frequency of the event, i.e., the frequency point where the energy is maximum, is calculated. Spectral envelope features are extracted using Mel-frequency cepstral coefficients. Combining the rise time, duration, and decay characteristics of the energy envelope, a pre-trained classifier based on a hybrid architecture of convolutional neural networks and bidirectional long short-term memory networks is used for classification. The input layer of this classifier receives a two-dimensional time-frequency feature map composed of Mel-frequency cepstral coefficient feature sequences and energy envelope features. Connected to the input layer is a two-layer two-dimensional convolutional feature extraction module. The first convolutional layer has 32 3x3 convolutional kernels with a stride of 1, using a modified linear unit activation function to extract local time-frequency texture features. The second convolutional layer has 64 convolutional kernels of the same size, and uses a max-pooling layer to reduce the feature dimensionality, outputting a high-level abstract feature representation. Following the convolutional module is a bidirectional long short-term memory network sequence modeling layer, containing a forward long short-term memory unit with 128 hidden units and a backward long short-term memory unit with 128 hidden units. A gating mechanism captures the forward and backward temporal dependencies of the feature sequences, establishing a dynamic pattern representation of the burst sound's temporal evolution. The bidirectional long short-term memory network output is connected to a fully connected feature fusion layer containing 256 neurons and a dropout regularization layer to prevent overfitting. Finally, it is connected to a Softmax classification layer with four output neurons. The four outputs correspond to the probability distributions of four types of environmental sudden sounds: loud noise, crowd commotion, alarm blaring, and mechanical failure. The envelope curves recording the start and end times, peak frequency, and energy changes over time are used as the environmental sudden sound type and time-frequency features.

[0039] S133. Perform power spectral density estimation on the distant baseline layer obtained from S12 decomposition. Use the periodogram method to set the frequency resolution and analysis time, and calculate the power spectral distribution of the distant baseline layer signal in different analysis time periods. Divide the frequency band into low frequency band (20 to 250 Hz), mid frequency band (250 to 2000 Hz), and high frequency band (2000 to 8000 Hz). Calculate the average power value of each frequency band in each time analysis frame, and plot the power change curve over time as the evolution trend of the noise power spectrum. This characterizes the energy fluctuations and long-term changes of the distant environmental noise in different frequency bands, providing background acoustic status information for subsequent auditory suspense pattern recognition.

[0040] S14. Auditory suspense pattern recognition and dependency matrix construction, specifically: S141. Potential accumulation candidate interval detection and energy slope threshold determination, specifically: The energy envelope of the mid-range event layer is smoothed using a moving average, and a sliding window of 300ms to 500ms is used for time-series filtering to eliminate instantaneous jitter. The energy slope is obtained by calculating the energy difference between adjacent windows. When the energy slope continuously exceeds a preset rise threshold of 3dB to 5dB per second, and the duration of this rise is greater than a first duration threshold of 800ms to 1200ms, the ambient sound energy is determined to be in an accumulation state. This period is marked as a candidate interval for energy accumulation, and its start and end timestamps are recorded.

[0041] S142. Confirmation of the charging mode and verification of the voice response delay threshold, specifically: The system detects near-field semantic layer speech activity after the end of the potential accumulation candidate interval and identifies the start time of the first speech segment. The time interval between the end timestamp of the potential accumulation candidate interval and the start time of the speech segment is calculated as the response delay. When this delay is less than a second duration threshold of 200ms to 500ms, a causal relationship between ambient sound accumulation and speech expression is confirmed. The potential accumulation candidate interval is then ultimately labeled as a potential accumulation pattern, establishing a mapping relationship between energy accumulation and speech response between the mid-field event layer and the near-field semantic layer.

[0042] S143. Release pattern recognition and burst response threshold determination, specifically: Instantaneous energy peak detection is performed on the mid-range event layer. When the detected energy peak exceeds a preset peak threshold of 10dB to 15dB above the average speech energy of the near-range semantic layer, it is determined to be a sudden impulse event, and the peak moment is recorded. The response delay between this peak moment and the start moment of the first speech segment immediately following it in the near-range semantic layer is calculated. When this delay is less than a third duration threshold of 100ms to 300ms, it is confirmed that the sudden environmental sound triggered an immediate language response, and this period is marked as the release mode, characterizing the interruption effect of the sudden environmental event on the narrative.

[0043] S144. Suspension mode detection and silent energy threshold test, specifically: After performing speech activity detection on the near-field semantic layer, silence segment recognition is performed by measuring the duration of the speechless interval between adjacent sentences. When the duration of the silence segment exceeds the fourth duration threshold of 1500ms to 2500ms, it is initially determined to be a semantic pause. Simultaneously, the average energy value of the mid-field event layer during this period is detected. When the average energy is lower than the silence energy threshold of 5dB to 10dB below the floor noise energy of the near-field semantic layer silence segment, it is confirmed that the scene is in an active narrative suspension state, which is marked as a suspension mode, representing a narrative blank space intentionally created by the speaker.

[0044] S145. Construction of the suspense dependency matrix and weight calculation, specifically: Record the start and end timestamps, along with the associated near-field semantic layer or mid-field event layer identifiers, for each of the three modes: build-up mode, release mode, and suspension mode. Calculate the reciprocal of the time interval between the end of the build-up mode and the peak time of the release mode as the causal dependency weight from build-up to release, with a value ranging from 0.5 to 1.0; calculate the reciprocal of the time interval between the peak time of the release mode and the start time of the suspension mode as the transition probability weight from release to suspension, with a value ranging from 0.3 to 0.8. Organize the time boundaries, layer identifiers, and weight values ​​of each mode into an adjacency matrix to generate an auditory suspense dependency matrix.

[0045] S15. Soundscape topology description vector generation and multimodal output, specifically: The spatial orientation labels of the near-field semantic layer, mid-field event layer, and far-field base layer are combined with the reverberation time estimates of each layer calculated by the reverberation time estimation algorithm, and the inter-layer energy coupling coefficients obtained in step S12. These are then concatenated or structurally combined to generate a soundscape topology description vector. This vector comprehensively represents the spatial distribution characteristics, acoustic environment attributes, and inter-layer interaction intensity of the soundscape at the news scene. Finally, the soundscape topology description vector, the acoustic positioning coordinates calculated in step S13, and the auditory suspense dependency matrix constructed in step S14 are simultaneously output. These three data entities together form the input data foundation for subsequent cross-modal reference resolution, rhetorical prosody transcoding, and implied meaning inference steps.

[0046] S2. Recognize near-field speech to generate original text, combine topological vectors to resolve prosody and generate semantic text, analyze prosody to construct rhetorical templates, and reconstruct a text skeleton with foreshadowing clusters, climax nodes, and blank spaces based on the suspense matrix; specifically including: S21. End-to-end speech recognition and original text sequence generation, specifically: This system is a speech recognition system based on an end-to-end deep neural network, which takes the speech content stream of the near-semantic layer as input. The system adopts an encoder-decoder architecture. The encoder consists of four layers of bidirectional long short-term memory networks stacked together to extract high-dimensional acoustic feature representations of speech frames and convert the input acoustic feature sequence into a high-level contextual representation. The encoder output connection is based on a content attention-based alignment mechanism, which calculates the attention weight distribution between the acoustic feature hidden state and the target character or sub-word unit, and establishes a soft alignment relationship between the acoustic frame and the text label.

[0047] The decoder employs a unidirectional long short-term memory network, predicting the probability distribution of characters or sub-word units frame by frame based on the attention-weighted acoustic context and historical decoding output. During the training phase, a joint optimization objective of the connection-time classification loss function and the attention mechanism loss function is used to update the network parameters through the backpropagation algorithm. During the inference phase, a bundle search decoding strategy is adopted to maintain the cumulative probability of candidate sequences and select the optimal text path. During the decoding process, each output word unit is mapped back to the original speech sampling point timestamp by recording its alignment position in the encoder output feature frame, generating an original text sequence with precise timestamp markings.

[0048] S22. Cross-modal reference resolution and spatial semantic completion, specifically: S221. Lexical and syntactic analysis and spatiotemporal alignment of pronouns, specifically: Part-of-speech tagging and dependency parsing are performed on the original text sequence. A bidirectional long short-term memory network-conditional random field sequence labeling model is used to identify the set of demonstrative pronouns and the set of spatially ambiguous references in the text. Demonstrative pronouns include personal demonstrative pronouns such as "this," "that," "this side," and "that side," as well as location demonstrative pronouns. Spatially ambiguous references include locative adverbs such as "here," "there," "just now," and "over there," as well as relative positional phrases. First, the text is converted into a distributed vector representation through a word embedding layer. The input is a bidirectional encoding layer composed of a forward long short-term memory network and a backward long short-term memory network in parallel. The forward layer extracts historical context features in forward order, and the backward layer extracts future context features in reverse order. The output vectors of the two layers are concatenated to form a hidden state sequence that integrates bidirectional contextual semantics. This sequence is then input into a conditional random field decoding layer. This layer models the transition probability matrix between adjacent labels and the emission probability of the current state. A Viterbi dynamic programming algorithm is used to search for the globally optimal label path. The decoded output is a fine-grained category label for each word, from which the set of demonstrative pronouns and the set of spatially ambiguous references are identified.

[0049] For each identified pronoun, its position index in the text sequence and its grammatical role in the dependency syntax tree are extracted. At the same time, based on the timestamp of the sentence in which the pronoun is located, the spatial azimuth and distance estimates of each sound source within the corresponding time window are retrieved from the acoustic positioning coordinate database to establish the spatiotemporal alignment relationship between text reference and acoustic positioning.

[0050] S222. Sector-shaped region mapping and spatial orientation resolution, specifically: The spatial orientation labels in the soundscape topology description vector are divided into 8 to 12 sector regions with an angular resolution of 30 to 45 degrees, covering the omnidirectional range of horizontal azimuth from 0 to 360 degrees. Each sector region is pre-associated with an orientation description phrase, such as left, right front, or directly behind. The sound source azimuth angle value at the corresponding time of the pronoun is calculated, the index of the sector region into which the value falls is determined, and the orientation description phrase associated with that region is extracted as the spatial mapping result of the pronoun.

[0051] S223. Text backfilling and semantic sequence generation, specifically: When the detected pronoun is a personal demonstrative pronoun and the azimuth of the sound source points to the identified specific speaker's location, and the azimuth matching error is less than 10 to 15 degrees, the pronoun is replaced with the corresponding speaker's identity identifier or role description to generate entity completion information.

[0052] The mapped spatial location descriptions or entity completion information are then filled back into the positions of the corresponding pronouns in the original text sequence. The original pronouns are deleted, maintaining the grammatical integrity and semantic coherence of the sentences, generating a semantic text sequence after dereference resolution. This sequence completes the missing spatial references in spoken language through acoustic evidence, transforming vague "this" and "that" into clear spatial location descriptions or speaker identifications, eliminating cross-modal referential ambiguity, and providing a spatially semantically clear textual foundation for subsequent rhetorical prosodic transcoding and narrative structure reconstruction.

[0053] S23. Prosodic Feature Extraction and Rhetorical Prosodic Template Construction, specifically: S231. Fundamental frequency extraction and pitch curve feature calculation, specifically: The cepstral method is used to extract the fundamental frequency trajectory from the near-field semantic layer speech content stream. First, pre-emphasis processing is performed on the speech signal, using a first-order high-pass filter to boost the energy of high-frequency components and compensate for the vocal tract transmission characteristics. Then, a frame-segmentation and windowing operation is performed using the Hanning window function, with a frame length set to 20ms to 30ms and a frame shift set to 10ms, dividing the continuous signal into a short-time stationary frame sequence. A fast Fourier transform is calculated for each frame to obtain the spectrum, and the logarithmic power spectrum is obtained by taking the logarithm. Finally, an inverse Fourier transform is used to convert the signal to the cepstral domain to obtain the cepstral sequence.

[0054] The peak position corresponding to the fundamental frequency period is searched in the cepstral domain. The cepstral coordinates corresponding to this peak are extracted as the estimated value of the fundamental frequency period, and the reciprocal is calculated to obtain the frame-by-frame pitch value. Median filtering is performed on the extracted fundamental frequency sequence for smoothing. A 5-point or 7-point median filtering window is selected to eliminate outliers and jump errors, generating a continuous and smooth pitch curve. Based on this pitch curve, global statistical features are calculated, including the global mean (average pitch level), the standard deviation (pitch fluctuation), and the peak value of the pitch change rate (identified by the maximum rise slope) is extracted through difference calculation to identify the rising segment.

[0055] S232. Pause duration statistics and breath sound feature extraction, specifically: First, the near-field semantic layer signal is processed using a speech activity detection algorithm. A dual-threshold detection mechanism based on short-time energy and zero-crossing rate is adopted. The short-time energy threshold is set to 10-20 dB above the mean background noise energy, and the zero-crossing rate threshold is set to 1.5 times the standard deviation above the static distribution mean. By comparing the signal parameters of each frame with the thresholds, audible and silent segments are determined, thereby identifying word boundaries. The length of the silent gap between adjacent words is measured to the millisecond level as pause duration data. The duration distribution of all pauses within each sentence is statistically analyzed. A threshold of 150 ms to 250 ms is set as the boundary between short and long pauses. The proportion of pauses shorter than this threshold is calculated to obtain the short pause percentage, and the duration of the longest pause in the sentence is recorded.

[0056] Subsequently, the speech rate change gradient is calculated according to a fixed duration window. The window length is set to 1s to 2s. The number of effective syllables in the window is counted by syllable kernel detection or hidden Markov model speech recognition. The ratio of the number of syllables to the window duration is calculated to obtain the number of syllables per second. The speech rate change gradient sequence is obtained by sliding the window with a step size of 100ms. The local peaks and local valleys in the sequence are recorded.

[0057] Finally, high-pass filtering is performed on the speech signal, with the cutoff frequency set to 3kHz to 5kHz. After extracting the high-frequency components, the short-time energy envelope is calculated. The segment with stable energy and a duration of more than 100ms is identified as the breathing noise segment. The average energy value in this segment is calculated as the average intensity of the breath sound, and the maximum energy value in the segment is extracted as the peak intensity of the breath sound.

[0058] S233. Conditions for constructing the tension-short sentence mapping pattern, specifically: When the maximum slope of the rising segment of the pitch curve exceeds the first threshold of 80Hz to 120Hz per second, and the proportion of short pauses in the pause duration distribution exceeds the second threshold of 70% to 85%, while the speech rate gradient is greater than the third threshold of 5 to 7 syllables per second, a tension-short sentence mapping pattern is constructed. This pattern instructs the corresponding semantic units to be converted into short sentences, using simple subject-predicate structures or condensed sentences with omitted modifiers, to simulate the auditory tightness and urgency, keeping the rhythm of the generated text synchronized with the tense prosody of the original speech, conveying the speaker's urgent emotions and the urgency of the situation.

[0059] S234. Conditions for constructing the soothing-long sentence mapping pattern, specifically: A soothing-long sentence mapping pattern is constructed when the standard deviation of the pitch curve is below the fourth threshold of 25Hz to 35Hz, the longest pause duration is between the fifth and sixth thresholds of 400ms to 600ms, and the breath sound intensity is below the seventh threshold of 15% to 25% of the average speech energy. This pattern indicates that the corresponding semantic units are converted into complex long sentences, using complex sentence structures containing multiple modifiers or clauses to simulate a soothing and smooth auditory experience, matching the rhythm of the text with the steady rhyme of the original speech, and conveying the speaker's composed attitude and the gentle tone of the narrative.

[0060] S235. Suspense-Sentence Break Mapping Pattern Construction Condition Determination, specifically: When the pitch curve experiences a sudden drop exceeding the eighth threshold of 100Hz to 150Hz, and the longest pause lasts beyond the ninth threshold of 1200ms to 1800ms, while the breath sound intensity suddenly increases by 50% to 80% of the baseline average breath sound intensity, a suspense-segmentation mapping pattern is constructed. This pattern instructs the insertion of segment markers or ellipses at the pause positions, employing truncated sentences or white space to simulate the auditory effect of suspense, ensuring that the text rhythm matches the suspense-building rhythm of the original speech, creating dramatic tension between narrative interruption and anticipation of continuation.

[0061] S24. Narrative structure reconstruction based on suspense matrix, specifically as follows: S241. Based on the time boundary information of the building-up mode, release mode, and suspension mode recorded in the auditory suspense dependency matrix, extract the start timestamp, end timestamp, and associated soundscape layer identifier for each mode. Based on these time boundaries, retrieve the semantic units within the corresponding time window in the semantic text sequence after referential resolution, and perform segmented mapping processing.

[0062] S242. For semantic units in the corresponding time stamp interval of the accumulation mode, identify them as background content for the core facts, and aggregate these semantic units into a cluster of background paragraphs in the original time order. This cluster is located at the bottom support position of the news text skeleton and is used to provide background information and preconditions for the occurrence of the event.

[0063] For semantic units in the corresponding release mode timestamp interval, identify them as key turning points or core fact statements, and mark these semantic units as climax statement nodes. These nodes are located at the top core position of the news text skeleton and constitute the main body of the news lead or key fact statement.

[0064] For semantic units in the corresponding suspension mode timestamp interval, identify them as narrative interruption or emotional suspension points, and mark these semantic units as blank description positions. These positions are embedded at narrative turning points and are used to insert environmental atmosphere descriptions or emotional rendering text.

[0065] S243. Extract the dependency edge weights between each mode in the auditory suspense dependency matrix, including the causal dependency edge from the build-up mode to the release mode and the transformation dependency edge from the release mode to the suspension mode. Based on these dependency edges, establish causal logical connections between the cluster of preparatory paragraphs and the climax statement node, indicating how the background preparatory work leads to the outbreak of the core facts; establish the transitional logical connections between the climax statement node and the blank description position, indicating the emotional suspension or narrative transition after the core statement; form a narrative logic network through directed edge connections to ensure that there are clear logical deductions and emotional transition relationships between the components of the text skeleton.

[0066] S244. The semantic text sequence, originally arranged linearly in time, is reorganized into a news text skeleton with a spatial hierarchy, realizing a paradigm shift from auditory timeline logic to text spatial hierarchy logic. The introductory paragraph clusters support the climax statement nodes, and the climax statement nodes lead to the blank description positions, forming a complete suspense narrative structure.

[0067] S3. Based on micro-prosody, infer the implied meaning, semanticize the mid-to-far view layer into atmospheric text to fill blank spaces, convert the implied meaning into paralinguistic markers to insert climax nodes, optimize the rhythm of the introductory cluster sentences, and generate news articles with soundscape intertextuality indexes; specifically including: S31. Microprosodic feature extraction and implied meaning type determination, specifically: S311. Multi-dimensional feature extraction and parameter calculation of micro-prosodics, specifically: Multidimensional microprosodic features are extracted from the near-semantic layer speech content stream. The fundamental frequency curve is extracted using the cepstral method. After pre-emphasis on the speech signal, it is framed and windowed with a frame length of 20ms to 30ms and a frame shift of 10ms. A cepstral sequence is obtained through Fast Fourier Transform and Inverse Fourier Transform. Peak values ​​are searched in the cepstral domain to extract the frame-by-frame fundamental frequency value. After smoothing with 5-point or 7-point median filtering, the fundamental frequency value corresponding to each syllable core is calculated. By comparing the absolute value of the fundamental frequency difference between adjacent syllables, points with a change amplitude exceeding a preset abrupt change threshold of 20Hz to 40Hz are marked as pitch abrupt change points, and the occurrence time of these abrupt change points is recorded.

[0068] The speech signal is subjected to a bandpass filter from 3kHz to 8kHz. The percentage of short-time energy in this frequency band relative to the total energy of the band is calculated. After smoothing by a 5-point or 7-point moving average, the time-series curve of breath noise intensity is obtained, which reflects the change in the intensity of breathing noise.

[0069] By detecting silent gaps between adjacent syllables using short-time energy and zero-crossing rate dual thresholds, the duration of silent segments is accurately measured. Gaps in the range of 50ms to 200ms are marked as micro-pauses. The frequency of micro-pauses in each sentence and their distribution patterns at the beginning, transition, or end of the sentence are statistically analyzed to establish a micro-prosodic feature dataset.

[0070] S312. Deterministic statement judgment and smooth rhythm recognition, specifically: When the fundamental frequency curve is detected to be generally flat, the global fundamental frequency standard deviation is below the first threshold of 20Hz to 40Hz, and the mean of the breath sound intensity temporal curve is below the second threshold of 10% to 20% of the average energy of the near-field semantic layer, it indicates that the speaker's voice is stable and even, with no signs of emotional fluctuation or hesitation, and the semantic expression is undoubtedly certain. The implied meaning of the corresponding statement is inferred to be a definite statement, and a definite statement label is assigned. A confidence score is calculated based on the degree of matching between the fundamental frequency standard deviation and the breath sound intensity features and the thresholds, for reference when inserting subsequent paralinguistic tags.

[0071] S313. Hesitant speculation judgment and hesitant prosodic recognition, specifically: When the frequency of micro-pauses exceeds the third threshold of 3 to 5 times per sentence, and these micro-pauses are mostly distributed at the beginning or transition points of sentences rather than in the middle, while the fundamental frequency curve shows a gradual upward trend of 5 Hz to 15 Hz per second rather than a sudden drop or stagnation, it indicates that the speaker is hesitant and lacks fluency, resulting in a lack of certainty in semantic expression. The implied meaning of the corresponding sentences is inferred as hesitant speculation, and a hesitant speculation label is assigned. A confidence score is calculated based on the degree of matching between the micro-pause frequency distribution and the gradual upward trend of the fundamental frequency with the threshold, for reference when inserting uncertain modifiers.

[0072] S314. Judgment of ironic allusions and identification of sharp rhythms, specifically: When a dense occurrence of pitch abrupt changes is detected, with the average time interval between adjacent abrupt changes less than the fourth threshold of 200ms to 400ms, and simultaneously, the breath sound intensity temporal curve shows an instantaneous peak exceeding 50% to 80% of the mean near the pitch abrupt changes (the fifth threshold), it indicates that the speaker's voice is sharp and volatile, accompanied by rapid breathing, and the semantic expression contains irony or suggestive meaning. The implied meaning of the corresponding statement is inferred to be an ironic suggestion, and an ironic suggestion label is assigned. A confidence score is calculated based on the degree of matching between the pitch abrupt change density and the breath sound peak value with the thresholds, for reference when inserting subsequent ironic cue tags.

[0073] S315. Implied meaning type label output and confidence calculation, specifically: The judgment results from steps S312 to S314 are summarized, and the final implied meaning type label is output for each statement in the near-field semantic layer. The label category includes one of three types: deterministic statement, hesitant speculation, or ironic implication. Simultaneously, a confidence score corresponding to this label is output, calculated based on the degree of matching of multi-dimensional micro-prosodic features. The score is calculated by weighting and summing multiple features, such as the fundamental frequency standard deviation, breath sound intensity, frequency and distribution of micro-pauses, and density of pitch abrupt changes, against their respective thresholds. The implied meaning type label and confidence score are organized into structured metadata, bound to the timestamp and text content of the corresponding statement, for use in subsequent sub-language tag insertion steps, thus achieving the transcoding of implied meaning text rhetoric.

[0074] S32. Semantic encoding of mid-to-far field soundscape and generation of ambient text, specifically: S321. Extract the environmental sudden sound type classification results and event start and end times obtained in step S13 of the mid-range event layer. For each sudden sound type, call the pre-built sentiment semantic mapping dictionary to perform lexical transformation. Map sudden loud noises such as bangs, impacts, and collapses to words describing near-field tension; map diffuse noises such as crowd shouts, screams, and commotions to words describing mid-range chaos; and map mechanical malfunction sounds, cracking sounds, and sharp friction sounds to words describing impending crisis. Establish a one-to-one mapping relationship between environmental sudden sound types and sentiment semantic tags.

[0075] S322. Extract the noise power spectrum evolution trend data obtained in step S13 from the background layer, focusing on analyzing the power change curve over time in the low-frequency band from 20Hz to 250Hz. Calculate the average power using a sliding window of 1s to 2s in length, and calculate the power slope by the power difference between adjacent windows. When a positive power slope is detected and it maintains an upward trend for more than 5s, it is determined to be a low-frequency power increase state, which is mapped to generate a description of a sense of oppression in the distance. When the power is detected to remain on a high-value plateau with fluctuations less than 3dB, it is determined to be a stable high-value state, which is mapped to generate a description of a continuous sense of envelopment. When a negative power slope is detected and the power decrease exceeds 6dB, it is determined to be a low-frequency sudden drop state, which is mapped to generate a description of a sudden silence.

[0076] S323. Align and fuse the emotional semantic mapping results of the mid-range event layer with the spatial distance encoding results of the far-range base layer based on the precise timestamp. Generate a spatial atmosphere description text containing three dimensions: near-range tension, mid-range chaos, and far-range oppression through template splicing or natural language generation algorithms. This text is used to fill the blank description space in the news text skeleton to enhance the on-site immersion and spatial depth of the manuscript.

[0077] S33. Integration of Contextual Elements and Optimization of Sentence Rhythm, specifically: S331. Retrieve the implicit meaning type label, confidence score, and timestamp information of the corresponding statement output from step S31, and perform the conversion of implicit meaning into paralinguistic markers. For statements judged as definitive statements, convert them into definitive reinforcement words such as explicitly stating, emphasizing, and confirming; for statements judged as hesitant speculations, convert them into uncertain modifiers such as seemingly, possibly, and probably; for statements judged as ironic hints, convert them into ironic prompts such as so-called, quotation marks, and notes. Based on the timestamp information, locate the climax statement node to which these statements belong in the news text skeleton, and precisely insert the converted paralinguistic markers into the corresponding statements to correct the definitive expression or emotional tendency of the original statements and enhance the critical depth of the text.

[0078] S332. Retrieve the spatial atmosphere description text generated in step S32. Based on the timestamp and position index of the blank description position in the news text skeleton, fill the blank description position with the generated description text of close-up tension, mid-range chaos, and long-range oppression. Insert environmental atmosphere rendering content at the narrative interruption position to connect the context narrative transition.

[0079] S333. Perform sentence rhythm optimization on the introductory paragraph clusters in the news text skeleton based on the rhetorical rhythm template constructed in step S23. According to the tense short sentence mapping mode, semantic units corresponding to rapid pitch rise and short pause duration are converted into short sentences. According to the relaxed long sentence mapping mode, semantic units corresponding to moderate pitch and moderate pause duration are converted into complex long sentences containing multiple modifiers or clause structures. According to the suspenseful sentence breaking mapping mode, semantic units corresponding to sudden pitch drop and abrupt pause duration are marked with sentence breaks or ellipses at the pause positions. By precisely adjusting the short sentence density and sentence breaking positions, a paradigm shift from auditory rhythm to text rhythm is achieved, generating a structured news text that is context-adaptive and has an auditory immersive feel.

[0080] S34. Construction of soundscape intertextual index and establishment of bidirectional source tracing links, specifically: S341. Document element traversal and character position encoding, specifically: The text content of structured news articles is parsed using a document object model, identifying and traversing all spatial atmosphere description text fragments and sub-linguistic marker elements. For each identified text fragment or marker element, its starting and ending character offsets within the entire article string are calculated. The starting offset corresponds to the index position of the first character of the element in the entire text, and the ending offset corresponds to the index of the next position after the last character of the element. Simultaneously, the paragraph number and sentence number of the element are recorded, establishing a multi-dimensional positional coordinate system to ensure accurate positioning of specific text areas within the article, providing text-side positional anchors for establishing a two-way mapping relationship.

[0081] S342. Generate source backtracking and soundscape spatiotemporal alignment, specifically: Soundscape layering is performed based on the source attributes of the spatial atmosphere description text or paralinguistic markers. For spatial atmosphere description text, the process traces back to step S32, generating the evolution trend of the noise power spectrum of the mid-range event layer environmental bursts or the far-range base layer noise, extracting the corresponding soundscape layer identifier (i.e., the mid-range event layer or the far-range base layer), as well as the start and end timestamps of the environmental bursts or the time window boundaries for noise power spectrum analysis. For paralinguistic markers, the process traces back to step S31, generating the inference results of the near-range semantic layer implied meaning, extracting the near-range semantic layer identifier and the start and end timestamps of the corresponding sentences, and establishing a mapping relationship between text elements and the spatiotemporal coordinates of the original audio.

[0082] S343. Establishment of bidirectional mapping entries for spatial atmosphere description, specifically: For each text describing the spatial atmosphere, a first bidirectional mapping entry is established. This entry records the start and end character offsets obtained in step S341, and the soundscape layer identifier determined by backtracking in step S342, i.e., the mid-range event layer or the far-range base layer. When the source is the mid-range event layer, the start and end timestamps of the specific environmental burst sound corresponding to the text are recorded, accurate to the millisecond level. When the source is the far-range base layer, the start and end times of the analysis time window of the noise power spectrum evolution trend corresponding to the text are recorded. This entry establishes a forward link and a reverse tracing path from a specific text range in the manuscript to a specific soundscape layer and time segment in the original audio.

[0083] S344. Establishment of bidirectional mapping entries for secondary language tags, specifically: For each paralinguistic marker, a second bidirectional mapping entry is established. This entry records the start and end character offsets of the marker in the text, obtained in step S341, and the near-field semantic layer identifier determined by backtracking in step S342. Simultaneously, the precise start and end timestamps of the corresponding speech sentences in the near-field semantic layer upon which the paralinguistic marker is based are recorded. These timestamps are obtained through the timestamp information generated by speech recognition in step S21, ensuring a one-to-one correspondence between the marker and the time period of the speaker's microprosodic features in the original audio. This entry establishes a bidirectional index relationship from paralinguistic rhetorical markers to the time period of near-field speech evidence.

[0084] S345. Soundscape Intertextuality Index Organization and Metadata Storage, specifically: All first and second bidirectional mapping entries are sorted and organized according to the text paragraph order or timestamp order to construct a structured data format soundscape intertextual index. This index is stored in a metadata file associated with the structured news text, recording the character position, soundscape layer identifier, and timestamp information of each entry in JSON or XML format. Hyperlinks are embedded in each spatial atmosphere description text and sub-language marker in the text display interface. When a user clicks, the parser queries the soundscape intertextual index to obtain the corresponding soundscape layer identifier and time range, calls the audio player to locate and play the acoustic content of the corresponding soundscape layer and time segment in the original audio stream, realizing the visual traceability and verification of descriptive text and audio evidence.

[0085] Example 2 like Figure 2 As shown, an AI multimodal audio acquisition and news transcript automatic generation system is used to implement an AI multimodal audio acquisition and news transcript automatic generation method, specifically including: A distributed microphone array is used to acquire multi-channel raw audio streams and perform timestamp synchronization and acoustic calibration on each channel; The soundscape topology decomposition module is used to decompose the synchronized calibrated multi-channel audio stream into a near-field semantic layer, a mid-field event layer, and a far-field base layer based on independent component analysis. It also extracts the speech content stream and acoustic localization coordinates of the near-field semantic layer, the environmental burst sound types and time-frequency characteristics of the mid-field event layer, and the noise power spectrum evolution trend of the far-field base layer. The auditory suspense map construction module is used to identify the building-up mode, release mode and suspension mode respectively based on the gradation slope of the energy envelope of the mid-field event layer, the time delay of the speech response in the near-field semantic layer and the length of the silence gap between adjacent sentences, generate an auditory suspense dependency matrix, and combine the spatial orientation labels, reverberation time estimates and interlayer energy coupling coefficients of the three sound scenes into a sound scene topology description vector. The cross-modal reference resolution module is used to perform speech recognition on the speech content stream of the near-field semantic layer to generate the original text sequence, extract indicator pronouns and spatially ambiguous references from the original text sequence, and perform cross-modal reference resolution by combining the spatial orientation labels in the sound scene topology description vector with the acoustic positioning coordinates to generate the semantic text sequence after reference resolution. The rhetorical prosody transcoding module is used to extract prosodic features of the close-up semantic layer, including pitch curves, pause duration distribution, speech rate change gradient and breath sound features. Based on the prosodic features, a rhetorical prosodic template is constructed, and the semantic text sequence is reconstructed into a news text skeleton with suspense levels based on the auditory suspense dependency matrix. The implied meaning inference module is used to extract micro-pause patterns, pitch change points and breath sound intensity from the close-up semantic layer to infer the speaker's implied meaning type, including definitive statements, hesitant speculations or ironic hints. The soundscape semantic coding module is used to perform soundscape semantic coding on the mid-range event layer and the far-range base layer, transforming the environmental sudden sound types and noise power spectrum evolution trends into spatial atmosphere description text; The document generation and indexing module is used to convert implied meaning types into corresponding sentences in the news text skeleton by inserting them into the sub-language tags, fill the blank description positions in the news text skeleton with spatial atmosphere description text, perform sentence rhythm optimization based on rhetorical rhythm templates on the introductory paragraph clusters, and generate structured news documents with soundscape intertextuality indexes. The soundscape intertextuality index is a two-way mapping table that records the start and end positions of the text and the corresponding soundscape layer identifiers and timestamp ranges in the original audio stream.

[0086] As can be seen from the above description, the embodiments of the present invention achieve the following technical effects: By deconstructing the raw audio stream into a near-field semantic layer, a mid-field event layer, and a far-field base layer with spatial topological relationships, this method overcomes the limitations of traditional techniques that treat audio as a one-dimensional speech stream. This layered analysis not only captures the speech content of highly directional target sound sources but also extracts the spatial orientation of diffuse environmental bursts and the dynamic evolution of persistent environmental noise bases. This enables the system to identify spatial acoustic events such as a distant alarm gradually increasing or a sudden commotion behind it. Combined with an auditory suspense map constructed based on acoustic energy flow patterns, acoustic suspense patterns such as build-up, release, and suspension are mapped to textual build-up clusters, climax nodes, and blank spaces. This transforms the generated news articles from a flat, narrative account of time into a narrative rhythm and dramatic tension controlled by a professional journalist, significantly enhancing the immersive experience and reading appeal of automated news products.

[0087] To address the issue of textual referential fragmentation caused by the reliance on shared acoustic context for demonstrative pronouns such as "this" and "that" in spoken language, this paper integrates spatial location labels from soundscape topology description vectors with acoustic positioning coordinates. This precisely maps ambiguous referents to spatial location descriptions such as the direction from which an alarm sound comes or the location of a noisy crowd, eliminating semantic ambiguity in automated transcription. Simultaneously, by analyzing pitch curves, pause duration distributions, and speech rate gradients to construct rhetorical prosodic templates, this paper encodes auditory compactness into short sentence clusters, semantic pauses into paragraph boundaries, and breathy vocal features into emotional cues. This achieves a paradigm shift from auditory timeline logic to textual spatial hierarchy logic, ensuring that the generated news skeleton conforms to professional editing standards of an inverted pyramid structure and resolving the structural contradiction between linear audio narrative and news reading habits.

[0088] By inferring the speaker's definitive statements, hesitant speculations, or ironic hints through micro-prosodic features, including micro-pause patterns, pitch abrupt changes, and breath intensity, these implicit meanings are transformed into paralinguistic markers such as "seems," "emphasizes," or "so-called." This allows the generated news articles to not only state facts but also convey the speaker's attitude, credibility, and underlying intentions, giving the automatically generated content a critical perspective akin to investigative reporting. Simultaneously, by establishing bidirectional links between the spatial atmosphere description text and specific soundscape layers and time segments in the original audio stream, a soundscape intertextual index is formed. This allows editors and readers to directly replay the original acoustic evidence by clicking on environmental descriptions in the text, verifying the accuracy of descriptions such as a silence falling or a gradually increasing background alarm. This establishes a verifiable news generation paradigm, fundamentally solving the credibility crisis and fact-checking challenges of AI-generated news.

[0089] The embodiments and / or implementation methods described above are merely preferred embodiments and / or implementation methods for implementing the technology of the present invention, and are not intended to limit the implementation methods of the technology of the present invention in any way. Any person skilled in the art may make some modifications or alterations to other equivalent embodiments without departing from the scope of the technical means disclosed in the present invention, but these should still be regarded as the technology or embodiments that are substantially the same as the present invention.

[0090] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. The above descriptions are only preferred embodiments of this application. It should be noted that due to the limitations of written expression, while there are objectively infinite specific structures, those skilled in the art can make several improvements, modifications, or changes without departing from the principles of this application, and can also combine the above technical features in an appropriate manner. These improvements, modifications, changes, or combinations, or the direct application of the inventive concept and technical solution to other situations without modification, should all be considered within the scope of protection of this application.

Claims

1. A method for AI-powered multimodal audio acquisition and automatic news text generation, characterized in that, include: The acquired audio stream is parsed into a three-layer soundscape, features are extracted to construct an auditory suspense map to identify suspense patterns, and topological vectors and suspense matrices are output. The system identifies near-field speech to generate original text, combines the topological vector to resolve referentials to generate semantic text, analyzes prosody to construct rhetorical templates, and reconstructs a text skeleton with foreshadowing clusters, climax nodes, and blank spaces based on the suspense matrix. Based on the inference of implied meanings from micro-prosody, the semantics of the mid-to-far view layer are transformed into atmospheric text to fill the blank spaces, the implied meanings are transformed into paralinguistic markers to insert climax nodes, the rhythm of the prelude cluster sentences is optimized, and news articles with sound and scene intertextuality indexes are generated.

2. The AI ​​multimodal audio acquisition and automatic news transcript generation method according to claim 1, characterized in that, The acquired audio stream is parsed into a three-layer soundscape, and features are extracted to construct an auditory suspense map to identify suspense patterns. The output includes a topological vector and a suspense matrix, specifically: At least four channels of raw audio stream are acquired using a distributed microphone array, and timestamp synchronization and acoustic calibration are performed on the audio signal of each channel. The synchronized calibrated multi-channel audio stream is input into the pre-trained sound scene topology decomposition network, which is then decomposed into a near-field semantic layer, a mid-field event layer, and a far-field basal layer based on independent component analysis. Beamforming and speech activity detection are performed on the near-field semantic layer to extract the speech content stream, and the spatial azimuth and distance estimates of each speech segment are calculated using the time difference of arrival algorithm as acoustic positioning coordinates; event boundary detection and type classification are performed on the mid-field event layer to extract the event start and end time, peak frequency and energy envelope of burst sounds as environmental burst sound type and time-frequency features; power spectral density estimation is performed on the far-field base layer to extract the power change curves of the low-frequency, mid-frequency and high-frequency bands as the evolution trend of noise power spectrum; Based on the energy gradient slope of the mid-range event layer, the near-range speech response delay, and the silence gaps between sentences, the building-up, release, and suspension patterns are identified to generate an auditory suspense dependency matrix. The spatial orientation labels, reverberation time estimates, and interlayer energy coupling coefficients of the near-field semantic layer, mid-field event layer, and far-field base layer are combined into a soundscape topology description vector, and the soundscape topology description vector, acoustic positioning coordinates, and auditory suspense dependency matrix are output.

3. The AI ​​multimodal audio acquisition and automatic news manuscript generation method according to claim 2, characterized in that, The synchronized, calibrated multi-channel audio stream is input into a pre-trained soundscape topology decomposition network, which is then decomposed into a near-field semantic layer, a mid-field event layer, and a far-field base layer based on independent component analysis. Specifically, this includes: A short-time Fourier transform is performed on the synchronized and calibrated multi-channel audio stream to convert it to the time-frequency domain and obtain the multi-channel complex spectrum matrix. The complex spectrum matrix is ​​input into a sound source separation network based on deep independent component analysis. The sound source separation network estimates the unmixing matrix iteratively by minimizing the mutual information loss between each output channel and outputs multiple independent sound source components. Spatial localization features are extracted for each independent sound source component, and the phase difference and amplitude difference of each component between different microphone channels are calculated to obtain the estimated spatial azimuth angle of each component. Based on the temporal stability of the estimated spatial azimuth angle and the time-frequency sparsity of the sound source components, the sound is classified into a near-field semantic layer, a mid-field event layer, and a far-field base layer. The three-layer sound scene signals are output after decomposition, and a corresponding spatial orientation label and inter-layer energy coupling coefficient are added to each layer signal.

4. The AI ​​multimodal audio acquisition and automatic news manuscript generation method according to claim 3, characterized in that, The process involves identifying build-up, release, and suspension patterns based on the energy gradient slope of the mid-range event layer, the near-range speech response delay, and the silence intervals between sentences, generating an auditory suspense dependency matrix. Specifically, this includes: The energy envelope of the mid-scene event layer is subjected to temporal smoothing, and the energy slope within the sliding window is calculated. When the energy slope continuously exceeds a preset rising threshold and the duration is greater than a first duration threshold, it is marked as a potential accumulation candidate interval. The start time of the first speech segment in the near-field semantic layer after the end of the potential candidate interval is detected, and the time difference between the end time of the potential candidate interval and the start time of the speech segment is calculated. If the time difference is less than the second duration threshold, the potential candidate interval is finally marked as the potential mode. Burst pulse detection is performed on the mid-range event layer. When an event with an energy peak exceeding a preset peak threshold is detected, the peak time of the event is recorded. The response delay between the peak time and the start time of the next adjacent speech segment in the near-range semantic layer is calculated. If the response delay is less than a third duration threshold, it is marked as release mode. The length of the silent segment between adjacent statements in the near-field semantic layer is detected. When the length of the silent segment exceeds the fourth duration threshold and the average energy of the mid-field event layer in the corresponding time period is lower than the silent energy threshold, it is marked as a suspended mode. Record the start timestamp, end timestamp, and associated soundscape layer identifier for each mode, and calculate the temporal dependency weights between the build-up mode and the release mode, and between the release mode and the suspension mode, to generate an auditory suspense dependency matrix.

5. The AI ​​multimodal audio acquisition and automatic news transcript generation method according to claim 4, characterized in that, Recognizing near-field speech to generate original text, combining the aforementioned topological vector to resolve referentiality and generate semantic text, analyzing prosody to construct a rhetorical template, and reconstructing a text skeleton with foreshadowing clusters, climax nodes, and blank spaces based on the aforementioned suspense matrix, specifically including: Speech recognition based on an end-to-end deep neural network is performed on the speech content stream of the near-field semantic layer to generate a raw text sequence with timestamps; Extract indicator pronouns and spatially ambiguous references from the original text sequence, align them with spatial orientation labels and acoustic positioning coordinates in the soundscape topology description vector, calculate the spatial orientation angles they point to, and generate a semantic text sequence after referential resolution. Extract the prosodic features of the near-field semantic layer, and construct a rhetorical prosodic template based on the numerical range of the prosodic features; Based on the auditory suspense dependency matrix, semantic units corresponding to the building-up mode timestamp are aggregated into a prelude paragraph cluster, semantic units corresponding to the release mode timestamp are marked as climax statement nodes, and semantic units corresponding to the suspension mode timestamp are marked as blank description positions. The logical relationship between the prelude paragraph cluster, climax statement nodes and blank description positions is established using the dependency edges in the auditory suspense dependency matrix, thus reconstructing the original time-linear semantic text sequence into a news text skeleton.

6. The AI ​​multimodal audio acquisition and automatic news transcript generation method according to claim 5, characterized in that, Extracting demonstrative pronouns and spatially ambiguous references from the original text sequence, aligning them with spatial orientation labels and acoustic positioning coordinates in the soundscape topology description vector, calculating the spatial azimuth angles of the points of reference, and generating a semantic text sequence after referential resolution, specifically including: The original text sequence is subjected to part-of-speech tagging and dependency parsing to identify the set of demonstrative pronouns and the set of spatial fuzzy referents. For each identified demonstrative pronoun, its position index and grammatical role in the text are extracted. At the same time, based on the timestamp of the sentence in which the demonstrative pronoun is located, the azimuth and distance estimates of each sound source within the corresponding time window are retrieved from the acoustic positioning coordinates. The spatial orientation labels in the soundscape topology description vector are divided into several sector regions, and each sector region is associated with a predefined orientation description phrase; the sector region to which the sound source azimuth angle corresponding to the pronoun belongs is calculated, and the orientation description phrase associated with the sector region is used as the spatial mapping result of the pronoun; When the pronoun is a personal demonstrative pronoun and the sound source azimuth points to a specific speaker, the pronoun is replaced with the corresponding speaker's identity identifier or role description, generating entity completion information; the mapped spatial orientation description or entity completion information is backfilled into the position of the corresponding pronoun in the original text sequence, the original pronoun is deleted, and a semantic text sequence after dereference resolution is generated.

7. The AI ​​multimodal audio acquisition and automatic news transcript generation method according to claim 6, characterized in that, Extracting the prosodic features of the near-field semantic layer, and constructing a rhetorical prosodic template based on the numerical range of the prosodic features, specifically including: The fundamental frequency trajectory is extracted from the speech content stream of the near-field semantic layer, and the frame-by-frame pitch value is obtained by cepstral method. After median filtering and smoothing, the pitch curve is generated, and the global mean, standard deviation and maximum slope of the rising segment of the pitch curve are calculated. The system detects the length of silent gaps between adjacent words, statistically analyzes the pause duration distribution within each sentence, calculates the ratio of short to long pauses and the duration of the longest pause, calculates the ratio of the number of syllables to the window duration according to a fixed time window, obtains the speech rate change gradient sequence, and records the local peaks and valleys of the speech rate; performs energy envelope analysis on the high-frequency band of the speech signal, and extracts the average intensity and peak intensity of breathing noise as breath sound features. When the maximum slope of the rising segment of the pitch curve exceeds the first threshold, the proportion of short pauses in the pause duration distribution exceeds the second threshold, and the speech rate change gradient is greater than the third threshold, a tension-short sentence mapping pattern is constructed, indicating that the corresponding semantic unit is converted into a short sentence. When the standard deviation of the pitch curve is lower than the fourth threshold, the longest pause duration is between the fifth and sixth thresholds, and the breath sound intensity is lower than the seventh threshold, a soothing-long sentence mapping pattern is constructed to indicate conversion to a compound long sentence. When the pitch curve drops sharply and the drop exceeds the eighth threshold, the longest pause duration exceeds the ninth threshold, and the breath sound intensity increases suddenly, a suspense-sentence-break mapping mode is constructed to indicate the insertion of a sentence break mark or ellipsis at the pause position.

8. The AI ​​multimodal audio acquisition and automatic news text generation method according to claim 7, characterized in that, The method of inferring implied meaning based on micro-prosody, semanticizing mid-to-far-field layers into atmospheric text to fill blank spaces, converting implied meaning into paralinguistic markers to insert climax nodes, optimizing the rhythm of prelude cluster sentences, and generating news articles with soundscape intertextuality indexes specifically includes: Micro-pauses, pitch abrupt changes, and breath sound intensity are extracted from the near-field semantic layer, and inferred from these to belong to definitive statements, hesitant speculations, or ironic hints; Emotional semantic mapping is performed on the environmental sudden sound types of the mid-range event layer, mapping plosive sounds to near-field tension and sudden noise from crowds to mid-range chaos. Spatial distance encoding is performed on the noise power spectrum evolution trend of the far-field base layer, mapping the rise of low-frequency power to far-field oppression, and combining them to generate spatial atmosphere description text. The implied meaning type is converted into a secondary language tag and inserted into the corresponding statement of the climax statement node. The spatial atmosphere description text is filled into the blank description space, and the sentence rhythm is optimized for each sentence in the preparatory paragraph cluster according to the rhetorical rhythm template. A soundscape intertextual index, or bidirectional mapping table, is established for each spatial atmosphere description text and sub-language mark. This table records the start and end positions of the text and the corresponding soundscape layer identifiers and timestamp ranges in the original audio stream, outputting a structured news article.

9. The AI ​​multimodal audio acquisition and automatic news text generation method according to claim 8, characterized in that, Extracting micro-pauses, pitch shifts, and breath sounds from the near-field semantic layer, and inferring whether they constitute definitive statements, hesitant speculations, or ironic hints, specifically including: The fundamental frequency curve is extracted from the speech content stream of the near-field semantic layer, the fundamental frequency value corresponding to each syllable is calculated, and the points where the fundamental frequency change between adjacent syllables exceeds the preset abrupt change threshold are detected and marked as pitch abrupt change points; high-frequency energy analysis is performed on the speech signal, the energy ratio is calculated, and the breath sound intensity time sequence curve is obtained after smoothing; silent gaps between adjacent syllables are detected, micro-pauses are marked, and the frequency and distribution pattern of micro-pauses within each sentence are statistically analyzed; When the fundamental frequency curve is generally flat, the global fundamental frequency standard deviation is lower than the first threshold, and the mean of the breath sound intensity time series curve is lower than the second threshold, the implied meaning of the corresponding statement is inferred to be a deterministic statement. When the frequency of the micro-pauses exceeds the third threshold, and the micro-pauses are mostly distributed at the beginning or turning point of the statement, and the fundamental frequency curve shows a gradual upward trend, the implied meaning of the corresponding statement is inferred to be a hesitant speculation. When the pitch change points appear densely and the average distance between adjacent change points is less than the fourth threshold, and the breath sound intensity time sequence curve shows an instantaneous peak value exceeding the fifth threshold near the pitch change point, the implied meaning of the corresponding statement is inferred to be an ironic suggestion. Output the implied meaning type label and corresponding confidence score for each statement, for subsequent use in inserting sub-language tags.

10. The AI ​​multimodal audio acquisition and automatic news text generation method according to claim 9, characterized in that, A soundscape intertextual index, or bidirectional mapping table, is established for each spatial atmosphere description text and sublingual marker. This table records the start and end positions of the text and the corresponding soundscape layer identifiers and timestamp ranges in the original audio stream. The output is a structured news article, specifically including: Traverse each spatial atmosphere description text and each secondary language mark in the structured news article to obtain the start and end character positions of the text or mark in the article; Based on the source of the spatial atmosphere description text or sub-language markers, trace back to the soundscape layering identifiers and timestamp ranges in the corresponding original audio stream; Establish a first bidirectional mapping entry for each spatial atmosphere description text, recording the start character position, end character position, corresponding soundscape layer identifier, and the start and end times of sudden environmental sounds in the mid-range event layer or the time window of the noise power spectrum evolution trend in the far-range base layer. A second bidirectional mapping entry is established for each sub-language tag, recording the starting character position, the ending character position, the corresponding near semantic layer identifier, and the start and end timestamps of the corresponding statement in the near semantic layer; All bidirectional mapping entries are organized into the aforementioned soundscape intertextual index and stored in a metadata file associated with the structured news article.

Images