Method and system for mobile sound source positioning and identity coherent identification in multi-node pickup network

Abstract

The application provides a mobile sound source positioning and identity coherent identification method and system in a multi-node pickup network, and relates to the technical field of distributed acoustic sensing and intelligent audio processing. The method comprises the following steps: collecting audio in real time and estimating local positioning and signal quality; dynamically selecting a main node, and performing cross-fading splicing output of smooth audio through time delay alignment and phase difference compensation when the node is switched; extracting voiceprint features in real time, and synchronously transmitting historical voiceprint vectors and identity labels to a relay node at the switching moment; the relay node calculates the Euclidean distance of new and old features to continue the identity label, and weights and fuses the updated feature template according to a smoothing factor; the positioning information and identity recognition results of each node are fused to construct a joint observation, a filtering algorithm is used for recursive estimation, and a continuous moving trajectory with a unique identity label is output in real time. The application realizes real-time deep coupling of spatial trajectory and identity features, and improves the accuracy of cross-node identity continuation.

Description

Technical Field

[0001] This invention relates to the field of distributed acoustic sensing and intelligent audio processing technology, and more specifically, to a method and system for locating and identifying mobile sound sources in a multi-node sound pickup network. Background Technology

[0002] With the rapid evolution of intelligent security, human-computer interaction, and IoT technologies, the need for real-time location and identification of mobile speakers in continuous, large-scale spatial scenarios is becoming increasingly urgent. These scenarios typically include airport waiting halls, large commercial complexes, convention centers, and underground utility tunnels. Their common characteristics include extremely vast spatial areas, often obstructed views by pillars or displays, dynamically changing speaker positions, and the reliance on a large number of distributed audio pickup devices working in a networked collaborative manner. In complex acoustic environments, ensuring that the spatial trajectory of a target is not lost during wide-area movement and that its identity label remains consistently accurate has become a core challenge that urgently needs to be overcome in the field of acoustic sensing.

[0003] To address the aforementioned need for sound perception in a wide-area continuous space, traditional technologies attempt to employ centralized microphone array solutions for sound source localization and identification. These solutions typically mount microphone arrays on a single fixed node or mobile robot platform, utilizing causal convolutional recurrent neural networks to predict the spatial characteristics of multiple sound sources, and combining this with beamformers to separate target sound sources and extract voiceprints for identity matching. However, the physical aperture of a single microphone array has inherent limitations, and its effective high-quality sound pickup range is usually limited to within a few meters. As the speaker moves away from the array center, the audio signal-to-noise ratio decreases exponentially, leading to a severe deterioration in localization accuracy and voiceprint recognition rate in far-field environments. Furthermore, in complex indoor environments, obstacles can easily create non-line-of-sight propagation paths between the sound source and a single microphone array. Once the direct sound is blocked, the localization algorithm will produce irreversible and significant errors, and the tracking trajectory can easily be completely lost when the speaker turns around or pauses briefly.

[0004] To address the challenge of wide-area coverage, some research has shifted towards fiber optic monitoring schemes based on distributed acoustic sensing. These schemes primarily acquire vibration and sound field information over a spatial range by laying long-distance sensing fibers and demodulating backscattered Rayleigh light signals. While this approach offers advantages in coverage, its working mechanism limits its sensitivity to high-energy structured vibration sources such as mechanical excavation and vehicle movement. For weak aerodynamic acoustic signals like human voices, fiber optic sensing suffers from severe inadequacy. More critically, as acoustic signals penetrate multiple layers of media to reach the fiber optic cable, high-frequency characteristic bands containing rich identity information undergo severe distortion and attenuation. This results in the system only being able to identify the location and general type of the event, completely losing the ability to distinguish the specific identity of similar sound-emitting entities, and failing to achieve biometric-level voiceprint authentication in multi-target scenarios.

[0005] In the area of ​​distributed node collaboration, the industry has also proposed some sound source localization systems based on multi-node microphone array networks. These systems employ multiple spatially distributed linear arrays. Each node independently estimates its two-dimensional direction of arrival (DOA) and then exchanges spatial geometric information between nodes through an adaptive diffusion strategy, ultimately calculating the three-dimensional spatial location of the sound source. The fatal flaw of this type of system lies in its technical approach, which focuses solely on pure sound source geometry calculation, achieving spatial coordinate output while completely ignoring the feature analysis of the sound content. When a speaker moves from the coverage area of ​​one pickup node to the coverage area of ​​an adjacent node, the system lacks a cross-node biometric association handshake mechanism, causing the identity tracking link established by the preceding node to be forcibly severed. When a new node captures the speaker, it can only forcibly classify them as a completely new target, resulting in a large number of broken, abrupt, and even overlapping invalid trajectories from a global monitoring perspective.

[0006] To compensate for the inability of pure positioning systems to identify individuals, another technical approach proposes a post-processing tracking scheme based on voiceprint embedding. This scheme employs a serial processing architecture of tracking first and then assigning identity. First, traditional multi-node positioning technology is used to passively collect and generate complete spatiotemporal trajectory segments. After trajectory generation, a beamformer is used to separate the audio from each trajectory segment, and finally, voiceprint features are extracted and compared with a registration database for identity assignment. This asynchronous post-processing architecture fundamentally sacrifices the system's real-time performance, failing to meet the millisecond-level response requirements of intelligent monitoring applications. Because voiceprint feature extraction is performed in isolation on fragmented spatiotemporal trajectory segments, there is a lack of dynamic calibration of the same speaker's acoustic features across time periods. This leads to significant differences in feature vectors extracted from adjacent trajectory segments of the same target when environmental noise changes, resulting in identity matching failures and trajectory chain breaks. Ultimately, the bottlenecks of existing technologies stem from the failure to achieve deep parallel coupling between spatial tracking and voiceprint identity authentication at the underlying architecture level, lacking a dynamic adaptive mechanism capable of seamlessly transferring, calibrating, and perpetuating identity context information among distributed nodes. Summary of the Invention

[0007] The present invention aims to solve at least one of the aforementioned technical problems existing in the prior art.

[0008] Therefore, the first aspect of the present invention provides a method for locating and identifying mobile sound sources in a multi-node sound pickup network.

[0009] The second aspect of the present invention provides a mobile sound source localization and identity coherent recognition system in a multi-node sound pickup network.

[0010] This invention provides a method for mobile sound source localization and coherent identity recognition in a multi-node sound pickup network, wherein the multi-node sound pickup network includes multiple spatially discrete sound pickup nodes, and the method includes: Each of the aforementioned sound pickup nodes acquires audio signals in real time and estimates the location information of moving sound sources within the coverage area and the signal quality of the corresponding audio based on a local sound source localization algorithm; Based on the location information and signal quality reported by each pickup node, the optimal pickup node is dynamically selected as the first pickup node and the primary audio is acquired. At the same time, the backup audio of the candidate second pickup node is cached. When the movement of the moving sound source is detected, which causes the primary pickup node to switch from the first pickup node to the second pickup node, the primary audio and the backup audio are time-aligned using cross-correlation delay estimation. After phase difference compensation by combining short-time Fourier transform, cross-gradient splicing is performed to output a smooth transition audio. Voiceprint features are extracted in real time from the output continuous audio stream to generate a high-dimensional voiceprint feature vector of the current speaker; during the switching of the pickup node, the historical voiceprint feature vector extracted by the first pickup node and the corresponding identity tag are synchronously transmitted to the second pickup node. The second voice pickup node calculates the Euclidean distance between the newly extracted voiceprint feature vector and the received historical voiceprint feature vector. If the Euclidean distance is less than a preset distance threshold, the identity association is determined to be successful, and the identity label continues on the second voice pickup node. After the identity association is successful, the high-confidence voiceprint features extracted subsequently are weighted and fused with the historical voiceprint feature vector according to a preset smoothing factor, and the voiceprint feature template of the current speaker is updated based on the fusion result. By integrating the location information reported by each sound pickup node and the results of identity continuation and recognition, the spatial coordinate observation and voiceprint comparison score are combined to form a joint observation. A filtering algorithm is used to recursively estimate the three-dimensional position, motion state and identity association state of the mobile sound source, and outputs a continuous movement trajectory with a unique identity tag in real time.

[0011] The method for mobile sound source localization and coherent identity recognition in a multi-node sound pickup network according to the above-described technical solution of the present invention may further have the following additional technical features: In the above technical solution, the step of dynamically selecting the optimal pickup node as the first pickup node based on the positioning information and signal quality reported by each pickup node includes: A normalized weighted score is calculated based on the signal-to-noise ratio and energy value of the audio corresponding to each pickup node; Determine if a primary pickup node exists. If it does, calculate the difference between the score of the highest-scoring node and the score of the current primary pickup node. When the highest-scoring node is different from the current primary pickup node and the score difference is greater than a preset switching threshold, pickup node switching preparation is triggered, the highest-scoring node is marked as a candidate second pickup node, and audio cache synchronization of the second pickup node is started.

[0012] In the above technical solution, after the step of extracting voiceprint features from the output continuous audio stream in real time, the method further includes: When the mobile sound source stops emitting sound and enters an intermittent period, the latest voiceprint feature vector extracted before the mobile sound source stopped and the corresponding identity tag are stored in the short-term voiceprint feature cache area. When a vocal segment is re-detected in the sound pickup network and candidate voiceprint features are extracted, the similarity between the candidate voiceprint features and the features stored in the short-term voiceprint feature cache is calculated first. If the feature similarity is greater than the preset recapture threshold, the original identity label is maintained to achieve recapture of the mobile sound source identity after the interval.

[0013] In the above technical solution, when the same mobile sound source is captured by multiple sound pickup nodes at the same time, the candidate speech segments are spatially clustered and merged into the same candidate target based on the positioning information reported by each sound pickup node. Each sound pickup node that captures the candidate target calculates the feature similarity between the current speech segment and each candidate identity, and generates a node confidence weight by combining the signal quality of the audio corresponding to the node. Based on the feature similarity and the node confidence weight, a collaborative weighted voting is performed to calculate a comprehensive score, and the candidate identity with the highest comprehensive score that is greater than a preset judgment threshold is determined as the final identity label of the candidate target.

[0014] In the above technical solution, the method for calculating the comprehensive score includes:

[0015] in, The total vote score represents the candidate identity m; This represents the similarity between the current speech features reported by the i-th pickup node and the features of the candidate identity m; The node confidence weight represents the i-th pickup node. The node confidence weight is determined by normalizing the corresponding node's signal-to-noise ratio, speech energy, sound source distance, or localization stability.

[0016] In the above technical solution, the step of using a filtering algorithm to recursively estimate the three-dimensional position, motion state, and identity association state of the mobile sound source includes: using the three-dimensional spatial coordinates and velocity of the mobile sound source as the state vector of the Kalman filter; using the position data reported by each node and the voiceprint comparison score when the identity association is successful as the observation vector for state update; maintaining the continuity of the spatial trajectory by relying on the Kalman prediction results when the mobile sound source pauses briefly; and maintaining the consistency of the identity label by combining the historical voiceprint feature vector.

[0017] In the above technical solution, after performing cross-gradient splicing on the primary audio and the backup audio to output a smooth transition audio, the smooth transition audio is enhanced by a beamforming algorithm or a speech separation algorithm to enhance the target sound source, so that the enhanced pure audio stream is input to the voiceprint feature extraction step.

[0018] In the above technical solution, the microphone nodes are synchronized using a precise time synchronization protocol or a global positioning system to ensure that the first microphone node and the second microphone node have a unified time reference when performing the cross-correlation delay estimation.

[0019] In the above technical solution, during the process of synchronously transmitting the historical voiceprint feature vector extracted by the first voice pickup node and the corresponding identity tag to the second voice pickup node, the transmitted historical voiceprint feature vector is encrypted, and the second voice pickup node decrypts it after receiving it to complete the calculation of the Euclidean distance.

[0020] This invention provides a mobile sound source localization and coherent identity recognition system in a multi-node sound pickup network, comprising: The node-level sound source perception module includes sound pickup nodes deployed discretely in various spaces, used to collect audio signals in real time, and estimate the location information of moving sound sources within the coverage area and the signal quality of the corresponding audio based on the local sound source localization algorithm. The dynamic microphone selection and audio splicing module is used to dynamically select the optimal microphone node as the first microphone node and acquire the primary audio based on the positioning information and signal quality reported by each microphone node, while caching the backup audio of the candidate second microphone node; when the movement of the moving sound source is detected, causing the primary microphone node to switch from the first microphone node to the second microphone node, the module uses cross-correlation delay estimation to time-align the primary audio and the backup audio, and performs phase difference compensation by combining short-time Fourier transform, and then performs cross-gradient splicing to output a smooth transition audio. The voiceprint feature extraction and transmission module is used to extract voiceprint features from the output continuous audio stream in real time and generate a high-dimensional voiceprint feature vector of the current speaker; during the switching of the pickup node, the historical voiceprint feature vector extracted by the first pickup node and the corresponding identity tag are synchronously transmitted to the second pickup node. The cross-node identity continuation and template update module is used to enable the second voice pickup node to calculate the Euclidean distance between its newly extracted voiceprint feature vector and the received historical voiceprint feature vector; if the Euclidean distance is less than a preset distance threshold, the identity association is determined to be successful, and the identity label is continued on the second voice pickup node; after the identity association is successful, the high-confidence voiceprint features extracted subsequently are weighted and fused with the historical voiceprint feature vector according to a preset smoothing factor, and the voiceprint feature template of the current speaker is updated based on the fusion result; The spatiotemporal trajectory and identity joint tracking module is used to integrate the positioning information reported by each of the sound pickup nodes and the identity continuation and recognition results. It combines spatial coordinate observation and voiceprint comparison score to form a joint observation. It uses a filtering algorithm to recursively estimate the three-dimensional position, motion state and identity association state of the mobile sound source, and outputs a continuous movement trajectory with a unique identity tag in real time.

[0021] In summary, due to the adoption of the above-mentioned technical features, the beneficial effects of the present invention are: This invention achieves real-time joint output of sound source spatial location and speaker identity features by deeply coupling and parallelizing the sound source localization unit and the voiceprint recognition unit at the system architecture level. This fundamentally overcomes the severe latency problem caused by the serial post-processing architecture of existing technologies, which first tracks the trajectory and then assigns the identity. The system can output joint data containing the target's identity identifier, three-dimensional spatial coordinates, and timestamps in real time with extremely low latency during speaker movement, effectively meeting the stringent requirements for immediate response in highly dynamic application scenarios such as intelligent security monitoring and real-time human-computer interaction.

[0022] To address the unique physical environment of multi-node network coverage in wide-area spaces, this invention innovatively introduces a seamless cross-node identity continuation mechanism based on voiceprint feature transmission. When target movement is detected, causing a switch in the primary voice pickup node, the system synchronously transmits the high-dimensional voiceprint feature vector extracted by the preceding node to the relay node for priority comparison and association. This mechanism breaks through the limitation of traditional networks that only transmit positional geometric information, completely solving the core pain point of identity tags being easily broken, changed, or misjudged as new targets in cross-regional tracking. This significantly increases the accuracy of cross-node identity continuation from 60% to 70% in existing technologies to over 95%, and significantly reduces the false alarm rate caused by identity fragmentation.

[0023] At the underlying processing level of the audio data stream, this invention constructs a dynamic microphone selection and feature caching smooth splicing mechanism driven by signal quality, effectively ensuring the absolute continuity and purity of the voice input source during the moment of physical node switching. Through time alignment and cross-gradual splicing of the audio from primary and backup nodes, the system successfully eliminates the phase abrupt changes and feature disturbances caused by microphone switching at different physical locations, resulting in a reduction of more than 80% in the amplitude of voiceprint feature fluctuations during node switching. This not only provides a highly consistent feature foundation for subsequent accurate identity association but also creates a completely seamless experience for users in practical application scenarios such as voice interaction.

[0024] To address the unavoidable intermittent vocalization behaviors such as pauses and turns in real-world applications, this invention designs an intermittent voiceprint memory and rapid recapture strategy. By constructing a short-term historical voiceprint feature cache for active speakers in memory, the system is endowed with short-term memory capabilities. When a speaker resumes speaking after a brief silence, the system can prioritize activating the cached features for matching, increasing the success rate of identity recapture after the intermittent period from less than 40% to over 90%, significantly enhancing the system's robustness and continuity in the face of complex movement behaviors and long-term tracking tasks.

[0025] In harsh acoustic environments facing reverberation, strong background noise, and simultaneous speech from multiple targets, this invention implements a multi-node collaborative identity voting mechanism based on feature similarity and node signal quality confidence weights. When the same sound source is captured simultaneously by multiple pickup nodes, this mechanism can intelligently discard poor observations from low-quality or interfered nodes, and obtain the optimal identity determination result through weighted fusion. This not only effectively eliminates the identity conflict problem caused by each node acting independently, but also improves the recognition accuracy in complex environments by 15 to 20 percentage points compared to single-node schemes, maintaining highly reliable recognition output even under extremely low signal-to-noise ratio conditions.

[0026] Based on the aforementioned decentralized distributed computing architecture, each pickup node possesses localized spatial computation and feature extraction capabilities. This drastically reduces the data transmitted from the nodes to the central processing unit from a massive raw audio stream to lightweight positioning coordinates and feature vectors, lowering network communication bandwidth usage by over 98%. This architecture not only fundamentally resolves the bottleneck of centralized processing's excessive reliance on central computing power and network bandwidth but also endows the system with extremely flexible horizontal scalability, easily supporting large-scale deployments of hundreds of nodes. Furthermore, a local failure in a single node will not affect the entire network, greatly ensuring the overall availability of the system. This perfectly meets the actual deployment needs of ultra-large-scale continuous spaces such as airport terminals, large commercial exhibition halls, and industrial safety production.

[0027] Additional aspects and advantages of the invention will become apparent in the following description or may be learned by practice of the invention. Attached Figure Description

[0028] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart of a method for locating and identifying mobile sound sources in a multi-node sound pickup network according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall architecture of a multi-node sound pickup network system according to an embodiment of the present invention; Figure 3 This is a block diagram of the internal structure of a pickup node according to an embodiment of the present invention; Figure 4 This is a flowchart illustrating the dynamic microphone selection and audio enhancement mechanism of one embodiment of the present invention. Figure 5 This is a schematic diagram of a cross-node voiceprint association and identity transfer mechanism according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the intermittent voiceprint memory and rapid recapture mechanism according to an embodiment of the present invention. Detailed Implementation

[0029] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0030] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0031] The following reference Figures 1 to 6 This invention describes a method and system for locating and identifying mobile sound sources in a multi-node sound pickup network, provided by some embodiments of the present invention.

[0032] Some embodiments of this application provide a method for locating and coherently identifying mobile sound sources in a multi-node sound pickup network.

[0033] The core technical concept of this invention lies in breaking away from the traditional pipeline architecture of spatial positioning and identity recognition in distributed acoustic perception, and constructing an intelligent audio collaborative processing scheme that deeply couples spatial trajectory and biometric voiceprint features and adaptively evolves. In terms of the overall network topology, the system relies on multiple spatially discrete pickup nodes with independent local computing power to push massive amounts of raw multi-channel audio computation down to the edge. By reporting ultra-lightweight local spatial coordinates and signal quality characteristics in real time, it fundamentally resolves the excessive dependence of centralized processing on backbone network bandwidth and central computing power in large-scale scenarios. Based on this, this invention, through the parallel driving of location and identity dual engines and a cross-node context handshake mechanism, not only transmits the conventional spatial geometric state when a moving sound source crosses the node coverage edge, but also explicitly uses historical high-dimensional voiceprint feature vectors as associated context for synchronous cross-regional relay. Furthermore, relying on primary and backup microphone delay alignment, phase difference compensation, and cross-gradient smooth splicing technology, it eliminates acoustic disturbances introduced by physical channel switching. To address the intermittent vocalization behaviors of speakers, such as frequent turning around and short pauses, in natural interactions, the system further introduces a short-term memory caching mechanism and a multi-node joint weighted voting model, forming a wide-area global identity tracking closed loop with anti-drift, high robustness, and low latency.

[0034] like Figure 1 As shown, the first embodiment of the present invention proposes a method for locating and identifying mobile sound sources in a multi-node sound pickup network, including the following steps S1 to S5.

[0035] S1. Each of the aforementioned sound pickup nodes collects audio signals in real time and estimates the location information of mobile sound sources within the coverage area and the signal quality of the corresponding audio based on the local sound source localization algorithm. S2. Based on the positioning information and signal quality reported by each of the pickup nodes, the optimal pickup node is dynamically selected as the first pickup node and the primary audio is obtained. At the same time, the backup audio of the candidate second pickup node is cached. When the movement of the moving sound source is detected, which causes the primary pickup node to switch from the first pickup node to the second pickup node, the primary audio and the backup audio are time-aligned using cross-correlation delay estimation. After phase difference compensation by combining short-time Fourier transform, cross-gradient splicing is performed to output a smooth transition audio. S3. Extract voiceprint features from the output continuous audio stream in real time to generate a high-dimensional voiceprint feature vector of the current speaker; during the switching of the pickup node, transmit the historical voiceprint feature vector extracted by the first pickup node and the corresponding identity tag to the second pickup node synchronously. S4. The second voice pickup node calculates the Euclidean distance between the newly extracted voiceprint feature vector and the received historical voiceprint feature vector. If the Euclidean distance is less than a preset distance threshold, the identity association is determined to be successful, and the identity label continues on the second voice pickup node. After the identity association is successful, the high-confidence voiceprint features extracted subsequently are weighted and fused with the historical voiceprint feature vector according to a preset smoothing factor, and the voiceprint feature template of the current speaker is updated based on the fusion result. S5. Integrate the positioning information reported by each of the sound pickup nodes and the above-mentioned identity continuation and recognition results, combine the spatial coordinate observation and voiceprint comparison score into a joint observation, and use a filtering algorithm to recursively estimate the three-dimensional position, motion state and identity association state of the mobile sound source, and output the continuous movement trajectory with a unique identity tag in real time.

[0036] Next, the implementation details of each step of the mobile sound source localization and identity coherent recognition method in the multi-node sound pickup network of the present invention will be described in detail with reference to the accompanying drawings.

[0037] For step S1, perform node-level sound source perception and quality assessment. Combined with... Figure 2 and Figure 3 As shown, to implement the method of this invention, a distributed network consisting of multiple spatially discrete pickup nodes is required. In one specific embodiment, combined with... Figure 2 The diagram shows the overall architecture of the multi-node microphone network system. The acoustic perception execution process at the front end of the method disclosed in this paper includes: each microphone node using multiple configured microphones (such as…). Figure 2The microphones (Mic1 to Mic4) shown capture ambient sound in real time and sequentially pre-amplify and filter it to initially remove background noise. Subsequently, an analog-to-digital conversion is performed to convert the analog acoustic signal into a digital audio signal, which is then stored in a local audio buffer, such as a circular buffer structure, for later use in subsequent processing. Regarding power supply at the hardware level, the operating voltage and status of each component are uniformly scheduled and guaranteed by the power management module within the microphone node.

[0038] During the aforementioned data flow, the central control unit within the audio pickup node is responsible for global data scheduling and step control. Specifically, the central control unit extracts digital audio signals from the audio buffer and distributes them to the local sound source localization unit and the signal quality assessment module. Subsequently, the local sound source localization unit, based on the received audio data, executes sound source localization algorithms such as time difference of arrival or beamforming to calculate in real time the spatial geometric features, such as the relative azimuth and distance, of moving sound sources within the current node's coverage area to form local positioning information. Simultaneously, the signal quality assessment module executes signal quality assessment calculation logic, extracting and measuring quality parameters such as the signal-to-noise ratio and energy value of the corresponding audio data segment in real time to quantitatively assess the purity of the current audio pickup link.

[0039] After completing the aforementioned local calculations, the central control unit reports lightweight node-level data, including positioning results and signal quality parameters, to the central processing unit or other node levels responsible for subsequent association and splicing via a communication interface (such as wired, Wi-Fi, or 5G interface). This effectively resolves the bottleneck of excessive reliance on backbone network bandwidth in large-scale scenarios. To ensure the rigor and consistency of multi-node collaborative processing, in some embodiments of the method, the time synchronization module within each pickup node receives external time synchronization commands and uses a precise time synchronization protocol (such as PTP / IEEE1588 protocol) or a GPS timing mechanism to achieve high-precision clock synchronization. This operation of constructing a unified global time base provides an indispensable timestamp constraint for the timing alignment of subsequent cross-node audio streams. It should be noted that the microphone array pickup technology, the sound source localization principle based on time difference of arrival, and the basic signal processing actions such as beamforming mentioned in the embodiments of this application are all known existing general technologies in the field of digital signal processing and pattern recognition. Those skilled in the art can implement the deployment of these basic underlying algorithms based on existing technical manuals or conventional technical means, so their internal mathematical derivations will not be discussed here.

[0040] Furthermore, combined Figure 3The flowchart illustrating the processing hierarchy shows that the execution steps in step S1 of this method specifically cover the data flow and computation from the "sound pickup layer" to the "processing layer." In the sound pickup layer, various sound pickup nodes (such as...) are distributed within a continuous large space... Figure 3 The pickup nodes (1, 2, ..., N) independently and in parallel acquire the aforementioned audio signals and transmit the audio stream and preliminary positioning reference features to the processing layer. At the processing layer, the distributed edge computing advantages of this method are fully realized. Each pickup node does not need to upload a large amount of raw audio stream to the central network; instead, it directly calls processing resources locally to perform calculations. Specifically, based on the audio data within its local circular buffer, the pickup node executes sound source localization algorithms such as time difference of arrival or beamforming to calculate in real time the spatial geometric features, such as the relative orientation and distance, of moving sound sources within the current node's coverage area, thus forming local positioning information. Simultaneously, through signal quality assessment calculation logic, the signal-to-noise ratio and energy value of the corresponding audio data segment are extracted and measured in real time to quantitatively assess the purity of the current pickup link. Finally, each pickup node reports lightweight node-level data containing the aforementioned positioning calculation results and signal quality parameters to the layer responsible for subsequent association and splicing via wired or wireless communication interfaces, effectively mitigating the bottleneck of excessive reliance on backbone network bandwidth in large-scale scenarios.

[0041] Combination Figure 3 The associated layer portion of the processing hierarchy logic flow diagram shown above, for step S2, performs dynamic microphone selection and smooth audio splicing operations. Combined with... Figure 4 The flowchart shown illustrates the dynamic microphone selection and audio enhancement workflow. This step aims to address the core problem in multi-node networks where sudden changes in audio sources due to physical switching lead to discontinuous and fluctuating voiceprint feature extraction. In traditional simple audio splicing schemes, feature perturbations are easily generated during switching. To address this, this invention introduces quality-driven and cache smoothing mechanisms to ensure the audio continuity and stability required for subsequent processing steps.

[0042] In one specific embodiment, step S2 first receives in real-time the location information and corresponding audio quality parameters such as signal-to-noise ratio (SNR) and energy value reported by each pickup node in the previous step. Then, a normalized weighted score is calculated based on the SNR and energy value of the audio corresponding to each pickup node. After scoring, it is determined whether a primary pickup node is currently in operation within the current judgment logic. If not, the node with the highest score is directly selected as the primary node. If it exists, the score difference between the current highest-scoring node and the current primary pickup node is calculated. To avoid frequent invalid switching of the pickup link due to slight fluctuations in sound over a short period, when determining that the highest-scoring node differs from the current primary pickup node, it is necessary to further determine whether the score difference is greater than a preset switching threshold. When the difference exceeds the threshold, the switching preparation for the pickup node is formally triggered. The highest-scoring node is marked as a candidate second pickup node, and the synchronization process of the audio buffer of the second pickup node is started. For example, the audio of the candidate node is cached for 200 milliseconds during the switching preparation period for a smooth transition. If the score difference does not reach the threshold requirement, the current primary pickup node is maintained to continue picking up the primary audio.

[0043] In some embodiments, when the movement of the moving sound source is detected, causing the primary microphone node to substantially switch from the first microphone node to the second microphone node, the system does not employ a rigid direct truncation and splicing strategy. Specifically, cross-correlation time delay estimation is used to align the time axis of the primary audio and the backup audio to eliminate the time delay caused by the difference in sound wave propagation distance between microphones in different spatial locations. Simultaneously, the frequency domain phase information of the signal is extracted using short-time Fourier transform, thereby performing precise phase difference compensation on the two signals. Based on this time and phase alignment, smooth window functions such as the Hanning window are used to perform cross-gradual splicing on the two audio signals, thereby achieving a seamless and smooth transition between microphone audio from different physical locations. This process effectively ensures that the audio stream sent to the subsequent voiceprint extraction stage is not only waveform-continuous but also does not produce characteristic jumps.

[0044] In one specific embodiment, after performing cross-fade splicing on the primary audio and the backup audio to output a smooth transition audio, in order to further improve the signal purity and signal-to-noise ratio of the signal fed into the voiceprint recognition engine, the method further applies beamforming algorithm or speech separation enhancement algorithm to the smooth transition audio for target sound source enhancement processing. The enhanced, clean, and smooth audio stream will be directly input into the subsequent real-time voiceprint feature extraction step.

[0045] It should be noted that the cross-correlation delay estimation, short-time Fourier transform (STFT), Hanning window function, beamforming algorithm for audio enhancement, and speech separation algorithm mentioned in the embodiments of this application are all known, existing, fundamental algorithms in the field of digital audio processing. Those skilled in the art can implement the specific transformation and windowing operations by calling standard open-source algorithm libraries or performing conventional mathematical time-frequency domain derivations based on general digital signal processing knowledge. Therefore, these fundamental mathematical principles and formulas will not be exhaustively elaborated upon here.

[0046] Combination Figure 3 In the related layer part of the processing hierarchy logic flow diagram shown, for step S3, real-time voiceprint feature extraction and context passing operations are performed. Combining the overall method flow, after the smooth splicing and enhancement steps described above, real-time voiceprint feature extraction is performed on the output continuous audio stream, thereby converting the sound signal into identifiable and transferable identity information and generating a high-dimensional voiceprint feature vector of the current speaker. In a specific embodiment, as audio data continuously flows in, the first pickup node continuously extracts high-dimensional feature representations from the current speech segment. It should be noted that the voiceprint feature extraction algorithms mentioned in this application embodiment, such as those using Mel-frequency cepstral coefficients, the x-vector model based on deep neural networks, or the ECAPA-TDNN network model, are all existing general technologies in the field of pattern recognition. Those skilled in the art can choose appropriate known extraction networks to achieve the mapping and transformation of sound signals to identity feature vectors based on the computing power conditions of the actual application scenario; therefore, a lengthy explanation of the principles is not provided here.

[0047] Combination Figure 5 The diagram illustrating the cross-node voiceprint association and identity transfer mechanism shows that when the speaker's movement is detected, causing the voice pickup network to trigger a physical switch of the primary node, the method, in order to overcome the shortcomings of traditional networks where identity tags are prone to jumps or breaks, will initiate a cross-node identity context transfer action. Specifically, as shown... Figure 5 As shown, when a switch from node A (i.e., the first microphone node) to node B (i.e., the candidate second microphone node) is detected at time point t2, the method disclosed herein is no longer limited to simply transmitting location coordinates. Instead, it synchronously transmits and distributes the historical voiceprint feature vector (i.e., feature V_A in the figure) extracted and solidified by the first microphone node at time point t1, along with its corresponding unique identity tag (i.e., identity tag ID1 in the figure), as a whole, as prior reference information for cross-node identity succession to the second microphone node about to take over. Through this context handshake mechanism, the synchronization of physical space transfer and biometric association is achieved.

[0048] In some embodiments, considering that voiceprint features directly involve the user's biometric privacy, during the process of synchronously transmitting the historical voiceprint feature vector extracted by the first voice pickup node and the corresponding identity tag to the second voice pickup node, the method also encrypts the transmitted historical voiceprint feature vector and identity information. Subsequently, the second voice pickup node decrypts the encrypted data packet upon receiving it to recover the original feature vector, thereby completing the subsequent Euclidean distance calculation for identity comparison. In closed or trusted network environments where strict encryption is not required, the above encryption steps can also be replaced by a lightweight data obfuscation algorithm to prevent the leakage of sensitive privacy data in cross-node network transmission links, thereby enhancing the data security of the system in actual deployment.

[0049] For step S4, perform cross-node identity association continuation and feature template adaptive evolution operations. Combined with... Figure 5 The diagram illustrating the cross-node voiceprint association and identity transfer mechanism shows that after the second pickup node receives the transmitted historical voiceprint feature vector and corresponding identity label, it locally calculates the Euclidean distance between the newly extracted voiceprint feature vector and the received historical voiceprint feature vector. This distance characterizes the proximity between the current speech's voiceprint embedding and historical identity features. When the Euclidean distance is less than a preset distance threshold, the current speaker is determined to be consistent with the historical identity, and the identity association is successful, allowing the identity label to be seamlessly continued on the newly switched second pickup node. After successful identity association, the method does not directly discard the transmitted historical features. Instead, it weights and fuses the high-confidence voiceprint features continuously extracted by the new node with the historical voiceprint feature vector according to a preset smoothing factor. Based on this fusion result, the voiceprint feature template of the current speaker is dynamically updated, enabling the updated template to adaptively retain both historical identity features and the latest voiceprint characteristics under the current new node's acoustic environment. This achieves smooth transfer of identity labels between different pickup nodes and dynamic evolution of the voiceprint feature library. It should be noted that the calculation of Euclidean distance and the weighted fusion update mechanism based on smoothing factor are both common numerical distance metrics and template iteration algorithms in this field. Those skilled in the art can set parameters and perform calculations using conventional technical means.

[0050] In one specific embodiment, combined with Figure 6 The diagram illustrates the intermittent voiceprint memory and rapid recapture mechanism. Considering that mobile speakers in real-world scenarios often engage in intermittent vocalization behaviors such as short pauses or turning around, this disclosed method introduces a short-term memory mechanism during feature extraction and identity association. Specifically, when the mobile sound source is detected to have stopped emitting sound and entered a pause period ( Figure 6At times t2-t3 (as shown), the latest voiceprint feature vector V1 extracted before the moving sound source stopped and its corresponding identity tag ID1 are stored in a dedicated short-term voiceprint feature buffer. Subsequently, when a new sound segment is detected again in the sound pickup network and a candidate new voiceprint feature V2 is extracted ( Figure 6 At times t3-t4 (as shown), the feature similarity between the candidate new voiceprint features and the historical features stored in the short-term voiceprint feature buffer is calculated first. If the feature similarity is greater than a preset recapture threshold, the current voice is determined to belong to the same speaker as the voice before the pause, thus maintaining the continuation of the original identity label. This mechanism effectively avoids the problem in the prior art where a voice re-speaking after a pause is misjudged as a new target, achieving fast and accurate identity recapture of a moving sound source after the interval.

[0051] In some embodiments, to address the conflict arising from independent identification by each node when multiple people are simultaneously moving and speaking within the coverage area of ​​multiple nodes, a multi-node collaborative identity voting mechanism is designed. When the same mobile sound source is simultaneously captured by multiple pickup nodes, candidate speech segments are first spatially clustered based on the positioning information reported by each pickup node, merging sound source observations belonging to the same spatial location into the same candidate target. Subsequently, each pickup node capturing the candidate target independently calculates the feature similarity between the current speech segment and each candidate identity, and combines parameters such as the signal-to-noise ratio, speech energy, sound source distance, or positioning stability of the corresponding audio of its node, and generates node confidence weights after normalization processing. Based on this, collaborative weighted voting is performed based on the feature similarity and the node confidence weights to calculate a comprehensive score. Specifically, for any candidate identity m, the formula for calculating its comprehensive score is:

[0052] in, The total vote score represents the candidate identity m; This represents the similarity between the current speech features reported by the i-th pickup node and the features of the candidate identity m; The node confidence weight represents the i-th pickup node.

[0053] Finally, the candidate identity with the highest comprehensive score that exceeds the preset judgment threshold is determined as the final identity label for that candidate target. This joint weighting strategy effectively reduces the interference of low-quality or distant nodes on the identity determination results, significantly improving the accuracy and stability of identity determination in complex environments.

[0054] In one specific embodiment, a global voiceprint registration library can be maintained at the system's underlying layer, supporting a dynamic update mechanism. Besides performing the aforementioned smooth weighted update of the existing speaker's voiceprint feature template after successful identity association, this mechanism also allows for the real-time addition or deletion of specific users' voiceprint models during system operation. For example, in actual business scenarios, new users' voiceprint features can be recorded at any time and simultaneously distributed to various voice pickup nodes in the network, or expired identity templates can be removed. The entire process does not require restarting the system or reloading the fixed registration library, greatly improving the system's business adaptability.

[0055] For step S5, perform joint tracking of spatiotemporal trajectory and identity. Figure 3 The output layer of the processing hierarchy logic flow diagram shown above, after completing spatial calculation and cross-node identity association in the aforementioned steps, deeply couples the positioning and recognition processes at the macro-logic level to overcome the shortcomings of traditional technologies, such as the separation of positioning and recognition and poor real-time performance caused by serial processing. In a specific embodiment, the sound source positioning information reported in real time by each of the sound pickup nodes and the voiceprint recognition results output in the above steps are integrated, breaking the previous isolated data format and forming a joint observation of spatial coordinate observation and voiceprint comparison score. Subsequently, a filtering algorithm is used to recursively estimate the three-dimensional position, motion state, and identity association state of the moving sound source.

[0056] Specifically, the three-dimensional spatial coordinates and velocity of the moving sound source are used as the state vector of the state-space model such as Kalman filtering, and the position data reported by each node and the voiceprint comparison score when identity association is successful are used as the observation vector for state updating. It should be noted that the Kalman filtering algorithm mentioned in this application is a known basic algorithm in the field of target tracking. Those skilled in the art can implement its prediction and update process based on existing technology, so its internal mathematical formulas will not be elaborated here. In some embodiments, when the speaker makes intermittent vocalization behaviors such as short pauses, the tracking link is not immediately cut off. Instead, the geometric continuity of the spatial trajectory is maintained by relying on the Kalman prediction results, and the continuity of identity association is maintained by combining the aforementioned historical voiceprint features.

[0057] Through the parallel fusion of spatiotemporal and identity information, the method ultimately generates and outputs a movement trajectory with a unique identity tag in real time. The structured data output naturally includes identity identifiers, three-dimensional spatial coordinates, and timestamps, achieving millisecond-level zero-latency response and providing complete, continuous, and uninterrupted basic perception data for higher-level applications such as large exhibition halls and intelligent monitoring.

[0058] In some embodiments, step S5 not only outputs structured data of continuous movement trajectories with unique identification tags in real time in the background for third-party systems to call, but also pushes this combined data to a visual trajectory display interface. This display interface can be deployed on a large screen in a monitoring center or on security terminal equipment, and can combine with an indoor electronic map to render movement trajectory lines with identification tags in real time, intuitively presenting the movement path and stopping area of ​​personnel. Furthermore, to cope with extremely complex application scenarios, the method also supports multimodal assisted positioning extension. Specifically, the system can access observation data from infrared sensors or cameras and perform joint data fusion of the three-dimensional spatial coordinates calculated purely acoustically and the human figure coordinates captured by the visual sensors. By introducing multimodal sensors as an auxiliary positioning source, the accuracy loss of single pure acoustic positioning in extreme acoustic obstruction or strong interference environments can be effectively compensated, thereby achieving more robust spatiotemporal trajectory joint tracking.

[0059] Based on the same inventive concept as the aforementioned mobile sound source localization and identity coherent recognition method in a multi-node audio pickup network, some other embodiments of the present invention also provide a mobile sound source localization and identity coherent recognition system in a multi-node audio pickup network. This system physically includes multiple spatially discretely deployed audio pickup nodes and a central processing unit communicatively connected to each of the aforementioned audio pickup nodes. Logically, the system is divided into a series of collaboratively working virtual functional modules, mainly including a node-level sound source perception module, a dynamic microphone selection and audio splicing module, a voiceprint feature extraction and transmission module, a cross-node identity continuation and template update module, and a spatiotemporal trajectory and identity joint tracking module. Specifically, the node-level sound source perception module is directly configured in each spatially discretely deployed bottom-level audio pickup node. It is used to collect multi-channel audio signals in real time through a microphone array and call a local sound source localization algorithm to estimate the relative orientation and distance of mobile sound sources within the coverage area. Simultaneously, this module is responsible for extracting the signal-to-noise ratio and energy value of the corresponding audio segment to generate signal quality parameters characterizing the purity of the audio pickup link.

[0060] The dynamic microphone selection and audio splicing module is mainly used to perform normalized scoring based on the real-time location information and signal quality reported by each microphone node. It dynamically maintains the microphone node with the best current pickup quality as the first microphone node and acquires its primary audio. Simultaneously, it caches backup audio from candidate second microphone nodes that meet neighborhood conditions in the background. When the movement of the moving sound source is detected, causing the primary microphone node to substantially switch from the first to the second microphone node, this module uses cross-correlation delay estimation to strictly align the primary and backup audio on the time axis, and combines short-time Fourier transform for frequency domain phase difference compensation. Based on this, the module uses a smooth window function to perform cross-gradient splicing to output a smoothly transitioning audio stream, thereby completely eliminating phase abrupt changes and feature disturbances introduced by physical node switching.

[0061] The voiceprint feature extraction and transmission module is used to extract voiceprint features from the spliced ​​and smoothed continuous audio stream in real time, generating a high-dimensional voiceprint feature vector that can represent the identity of the current speaker. During the physical switching of the aforementioned pickup nodes, this module uses the historical voiceprint feature vector and corresponding unique identity label already fixed in the first pickup node as identity context reference information, and synchronously transmits it to the second pickup node that is about to take over. The cross-node identity continuation and template update module is used to receive the above context information and enable the second pickup node to locally calculate the Euclidean distance between its newly extracted voiceprint feature vector and the received historical voiceprint feature vector. If the comparison shows that the Euclidean distance is less than a preset distance threshold, the module determines that the identity association is successful, so that the identity label can be seamlessly continued on the second pickup node. After the identity association is successfully continued, the module further performs weighted fusion of the high-confidence voiceprint features continuously extracted by the new node and the historical voiceprint feature vector according to a preset smoothing factor, and dynamically and adaptively updates the global voiceprint feature template of the current speaker based on the fusion result.

[0062] The spatiotemporal trajectory and identity joint tracking module is used for data coupling at the highest output layer of the system architecture. It integrates the positioning coordinate information reported by each sound pickup node with the identity continuation and recognition results output from the above steps, and constructs a joint observation by combining the spatial coordinate observation matrix and the voiceprint comparison score. Subsequently, this module uses state-space recursive filtering algorithms such as Kalman filtering to uniformly recursively estimate and continuously update the three-dimensional spatial position, dynamic motion state, and identity association state of the moving sound source, and finally outputs a continuous movement trajectory with a unique identity tag in real time.

[0063] It should be noted that the system and its various virtual functional modules provided in this embodiment have the same mechanism for solving technical problems and produce the same technical effects as the corresponding method embodiments described above. The specific workflow of each module, parameter calculation methods, and logical extensions for handling intermittent voiceprint memory or multi-node collaborative voting can all be found in the detailed descriptions in the aforementioned method embodiments, and will not be repeated here. Those skilled in the art will understand that the above-mentioned logical modules can be implemented through… Figure 2 The central control unit and central processing unit in the system architecture shown execute software code programs stored in memory. They can also be implemented in pure hardware or a combination of hardware and software through application-specific integrated circuits, digital signal processors or programmable logic devices. All of these fall within the protection scope of this invention.

[0064] In this specification, the illustrative expressions of the terms used do not necessarily refer to the same embodiments or examples. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0065] Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention shall be included within the scope of protection of this invention.

Claims

1. A method for localizing and coherently identifying mobile sound sources in a multi-node sound pickup network, characterized in that, The multi-node sound pickup network includes multiple spatially discrete sound pickup nodes, and the method includes: Each of the aforementioned sound pickup nodes acquires audio signals in real time and estimates the location information of moving sound sources within the coverage area and the signal quality of the corresponding audio based on a local sound source localization algorithm; Based on the location information and signal quality reported by each pickup node, the optimal pickup node is dynamically selected as the first pickup node and the primary audio is acquired. At the same time, the backup audio of the candidate second pickup node is cached. When the movement of the moving sound source is detected, which causes the primary pickup node to switch from the first pickup node to the second pickup node, the primary audio and the backup audio are time-aligned using cross-correlation delay estimation. After phase difference compensation by combining short-time Fourier transform, cross-gradient splicing is performed to output a smooth transition audio. Voiceprint features are extracted in real time from the output continuous audio stream to generate a high-dimensional voiceprint feature vector of the current speaker; during the switching of the pickup node, the historical voiceprint feature vector extracted by the first pickup node and the corresponding identity tag are synchronously transmitted to the second pickup node. The second voice pickup node calculates the Euclidean distance between the newly extracted voiceprint feature vector and the received historical voiceprint feature vector. If the Euclidean distance is less than a preset distance threshold, the identity association is determined to be successful, and the identity label continues on the second voice pickup node. After the identity association is successful, the high-confidence voiceprint features extracted subsequently are weighted and fused with the historical voiceprint feature vector according to a preset smoothing factor, and the voiceprint feature template of the current speaker is updated based on the fusion result. By integrating the location information reported by each of the sound pickup nodes and the results of identity continuation and recognition, the spatial coordinate observation and voiceprint comparison score are combined to form a joint observation. A filtering algorithm is used to recursively estimate the three-dimensional position, motion state and identity association state of the mobile sound source, and output the continuous movement trajectory with a unique identity tag in real time. The step of dynamically selecting the optimal microphone node as the first microphone node based on the positioning information and signal quality reported by each microphone node includes: A normalized weighted score is calculated based on the signal-to-noise ratio and energy value of the audio corresponding to each pickup node; Determine if a primary pickup node exists. If it does, calculate the difference between the score of the highest-scoring node and the score of the current primary pickup node. When the highest-scoring node is different from the current primary pickup node and the score difference is greater than a preset switching threshold, pickup node switching preparation is triggered, the highest-scoring node is marked as a candidate second pickup node, and audio cache synchronization of the second pickup node is started. After the step of extracting voiceprint features from the output continuous audio stream in real time, the method further includes: When the mobile sound source stops emitting sound and enters an intermittent period, the latest voiceprint feature vector extracted before the mobile sound source stopped and the corresponding identity tag are stored in the short-term voiceprint feature cache area. When a vocal segment is re-detected in the sound pickup network and candidate voiceprint features are extracted, the similarity between the candidate voiceprint features and the features stored in the short-term voiceprint feature cache is calculated first. If the feature similarity is greater than the preset recapture threshold, the original identity label is maintained to achieve recapture of the mobile sound source identity after the interval. The step of using a filtering algorithm to recursively estimate the three-dimensional position, motion state, and identity association state of the mobile sound source includes: using the three-dimensional spatial coordinates and velocity of the mobile sound source as the state vector of the Kalman filter; using the position data reported by each node and the voiceprint comparison score when identity association is successful as the observation vector for state update; maintaining the continuity of the spatial trajectory by relying on the Kalman prediction results when the mobile sound source pauses briefly; and maintaining the consistency of the identity label by combining the historical voiceprint feature vector.

2. The method for mobile sound source localization and coherent identity recognition in a multi-node sound pickup network according to claim 1, characterized in that, When the same mobile sound source is captured by multiple pickup nodes at the same time, the candidate speech segments are spatially clustered and merged into the same candidate target based on the positioning information reported by each pickup node. Each sound pickup node that captures the candidate target calculates the feature similarity between the current speech segment and each candidate identity, and generates a node confidence weight by combining the signal quality of the audio corresponding to the node. Based on the feature similarity and the node confidence weight, a collaborative weighted voting is performed to calculate a comprehensive score, and the candidate identity with the highest comprehensive score that is greater than a preset judgment threshold is determined as the final identity label of the candidate target.

3. The method for mobile sound source localization and coherent identity recognition in a multi-node sound pickup network according to claim 2, characterized in that, The method for calculating the overall score includes: in, The total vote score represents the candidate identity m; This represents the similarity between the current speech features reported by the i-th pickup node and the features of the candidate identity m; The node confidence weight represents the i-th pickup node. The node confidence weight is determined by normalizing the corresponding node's signal-to-noise ratio, speech energy, sound source distance, or localization stability.

4. The method for mobile sound source localization and coherent identity recognition in a multi-node sound pickup network according to claim 1, characterized in that, After performing cross-gradient splicing on the primary audio and the backup audio to output a smooth transition audio, the smooth transition audio is enhanced by target sound source using a beamforming algorithm or a speech separation algorithm, so that the enhanced clean audio stream is input to the voiceprint feature extraction step.

5. The method for mobile sound source localization and coherent identity recognition in a multi-node sound pickup network according to claim 1, characterized in that, The microphone nodes are synchronized using a precise time synchronization protocol or a global positioning system to ensure that the first microphone node and the second microphone node have a unified time reference when performing the cross-correlation delay estimation.

6. The method for mobile sound source localization and coherent identity recognition in a multi-node sound pickup network according to claim 1, characterized in that, During the process of synchronously transmitting the historical voiceprint feature vector extracted by the first voice pickup node and the corresponding identity tag to the second voice pickup node, the transmitted historical voiceprint feature vector is encrypted. After receiving it, the second voice pickup node decrypts it to complete the calculation of the Euclidean distance.

7. A mobile sound source localization and identity recognition system in a multi-node sound pickup network, characterized in that, The method for mobile sound source localization and coherent identity recognition in a multi-node sound pickup network as described in any one of claims 1 to 6, the system comprising: The node-level sound source perception module includes sound pickup nodes deployed discretely in various spaces, used to collect audio signals in real time, and estimate the location information of moving sound sources within the coverage area and the signal quality of the corresponding audio based on the local sound source localization algorithm. The dynamic microphone selection and audio splicing module is used to dynamically select the optimal microphone node as the first microphone node and acquire the primary audio based on the positioning information and signal quality reported by each microphone node, while caching the backup audio of the candidate second microphone node; when the movement of the moving sound source is detected, causing the primary microphone node to switch from the first microphone node to the second microphone node, the module uses cross-correlation delay estimation to time-align the primary audio and the backup audio, and performs phase difference compensation by combining short-time Fourier transform, and then performs cross-gradient splicing to output a smooth transition audio. The voiceprint feature extraction and transmission module is used to extract voiceprint features from the output continuous audio stream in real time and generate a high-dimensional voiceprint feature vector of the current speaker; during the switching of the pickup node, the historical voiceprint feature vector extracted by the first pickup node and the corresponding identity tag are synchronously transmitted to the second pickup node. The cross-node identity continuation and template update module is used to enable the second voice pickup node to calculate the Euclidean distance between its newly extracted voiceprint feature vector and the received historical voiceprint feature vector; if the Euclidean distance is less than a preset distance threshold, the identity association is determined to be successful, and the identity label is continued on the second voice pickup node; after the identity association is successful, the high-confidence voiceprint features extracted subsequently are weighted and fused with the historical voiceprint feature vector according to a preset smoothing factor, and the voiceprint feature template of the current speaker is updated based on the fusion result; The spatiotemporal trajectory and identity joint tracking module is used to integrate the positioning information reported by each of the sound pickup nodes and the identity continuation and recognition results. It combines spatial coordinate observation and voiceprint comparison score to form a joint observation. It uses a filtering algorithm to recursively estimate the three-dimensional position, motion state and identity association state of the mobile sound source, and outputs a continuous movement trajectory with a unique identity tag in real time.

Images