Method for determining data representative of emergency situations, corresponding system and program.

Abstract

The invention relates to a method and system for determining emergency situations from audio signals received from a set of transmission channels used by a set of intervention teams, the method and system comprising at least the implementation of the steps: acquisition (S01) of an audio signal from a current transmission channel among the set of transmission channels; processing (S02) of the audio signal comprising: sampling at a predetermined frequency; segmentation of the sampled signal according to a predetermined interval, delivering a sequence of audio segments; and for each audio segment, extraction of spectral features, delivering a sequence of segments of spectral features; determination (S03) of a first data representative of an emergency situation of the current transmission channel as a function of the sequence of segments of spectral features of the audio signal.Fig 2.

Description

Title of the invention: Method for determining data representative of emergency situations, corresponding system and program. Scope of the invention

[0001] The invention relates to a communication system designed to improve communication management between response teams in emergency situations. More specifically, the invention relates to a device and methods enabling a human operator to prioritize the communication channel requiring immediate attention from among a plurality of available channels. Prior art

[0002] In the field of emergency response, the person in command faces numerous challenges when managing several response teams in the field. Each team uses a separate communication channel to exchange information. The person in command must therefore simultaneously monitor several channels to understand the issues encountered by each team and provide appropriate instructions. This task is particularly difficult during major crises where communication channels are highly active.

[0003] Currently, existing solutions rely primarily on manual monitoring of channels by the person in command. This method has several limitations. First, it is time-consuming, as the person in command (the dispatcher) must select and monitor each channel individually. Second, it is suboptimal, as the dispatcher can easily miss events or emergencies due to information overload.

[0004] Thus, critical communication systems, such as those used by police forces, firefighters, and other emergency response teams, often rely on push-to-talk (PTT) technology. This technology allows users to communicate instantly by pressing a button, which activates their communication device and enables them to speak directly to other team members or a command center. PTT systems are used in environments where the speed and clarity of communication are essential. The technical components of such a push-to-talk solution include, in particular, communication devices, with which members of emergency response teams are equipped, such as walkie-talkies or PTT radios. These devices allow a Instant two-way communication is achieved by simply pressing a button to speak. Each response team (for example, a brigade) uses a separate communication channel, which might be a specific communication frequency from a set of available frequencies, a digital channel from a set of channels, or another data transmission technology. This allows for separate communications between different teams and reduces communication difficulties within the same team. The person in command, the dispatcher, is located in a command center, which could be a mobile vehicle or a fixed control room. The command center is equipped with a monitoring console that allows the dispatcher to select and listen to different communication channels. A centralized communications server manages the connections between communication devices and communication channels. It ensures the transmission of voice messages and data between members of the response teams and the command center. In this configuration, response team members are in the field and use communication devices to exchange information and coordinate their actions. The dispatcher is responsible for coordinating communications and actions among the response teams. They monitor the communication channels, listen to the exchanges between team members, and give instructions based on the information received and other available information. The typical "push-to-talk" solution has several limitations.The dispatcher must manually select the channels they wish to monitor (or may attempt to monitor multiple channels simultaneously), which is time-consuming and suboptimal. When numerous teams are in the field, the dispatcher may miss important events and emergencies due to information overload, given the complexity of simultaneously monitoring multiple channels. Furthermore, human capacity to manage multiple channels simultaneously is limited, often to just a few, which is insufficient in complex situations involving numerous teams, such as large-scale demonstrations or crisis situations like riots or fires (such as the Notre Dame de Paris fire).

[0005] US11615252 describes a virtual assistant system to help emergency dispatchers manage live incidents by using natural language processing and machine learning technologies to analyze incident logs, track incident status, and generate action recommendations. The virtual assistant can also respond to dispatcher requests and generate incident reports. This system aims to reduce responder workload, improve response times, and minimize human error. The solution proposed in this document, however, does not simplify the dispatcher's task in managing response teams because it focuses primarily on analyzing incident logs and generating recommendations, rather than on real-time monitoring and management of multiple communication channels. The dispatcher must still manually select which channels to monitor, which is time-consuming and can lead to the omission of critical events. Furthermore, the human capacity to manage multiple channels simultaneously remains limited, a limitation not addressed by the solution proposed in US document 11615252.

[0006] Consequently, it is necessary to develop support systems capable of more easily identifying critical situations and reporting them to the person in command. These systems must allow for effective and rapid monitoring of communications, thereby reducing the risk of missing emergencies and improving the responsiveness and effectiveness of interventions. Current solutions, while useful, do not fully meet the needs of dispatchers in emergency situations, hence the need for innovation in this area. Summary of the invention

[0007] The invention provides an improvement on these prior art systems. More specifically, a method is proposed for determining data representative of emergency situations from audio signals, said audio signals being received from a transmission channel belonging to a set of transmission channels used by a set of response teams, each comprising at least one transmitter and at least one member, each response team using a different transmission channel, said method being implemented via an electronic device comprising at least one processing unit and a memory. Such a method comprises at least one iteration of the following steps: - acquisition of an audio signal from a current transmission channel among all transmission channels; - audio signal processing including: sampling at a predetermined frequency; segmentation of the sampled signal according to a predetermined interval, delivering a sequence of audio segments; and for each audio segment, extraction of spectral features, delivering a sequence of segments with spectral features; - determination of a first representative data of an emergency situation of the current transmission channel as a function of the sequence of segments of spectral characteristics of the audio signal.

[0008] According to a particular feature, the steps of audio signal acquisition, audio signal processing and determination of a first data representative of an emergency situation are implemented in parallel for all audio channels.

[0009] According to a particular feature, the extraction of spectral features includes a step of transforming each segment into a visual representation of the energy distribution of the sound signal segment.

[0010] According to a particular feature, the step of determining the data representative of an emergency situation of the current transmission channel as a function of the sequence of segments of spectral characteristics of the audio signal includes at least one step of inference, by a classification module, of a pre-trained neural network.

[0011] According to a particular characteristic, the pre-trained neural network is of the recurrent neural network type.

[0012] According to a particular feature, the step of determining the data representative of an emergency situation of the current transmission channel as a function of the sequence of segments of spectral characteristics of the audio signal takes into account at least one prior audio signal classification decision by a dispatcher.

[0013] According to a particular feature, the method further comprises, for said current channel: - a step of transcribing the voices present in the audio signal into a text representative of the words spoken on said current channel; - a step involving the determination, using a second pre-trained neural network, of a second piece of data representative of an emergency situation - a step of merging the first data point representing an emergency situation with a second data point representing an emergency situation; and - a step of determining data representative of a final emergency situation using a classification module.

[0014] According to a particular feature, a priority transmission channel, among the set of transmission channels, is selected based on data representative of transmission channel emergency situations determined for each channel used from the set of transmission channels.

[0015] According to a particular feature, the method further includes a step of transmitting, to a dispatcher, via a monitoring console, a signal representative of the priority transmission channel.

[0016] According to another aspect, the disclosure also relates to a system for determining data representative of emergency situations from audio signals, said audio signals being received from a transmission channel belonging to a set of transmission channels used by a set of response teams, each comprising at least one transmitter and at least one member, each response team using a different transmission channel, said system comprising at least one electronic device including at least one processing unit and a memory. Such a system includes means for iteratively implementing the following steps: - Acquisition of an audio signal from a current transmission channel among all transmission channels, - audio signal processing including: sampling at a predetermined frequency; segmentation of the sampled signal according to a predetermined interval, delivering a sequence of audio segments; and for each audio segment, extraction of spectral features, delivering a sequence of segments with spectral features; - determination of a data representative of an emergency situation of the current transmission channel as a function of the sequence of segments of spectral characteristics of the audio signal.

[0017] According to a preferred implementation, the various steps of the processes according to this disclosure are implemented by one or more software or computer programs, comprising software instructions intended to be executed by a data processor of an electronic channel management device or, more generally, of a communicating object according to this technique and designed to control the execution of the various steps of the processes, implemented at the level of an electronic channel management device or a communicating object, a remote server and / or a channel or communicating object management / monitoring system, within the framework of a distribution of the processing to be carried out and determined by a scripted source code or a compiled code.

[0018] Consequently, the present technique also relates to programs, capable of being executed by a computer or by a data processor, these programs comprising instructions to control the execution of the steps of the processes as mentioned above.

[0019] A program may use any programming language, and be in the form of source code, object code, or intermediate code between source code and object code, such as in a partially compiled form, or in any other desirable form.

[0020] The present technique also relates to an information carrier readable by a data processor, and comprising instructions of a program as mentioned above.

[0021] The information medium can be any entity or terminal capable of storing the program. For example, the medium can include a storage means, such as a ROM, for example a CD ROM or a microelectronic circuit ROM, or a magnetic recording means, for example a mobile medium (memory card) or a hard disk or an SSD.

[0022] On the other hand, the information medium can be a transmissible medium such as an electrical or optical signal, which can be transmitted via an electrical or optical cable, by radio, or by other means. The program according to the present technique can, in particular, be downloaded from an Internet-type network.

[0023] Alternatively, the information carrier may be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the process in question.

[0024] According to one embodiment, the present technique is implemented using software and / or hardware components. In this context, the term "module" in this document may refer to a software component, a hardware component, or a set of hardware and software components.

[0025] A software component corresponds to one or more computer programs, one or more subroutines of a program, or more generally to any element of a program or software capable of implementing a function or set of functions, as described below for the module concerned. Such a software component is executed by a data processor of a physical entity (terminal, server, gateway, set-top box, router, etc.) and is capable of accessing the hardware resources of that physical entity (memory, storage media, communication bus, input / output electronic cards, user interfaces, etc.).

[0026] Similarly, a hardware component corresponds to any element of a hardware assembly capable of implementing a function or a set of functions, as described below for the module concerned. It may be a programmable hardware component or one with an integrated processor for software execution, for example, an integrated circuit, a smart card, a memory card, an electronic card for executing firmware, etc.

[0027] Each component of the system described above naturally implements its own software modules.

[0028] The different embodiments mentioned above can be combined with each other for the implementation of the present technique. Brief description of the figures

[0029] Other objects, features and advantages of the invention will become more apparent upon reading the following description, given by way of simple illustrative, and not limiting, example, in relation to the figures, among which: - [Fig.l] schematically illustrates a system for implementing the disclosure process; - [Fig.2] illustrates the algorithm implemented within a channel selection engine based on disclosure; - [Fig.3] illustrates a portion of the additional classification module architecture according to an example of implementation.

[0030] Description of an embodiment

[0031] In general, the invention relates to an audio channel processing system, comprising a pre-trained classification module within critical communications solutions. This system is designed to analyze in real time a greater number of communication channels than a human dispatcher can handle alone.

[0032] The system is capable of detecting sounds, behavioral patterns, geographic locations, and their evolution. Optionally, it implements speech recognition features, as well as keyframes in the case of video communications. This analysis makes it possible to assess the criticality level of each situation. Based on this assessment, the system can trigger alerts of varying intensities on a console in the dispatcher. These alerts can range from simply highlighting a channel to immediate monitoring, accompanied by an increase in volume and an alarm. Furthermore, information on the analysis that led to the alert can be provided, such as a voice playback for audio channels.

[0033] The system also includes means of continuous improvement through a self-learning process. This process is based on machine learning, which adapts the system to the specific needs of each type of dispatcher. The system learns from the real reactions of human dispatchers after system installation, thus enabling continuous personalization and optimization.

[0034] Integrating this system into critical communications solutions offers several advantages. It significantly increases dispatcher monitoring capacity, thereby reducing the risk of missing emergency situations. Furthermore, this system can process and analyze complex data more quickly and accurately, improving the responsiveness and effectiveness of interventions. In addition, the system is designed to be flexible and adaptable, able to It can be adjusted to meet the specific needs of different organizations or situations. This flexibility ensures that the system remains relevant and effective in constantly evolving environments. The data used includes, in particular, audio signals (but especially voice signals) from the communication channels of the various teams involved in the response. The following data can also be used: - User ID and Role; - Geographical location of users and its derivatives (speed,...); - All text and video communications; - Frequency of speaking; - Distributor identifier.

[0035] This data can be processed, in whole or in part, by the system, which then deduces a probability of an emergency situation (alert) among all channels, along with an associated criticality index. This additional data can be processed via an additional processing module or included in the data provided to the statistical processing module described below. In any case, when the probability of a channel exceeds a certain threshold, the elements justifying the alert (recording, image, etc.) are provided. Based on this pair of parameters, the system establishes an alert type (from "no alert" to "maximum emergency"). This may include channel scheduling for the dispatcher, highlighting the channel to be monitored, a forced switch to the channel in question, and, in all alert cases, the originating elements enabling the differentiation of false positives.

[0036] The actions of the dispatcher are also transmitted to the system to feed self-learning and adapt to the specific practices of each dispatcher and each usage environment.

[0037] In relation to [Fig. 1], a system for processing representative SysEv voice exchange data is described for determining a priority communication channel to listen to from among a plurality of available channels. The SysEv system of [Fig. 1] comprises an SrvTI server which includes a DetEng engine for selecting channels requiring attention, links (direct or indirect) with at least one DigRecrv receiver for receiving voice signals from at least one walkie-talkie and / or a DPtT push-to-talk communication device, and a VocDB database including means for recording and storing audio streams from said at least one DigRecrv receiver. The database further includes means for recording and storing representative data of the listening decisions made (current and past decisions) by the dispatcher. The system further includes a console The SurvC monitoring system for the RepC dispatcher is connected to the SrvTI server via a communication network (Ntwk). The SrvTI server also includes means for implementing a deep learning-based statistical processing model capable of analyzing audio streams in real time. This system allows the dispatcher to efficiently manage communications between emergency response teams by enabling the SrvTI server to signal, via the SurvC console, one or more channels requiring the RepC dispatcher's attention.

[0038] Thus, the disclosure system comprises several elements enabling the efficient management of communications between emergency response teams. This system includes an SrvTI server that integrates a channel selection engine, DetEng, for identifying channels requiring attention. This detection engine uses a pre-trained MTS statistical processing module to analyze the audio streams of multiple communication channels simultaneously in real time.

[0039] The SrvTI server is, for example, connected, at least indirectly, to one or more DigRecrv receivers (which may be radio receivers, or in any case one or more receivers capable of receiving voice signals, via physically separate transmission channels – different frequencies or frequency bands – or logically separate channels – for example, data packets from sets of data packets). These DigRecrv receivers allow the parallel reception of the various voice signals exchanged on the communication channels used by the intervention teams with their push-to-talk devices. Other data may also be transmitted, such as identification data for the speaker(s) and / or location data for the speaker(s) or the team, etc. In the embodiment shown, the audio streams from the channels are transmitted to the DetEng selection engine for analysis.In this implementation example, the DetEng selection engine does not perform speech-to-text conversion of words or speech exchanges. Instead, it analyzes the audio streams of the channels to probabilistically detect those requiring immediate attention, without transforming them into text (without analyzing the spoken words). Therefore, the processed data is audio, and the statistical processing module's training is primarily performed on audio streams. The statistical processing module implemented by the DetEng selection engine can process multiple channels in parallel. To achieve this, the detection engine can implement several instances of the statistical processing model, although these multiple instances are not mandatory. This capability ensures simultaneous and efficient analysis of the different communication channels.The system also includes the VocDB database for recording and . The VocDB database stores the audio streams of the different channels. It also records the times when the DetEng selection engine suggests that the dispatcher listen to a specific channel, as well as the times when the dispatcher decides to listen to a channel and the identifier of the channel being listened to. This functionality allows for maintaining a history of the dispatcher's decisions and actions, while also enabling, as explained later, a refinement of the statistical processing model.

[0040] The RepC dispatcher uses the SurvC monitoring console connected to the SrvTI server to monitor communication channels. The SurvC monitoring console displays suggestions from the DetEng selection engine and allows the RepC dispatcher to select and listen to communication channels requiring immediate attention. The SurvC monitoring console is also equipped with commands to allow the RepC dispatcher to give instructions to response teams.

[0041] For real-time audio stream analysis, the appropriate MTS statistical processing module is configured to process the raw audio signals, extract relevant features, and perform inferences to detect channels requiring immediate attention. An example of a process and a neural network model adapted for implementing audio streams using the MTS statistical processing module is then presented in relation to [Fig. 2]. For simplicity, the example in [Fig. 2] deals with a single channel. This example includes several steps implemented to determine whether audio streams are prioritized (or not).

[0042] The first step implemented by the MTS statistical processing module involves transforming the audio signals into data ready for the MCI classification module (in this case, a neural network implemented according to a particular model, such as one of those described below). This first step includes an acquisition step (SOI) of the audio signals (Sig). These are acquired in real time by the DigRecrv receivers (and transmitted to the SrvTI server or received directly by the server when the DigRecrv receiver(s) are integrated into this SrvTI server), followed by a preprocessing step (S02) of the audio signals. This step may include normalization to ensure a constant amplitude. This step also includes sampling at a fixed frequency, for example, 16 kHz or 44.1 kHz, which captures the range of speech frequencies.The audio signals of a channel are segmented (Segs) into short time intervals, typically from 20 ms (milliseconds) to 40 ms, depending on the requirements. These time intervals partially overlap (e.g., 50% overlap) to ensure that no critical information is lost between segments. Then, . The characteristics of these segments are extracted. Thus, the audio segments are transformed into spectral features (Scs), such as spectrograms, Mel frequency costral coefficients (MFCCs), or Mel spectrograms. More specifically, this transformation involves obtaining a visual representation of the energy distribution of the audio signal segment. These features capture the frequency and / or time information of the audio signal. In one example, each audio segment is transformed into a spectrogram representing the frequencies present in the audio signal over time. The spectrogram is obtained by applying a Fourier transform (e.g., STFT) or a Mel-type transformation to the audio segments for each channel. The spectrograms are normalized and resized to be compatible with the input of the neural network model being used.

[0043] The next step, implemented by the statistical processing module MTS, consists of using these segments of spectral features (for example spectrograms) to determine (S03) a data representative of an emergency situation (Dru) by performing one or more inferences of the neural network (of the MCI classification module), at least one inference per channel.

[0044] More specifically, as a first example, a UNet+ type neural network architecture is suitable for implementing the classification according to this disclosure. The inventors have determined that this architecture has an interesting feature in the context of this disclosure, since it allows for a certain degree of historical data retention. A description of the UNet+ architecture used in an example implementation of this disclosure is as follows: - the model takes as input the spectrograms of the audio segments, for a given range. Each spectrogram is represented as a 2D matrix, where the horizontal axis represents time and the vertical axis represents frequencies; - The encoder consists of several convolution layers followed by pooling layers. Each convolution layer applies filters to extract local features from the spectrograms, while the pooling layers reduce the dimensionality of the data, to obtain more abstract information; - the central part of the network, called the bottleneck, consisting of several convolution layers without pooling, captures the most abstract and important features of the spectrograms. The decoder consists of transposed convolution (or deconvolution) layers and upsampling layers. It reconstructs the features extracted by the encoder and bottleneck to produce an output of the same quality. dimension than the input. The decoder also uses skip connections to combine the characteristics of the encoder with those of the decoder, thus improving the accuracy of the analysis. - The model output is a probability map indicating, for each audio segment, the probability that the channel requires immediate attention. This probability map (along with those corresponding to the other audio channels) is then used by the DetEng selection engine to signal priority channels to the RepC dispatcher.

[0045] By way of a second example, a convolutional neural network (CNN) or recurrent neural network (RNN) architecture is implemented. Such an architecture, adapted for determining a priority channel, includes, for example: - convolutional layers, of type 1D or 2D, in order to extract temporal and / or frequency patterns from spectrograms; - pooling layers enabling dimensionality reduction and the capture of salient features; - LSTM or GRU layers to model long-term temporal dependencies in the audio signal; - dense layers allowing for a final integration of features and classification; and - a dense output layer with sigmoid activation to predict the probability of voice activity representative of an emergency situation or requiring the dispatcher's attention. This probability (as well as those corresponding to the other audio channels) is then used by the DetEng selection engine to signal priority channels to the RepC dispatcher.

[0046] As previously stated, inference is performed over longer ranges, for example 1 to 2 seconds, composed of several consecutive segments (spectrograms), and it is this set that allows the representative data of an emergency situation (DRU) to be obtained for the given range, it being understood that the consecutive segments (spectrograms) can continuously feed the process. This segmentation allows for detailed analysis while maintaining temporal continuity, ensuring that no information is lost between the analyzed segments. Using this approach, the classification module (MCI) can analyze the continuous audio stream and detect peaks in vocal activity, for example when energy increases, with good temporal accuracy.For each channel, a data representative of an emergency situation (Dru) is obtained and the process then includes, for the channel selection engine requiring attention DetEng, a selection (S04) of the channel which should attract the attention of the dispatcher (according to the scores given) and then a transmission (S05), to the dispatcher RepC, via the console of. SurvC monitoring, of a signal representative of the priority radio transmission channel (video signal, display on a screen, etc.).

[0047] As previously stated, depending on the operational implementation conditions, the DetEng channel selection engine can instantiate (i.e., create instances—for example, threads in a multithreaded implementation or virtual machines in a suitable implementation) several statistical processing modules, each statistical processing module being responsible for processing one communication channel from among the plurality of communication channels. In such a case, each instance of a statistical processing module provides a result for the channel it processes.These results are sorted and processed (for example, using a mathematical classification function or a specific neural network model designed to detect the most urgent channel) by the DetEng channel selection engine to determine, at any given time, which channel, among all the analyzed channels, has the highest probability of being listened to. In another implementation, the DetEng channel selection engine implements a single statistical processing module, which therefore handles all communication channels either serially or in parallel. The inventors determined that processing one-second audio segments for inference on a server equipped with a recent graphics card (e.g., an NVidia® 40 series card) requires approximately 5 to 10 ms.The total processing time for 10 channels in series is therefore generally close to 50 to 100 ms, which is perfectly compatible with real-time processing.

[0048] Regardless of the classification module architecture and its implementation, training is performed. For example, the classification module, based on a Gated Recurrent Unit (GRU) architecture, is designed to analyze audio streams in real time and detect channels requiring immediate attention. According to this description, the classification module is trained in two phases: supervised pre-training and continuous training. These two phases ensure that the classification module is immediately usable upon deployment and that it continuously improves based on the dispatcher's decisions during interventions.

[0049] Thus, supervised pre-training comprises several steps. The first of these consists of collecting a set of audio data representative of emergency situations. This data may come from historical recordings of past emergency communications, including annotations (made after the fact or representative of channel selections made by the dispatcher). at this point in the intervention) indicating the times when immediate attention was required.

[0050] Once collected (and possibly annotated), the audio data is preprocessed to be compatible with the input of the classification module. This includes implementing transformations identical to those previously described, such as normalization of the audio signals, segmentation into time intervals with partial overlap (50%), and transformation into spectrograms according to the chosen methodology.

[0051] The audio tracks (transformed, i.e., the set of accumulated spectrograms representing a defined time period of 1 to 2 seconds) are labeled according to annotations indicating whether a track requires immediate attention or not. These labels serve as ground truths for supervised training.

[0052] Using these audio tracks, the classification module is trained using the labeled tracks as input and the labels as output. Training is performed by minimizing a loss function (for example, cross-entropy) to adjust the weights of the GRU model. Regularization techniques, such as dropout, can be used to avoid overfitting.

[0053] Finally, in a conventional manner, the model is validated and evaluated on a separate validation dataset to ensure that it generalizes the new data correctly. Performance metrics, such as accuracy, recall, and ROC curve, are used to assess the quality of the model.

[0054] This pre-trained classification module (MCI) is provided to the statistical processing module implemented by the DetEng selection engine for in situ use.

[0055] The pre-trained classification module is then used in operation and, according to this document, is modified by continuous training. More specifically, once the statistical processing module is deployed, the decisions made by the dispatcher during interventions are recorded: these decisions concern channel selection from among several available channels. This includes the channels actually selected by the dispatcher, independently of any (potential) suggestions from the statistical processing module. The audio segments corresponding to the times and channels successively selected by the dispatcher are relabeled according to the new decisions. These new labels serve as ground truths for refining the training.

[0056] The classification module is retrained using the new labels and corresponding audio segments. Continuous training is performed by minimizing the same loss function used during supervised pre-training. The model weights are adjusted to better reflect the dispatcher's actual decisions. A feedback loop is implemented to integrate the dispatcher's new decisions. in the training process. This loop allows the classification module to adapt to the specific preferences of the dispatcher and the changing conditions of emergency interventions.

[0057] Of course, the model is regularly evaluated on validation datasets to ensure that it continues to behave as expected. Performance metrics are monitored to detect any signs of model quality degradation.

[0058] Thus, according to the present, by combining supervised pre-training and continuous training, the classification module is able to provide channel listening proposals from its initial deployment and to continuously improve based on the actual decisions of the dispatcher.

[0059] In a complementary example, the various channels used are processed to obtain data representative of the words, phrases, or expressions spoken by the different members of the different teams. More specifically, the communication channels are digitally processed by the SrvTI server in addition to, or instead of, the processing described above, preferably in parallel with that processing. In this situation, for each channel, the audio signal is processed by a speech recognition engine. More specifically, to achieve this objective, an end-to-end (E2E) transcription module is implemented. The distinctive feature of this transcription module, adapted for this implementation, lies in its ability to process noisy speech data. In such a transcription module, audio preprocessing is performed by a preprocessing sub-module to reduce background noise.This sub-module uses adaptive filtering and source separation techniques to isolate speakers' voices. Although reduced, some background noise remains, depending on the filtering performance and the initial amount of noise present. The pre-processed audio stream is then passed to a "Transformer" sub-module specializing in speech recognition. The transcription module (and the "Transformer" sub-module) also uses digital tagging of the audio stream to identify speakers (when such tagging is present). This information is integrated into the transcription process via a speaker-specific embedding mechanism.The output of the "Transformer" sub-module is processed by a post-processing sub-module that refines the transcription by applying linguistic rules and correcting common errors, particularly when transcribing technical terms or specific words. Speech overlap is handled using a time-separation technique. To achieve this, the transcription module segments the audio stream into short intervals. These short speech intervals are then processed separately. to be merged. Finally, a final formatting sub-module associates each transcribed text segment with the corresponding speaker identifier, when this identifier is available.

[0060] The Transformer model used for recognition was preferentially trained on a large corpus of noisy, multi-speaker audio data. It learned to handle speech overlaps and interference. In particular, the noise can be naturally occurring or specifically added to the audio data corpus: in which case the added noises are those typically encountered in demonstration-type interventions, for example. The Transformer model uses attention mechanisms to focus on the relevant parts of the audio signal. This allows for better handling of long sequences and long-term dependencies in speech extracted from different channels, and in particular for taking into account jargon or acronyms used by speakers, in order to obtain the most faithful transcription of the transcribed speech.To improve its robustness, the model can incorporate a "data augmentation" technique, in which, during training, artificial noise is added to clean audio samples used for training. According to this model, the added noise corresponds to noise that is actually representative of the noise captured in the channel during the interventions, such as crowd noise (shouting), explosions, and vehicle noise. In this way, the model is able to generalize better to real-world noisy conditions.

[0061] In any event, once the text is transcribed, it is provided by the speech recognition engine to the SrvTI server. This provision of the transcribed text may occur in addition to, or instead of, the audio stream, as described above, depending in particular on the operational implementation conditions and the latency required to obtain the transcribed text (real-time compatibility). When implemented in conjunction with the audio stream, the MTS statistical processing module uses both the transcriptions provided by the speech recognition engine and the audio stream from the channels to determine data representative of emergency situations. The methodology for processing the audio streams is identical to that described above. Consequently, the MCI classification module uses the textual transcription data in addition to the sequences of spectral feature segments.To achieve this, in one example implementation, the neural network architecture of the MCI classification module is modified to allow for the consideration of both spectral features and the transcribed text associated with these features, by two neural networks inferring the data in parallel. More specifically, a CNN of the type already described determines a first vector representing an emergency level based on the spectral features and a... A second neural network, of the RNN type, determines a representative vector of a second urgency level based on the transcribed text. Figure 3 illustrates an example of the architecture of this implementation.

[0062] The detail of a possible architecture is as follows. It comprises several interconnected modules allowing the capture of the characteristics specific to each modality, then merging them to produce a robust prediction of the class associated with an audio stream.

[0063] The system inputs include, on the one hand, spectrogram segments extracted from an audio stream and, on the other hand, associated text transcripts. The spectrograms, which are time- and frequency-domain representations, are processed by a specific module. This module comprises convolutional neural networks (CNNs) that extract relevant local patterns in the time-frequency dimensions. The resulting features are fed to a bidirectional random neural network (RNN) or a transformer, enabling the capture of global temporal relationships in the audio segment. The final result is a first global vector representing a level of urgency.

[0064] In parallel, the transcribed text is processed by a separate module. This module converts the tokens into embedding vectors, for example using pre-trained models such as Word2Vec or BERT, or via a specific embedding (allowing for better handling of acronyms or slang words). These vectors are provided in an RNN (such as a GRU or an LSTM) or in a Transformer, which encodes the sequential and contextual structure of the text. This produces a vector representing the second level of urgency.

[0065] Then the audio and text vectors are merged in a multimodal step. The merging can be done simply, by concatenating the vectors, or by a more relevant approach, such as a weighted combination of the modalities (aD_a + |3D_t), which can take into account, for example, the quality of the input transcription. An alternative method would be to use an intermodal attention mechanism to allow the module to automatically determine the relative importance of each modality for the classification task.

[0066] The merged vector is provided in a final classifier composed of successive dense layers with ReLU activations and regularization techniques such as dropout. Finally, an output layer with a softmax activation produces the probabilities associated with the target class (emergency situation or not), while also providing the associated probability to enable subsequent channel classification and inform the dispatcher.

Claims

Demands

1. A method for determining data representative of emergency situations from audio signals, said audio signals being received from a transmission channel belonging to a set of transmission channels used by a set of response teams, each comprising at least one transmitter and at least one member, each response team using a different transmission channel, said method being implemented via an electronic device (SrvTI) comprising at least one processing unit and a memory, said method comprising at least one iteration of the following steps: - acquisition (SOI) of an audio signal from a current transmission channel among the set of transmission channels; - processing (S02) of the audio signal comprising: sampling at a predetermined frequency;a segmentation of the sampled signal according to a predetermined interval, delivering a sequence of audio segments; and for each audio segment, an extraction of spectral features, delivering a sequence of segments of spectral features; - determination (S03) of a first data representative of an emergency situation of the current transmission channel as a function of the sequence of segments of spectral features of the audio signal.;

2. Method of determination, according to claim 1, characterized in that the steps of acquisition (SOI) of audio signal, processing (S02) of audio signal and determination (S03) of a first data representative of an emergency situation are carried out in parallel for all audio channels.

3. A method of determination, according to any one of the preceding claims, characterized in that the extraction of spectral characteristics includes a step of transforming each segment into a visual representation of the energy distribution of the sound signal segment.

4. A determination method according to any one of the preceding claims, characterized in that the determination step (S03) of the data representative of an emergency situation of the current transmission channel as a function of the sequence of segments of spectral characteristics of the audio signal includes at least one inference step, by a classification module (MCI), of a pre-trained neural network.

5. Method of determination, according to claim 4, characterized in that the pre-trained neural network is of the recurrent neural network type.

6. A method of determination, according to any one of the preceding claims, characterized in that the determination step (S03) of the data representing an emergency situation of the current transmission channel as a function of the sequence of segments of spectral characteristics of the audio signal takes into account at least one prior audio signal classification decision by a dispatcher.

7. A method of determination, according to any one of the preceding claims, characterized in that it further comprises, for said current channel: - a step of transcribing the voices present in the audio signal into a text representative of the words spoken on said current channel; - a step of determining, using a second pre-trained neural network, a second data point representative of an emergency situation; - a step of merging the first data point representative of an emergency situation and that of a second data point representative of an emergency situation; and - a step of determining a final data point representative of an emergency situation using a classification module.

8. A method of determination, according to any one of the preceding claims, characterized in that a priority transmission channel, among the set of transmission channels, is selected (S04) based on representative data of transmission channel emergency situations determined for each channel used from the set of transmission channels.

9. Method of determination, according to claim 8, characterized in that it further comprises a transmission step (S05), to a dispatcher (RepC), via a monitoring console (SurvC), of a signal representative of the priority transmission channel.

10. A system for determining data representative of emergency situations from audio signals, said audio signals being received from a transmission channel belonging to a set of transmission channels used by a set of response teams, each comprising at least one transmitter and at least one member, each response team using a different transmission channel, said system comprising at least one electronic device including at least one processing unit and a memory, which includes means for iteratively implementing the following steps: - acquiring an audio signal from a current transmission channel among the set of transmission channels, - processing the audio signal including: sampling at a predetermined frequency; segmenting the sampled signal according to a predetermined interval, delivering a sequence of audio segments;and for each audio segment, an extraction of spectral features, delivering a sequence of spectral feature segments; - determination of data representative of an emergency situation of the current transmission channel as a function of the sequence of spectral feature segments of the audio signal.

11. Computer program comprising instructions for carrying out the method according to any one of claims 1 to 9, when said instructions are executed by a processor of a computer processing circuit.

Images