Method and system for visual monitoring of the operation of an emergency conference system device

Abstract

The application provides a method and system for visual monitoring of emergency conference system equipment operation, audio signals at multiple positions in a target conference room are collected; a mixed signal formed by the audio signals is retrieved, the mixed signal is separated into multiple audio channels to form a matrix; a buffer area is set for each audio channel; the order of audio data input and output is controlled, left and right channel audio data of each channel is stored in a corresponding slave buffer area, and the slave buffer area is recalled to a master buffer area; volume size analysis, echo detection, howling detection and bottom noise detection are performed on audio data of the master buffer area; and audio quality abnormal conditions are recorded and displayed. The application realizes detection of the running state of an audio processor and a public address system, facilitates monitoring of the running state of equipment by operation and maintenance personnel, and improves operation and maintenance efficiency.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of digital audio signals, and particularly relates to a method and system for visual monitoring of operation of emergency conference system equipment. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.

[0003] The device operation visualization technology is an intelligent monitoring technology integrating panoramic monitoring of operation of emergency communication system equipment, rapid positioning of faults, real-time monitoring and analysis of video quality, one-key inspection of equipment, and other requirements. Through information support tools, the problems of non-uniform management of emergency communication system equipment, untimely discovery of fault alarms, slow positioning of fault causes, great difficulty in inspection, and difficulty in analysis of operation data are solved, the operation and maintenance cost of the emergency communication system is reduced, and the quality and efficiency of emergency communication management services are improved. Therefore, the audio signals at multiple positions in the conference room are collected, separated, stored, and analyzed in real time to detect the operation state of the audio processor and the public address system, which facilitates the monitoring of the operation state of the equipment by the operation and maintenance personnel.

[0004] Chinese patent document CN114339130A discloses a method for monitoring the operation state of conference room equipment. The method receives a preset prompt sound sent by a main conference equipment, performs sound recognition on the preset prompt sound, generates feedback information indicating that the sound is unobstructed if the sound recognition result is the same as a pre-stored sound, and sends the feedback information to the main conference equipment. If the sound recognition result is not the same as the pre-stored sound, feedback information indicating that the sound is obstructed is generated and sent to the main conference equipment. This method lacks quantitative and qualitative analysis of sound signals, and cannot intuitively and comprehensively display the sound quality at each place in the conference room, and has certain limitations. SUMMARY

[0005] To solve the above problems, the present application provides a method and system for visual monitoring of operation of emergency conference system equipment. The present application detects the operation state of the audio processor and the public address system, facilitates the monitoring of the operation state of the equipment by the operation and maintenance personnel, and improves the operation and maintenance efficiency.

[0006] According to some embodiments, the present application adopts the following technical solutions:

[0007] A method for visual monitoring of operation of emergency conference system equipment, comprising the following steps:

[0008] Collecting audio signals at multiple positions in a target conference room;

[0009] Retrieving a mixed signal formed by the audio signals, separating the mixed signal into multiple audio channels to form a matrix;

[0010] Setting a buffer for each audio channel;

[0011] Controlling the input and output order of the audio data, so that the left and right channel audio data of each channel is stored in the corresponding slave buffer, and the slave buffer is called back to the master buffer;

[0012] Performing volume size analysis, echo detection, howling detection, and noise floor detection on the audio data of the master buffer;

[0013] Recording and displaying the audio quality abnormality.

[0014] As an optional implementation, the specific process of collecting audio signals at multiple positions in the target conference room includes installing ceiling microphones at multiple positions on the top of the conference room to collect audio signals at those positions, and collecting multiple audio signals through different audio input terminals.

[0015] As an optional implementation, the specific process of retrieving a mixed signal formed by the audio signals, separating the mixed signal into multiple audio channels includes dividing N microphone signals into N / 2 audio channels in the mixed signal, each channel containing left and right channels, inputting the microphone signals in vector form, and inputting the audio channels in matrix form.

[0016] As an optional implementation, it further includes setting the audio format, setting the audio sampling format to pulse code modulation, setting the number of channels to left and right channels, setting the sampling interval and sampling rate, converting the sampled signal into a discrete time and discrete amplitude digital signal, setting the sampling bit number, setting the number of sampling bytes per second, and setting the block alignment mode.

[0017] As an optional implementation, the specific process of setting a buffer for each audio channel includes opening two slave buffers for each audio channel, setting a master buffer, and outputting data from the slave buffer to the master buffer.

[0018] As an optional implementation, the specific process of performing volume size analysis, echo detection, howling detection, and noise floor detection includes performing corresponding analysis through a volume analysis sub-thread, an echo detection sub-thread, a howling detection sub-thread, and a noise floor detection sub-thread, respectively.

[0019] As an optional implementation, the specific process of displaying the audio quality analysis result includes drawing a visual graphical interface, drawing a separate display area for each position where an audio signal is collected in the meeting venue, displaying the volume size and change, and abnormal conditions, converting the analysis results into a form, and displaying them in the corresponding area.

[0020] An emergency conference system device operation visual monitoring system comprises:

[0021] An audio acquisition module is configured to acquire audio signals at multiple positions in a target conference room.

[0022] A channel sound track separation module is configured to retrieve a mixed signal formed by the audio signals, separate the mixed signal into multiple audio channels, and form a matrix.

[0023] A channel control module is configured to control the input and output order of the audio data, store left and right channel audio data of each channel into a corresponding slave buffer, and call back the slave buffer to a master buffer.

[0024] An audio quality analysis module is configured to perform volume size analysis, echo detection, howling detection, and background noise detection on the audio data in the master buffer.

[0025] An interface display module is configured to display the audio analysis results.

[0026] A record storage module is configured to record audio quality abnormal conditions.

[0027] A computer readable storage medium is configured to store computer instructions, and the computer instructions are executed by a processor to complete the steps in the above method.

[0028] An electronic device comprises a memory and a processor, and computer instructions stored in the memory and running on the processor, and the computer instructions are executed by the processor to complete the steps in the above method.

[0029] Compared with the prior art, the present application has the following advantages:

[0030] The present application can detect the running state of the audio processor and the public address system, facilitate the monitoring of the running state of the device by the operation and maintenance personnel, and improve the operation and maintenance efficiency.

[0031] The present application lacks quantitative and qualitative analysis of sound signals, and cannot intuitively and comprehensively display the sound quality at each position in the venue, and has certain limitations.

[0032] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0033] The drawings accompanying the specification of the present application form part of the present application and are used to provide a further understanding of the present application, the schematic embodiments of the present application and the description thereof are used to explain the present application, and do not constitute an improper limitation on the present application.

[0034] Figure 1A system flowchart of the embodiment;

[0035] Figure 2 A flowchart for programming to realize multi-channel signal separation and storage;

[0036] Figure 3 A display interface diagram of the audio quality detection result;

[0037] Figure 4 An abnormal situation record interface diagram. DETAILED DESCRIPTION

[0038] The application will be further described below in conjunction with the accompanying drawings and embodiments.

[0039] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the application. Unless otherwise indicated, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the application pertains.

[0040] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and it should also be understood that when the terms "comprise" and / or "include" are used in the specification, there is a presence of a feature, step, operation, device, component, and / or combinations thereof.

[0041] Embodiment one

[0042] An emergency conference system audio device operation visual monitoring system belongs to the technical field of digital audio signal processing. As shown in the figure, the system comprises the following modules: an audio acquisition module, a channel control module, a signal storage module, a channel sound track separation module, an audio quality analysis module, an interface display module, and a record storage module. Figure 1 The audio acquisition module acquires audio signals at multiple positions and inputs the mixed signals to the host computer; the channel control module programs the sound card to control the input order of each channel signal and stores the acquired signals in the signal storage module; the channel sound track separation module restores the mixed signals in the signal storage module to multiple single-channel signals; the audio quality analysis module analyzes the audio quality of each signal; the interface display module displays the analysis results in real time on the QT interface; and the record storage module records abnormal situations and combines the QT to visually display and locally store the abnormal records.

[0043] The audio acquisition module acquires audio signals at multiple positions and inputs the mixed signals to the host computer; the channel control module programs the sound card to control the input order of each channel signal and stores the acquired signals in the signal storage module; the channel sound track separation module restores the mixed signals in the signal storage module to multiple single-channel signals; the audio quality analysis module analyzes the audio quality of each signal; the interface display module displays the analysis results in real time on the QT interface; and the record storage module records abnormal situations and combines the QT to visually display and locally store the abnormal records.

[0044] Embodiment two

[0045] An emergency conference system audio equipment operation visual monitoring method, comprising the following steps:

[0046] 1) Collect audio signals at multiple positions

[0047] a. Install ceiling microphones at multiple positions on the top of the conference room to collect audio signals at the positions;

[0048] b. Connect the multiple microphones to the multiple audio input terminals of a multi-interface sound card device through a Karon line;

[0049] c. Connect the output terminal of the sound card device to the input terminal of the host computer through a USB interface to complete the collection of the multiple microphone signals to the host computer;

[0050] 2) Retrieve the mixed signals input to the host computer

[0051] a. For convenience of description, the number of input microphones is denoted as N, and N is an even number.

[0052] b. In the mixed signals input to the host computer, the sound card device divides the N microphone signals into N / 2 audio channels, and each channel contains left and right sound channels;

[0053] c. The microphone signal input is denoted as the following vector:

[0054]

[0055] where x i , i = 1, 2, …, N represents each microphone input signal;

[0056] d. The audio channel input is denoted as the following matrix:

[0057]

[0058] where M = N / 2, X i represents each audio channel, the left sound channel and the right sound channel

[0059] 3) Set the audio format

[0060] a. Create an audio format structure instance wavform, and set the audio sampling format wavform.wFormatTag to WAVE_FORMAT_PCM, i.e. pulse code modulation (PCM). Pulse code modulation includes three steps of sampling, quantization and encoding. The sampling process converts a continuous time analog signal into a discrete time and continuous amplitude sampled signal; the quantization process converts the sampled signal into a discrete time and discrete amplitude digital signal; and the encoding process encodes the quantized signal into a binary code group output.

[0061] b. Set the number of channels as waveform.nChannels to 2, corresponding to two channels of left and right, corresponding to

[0062] c. Set the sampling rate as waveform.nSamplesPerSec to 44100, the relationship between the sampling interval and the sampling rate is expressed as the following formula:

[0063]

[0064] Where T is the sampling interval, f is the sampling rate, in Hz;

[0065] d. The sampling process of the signal can be expressed as the following formula:

[0066]

[0067]

[0068] Where, F PCM represents sampling by the PCM method, represents the sampled microphone input signal vector, represents the sampled audio channel input signal matrix, x i (nT), i = 1, 2, …, N represents the instantaneous value obtained by sampling the i-th microphone signal, respectively represent the instantaneous values obtained by sampling the left and right channel signals of the j-th channel, and nT represents the sampling time;

[0069] d. Set the sampling bit number waveform.wBitsPerSample to 16, which means that the instantaneous value obtained by sampling is quantized according to its amplitude, using a set of 16 values to quantize it.

[0070] e. Set the number of bytes sampled per second waveform.nAvgBytesPerSec to 44100x16x2 / 8, the calculation method is as follows:

[0071] nAvgBytesPerSec = nSamplesPerSec x wBitsPerSample x nChannels / 8

[0072] Where nSamplesPerSec is the sampling rate, wBitsPerSample is the sampling bit number, and nChannels is the number of channels;

[0073] f. Set the block alignment waveform.nBlockAlign to 2x16 / 8, the calculation method is as follows:

[0074] nBlockAlign = nChannelsw x wBitsPerSample / 8

[0075] where nChannelsw is the number of channels, wBitsPerSample is the number of bits per sample;

[0076] 4) Set up the buffer

[0077] a. In order to prevent the loss of audio data, two slave buffers are set up for each audio channel considering the limited size of the buffer. All slave buffers can be represented by the following matrix:

[0078]

[0079] where b i1 , b i2 , i = 1, 2, …, M represent the two slave buffers corresponding to the i-th input channel;

[0080] b. Set up the master buffer B, and design the callback function waveInProc() to capture the callback signal WIM_DATA, output the data in the slave buffer to the master buffer, which is used for subsequent data storage and audio quality analysis.

[0081] 5) Control the order of audio data input and output

[0082] a. Design the timing function Time() to control the order of opening the audio channels, and store the left and right channel audio data of M channels into the corresponding slave buffer every 1 second.

[0083] b. Output the audio data in the slave buffer to the master buffer through the callback function waveInProc().

[0084] 6) Store the audio signal

[0085] Use the CFile::Write function provided by the base class CFile of the MFC file class to access the master buffer, and store each route of microphone audio data as a wav format audio file respectively;

[0086] 7) Audio quality analysis

[0087] a. Design the volume analysis sub-thread VolumeMeasureThread to analyze the volume size of the audio data in the master buffer;

[0088] b. Design the echo detection sub-thread EchoDetectionThread to detect the echo of the audio data in the master buffer. Set the echo detection threshold Th echo , if the detection result is greater than Th echoIf so, then an echo is considered to exist;

[0089] c. Design a feedback detection sub-thread, WhistleDetectionThread, to perform feedback detection on the audio data in the main buffer. Set the feedback detection threshold Th. whist If the detection result is greater than Th whist If so, it is assumed that a howling sound exists;

[0090] d. Design a noise detection sub-thread, NoiseDetectionThread, to perform noise detection on the audio data in the main buffer; set the noise detection threshold Th. noise If the detection result is greater than Th noise If so, then it is considered that there is background noise;

[0091] 8) Display audio quality analysis results

[0092] a. Use Qt Creator to create a visual graphical interface, drawing a separate display area for each location in the meeting room where audio signals are being collected, such as... Figure 3 As shown, the volume level and changes are displayed in the form of a dynamic volume bar chart, and abnormal situations are indicated by changing the text color.

[0093] b. Design a volume display sub-thread, VolumeThread, to receive the volume analysis results sent by the VolumeMeasureThread sub-thread, and convert the volume into a bar chart and display it in the corresponding display area;

[0094] c. Design an echo display sub-thread EchoThread to receive the echo detection results sent by the EchoDetectionThread sub-thread, convert the results into text color format, and display them in the corresponding display area;

[0095] d. Design a howling detection sub-thread WhistleThread to receive howling detection results sent by the WhistleDetectionThread sub-thread, convert the results into text color format, and display them in the corresponding display area;

[0096] e. Design a background noise display sub-thread NoiseThread to receive the background noise detection results sent by the NoiseDetectionThread sub-thread, and convert the results into text color and display them in the corresponding display area;

[0097] 9) Record audio quality anomalies.

[0098] a. Using the QXlsx::Document function provided by the QtXlsxWriter library, create an Excel file to the local folder, and use the date as the file name;

[0099] b. Using the QXlsx::write function provided by the QtXlsxWriter library, create 6 table items in the Excel table, as shown in Figure 4 , respectively, "time", "conference room", "location", "case classification", "abnormal time" and "operation";

[0100] c. Using the QXlsx::saveAs function provided by the QtXlsxWriter library, save all detection records inserted into the Excel table as 1 record for 1 abnormal case.

[0101] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principles of the present application without creative labor shall be included in the protection scope of the present application.

Claims

1. A method for emergency conference system equipment operation visualized monitoring, characterized in that, The method comprises the following steps: Collecting audio signals at multiple positions in a target conference room; The specific process of collecting audio signals at multiple positions in a target conference room comprises installing ceiling microphones at multiple positions on the top of the conference room to collect audio signals there, and collecting multiple audio signals through different audio input terminals respectively; Retrieving a mixed signal formed by the audio signals and separating the mixed signal into multiple audio channels to form a matrix; Setting a buffer for each audio channel; The specific process of setting a buffer for each audio channel comprises opening two slave buffers for each audio channel, setting a master buffer, and outputting data from the slave buffer to the master buffer; Controlling the input and output order of audio data so that left and right channel audio data of each channel is stored in the corresponding slave buffer and recalled from the slave buffer to the master buffer; Performing volume size analysis, echo detection, howling detection, and background noise detection on the audio data in the master buffer; The specific process of performing volume size analysis, echo detection, howling detection, and background noise detection comprises performing corresponding analysis through a volume analysis sub-thread, an echo detection sub-thread, a howling detection sub-thread, and a background noise detection sub-thread respectively; Recording and displaying audio quality abnormal conditions; The specific process of displaying audio quality analysis results comprises drawing a visual graphical interface, drawing a separate display area for each position where an audio signal is collected in the conference, displaying volume size and changes and abnormal conditions, and converting the analysis results into a form for display in the corresponding area.

2. A method of operating visual monitoring of an emergency conference system device as claimed in claim 1, characterized in that The specific process of retrieving the mixed signal formed by the individual audio signals and separating it into multiple audio channels is included in the mixed signal. Road microphone signal split Each audio channel contains two channels, left and right. The microphone signal input is represented as a vector, and the audio channel input is represented as a matrix.

3. A method for operating visual monitoring of an emergency conference system device according to claim 1, characterized in that, Further comprising setting an audio format, setting the audio sampling format to pulse code modulation, setting the number of sound channels to two left and right channels, setting the sampling interval and sampling rate, converting the sampling signal into a discrete time and discrete amplitude digital signal, setting the sampling bit number, setting the number of sampling bytes per second, and setting the block alignment mode.

4. An emergency conference system equipment operation visualization monitoring system applying the emergency conference system equipment operation visualization monitoring method of any one of claims 1-3, characterized in that it comprises: An audio collection module for collecting audio signals at multiple positions in a target conference room; A channel sound separation module for retrieving a mixed signal formed by the audio signals and separating the mixed signal into multiple audio channels to form a matrix; A channel control module for controlling the input and output order of audio data so that left and right channel audio data of each channel is stored in the corresponding slave buffer and recalled from the slave buffer to the master buffer; An audio quality analysis module for performing volume size analysis, echo detection, howling detection, and background noise detection on the audio data in the master buffer; An interface display module for displaying audio analysis results; A record storage module for recording audio quality abnormal conditions.

5. A system for operating visual monitoring of an emergency conference system according to claim 4, c h a r a c t e r i s e d i n that The specific process of retrieving the mixed signal formed by the individual audio signals and separating it into multiple audio channels is included in the mixed signal. Road microphone signal split Each audio channel contains two channels, left and right. The microphone signal input is represented as a vector, and the audio channel input is represented as a matrix.

6. A system for operating visual monitoring of an emergency conference system according to claim 4, characterized in that Further comprising setting an audio format, setting the audio sampling format to pulse code modulation, setting the number of sound channels to two left and right channels, setting the sampling interval and sampling rate, converting the sampling signal into a discrete time and discrete amplitude digital signal, setting the sampling bit number, setting the number of sampling bytes per second, and setting the block alignment mode.

7. A computer readable storage medium characterized by, Computer program product for storing computer instructions which, when executed by a processor, perform the steps of the method of any one of claims 1-3.

8. A computer readable storage medium as in claim 7, wherein, The specific process of retrieving the mixed signal formed by the individual audio signals and separating it into multiple audio channels is included in the mixed signal. Road microphone signal split Each audio channel contains two channels, left and right. The microphone signal input is represented as a vector, and the audio channel input is represented as a matrix.

9. An electronic device, characterized by comprising: Computer program product comprising a memory and a processor and computer instructions stored on the memory and running on the processor, which, when executed by the processor, perform the steps of the method of any one of claims 1-3.

10. An electronic device as claimed in claim 9, characterized in that The specific process of retrieving the mixed signal formed by the individual audio signals and separating it into multiple audio channels is included in the mixed signal. Road microphone signal split Each audio channel contains two channels, left and right. The microphone signal input is represented as a vector, and the audio channel input is represented as a matrix.

Images