Security monitoring situation awareness method and system based on big data

Abstract

The application is suitable for the field of monitoring situation awareness technology, and particularly relates to a security monitoring situation awareness method and system based on big data, which comprises the following steps: delimiting a control area of security monitoring, selecting a plurality of monitoring points, wherein the monitoring points at least include sensitive points and protective points, collecting audio data by using a sound pickup device pre-deployed in the monitoring points, and preprocessing the audio data; constructing a neural network model, inputting the preprocessed audio data into the neural network model, outputting a plurality of audio features, creating a sample set composed of abnormal features and background features, and comparing the audio features with the sample set. By identifying abnormal features, the application can timely discover potential hidden dangers and avoid security risks. By establishing a dialogue, the application can verify abnormal features, reduce false positive rates, reduce unnecessary intervention, avoid interfering with normal activities in the control area, and greatly improve the user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of situational awareness, in particular to a security monitoring situational awareness method and system based on big data. BACKGROUND

[0002] Security monitoring devices are widely used in daily monitoring, most of which are composed of cameras and corresponding transmission and processing modules. By using security monitoring devices, the pictures within the monitoring range are transmitted to terminals such as computers and mobile phones in real time, and after analysis and processing, the security risks are identified to realize situational awareness.

[0003] When identifying behaviors of monitoring data, errors may be caused by environmental changes (such as light, angle, background, etc.), which may further cause false positives or false negatives. In addition, the massive storage data and computing resource consumption also greatly limit the use scenarios of the camera. Therefore, how to identify risks and verify using a pickup device is a technical problem to be solved by the present application. SUMMARY

[0004] The present application aims to provide a security monitoring situational awareness method and system based on big data to solve the problem of how to identify risks and verify using a pickup device as mentioned in the background.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0006] The security monitoring situational awareness method based on big data comprises:

[0007] The control area of security monitoring is delineated, and a plurality of monitoring points are selected, wherein the monitoring points at least include sensitive points and protective points. Audio data is collected by using pickup devices pre-deployed in the monitoring points, and pre-processing is performed.

[0008] A neural network model is constructed, and the pre-processed audio data is input into the neural network model to output a plurality of audio features. A sample set composed of abnormal features and background features is created. The audio features and the sample set are compared to determine whether there are abnormal features in the audio data.

[0009] If yes, the speaker pre-integrated in the pickup device is used to play interactive voice and establish a dialogue. Real-time audio data is continuously collected to identify target features. The voiceprint library pre-constructed is compared to determine whether a reply from authorized personnel is received. If yes, the dialogue is disconnected. If no, the risk coordinates are located based on the monitoring points, and the risk coordinates are sent to a preset terminal to start an alarm mechanism.

[0010] Further, the step of collecting audio data by using the sound pickup device pre-deployed in the monitoring point comprises:

[0011] The management platform of the audio data is established, the unique identifier of the sound pickup device is configured, and the sound pickup device is connected to the management platform;

[0012] The mapping between the unique identifier and the monitoring point is established, a location query table is constructed, and uploaded to the management platform.

[0013] Further, the step of collecting audio data and preprocessing comprises:

[0014] The audio data is preprocessed, wherein the preprocessing at least includes data cleaning, format unification and noise reduction;

[0015] The sound pickup device is used to establish a cooperative array and embed a time stamp;

[0016] The cooperative array and the time stamp are integrated to calculate the time difference and locate the generation position of the audio data.

[0017] Further, the step of constructing a neural network model and inputting the preprocessed audio data into the neural network model to output a plurality of audio features comprises:

[0018] The multi-modal data at the generation position is collected, wherein the multi-modal data at least includes video data and temperature data, and the audio features are corrected;

[0019] The priority of the multi-modal data and the audio features is configured.

[0020] Further, the step of determining whether there is an abnormal feature in the audio data comprises:

[0021] The environmental change at the monitoring point is obtained, and the sample set is dynamically updated;

[0022] The risk level of each abnormal feature is set, and the disposal order is determined.

[0023] Further, the step of playing interactive voice by using the speaker pre-integrated in the sound pickup device and establishing a dialogue comprises:

[0024] The deactivation authority of the speaker is issued to the authorized personnel, and a silence mechanism based on the dialogue is established;

[0025] The sound pickup device and the speaker are integrated into a preset multi-task platform.

[0026] Further, the method further comprises:

[0027] In the management area, the edge device is traversed, and the management platform, neural network model, sample set and voiceprint library are deployed to the edge device;

[0028] The risk value is embedded into the edge device, and a balanced strategy is constructed.

[0029] Further, the system comprises:

[0030] The acquisition module is configured to delimit a management area of security monitoring, select a plurality of monitoring points, wherein the monitoring points at least include sensitive points and protective points, collect audio data by using a sound pickup device pre-deployed in the monitoring points, and perform preprocessing on the audio data;

[0031] The identification module is configured to construct a neural network model, input the preprocessed audio data into the neural network model, output a plurality of audio features, create a sample set composed of abnormal features and background features, and compare the audio features with the sample set.

[0032] The judgment module is configured to determine whether there is an abnormal feature in the audio data, if yes, play interactive voice by using a loudspeaker pre-integrated in the sound pickup device, establish a dialogue, continue to collect real-time audio data, identify target features, compare a pre-constructed voiceprint library, determine whether a reply of an authorized person is received, if yes, disconnect the dialogue, and if no, locate a risk coordinate based on the monitoring points, and send the risk coordinate to a preset terminal to start an alarm mechanism.

[0033] Further, the acquisition module comprises:

[0034] The access unit is configured to establish a management platform of the audio data, configure a unique identifier of the sound pickup device, and access the sound pickup device to the management platform.

[0035] The uploading unit is configured to establish a mapping between the unique identifier and the monitoring points, construct a location query table, and upload the location query table to the management platform.

[0036] The preprocessing unit is configured to preprocess the audio data, wherein the preprocessing at least includes data cleaning, format unification and noise reduction.

[0037] The embedding unit is configured to use the sound pickup device to establish a cooperative array and embed a timestamp.

[0038] The positioning unit is configured to integrate the cooperative array and the timestamp, calculate a time difference, and locate a generation position of the audio data.

[0039] Further, the identification module comprises:

[0040] A correction unit is configured to collect multi-modal data at the generation position, wherein the multi-modal data at least includes video data and temperature data, and correct the audio feature;

[0041] A configuration unit is configured to configure the priority of the multi-modal data and the audio feature.

[0042] Compared with the prior art, the present application has the following beneficial effects:

[0043] By delimiting the monitoring area, the monitoring dead angle can be eliminated, and the monitoring accuracy can be improved. By deploying the sound pickup device, the control area can be monitored all day long without being affected by the environment. By constructing the neural network model, the processing efficiency of the audio data is greatly improved, while the accuracy of the posture perception data is ensured. By identifying the abnormal features, potential hidden dangers can be found in time, and safety risks can be avoided. By establishing the dialogue, the abnormal features can be verified, the false positive rate can be reduced, unnecessary intervention can be reduced, normal activities in the control area can be avoided, and the use experience is greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 The flowchart of the security monitoring situation awareness method based on big data provided by the embodiment of the present application is provided.

[0045] Figure 2 The first sub-flowchart of the security monitoring situation awareness method based on big data provided by the embodiment of the present application is provided.

[0046] Figure 3 The second sub-flowchart of the security monitoring situation awareness method based on big data provided by the embodiment of the present application is provided.

[0047] Figure 4 The third sub-flowchart of the security monitoring situation awareness method based on big data provided by the embodiment of the present application is provided.

[0048] Figure 5 The composition diagram of the security monitoring situation awareness system based on big data provided by the embodiment of the present application is provided.

[0049] Figure 6 The composition diagram of the acquisition module in the security monitoring situation awareness system based on big data provided by the embodiment of the present application is provided.

[0050] Figure 7 The composition diagram of the identification module in the security monitoring situation awareness system based on big data provided by the embodiment of the present application is provided.

[0051] Figure 8 The composition diagram of the judgment module in the security monitoring situation awareness system based on big data provided by the embodiment of the present application is provided. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0053] In Example 1, Figure 1 The implementation process of the security monitoring situation awareness method based on big data provided by the embodiment of the present application is shown, and the following is described in detail as follows:

[0054] S100: delineate a control area of security monitoring, select a plurality of monitoring points, wherein the monitoring points at least include: sensitive points and protective points, use the sound pickup device pre-deployed in the monitoring points to collect audio data and perform preprocessing.

[0055] The area that needs to be monitored for security, i.e. the control area, can be a building, a community, an industrial park, etc. According to the building structure and equipment distribution of the control area, a plurality of monitoring points are selected from the control area, which are the deployment positions of the sound pickup devices. The monitoring points include sensitive points and protective points, etc. The sensitive points usually refer to areas prone to security incidents, such as entrances, important passages, and crowded places, etc. The protective points are mainly fences, building walls, and parking lots, etc. In this embodiment, the division of monitoring points is not limited, in other words, the monitoring points can be composed of a single type or multiple different types of points. Sound pickup devices are deployed at the monitoring points, which can be sound pickups or microphones, etc. The sound pickup devices are used to collect audio data at the deployment points and perform preprocessing, which includes data cleaning, format unification, and noise reduction, etc.

[0056] S200: construct a neural network model, input the preprocessed audio data into the neural network model, output a plurality of audio features, create a sample set composed of abnormal features and background features, and compare the audio features with the sample set.

[0057] The neural network model is created, which adopts a convolutional neural network architecture and can accurately identify the timing features and spatial features in the audio data. After the neural network model is constructed, it also needs to be trained using historical data. The preprocessed audio data is input into the trained neural network model, and a plurality of audio features are generated in the output layer of the neural network, including glass breaking sound, door opening sound, and human voice, etc.

[0058] A large number of existing audio features are collected and labeled to obtain a sample set, in other words, the sample set is also a set of audio features; the audio features in the sample set are classified into abnormal features and background features, wherein the abnormal features include explosion sound, glass breaking sound, scream and door opening sound, and the background features include normal environmental sound such as wind and rain sound, human voice and mechanical operation sound; specifically, the audio features can also be refined, for example: the door opening sound is refined, the normal door opening sound is determined as a background feature, and the door opening sound with multiple attempts or accompanied by lock core abnormal sound is determined as an abnormal feature.

[0059] S300: determining whether there is an abnormal feature in the audio data, if yes, playing interactive voice by using the speaker integrated in the sound pickup device, establishing a dialogue, continuing to collect real-time audio data, identifying a target feature, comparing with a pre-constructed voiceprint library, determining whether a reply of an authorized person is received, if yes, disconnecting the dialogue, and if no, locating a risk coordinate based on the monitoring point, and sending the risk coordinate to a preset terminal to start an alarm mechanism.

[0060] determining whether there is an abnormal feature in the collected audio feature, if yes, playing interactive voice by using the speaker, wherein the interactive voice can be: “what do you need help with?”, or “please say your employee number”; using the sound pickup device and the speaker to interact with the visitor through voice recognition and voice synthesis technology, and establishing a dialogue; identifying the voice feature of the visitor from the dialogue, and defining the voice feature as a target feature, comparing the target feature with known samples in the voiceprint library, determining whether there is a same sample in the voiceprint library, if yes, indicating that the visitor is an authorized person, and disconnecting the dialogue; if there is no same sample in the voiceprint library, there can be two cases, one is that the visitor replies, but the visitor is not authorized, and the other is that no reply is received; finding out a risk coordinate according to the position of the monitoring point, wherein the risk coordinate is also the position of the abnormal feature, for example, the abnormal feature is a door opening sound, and the risk coordinate is also the position of the door; sending the risk coordinate to a preset terminal, wherein the preset terminal can be a terminal of a security personnel in a control area, and simultaneously starting an alarm mechanism, wherein the alarm mechanism is a specific alarm method, for example, using a buzzer deployed near the risk coordinate to alarm.

[0061] In embodiment 2, Figure 2 The implementation process of the security monitoring situation awareness method based on big data provided by the embodiment of the application is shown, and the step of collecting audio data by using the sound pickup device pre-deployed in the monitoring point is described in detail as follows:

[0062] S101: Assemble the management platform of the audio data, configure the unique identifier of the sound pickup device, and access the sound pickup device to the management platform.

[0063] A unique identifier is assigned to each sound pickup device, which can be the serial number, MAC address or device number of the device, and the sound pickup device is accessed to the management platform through a wireless or wired network; the management platform is responsible for receiving, storing, processing and analyzing the audio data collected by each sound pickup device, and binding the audio device with the unique identifier for subsequent data tracking and management; the management platform can also perform real-time data processing, audio data storage, backup and sound pickup device state monitoring, etc.

[0064] S102: Establish the mapping between the unique identifier and the monitoring point, build a location query table, and upload it to the management platform.

[0065] The correspondence between the unique identifier and the monitoring point is established, and the table form is used for storage to obtain a location query table, wherein the location query table is composed of the unique identifier of the sound pickup device and the deployment location of the sound pickup device.

[0066] In embodiment 3, Figure 2 The implementation process of the security monitoring situation awareness method based on big data is shown, and the steps of collecting and preprocessing the audio data are described as follows:

[0067] S103: Preprocessing the audio data, wherein the preprocessing at least includes data cleaning, format unification and noise reduction.

[0068] The audio data is preprocessed, and the specific steps of preprocessing are not limited; data cleaning is to delete useless data in the audio data, such as meaningless blank paragraphs; format unification is performed on all audio data to ensure that audio data from different devices or sources can be processed on the same platform.

[0069] S104: Assemble a collaborative array using the sound pickup device, and embed a time stamp.

[0070] According to the layout and pickup range of the sound pickup device, a sound pickup network with collaborative capability, i.e. a collaborative array, is assembled; a time stamp is inserted into each sound pickup device in the collaborative array to record the collection time of each piece of audio data, so as to locate the data in the subsequent data processing process.

[0071] S105: Integrate the collaborative array and the time stamp, calculate the time difference, and locate the generation position of the audio data.

[0072] In the cooperative array, the time difference of the sound signal reaching each pickup device is calculated by comparing the time stamps of the audio signals in multiple pickup devices, and the generation position of the audio data is calculated by the triangulation method in the prior art.

[0073] In embodiment 4, Figure 3 The implementation process of the security monitoring situation awareness method based on big data is shown, and the steps of constructing a neural network model and inputting the preprocessed audio data into the neural network model to output a plurality of audio features are described as follows:

[0074] S201: Collecting multi-modal data at the generation position, wherein the multi-modal data at least includes video data and temperature data, and correcting the audio features.

[0075] When judging whether the audio features are abnormal features, video data and temperature data at the generation position are collected, and the judgment result is verified.

[0076] S202: Configuring the priority of the multi-modal data and the audio features.

[0077] In addition, the multi-modal data can also be used for security monitoring, and the priority of the multi-modal data and the audio features is determined; for example, the priority of the video data is set to high, and the priority of the audio features is set to medium. If it is found through analysis of the video data that a visitor enters the control area by illegal means, the alarm mechanism can be directly started without further analyzing the audio features.

[0078] In embodiment 5, Figure 4 The implementation process of the security monitoring situation awareness method based on big data is shown, and the steps of judging whether the audio data contains abnormal features are described as follows:

[0079] S301: Obtaining the environmental changes at the monitoring point and dynamically updating the sample set.

[0080] In actual life, when the environment at the monitoring point changes, the sample set should also change; for example, the door at the monitoring point is replaced from a hinge door to a roller shutter door, and the sample set should be updated at the same time. The advantage of this is that it can improve the efficiency of audio data processing while increasing the accuracy of abnormal features.

[0081] S302: Setting the risk level of each abnormal feature and determining the disposal order.

[0082] If multiple pickup devices detect abnormal features at the same time, the processing order should be determined according to the corresponding risk level, and further, the abnormal features should be disposed in order from high to low risk level.

[0083] In embodiment 6, Figure 4 The method for security monitoring situation awareness based on big data provided by the embodiment of the application is shown. The steps of playing interactive voice and establishing a dialogue by using the loudspeaker pre-integrated in the sound pickup device are described in detail as follows.

[0084] S303: issuing a deactivation permission of the loudspeaker to the authorized personnel, and establishing a silence mechanism based on the dialogue.

[0085] In order not to affect normal activities, the authorized personnel can deactivate the loudspeaker by using the silence mechanism in various ways. The silence mechanism is a specific method for deactivating the loudspeaker.

[0086] For example, a silence mechanism is that when the loudspeaker plays interactive voice, the authorized personnel can temporarily deactivate the loudspeaker through the management platform. This has the advantage that the authorized personnel does not need to reply and does not start the alarm mechanism. In actual life, the authorized personnel needs to reply to the interactive voice when entering the control area, which may make the authorized personnel feel embarrassed and cause aversion. Therefore, the silence mechanism can be used to deactivate the loudspeaker.

[0087] S304: integrating the sound pickup device and the loudspeaker into a preset multitask platform.

[0088] The sound pickup device can not only be used for audio acquisition, but also can perform other tasks such as voice control of the device and clock-in and clock-out, and the loudspeaker can not only be used for playing interactive voice, but also can broadcast information.

[0089] In embodiment 7, different from embodiment 1, in the embodiment of the application, the method further comprises:

[0090] In the control area, edge devices are traversed, and the management platform, the neural network model, the sample set and the voiceprint library are deployed to the edge devices.

[0091] A risk value is embedded into the edge device to construct a balanced strategy.

[0092] In the management and control area, find out the intelligent gateway, the computing server and other computing devices, and determine as edge devices, deploy the management platform, the neural network model, the sample set and the voiceprint library into the edge devices; determine the load of each edge device, and define the load as a risk value, wherein the risk value can be determined by indicators such as memory, storage and network bandwidth; if the risk value is greater than a threshold value, start the balancing strategy, wherein the balancing strategy is: migrating tasks in the edge device with a risk value greater than the threshold value to other edge devices; for example, if the CPU usage of an edge device exceeds 80%, that is, the risk value is 80%, and the threshold value is 60%, the balancing strategy should be started, and part of the tasks in the edge device should be migrated.

[0093] Figure 5 The composition structure block diagram of the security monitoring situation awareness system based on big data provided by the embodiment of the application is shown, and the security monitoring situation awareness system based on big data 1 comprises:

[0094] The acquisition module 11 is used for delimiting a management and control area of security monitoring, selecting a plurality of monitoring points, wherein the monitoring points at least comprise sensitive points and protective points, using a sound pickup device pre-deployed in the monitoring points, acquiring audio data, and pre-processing;

[0095] The identification module 12 is used for constructing a neural network model, inputting the pre-processed audio data into the neural network model, outputting a plurality of audio features, creating a sample set composed of abnormal features and background features, and comparing the audio features with the sample set.

[0096] The judgment module 13 is used for judging whether there is an abnormal feature in the audio data, if yes, playing interactive voice using a loudspeaker pre-integrated in the sound pickup device, establishing a dialogue, continuing to acquire real-time audio data, identifying target features, comparing a pre-constructed voiceprint library, judging whether a reply of an authorized person is received, if yes, disconnecting the dialogue, and if no, positioning risk coordinates based on the monitoring points, sending the risk coordinates to a preset terminal, and starting an alarm mechanism.

[0097] Figure 6 The composition structure block diagram of the security monitoring situation awareness system based on big data provided by the embodiment of the application is shown, and the acquisition module 11 comprises:

[0098] The access unit 111 is used for establishing a management platform of the audio data, configuring a unique identifier of the sound pickup device, and accessing the sound pickup device to the management platform;

[0099] The uploading unit 112 is used for establishing a mapping between the unique identifier and the monitoring points, constructing a location query table, and uploading to the management platform;

[0100] a preprocessing unit 113, configured to preprocess the audio data, wherein the preprocessing at least includes data cleaning, format unification and noise reduction;

[0101] an embedding unit 114, configured to use the sound pickup device to form a cooperative array and embed a timestamp;

[0102] a positioning unit 115, configured to integrate the cooperative array and the timestamp, calculate a time difference and locate a generation position of the audio data.

[0103] Figure 7 A component structure block diagram of the security monitoring situation awareness system based on big data is shown, and the identification module 12 includes:

[0104] a correction unit 121, configured to collect multi-modal data at the generation position, wherein the multi-modal data at least includes video data and temperature data, and correct the audio feature;

[0105] a configuration unit 122, configured to configure a priority of the multi-modal data and the audio feature;

[0106] Figure 8 A component structure block diagram of the security monitoring situation awareness system based on big data is shown, and the judgment module 13 includes:

[0107] a more fine unit 131, configured to obtain an environmental change at a monitoring point and dynamically update the sample set;

[0108] a setting unit 132, configured to set a risk level of each of the abnormal features and determine a disposal sequence;

[0109] an establishment unit 133, configured to issue a stop permission of a loudspeaker to the authorized personnel and establish a silence mechanism based on the dialogue;

[0110] an integration unit 134, configured to integrate the sound pickup device and the loudspeaker into a preset multi-task platform.

[0111] The collection module 11 is mainly used for completing the step S100, the identification module 12 is mainly used for completing the step S200, and the judgment module 13 is mainly used for completing the step S300;

[0112] The access unit 111 is mainly used for completing the step S101, the uploading unit 112 is mainly used for completing the step S102, the preprocessing unit 113 is mainly used for completing the step S103, the embedding unit 114 is mainly used for completing the step S104, and the positioning unit 115 is mainly used for completing the step S105;

[0113] The correction unit 121 is mainly configured to complete step S201, and the configuration unit 122 is mainly configured to complete step S202.

[0114] The finer unit 131 is mainly configured to complete step S301, the setting unit 132 is mainly configured to complete step S302, the establishing unit 133 is mainly configured to complete step S303, and the integrating unit 134 is mainly configured to complete step S304.

[0115] The technical features of the above-described embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above-described embodiments are not described, but as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present disclosure.

[0116] The above-described embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it should not be understood as limiting the scope of the present application. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of protection of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

[0117] The above-described only the preferred embodiments of the present application, and not to limit the present application, any modification, equivalent replacement and improvement within the spirit and principles of the present application, should be included in the scope of protection of the present application.

Claims

1. A security monitoring situational awareness method based on big data, characterized in that, The method includes: The control area for security monitoring is delineated, and several monitoring points are selected, including at least sensitive points and protected points. Audio data is collected using audio pickup devices pre-deployed at the monitoring points and pre-processed. A neural network model is constructed, and the preprocessed audio data is input into the neural network model to output several audio features. A sample set consisting of abnormal features and background features is created, and the audio features are compared with the sample set. Determine if there are any abnormal features in the audio data. If so, use the speaker pre-integrated in the sound pickup device to play interactive voice and establish a dialogue. Continue to collect real-time audio data, identify target features, compare with the pre-built voiceprint database, and determine if an authorized person's reply has been received. If so, disconnect the dialogue. If not, locate the risk coordinates based on the monitoring point and send the risk coordinates to a preset terminal to activate the alarm mechanism. The step of determining whether there are abnormal features in the audio data includes: The system acquires environmental changes at monitoring points and dynamically updates the sample set. Set the risk level for each of the aforementioned abnormal characteristics and determine the order of handling them; The steps of playing interactive voice and establishing a dialogue using a speaker pre-integrated in the sound pickup device include: Grant the authorized personnel permission to disable the speaker, establishing a silence mechanism centered on the dialogue; The sound pickup device and speaker are integrated into a preset multi-tasking platform; The method further includes: Within the controlled area, edge devices are identified, and the management platform, neural network model, sample set, and voiceprint library are deployed to the edge devices. Risk values ​​are embedded into the edge devices to construct a balancing strategy.

2. The security monitoring situational awareness method based on big data according to claim 1, characterized in that, The step of collecting audio data using the audio pickup devices pre-deployed at the monitoring points includes: Establish a management platform for the audio data, configure a unique identifier for the pickup device, and connect the pickup device to the management platform; Establish a mapping between the unique identifier and the monitoring point, construct a location query table, and upload it to the management platform.

3. The security monitoring situational awareness method based on big data according to claim 2, characterized in that, The steps of acquiring audio data and preprocessing it include: The audio data is preprocessed, wherein the preprocessing includes at least: data cleaning, format unification, and noise reduction; Using the aforementioned pickup device, a cooperative array is constructed and a timestamp is embedded; By integrating the cooperative array and timestamp, the time difference is calculated, and the location where the audio data was generated is determined.

4. The security monitoring situational awareness method based on big data according to claim 3, characterized in that, The steps of constructing a neural network model, inputting the preprocessed audio data into the neural network model, and outputting several audio features include: Collect multimodal data at the location where the data was generated, wherein the multimodal data includes at least video data and temperature data, and correct the audio features accordingly; Configure the priority of the multimodal data and audio features.

Images