Security and protection monitoring situation awareness method and system based on big data

Abstract

The invention is suitable for the technical field of monitoring situation awareness, and particularly relates to a security and protection monitoring situation awareness method and system based on big data, and the method comprises the steps: delimiting a security and protection monitoring management and control region, and selecting a plurality of monitoring point locations which at least comprise sensitive point locations and protection point locations, collecting audio data by using a pickup device pre-deployed in the monitoring point location, and preprocessing the audio data; the method comprises the steps of preprocessing audio data, constructing a neural network model, inputting the preprocessed audio data into the neural network model, outputting to obtain a plurality of audio features, creating a sample set composed of abnormal features and background features, and comparing the audio features with the sample set. According to the method, potential hazards can be found in time and safety risks can be avoided by identifying the abnormal features, the abnormal features can be verified by establishing dialogues, the false alarm rate is reduced, unnecessary intervention is reduced, normal activities in a control area are prevented from being interfered, and the use experience is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of monitoring situation awareness, and in particular to a security monitoring situation awareness method and system based on big data. Background Art

[0002] Security monitoring devices are widely used in daily monitoring. Among them, most security monitoring devices are composed of cameras and corresponding transmission and processing modules. By using security monitoring devices, the pictures within the monitoring range are transmitted to terminals such as computers and mobile phones in real time, and after analysis and processing, the security risks therein are identified, so as to realize situation awareness.

[0003] When identifying behaviors from monitoring data, incorrect judgments may be caused due to environmental changes (such as light, angle, and background, etc.), which may lead to false alarms or missed alarms. Moreover, the consumption of massive storage data and computing resources also greatly limits the usage scenarios of cameras. Therefore, "how to use voice pickup devices to identify risks and verify them" is the technical problem to be solved by the present invention. Summary of the Invention

[0004] The purpose of the present invention is to provide a security monitoring situation awareness method and system based on big data to solve the problem of "how to use voice pickup devices to identify risks and verify them" proposed in the above background art.

[0005] To achieve the above purpose, the present invention provides the following technical solutions:

[0006] A security monitoring situation awareness method based on big data, the method comprising:

[0007] Define the control area of security monitoring, select a number of monitoring points, where the monitoring points at least include: sensitive points and protection points, and use the voice pickup devices pre-deployed at the monitoring points to collect audio data and perform preprocessing;

[0008] Construct a neural network model, input the preprocessed audio data into the neural network model, output a number of audio features, create a sample set composed of abnormal features and background features, compare the audio features with the sample set, and determine whether there are abnormal features in the audio data;

[0009] If so, use the speaker pre-integrated in the voice pickup device to play interactive voice, establish a dialogue, continue to collect real-time audio data, identify the target features, compare with the pre-constructed voiceprint library, and determine whether a reply from an authorized person is received. If so, disconnect the dialogue. If not, based on the monitoring point, locate the risk coordinates and send the risk coordinates to a preset terminal to activate the alarm mechanism.

[0010] Further, the step of collecting audio data by using the sound pickup device pre-deployed at the monitoring point includes:

[0011] Set up a management platform for the audio data, configure a unique identifier for the sound pickup device, and connect the sound pickup device to the management platform;

[0012] Establish a mapping between the unique identifier and the monitoring point, construct a location query table, and upload it to the management platform.

[0013] Further, the step of collecting audio data and performing preprocessing includes:

[0014] Perform preprocessing on the audio data, where the preprocessing at least includes: data cleaning, format unification, and noise reduction;

[0015] Use the sound pickup device to form a collaborative array and embed a timestamp;

[0016] Integrate the collaborative array and the timestamp, calculate the time difference, and locate the generation position of the audio data.

[0017] Further, the step of constructing a neural network model and inputting the preprocessed audio data into the neural network model to output a number of audio features includes:

[0018] Collect multi-modal data at the generation position, where the multi-modal data at least includes: video data and temperature data, and correct the audio features;

[0019] Configure the priorities of the multi-modal data and the audio features.

[0020] Further, the step of determining whether there are abnormal features in the audio data includes:

[0021] Obtain the environmental changes at the monitoring point and dynamically update the sample set;

[0022] Set the risk level of each abnormal feature and determine the handling order.

[0023] Further, the step of using the speaker pre-integrated in the sound pickup device to play interactive voice and establish a conversation includes:

[0024] Grant the authorization personnel the permission to disable the speaker and establish a silence mechanism centered on the conversation;

[0025] Integrate the sound pickup device and the speaker into a preset multi-task platform.

[0026] Further, the method further includes:

[0027] In the control area, edge devices are traversed, and the management platform, neural network model, sample set, and voiceprint library are deployed to the edge devices;

[0028] A risk value is embedded into the edge device to construct an equilibrium strategy.

[0029] Furthermore, the system includes:

[0030] A collection module, used to delimit the control area of security monitoring, select several monitoring points, where the monitoring points at least include: sensitive points and protection points, use the sound pickup devices pre-deployed at the monitoring points to collect audio data, and perform preprocessing;

[0031] An identification module, used to construct a neural network model, input the preprocessed audio data into the neural network model, output several audio features, create a sample set composed of abnormal features and background features, and compare the audio features with the sample set;

[0032] A judgment module, used to judge whether there are abnormal features in the audio data. If so, use the speaker pre-integrated in the sound pickup device to play interactive voice, establish a dialogue, continue to collect real-time audio data, identify target features, compare with the pre-constructed voiceprint library, judge whether a reply from an authorized person is received. If so, disconnect the dialogue. If not, based on the monitoring points, locate the risk coordinates and send the risk coordinates to a preset terminal to start an alarm mechanism.

[0033] Furthermore, the collection module includes:

[0034] An access unit, used to set up the management platform for the audio data, configure the unique identifier of the sound pickup device, and connect the sound pickup device to the management platform;

[0035] An upload unit, used to establish the mapping between the unique identifier and the monitoring points, construct a location query table, and upload it to the management platform;

[0036] A preprocessing unit, used to preprocess the audio data, where the preprocessing at least includes: data cleaning, format unification, and noise reduction;

[0037] An embedding unit, used to use the sound pickup device to form a collaborative array and embed a timestamp;

[0038] A positioning unit, used to integrate the collaborative array and the timestamp, calculate the time difference, and locate the generation position of the audio data.

[0039] Furthermore, the identification module includes:

[0040] A correction unit, configured to collect multi-modal data at the generation position, where the multi-modal data at least includes: video data and temperature data, and correct the audio features;

[0041] A configuration unit, configured to configure the priorities of the multi-modal data and the audio features.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0043] By delimiting the monitoring area, monitoring dead angles can be eliminated and monitoring accuracy can be improved. By deploying sound pickup devices, the controlled area can be monitored all day long without being affected by the environment. By constructing a neural network model, the processing efficiency of audio data is greatly improved, and at the same time, the accuracy of posture perception data is ensured. By identifying abnormal features, potential hidden dangers can be discovered in time to avoid security risks. By establishing a dialogue, the abnormal features can be verified, the false alarm rate can be reduced, unnecessary interventions can be reduced, and the normal activities in the controlled area can be avoided from being interfered, greatly improving the user experience. Description of the Drawings

[0044] Figure 1 It is a flowchart of the security monitoring situation awareness method based on big data provided by an embodiment of the present invention;

[0045] Figure 2 It is a first sub-flowchart of the security monitoring situation awareness method based on big data provided by an embodiment of the present invention;

[0046] Figure 3 It is a second sub-flowchart of the security monitoring situation awareness method based on big data provided by an embodiment of the present invention;

[0047] Figure 4 It is a third sub-flowchart of the security monitoring situation awareness method based on big data provided by an embodiment of the present invention;

[0048] Figure 5 It is a block diagram of the composition of the security monitoring situation awareness system based on big data provided by an embodiment of the present invention;

[0049] Figure 6 It is a block diagram of the composition of the acquisition module in the security monitoring situation awareness system based on big data provided by an embodiment of the present invention;

[0050] Figure 7 It is a block diagram of the composition of the recognition module in the security monitoring situation awareness system based on big data provided by an embodiment of the present invention;

[0051] Figure 8 It is a block diagram of the composition of the judgment module in the security monitoring situation awareness system based on big data provided by an embodiment of the present invention. Specific Embodiments

[0052] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0053] In Embodiment 1, Figure 1 The implementation process of the security monitoring situation awareness method based on big data provided by the embodiments of the present invention is shown, and the details are as follows:

[0054] S100: Define the control area of security monitoring, select a number of monitoring points, where the monitoring points at least include: sensitive points and protection points, and use the sound pickup devices pre-deployed at the monitoring points to collect audio data and perform preprocessing.

[0055] Define the area that needs to be monitored for security, that is, the control area. The control area can be a building, a community, an industrial park, etc.; according to the building structure and equipment distribution of the control area, etc., select a number of monitoring points from the control area. The monitoring points are also the deployment positions of the sound pickup devices. The monitoring points include: sensitive points and protection points, etc. Among them, sensitive points usually refer to areas where security incidents are likely to occur, such as entrances and exits, important passages, and crowded places, etc. Protection points are mainly fences, building facades, parking lots, etc.; in this embodiment, the division of monitoring points is not limited. In other words, the monitoring points can be composed of single or multiple different types of points; deploy sound pickup devices at the monitoring points, where the sound pickup devices can be microphones or microphones, etc.; use the sound pickup devices to collect audio data at the deployment points and perform preprocessing. The preprocessing steps include: data cleaning, format unification, noise reduction, etc.

[0056] S200: Build a neural network model, input the preprocessed audio data into the neural network model, output a number of audio features, create a sample set composed of abnormal features and background features, and compare the audio features with the sample set.

[0057] Create a neural network model. The neural network model adopts a convolutional neural network architecture and can accurately identify the temporal features and spatial features in the audio data; after the neural network model is built, historical data is also needed for training; input the preprocessed audio data into the trained neural network model, and generate a number of audio features in the output layer of the neural network. These features include: glass breaking sound, door opening sound, and human voice, etc.

[0058] Collect a large number of existing audio features and label them to obtain a sample set. In other words, the sample set is a collection of audio features. Classify the audio features in the sample set into abnormal features and background features. The abnormal features include explosion sounds, glass breaking sounds, screaming sounds, door opening sounds, etc. The background features include normal environmental sounds, such as wind and rain sounds, human voices, and mechanical operation sounds, etc. Specifically, the audio features can be further refined. For example, refine the door opening sound. Determine the normal door opening sound as a background feature, and determine the door opening sound with multiple attempts or accompanied by abnormal sounds of the lock cylinder as an abnormal feature.

[0059] S300: Determine whether there are abnormal features in the audio data. If so, use the speaker pre-integrated in the sound pickup device to play interactive voice, establish a conversation, continue to collect real-time audio data, identify the target feature, compare it with the pre-constructed voiceprint library, and determine whether a reply from an authorized person is received. If so, disconnect the conversation. If not, based on the monitoring point, locate the risk coordinates and send the risk coordinates to a preset terminal to activate the alarm mechanism.

[0060] Determine whether there are abnormal features in the collected audio features. If so, use the speaker to play interactive voice. The interactive voice can be, for example, "May I help you?" or "Please state your employee number." Use the sound pickup device and the speaker to conduct natural language interaction with the visitor through speech recognition and speech synthesis technologies and establish a conversation. Identify the voice feature of the visitor from the conversation and define the voice feature as the target feature. Compare the target feature with the known samples in the voiceprint library to determine whether there are the same samples in the voiceprint library. If so, it means that the visitor is an authorized person and disconnect the conversation. If there are no same samples in the voiceprint library, there may be two situations. One is that the visitor has replied, but the visitor is not authorized. The other is that no reply is received. According to the location of the monitoring point, find the risk coordinates. The risk coordinates are the generation location of the abnormal feature. For example, if the abnormal feature is the door opening sound, the risk coordinates are the location of the door. Send the risk coordinates to a preset terminal. The preset terminal can be the terminal of the security personnel in the controlled area, and at the same time activate the alarm mechanism. The alarm mechanism is a specific alarm method. For example, use the buzzer deployed near the risk coordinates to give an alarm.

[0061] In Embodiment 2, Figure 2 The implementation process of the security monitoring situation awareness method based on big data provided by the embodiments of the present invention is shown. The following details the step of collecting audio data using the sound pickup device pre-deployed at the monitoring point as follows:

[0062] S101: Set up the management platform for the audio data, configure the unique identifier of the sound pickup device, and connect the sound pickup device to the management platform.

[0063] Assign a unique identifier to each sound pickup device. This identifier can be the device's serial number, MAC address, or device number, etc. Connect the sound pickup device to the management platform through wireless or wired network. The management platform is responsible for receiving, storing, processing, and analyzing the audio data collected by each sound pickup device, and binding the audio device with the unique identifier for subsequent data tracking and management. The management platform can also perform real-time data processing, audio data storage, backup, and monitoring of the status of the sound pickup device, etc.

[0064] S102: Establish the mapping between the unique identifier and the monitoring point, construct the location query table, and upload it to the management platform.

[0065] Establish the corresponding relationship between the unique identifier and the monitoring point, and store it in the form of a form to obtain the location query table, where the location query table consists of the unique identifier of the sound pickup device and the deployment location of the sound pickup device.

[0066] In Embodiment 3, Figure 2 The implementation process of the security monitoring situation awareness method based on big data provided by the embodiment of the present invention is shown. The steps of collecting audio data and performing preprocessing are described in detail as follows:

[0067] S103: Preprocess the audio data, where the preprocessing at least includes: data cleaning, format unification, and noise reduction.

[0068] Preprocess the audio data. The specific steps of the preprocessing are not limited. Data cleaning is to delete the useless data in the audio data, such as meaningless blank paragraphs. Unifying the format of all audio data is to ensure that audio data from different devices or sources can be processed on the same platform.

[0069] S104: Use the sound pickup device to form a collaborative array and embed a timestamp.

[0070] According to the layout and pickup range of the sound pickup device, etc., form a sound pickup network with collaborative capabilities, that is, a collaborative array. Insert a timestamp into each sound pickup device in the collaborative array to record the acquisition time of each piece of audio data for data positioning in subsequent data processing.

[0071] S105: Integrate the collaborative array and the timestamp, calculate the time difference, and locate the generation position of the audio data.

[0072] In a collaborative array, by comparing the timestamps of audio signals in multiple pickup devices, the time difference for the sound signal to reach each pickup device is calculated, and the generation location of the audio data is deduced through the triangulation method in the prior art.

[0073] In Embodiment 4, Figure 3 The implementation process of the security monitoring situation awareness method based on big data provided by the embodiment of the present invention is shown. The steps of constructing a neural network model and inputting the preprocessed audio data into the neural network model to output several audio features are described in detail as follows:

[0074] S201: Collect multimodal data at the generation location, where the multimodal data at least includes: video data and temperature data, and correct the audio features.

[0075] When judging whether the audio feature is an abnormal feature, video data, temperature data, etc. at the generation location are collected to verify the judgment result.

[0076] S202: Configure the priorities of the multimodal data and the audio features.

[0077] In addition, multimodal data can also be used for security monitoring, and the priorities of the multimodal data and the audio features are determined; for example, the priority of the video data is set to high, and the priority of the audio feature is set to medium. If it is found through analyzing the video data that a certain visitor enters the controlled area by illegal means, the alarm mechanism can be directly activated at this time without further analyzing the audio feature.

[0078] In Embodiment 5, Figure 4 The implementation process of the security monitoring situation awareness method based on big data provided by the embodiment of the present invention is shown. The steps of judging whether there are abnormal features in the audio data are described in detail as follows:

[0079] S301: Obtain the environmental changes at the monitoring point and dynamically update the sample set.

[0080] In real life, when the environment at the monitoring point changes, the sample set should also change accordingly; for example, when the door at the monitoring point is changed from a hinged door to a rolling shutter door, the sample set should be updated simultaneously; the advantage of doing this is that it can improve the processing efficiency of audio data while increasing the accuracy of abnormal features.

[0081] S302: Set the risk level of each abnormal feature and determine the disposal order.

[0082] If multiple pickup devices detect abnormal features at the same time, the processing order should be determined according to the corresponding risk levels; further, the abnormal features should be disposed of in the order from high to low risk level.

[0083] In Embodiment 6, Figure 4 The implementation process of the security monitoring situation awareness method based on big data provided by the embodiments of the present invention is shown. The following details the steps of playing interactive voice using the speaker pre-integrated in the sound pickup device and establishing a conversation, as follows:

[0084] S303: Grant the authorization personnel the permission to deactivate the speaker, and establish a silence mechanism centered on the conversation.

[0085] In order not to affect normal activities, the authorization personnel can use various methods to deactivate the speaker using the silence mechanism, which is the specific method of deactivating the speaker.

[0086] For example, a certain silence mechanism is that when the speaker plays interactive voice, the authorization personnel can temporarily deactivate the speaker through the management platform. The advantage of this is that the authorization personnel neither need to reply nor trigger the alarm mechanism. In real life, when the authorization personnel enter the controlled area and need to reply to the interactive voice, this may make the authorization personnel feel embarrassed and cause disgust. Therefore, the silence mechanism can be used to deactivate the speaker.

[0087] S304: Integrate the sound pickup device and the speaker into a preset multi-task platform.

[0088] The sound pickup device can not only be used for audio collection, but also perform other tasks, such as voice control of the device and punching in and out for work. The speaker can not only be used to play interactive voice, but also broadcast information.

[0089] In Embodiment 7, different from Embodiment 1, in the embodiments of the present invention, the method further includes:

[0090] In the controlled area, traverse the edge devices, and deploy the management platform, neural network model, sample set, and voiceprint library to the edge devices;

[0091] Embed a risk value into the edge device to construct an equilibrium strategy.

[0092] In the controlled area, find intelligent gateways, computing servers, and other computing devices, and identify them as edge devices. Deploy the management platform, neural network model, sample set, voiceprint library, etc. to the edge devices; determine the load of each edge device, and define the load as a risk value, where the risk value can be determined by metrics such as memory, storage, and network bandwidth; if the risk value is greater than the threshold, start the balancing strategy, where the balancing strategy is: migrate the tasks in the edge device with a risk value greater than the threshold to other edge devices; for example, if the CPU usage rate of an edge device exceeds 80%, that is, the risk value is 80%, and if the threshold is 60%, the balancing strategy should be started to migrate some tasks in the edge device.

[0093] Figure 5 The block diagram of the composition structure of the security monitoring situation awareness system based on big data provided by the embodiment of the present invention is shown. The security monitoring situation awareness system 1 based on big data includes:

[0094] The acquisition module 11 is used to delimit the controlled area of security monitoring, select a number of monitoring points, where the monitoring points at least include: sensitive points and protection points, and use the sound pickup devices pre-deployed at the monitoring points to collect audio data and perform preprocessing;

[0095] The recognition module 12 is used to build a neural network model, input the preprocessed audio data into the neural network model, output a number of audio features, create a sample set composed of abnormal features and background features, and compare the audio features with the sample set;

[0096] The judgment module 13 is used to judge whether there are abnormal features in the audio data. If so, use the speaker pre-integrated in the sound pickup device to play interactive voice, establish a dialogue, continue to collect real-time audio data, identify the target features, compare with the pre-constructed voiceprint library, and judge whether the reply of the authorized person is received. If so, disconnect the dialogue. If not, based on the monitoring point, locate the risk coordinates and send the risk coordinates to the preset terminal to start the alarm mechanism.

[0097] Figure 6 The block diagram of the composition structure of the security monitoring situation awareness system based on big data provided by the embodiment of the present invention is shown. The acquisition module 11 includes:

[0098] The access unit 111 is used to build the management platform of the audio data, configure the unique identifier of the sound pickup device, and connect the sound pickup device to the management platform;

[0099] The upload unit 112 is used to establish the mapping between the unique identifier and the monitoring point, build a location query table, and upload it to the management platform;

[0100] A preprocessing unit 113 for preprocessing the audio data, where the preprocessing at least includes: data cleaning, format unification, and noise reduction;

[0101] An embedding unit 114 for using the pickup device to form a collaborative array and embed timestamps;

[0102] A positioning unit 115 for integrating the collaborative array and timestamps, calculating a time difference, and locating the generation position of the audio data.

[0103] Figure 7 The block diagram of the composition structure of the security monitoring situation awareness system based on big data provided by the embodiment of the present invention is shown. The recognition module 12 includes:

[0104] A correction unit 121 for collecting multimodal data at the generation position, where the multimodal data at least includes: video data and temperature data, and correcting the audio features;

[0105] A configuration unit 122 for configuring the priorities of the multimodal data and audio features;

[0106] Figure 8 The block diagram of the composition structure of the security monitoring situation awareness system based on big data provided by the embodiment of the present invention is shown. The judgment module 13 includes:

[0107] A refinement unit 131 for obtaining the environmental changes at the monitoring point and dynamically updating the sample set;

[0108] A setting unit 132 for setting the risk level of each abnormal feature and determining the disposal order;

[0109] An establishment unit 133 for distributing the disabling permission of the speaker to the authorized personnel and establishing a silence mechanism centered on the conversation;

[0110] An integration unit 134 for integrating the pickup device and the speaker into a preset multi-task platform.

[0111] Among them, the acquisition module 11 is mainly used to complete step S100, the recognition module 12 is mainly used to complete step S200, and the judgment module 13 is mainly used to complete step S300;

[0112] The access unit 111 is mainly used to complete step S101, the upload unit 112 is mainly used to complete step S102, the preprocessing unit 113 is mainly used to complete step S103, the embedding unit 114 is mainly used to complete step S104, and the positioning unit 115 is mainly used to complete step S105;

[0113] The correction unit 121 is mainly used to complete step S201, and the configuration unit 122 is mainly used to complete step S202;

[0114] The refinement unit 131 is mainly used to complete step S301, the setting unit 132 is mainly used to complete step S302, the establishment unit 133 is mainly used to complete step S303, and the integration unit 134 is mainly used to complete step S304.

[0115] The technical features of the above-described embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0116] The above-described embodiments only represent several implementation manners of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention should be subject to the appended claims.

[0117] The above is only the preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A security monitoring situation awareness method based on big data, characterized in that: The method comprises: Delineate a control area for security monitoring, select a number of monitoring points, where the monitoring points include at least sensitive points and protection points, use the sound pickup equipment pre-deployed at the monitoring points to collect audio data, and perform pre-processing; Constructing a neural network model, and inputting the preprocessed audio data into the neural network model, outputting a number of audio features, creating a sample set consisting of abnormal features and background features, and comparing the audio features and the sample set; Determine whether there are abnormal features in the audio data. If so, use the speaker pre-integrated in the sound pickup device to play interactive voice and establish a dialogue. Continue to collect real-time audio data, identify target features, compare with the pre-built voiceprint library, and determine whether a reply from the authorized person is received. If so, disconnect the dialogue. If not, locate the risk coordinates based on the monitoring point, send the risk coordinates to the preset terminal, and start the alarm mechanism.

2. The security monitoring situation awareness method based on big data according to claim 1 is characterized in that: The step of collecting audio data by using the sound pickup device pre-deployed at the monitoring point includes: Establishing a management platform for the audio data, configuring a unique identifier for the sound pickup device, and connecting the sound pickup device to the management platform; A mapping between the unique identifier and the monitoring point is established, a location query table is constructed, and uploaded to the management platform.

3. The security monitoring situation awareness method based on big data according to claim 2 is characterized in that: The steps of collecting audio data and performing preprocessing include: Preprocessing the audio data, wherein the preprocessing at least includes: data cleaning, format unification and noise reduction; Using the sound pickup device, a collaborative array is formed and a timestamp is embedded; The collaborative array and the timestamp are integrated to calculate the time difference and locate the generation position of the audio data.

4. The security monitoring situation awareness method based on big data according to claim 3 is characterized in that: The steps of constructing a neural network model, inputting the preprocessed audio data into the neural network model, and outputting a plurality of audio features include: Collecting multimodal data at the generation location, wherein the multimodal data at least includes: video data and temperature data, and modifying the audio feature; Priorities of the multimodal data and audio features are configured.

5. The security monitoring situation awareness method based on big data according to claim 1 is characterized in that: The step of determining whether there is an abnormal feature in the audio data comprises: Obtaining environmental changes at monitoring points and dynamically updating the sample set; The risk level of each abnormal feature is set and the order of treatment is determined.

6. The security monitoring situation awareness method based on big data according to claim 1 is characterized in that: The step of using a speaker pre-integrated in the sound pickup device to play interactive voice and establish a conversation includes: issuing the permission to deactivate the loudspeaker to the authorized personnel, and establishing a silencing mechanism centered on the conversation; The sound pickup device and the loudspeaker are integrated into a pre-set multi-tasking platform.

7. The security monitoring situation awareness method based on big data according to claim 2 is characterized in that: The method further comprises: In the control area, traverse the edge devices and deploy the management platform, neural network model, sample set and voiceprint library to the edge devices; A risk value is embedded into the edge device to build a balancing strategy.

8. The security monitoring situation awareness system based on big data is characterized by: The system comprises: The acquisition module is used to define the control area of ​​security monitoring, select a number of monitoring points, where the monitoring points include at least sensitive points and protection points, use the sound pickup equipment pre-deployed at the monitoring points to collect audio data, and perform pre-processing; A recognition module is used to construct a neural network model, and input the preprocessed audio data into the neural network model, output a number of audio features, create a sample set consisting of abnormal features and background features, and compare the audio features and the sample set; The judgment module is used to judge whether there are abnormal features in the audio data. If so, the interactive voice is played by using the speaker pre-integrated in the sound pickup device, and a dialogue is established. The real-time audio data is continued to be collected, the target features are identified, and the pre-built voiceprint library is compared to judge whether a reply from the authorized person is received. If so, the dialogue is disconnected. If not, the risk coordinates are located based on the monitoring points, and the risk coordinates are sent to a preset terminal to start the alarm mechanism.

9. The security monitoring situation awareness system based on big data according to claim 8 is characterized in that: The acquisition module comprises: An access unit, used to establish a management platform for the audio data, configure a unique identifier for the sound pickup device, and access the sound pickup device to the management platform; An uploading unit, used to establish a mapping between the unique identifier and the monitoring point, construct a location query table, and upload it to the management platform; A preprocessing unit, used for preprocessing the audio data, wherein the preprocessing at least includes: data cleaning, format unification and noise reduction; An embedding unit, used to use the sound pickup device to form a collaborative array and embed a timestamp; The positioning unit is used to integrate the collaborative array and the timestamp, calculate the time difference, and locate the generation position of the audio data.

10. The security monitoring situation awareness system according to claim 9, characterized in that: The identification module comprises: A correction unit, configured to collect multimodal data at the generation location, wherein the multimodal data at least includes: video data and temperature data, and to correct the audio feature; A configuration unit is used to configure the priorities of the multimodal data and audio features.

Images