A data mining method based on public safety events

Abstract

The invention discloses a data mining method based on public safety events, relates to the technical field of data mining, and solves the technical problems that it is difficult to fully analyze data, when the amount of retrieved data is too large, it is difficult to comprehensively analyze the relevance of the data, and it is not convenient to quickly lock the data with stronger relevance to safety time while comprehensively retrieving a large amount of related data; the method comprises the following steps: obtaining multimodal data from different sources, including public area monitoring data, radar sensor data and vehicle sensor data, pre-processing the multimodal data from different sources to form a public safety event data set; according to the spatiotemporal characteristics of the public safety event data and the relevance analysis results of the data in the public safety event data set, establishing a security event relevance grouping; by associating the data in the data set, it is convenient to deeply understand the connection and influence between different events.

Description

Technical Field

[0001] The invention belongs to the field of data mining, and in particular to a data mining method based on public safety events. Background Art

[0002] With the development of society and the process of urbanization, the types and number of public safety incidents are increasing, and more effective methods are needed to handle and respond to these incidents. The development of data mining technology has made it possible to process large-scale data, making it possible to conduct in-depth analysis of public safety incident data. Government departments and security agencies need more accurate and timely information to guide decision-making and resource allocation to meet various security challenges. The data mining method of public safety incidents is a method of mining public safety incident data using data mining technology. By exploring the potential patterns and associations in large-scale event data, the relationship and laws between events can be revealed, thereby improving public safety management and response capabilities.

[0003] Most of the existing data mining methods for public safety incidents are only established to retrieve data related to safety time, which makes it difficult to fully analyze the data. When the amount of retrieved data is too large, it is difficult to conduct a comprehensive analysis of the data's relevance. It is not convenient to quickly lock in data that is more strongly related to safety time while comprehensively retrieving a large amount of related data. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a data mining method based on public safety events, which is used to solve the technical problems that it is difficult to fully analyze the data. When the amount of retrieved data is too large, it is difficult to conduct a comprehensive analysis of the correlation of the data, and it is not convenient to quickly lock the data with a stronger correlation with the safety time while comprehensively retrieving a large amount of related data.

[0005] To solve the above problems, a first aspect of the present invention provides a data mining method based on public safety events, comprising the following steps:

[0006] Acquire multimodal data from different sources, including public area surveillance data, radar sensor data, and vehicle sensor data, and pre-process the multimodal data from different sources to form a public safety event dataset;

[0007] By combining natural language processing and image processing technologies, an index structure is constructed to retrieve structured and unstructured data in public safety event datasets;

[0008] By accessing the data source of the public news network, relevant data of public safety events are collected from the security events in the data source, and the popularity of public safety events is ranked;

[0009] According to the ranking order of the popularity of security events, multimodal data retrieval is performed on structured data and unstructured data in the public security event dataset, and data on the same public security event in the public security event dataset is associated;

[0010] By correlating the data in the public security event data set, and according to the security event characteristics, performing correlation analysis on the data in the correlated public security event data set;

[0011] According to the spatiotemporal characteristics of public security event data and the correlation analysis results of the data in the public security event data set, security event correlation groups are established.

[0012] As a further solution of the present invention, multimodal data from different sources are preprocessed to form a public safety event data set, including the following steps:

[0013] Preprocess the multimodal data of public area monitoring data, radar sensor data, and vehicle sensor data to remove noise, outliers, and redundant information;

[0014] Divide multimodal data into text data, image data, audio data, and video data, and convert each type of data into a unified format and structure;

[0015] The time and location information of each text data, image data, audio data and video data collection is extracted as data labels.

[0016] As a further solution of the present invention: by combining natural language processing and image processing technology, an index structure is constructed to retrieve structured data and unstructured data in the public security event data set, including the following steps:

[0017] The text information in image data and video data is extracted from the image through OCR optical character recognition technology and converted into text data. At the same time, the voice information in the audio data is converted into text data through voice recognition tools. The OCR optical character recognition technology uses Tesseract OCR optical character recognition tool. Tesseract is an open source OCR engine that supports multiple languages ​​and can process multiple image formats.

[0018] The text data obtained by converting the image data, audio data and video data is labeled according to the time and location information of the collection of the image data, audio data and video data, and added to the text data;

[0019] For text data, the keyword extraction algorithm is used to extract keywords from each text data, and the extracted keywords are added to the data label as feature representations; TextRank is a graph-based ranking algorithm, which is similar to the PageRank algorithm and determines the importance of words by calculating the relationship between words. In this algorithm, the relationship between words is usually represented by their co-occurrence frequency.

[0020] For image data, the convolutional neural network model is used to apply convolution and pooling operations on the image to extract feature maps. The feature maps output by the convolutional layer are flattened into feature vectors through the fully connected layer of the convolutional neural network model. The feature vectors of the image are mapped to the output space of the image through the fully connected operation to obtain the feature representation of the image. The feature representation of the image is added to the data label.

[0021] For audio data, the audio signal is decomposed into spectral bands through MFCC Mel-frequency cepstrum coefficients, and then the energy of the spectral bands is extracted; the audio signal is divided into short-time windows through short-time Fourier transform, and then each short-time window is transformed to obtain spectrum information;

[0022] The energy data and spectrum information of the obtained spectrum band are added to the data label as feature representation;

[0023] For video data, the Lucas-Kanade optical flow algorithm is used to capture the motion information between adjacent frames in the video and identify the image information of the moving object. The convolutional neural network model is used to apply convolution and pooling operations on the image information of the moving object to extract the feature map. The fully connected layer of the convolutional neural network model is used to flatten the feature map output by the convolutional layer into a feature vector. The feature vector of the image is mapped to the output space of the image through the fully connected operation to obtain the feature representation of the image information of the moving object, and the obtained feature representation is added to the data label.

[0024] According to the data labels, index structures for text data, image data, audio data and video data are constructed respectively to retrieve the structured data and unstructured data in the public safety incident dataset.

[0025] As a further solution of the present invention, relevant data of public safety events are collected from the security events in the data source, and the public safety events are ranked by popularity, including the following steps:

[0026] Collect reports on security incidents from public news network data sources, clean the data, and remove irrelevant information and duplicate data; obtain relevant data through web crawlers, APIs, etc.

[0027] According to the time and location information of the security incident, all areas where public security incidents occurred are divided into blocks, the collected news reports are grouped according to security incidents, and security incidents that occurred in the same time period and the same block are grouped into the same security data group;

[0028] Calculate the popularity of security events in each group based on the number of reports, exposure, and news reporting duration of security events;

[0029] The security events in each group are ranked according to their heat values, and the ranking order of the heat values ​​is used as the heat ranking order.

[0030] As a further solution of the present invention: according to the order of the popularity ranking of security events, multimodal data retrieval is performed on structured data and unstructured data in the public security event data set, and data related to the same public security event in the public security event data set is associated, including the following steps:

[0031] According to the order of the heat ranking of the security events in each group of public security events, according to the order of the heat ranking of the security events, extract the text data, image data, audio data or video data in the security event report data corresponding to each security event in the data source of the news network;

[0032] Convert text information in image data and video data, and voice information in audio data, into text data;

[0033] Extracting the keywords of the text data, the feature representation of the image data, the energy data and spectrum information of the spectrum band of the audio data, or the feature representation of the image information of the moving object of the video data from the security incident report data corresponding to each security incident in the data source of the news network;

[0034] The text data, image data, audio data or video data of the security incident are retrieved respectively through the index structure of the corresponding type of data, and the retrieved data are correlated with each other as data about the same public security incident.

[0035] As a further solution of the present invention, by correlating the data in the public security event data set, according to the security event characteristics, a correlation analysis is performed on the data in the correlated public security event data set, including the following steps:

[0036] Obtain text data from the corresponding security incident report data in the data source of the news network, extract keywords from each text data through a keyword extraction algorithm, and count the frequency of occurrence of all keywords in the text data associated with the public security incident data set, and count the frequency of occurrence of all keywords in each text data through a weighted average method to obtain a correlation evaluation value of the text data;

[0037] Obtaining the feature representation of the image of the image data or the feature representation of the image information of the moving object of the video data in the corresponding security incident report data in the data source of the news network, and the feature representation of the image data or video data associated with the corresponding public security incident data set;

[0038] The correlation between image features is evaluated by training a Simesian neural network model, and the correlation evaluation value of the image data or video data associated in the public security event data set is evaluated by the trained Simesian neural network model;

[0039] Obtain an audio spectrum graph of an image of audio data in the corresponding security incident report data in the data source of the news network, and an audio spectrum graph of the audio data associated with the corresponding public security incident data set;

[0040] The correlation between audio spectrograms is evaluated by training the Simese neural network model, and the correlation evaluation value of the audio data associated in the public safety event dataset is evaluated by the trained Simese neural network model.

[0041] As a further solution of the present invention: according to the spatiotemporal characteristics of the public security event data and the correlation analysis results of the data in the public security event data set, a security event correlation group is established, including the following steps:

[0042] Count the time difference between the data collection time information of the associated data in the public security incident data set and the time of occurrence of the corresponding security incident report data in the data source of the news network;

[0043] Count the distance difference between the data collection location information of the associated data in the public security incident data set and the time occurrence location of the corresponding security incident report data in the data source of the news network;

[0044] According to the spatiotemporal characteristics of public safety event data and the correlation analysis results of the data in the public safety event data set, the spatiotemporal correlation correlation evaluation values ​​of different public safety events are calculated;

[0045] According to the spatiotemporal correlation evaluation values ​​of different public safety events, the public safety events are grouped into high spatiotemporal correlation groups and low spatiotemporal correlation groups.

[0046] As a further solution of the present invention: the spatiotemporal correlation correlation evaluation values ​​of different public safety events are calculated by the following formula:

[0047]

[0048] Where K is the spatiotemporal correlation evaluation value of the data to be associated in the public security event dataset, E is the correlation evaluation value of the text data, image data, audio data or video data in the public security event dataset, E0 is the average correlation evaluation value of the text data, image data, audio data or video data in the public security event dataset, and E max is the maximum value of the association evaluation value of text data, image data, audio data or video data in the public security event data set, T is the time difference of the associated data in the public security event data set, T0 is the average time difference of all the associated data in the public security event data set, T max is the maximum value of the time difference of all the associated data in the statistical public safety event data set, L is the distance difference of the associated data in the statistical public safety event data set, L0 is the average value of the distance difference of all the associated data in the statistical public safety event data set, and L max It is the maximum value of the distance difference of all the associated data in the public safety event dataset.

[0049] As a further solution of the present invention: according to the spatiotemporal correlation evaluation values ​​of different public safety events, the public safety events are grouped into high spatiotemporal correlation grouping and low spatiotemporal correlation grouping, including the following steps:

[0050] According to the spatiotemporal correlation evaluation values ​​of different public safety events, the public safety events whose spatiotemporal correlation evaluation values ​​are greater than a preset threshold are grouped into a high spatiotemporal correlation group;

[0051] Public safety events whose spatiotemporal correlation evaluation values ​​are less than or equal to a preset threshold are grouped into low spatiotemporal correlation groups.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] The present invention retrieves multimodal data from structured data and unstructured data in a public security event data set according to the order of the popularity ranking of security events, and associates data about the same public security event in the public security event data set; combines natural language processing and image processing technology to build an index structure, and realizes efficient retrieval of structured and unstructured data in the public security event data set; at the same time, after the data source connected to the public news network can rank the popularity of security events, multimodal data retrieval is performed according to the ranking order, and security events with higher popularity are retrieved preferentially according to time popularity, so as to facilitate rapid association and aggregation of relevant data of security events of public concern, facilitate a more comprehensive understanding of relevant information of the same event, and promote data association and comprehensive analysis.

[0054] The present invention performs correlation analysis on the data in the public security event data set that is correlated according to the characteristics of the security event; at the same time, according to the spatiotemporal characteristics of the public security event data and the correlation analysis results of the data in the public security event data set, a security event correlation group is established; by correlating the data in the data set, it is convenient to deeply understand the connection and influence between different events, thereby providing a deeper insight and analysis; at the same time, the associated data helps to establish an association network between events, thereby promoting a comprehensive analysis of the events, rather than being limited to an independent analysis of a single data. According to the spatiotemporal characteristics and features of the public security event data, the data is analyzed for correlation, and correlation groups are established, so as to facilitate more timely discovery of data with different correlations to security events. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 It is a schematic diagram of the process of the present invention;

[0057] Figure 2 The figure is a flow chart of associating data on the same public safety event according to the present invention. DETAILED DESCRIPTION

[0058] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] See also Figure 1-Figure 2 The first aspect of the present invention provides a data mining method based on public safety events, comprising the following steps:

[0060] Acquire multimodal data from different sources, including public area surveillance data, radar sensor data, and vehicle sensor data, and pre-process the multimodal data from different sources to form a public safety event dataset;

[0061] By combining natural language processing and image processing technologies, an index structure is constructed to retrieve structured and unstructured data in public safety event datasets;

[0062] By accessing the data source of the public news network, relevant data of public safety events are collected from the security events in the data source, and the popularity of public safety events is ranked;

[0063] According to the ranking order of the popularity of security events, multimodal data retrieval is performed on structured data and unstructured data in the public security event dataset, and data on the same public security event in the public security event dataset is associated;

[0064] By correlating the data in the public security event data set, and according to the security event characteristics, performing correlation analysis on the data in the correlated public security event data set;

[0065] According to the spatiotemporal characteristics of public security event data and the correlation analysis results of the data in the public security event data set, security event correlation groups are established.

[0066] Specifically, in this embodiment, by combining natural language processing and image processing technology, an index structure is constructed to retrieve structured data and unstructured data in a public security event data set; at the same time, by accessing a data source of a public news network, relevant data of public security events are collected from security events in the data source, and the public security events are ranked by popularity; and according to the order of the popularity ranking of security events, multimodal data is retrieved from the structured data and unstructured data in the public security event data set, and data about the same public security event in the public security event data set is associated; an index structure is constructed by combining natural language processing and image processing technology to achieve efficient retrieval of structured and unstructured data in the public security event data set; at the same time, after the data source accessing the public news network can rank the popularity of security events, multimodal data retrieval is performed according to the ranking order, and security events with higher popularity are retrieved according to the time popularity priority, so as to facilitate the rapid association and aggregation of relevant data of security events of public concern, facilitate a more comprehensive understanding of the relevant information of the same event, and promote data association and comprehensive analysis.

[0067] By associating the data in the public security incident data set, and according to the characteristics of the security incidents, the data in the associated public security incident data set is analyzed for correlation; at the same time, according to the spatiotemporal characteristics of the public security incident data, the correlation analysis results of the data in the public security incident data set are used to establish security incident correlation groups; by associating the data in the data set, it is easy to gain a deeper understanding of the connection and impact between different events, thereby providing deeper insights and analysis; at the same time, the associated data helps to establish a correlation network between events, thereby promoting a comprehensive analysis of events, rather than being limited to independent analysis of single data. According to the spatiotemporal characteristics and features of the public security incident data, the data is analyzed for correlation and correlation groups are established, so as to facilitate more timely discovery of data with different correlations to security incidents.

[0068] In one embodiment of the present invention, multimodal data from different sources are preprocessed to form a public safety event data set, including the following steps:

[0069] Preprocess the multimodal data of public area monitoring data, radar sensor data, and vehicle sensor data to remove noise, outliers, and redundant information;

[0070] Divide multimodal data into text data, image data, audio data, and video data, and convert each type of data into a unified format and structure;

[0071] The time and location information of each text data, image data, audio data and video data collection is extracted as data labels.

[0072] In one embodiment of the present invention, by combining natural language processing and image processing technology, an index structure is constructed to retrieve structured data and unstructured data in a public safety event data set, including the following steps:

[0073] The text information appearing in the image data and the video data is extracted from the image by using the OCR optical character recognition technology and converted into text data. At the same time, the voice information in the audio data is converted into text data by using the voice recognition tool. In this embodiment, the OCR optical character recognition technology adopts the Tesseract OCR optical character recognition tool. Tesseract is an open source OCR engine that supports multiple languages ​​and can process multiple image formats.

[0074] The text data obtained by converting the image data, audio data and video data is labeled according to the time and location information of the collection of the image data, audio data and video data, and added to the text data;

[0075] For text data, keywords in each text data are extracted through keyword extraction algorithm, and the extracted keywords are added to the data tag as feature representation; in this embodiment, TextRank is a graph-based ranking algorithm, which is similar to the PageRank algorithm and determines the importance of words by calculating the relationship between words. In this algorithm, the relationship between words is usually represented by the co-occurrence frequency between them.

[0076] For image data, the convolutional neural network model is used to apply convolution and pooling operations on the image to extract feature maps. The feature maps output by the convolutional layer are flattened into feature vectors through the fully connected layer of the convolutional neural network model. The feature vectors of the image are mapped to the output space of the image through the fully connected operation to obtain the feature representation of the image. The feature representation of the image is added to the data label.

[0077] For audio data, the audio signal is decomposed into spectral bands through MFCC Mel-frequency cepstrum coefficients, and then the energy of the spectral bands is extracted; the audio signal is divided into short-time windows through short-time Fourier transform, and then each short-time window is transformed to obtain spectrum information;

[0078] The energy data and spectrum information of the obtained spectrum band are added to the data label as feature representation;

[0079] For video data, the Lucas-Kanade optical flow algorithm is used to capture the motion information between adjacent frames in the video and identify the image information of the moving object. The convolutional neural network model is used to apply convolution and pooling operations on the image information of the moving object to extract the feature map. The fully connected layer of the convolutional neural network model is used to flatten the feature map output by the convolutional layer into a feature vector. The feature vector of the image is mapped to the output space of the image through the fully connected operation to obtain the feature representation of the image information of the moving object, and the obtained feature representation is added to the data label.

[0080] According to the data labels, index structures for text data, image data, audio data and video data are constructed respectively to retrieve the structured data and unstructured data in the public safety incident dataset.

[0081] In one embodiment of the present invention, collecting relevant data of public safety events from security events in a data source and ranking the popularity of public safety events include the following steps:

[0082] Collect reports on security incidents from public news network data sources, clean the data, and remove irrelevant information and duplicate data; obtain relevant data through web crawlers, APIs, etc.

[0083] According to the time and location information of the security incident, all areas where public security incidents occurred are divided into blocks, the collected news reports are grouped according to security incidents, and security incidents that occurred in the same time period and the same block are grouped into the same security data group;

[0084] Calculate the popularity of security events in each group based on the number of reports, exposure, and news reporting duration of security events;

[0085] The security events in each group are ranked according to their heat values, and the ranking order of the heat values ​​is used as the heat ranking order.

[0086] In this example, a method based on the number of news reports is used, such as counting the number of reports on each security incident and calculating the corresponding heat values ​​of different reports; a method based on exposure is used, such as counting the number of clicks, reposts, comments, etc. on each security incident news report, and setting corresponding heat values ​​according to different numbers; a method based on the length of news reports is used, and the length of news reports on each security incident is statistically calculated, and a corresponding heat value is set according to the length of news reports. The total heat value of each security incident is calculated by the weighted average method based on the heat values ​​of the number of reports, exposure, and length of news reports.

[0087] Each security event is ranked according to its popularity value and can be presented in a table or chart to provide a more intuitive understanding of the popularity ranking of each security event.

[0088] In one embodiment of the present invention, according to the order of popularity ranking of security events, multimodal data retrieval is performed on structured data and unstructured data in a public security event data set, and data related to the same public security event in the public security event data set is associated, including the following steps:

[0089] According to the order of the heat ranking of the security events in each group of public security events, according to the order of the heat ranking of the security events, extract the text data, image data, audio data or video data in the security event report data corresponding to each security event in the data source of the news network;

[0090] Convert text information in image data and video data, and voice information in audio data, into text data;

[0091] Extracting the keywords of the text data, the feature representation of the image data, the energy data and spectrum information of the spectrum band of the audio data, or the feature representation of the image information of the moving object of the video data from the security incident report data corresponding to each security incident in the data source of the news network;

[0092] The text data, image data, audio data or video data of the security incident are retrieved respectively through the index structure of the corresponding type of data, and the retrieved data are correlated with each other as data about the same public security incident.

[0093] In one embodiment of the present invention, by correlating data in a public security event data set, performing correlation analysis on data in the correlated public security event data set according to security event characteristics, the steps include:

[0094] Obtain text data from the corresponding security incident report data in the data source of the news network, extract keywords from each text data through a keyword extraction algorithm, and count the frequency of occurrence of all keywords in the text data associated with the public security incident data set, and count the frequency of occurrence of all keywords in each text data through a weighted average method to obtain a correlation evaluation value of the text data;

[0095] Obtaining the feature representation of the image of the image data or the feature representation of the image information of the moving object of the video data in the corresponding security incident report data in the data source of the news network, and the feature representation of the image data or video data associated with the corresponding public security incident data set;

[0096] The correlation between image features is evaluated by training a Simesian neural network model, and the correlation evaluation value of the image data or video data associated in the public security event data set is evaluated by the trained Simesian neural network model;

[0097] Obtain an audio spectrum graph of an image of audio data in the corresponding security incident report data in the data source of the news network, and an audio spectrum graph of the audio data associated with the corresponding public security incident data set;

[0098] The correlation between audio spectrograms is evaluated by training the Simese neural network model, and the correlation evaluation value of the audio data associated in the public safety event dataset is evaluated by the trained Simese neural network model.

[0099] In this embodiment, the Simese neural network model is trained by the following method to evaluate the correlation between image features:

[0100] Collect paired image data as training sets. The similarities of these image pairs are different, and a correlation evaluation value is artificially set for each image pair.

[0101] Assign a label to each image pair, indicating their association evaluation value;

[0102] Build a neural network structure as a branch of the Simese network. This network will be used to extract features of a single image;

[0103] The Simese network consists of two identical branches that share the same weights and architecture. Each branch processes an image input; at the top layer of the Simese network, the feature vectors extracted by the two branches are compared to calculate the similarity between them.

[0104] The training set image data is input into the two branches of the Simese network, and the Simese neural network model is trained to evaluate the association evaluation value.

[0105] In one embodiment of the present invention, according to the spatiotemporal characteristics of the public security event data and the correlation analysis results of the data in the public security event data set, a security event correlation group is established, including the following steps:

[0106] Count the time difference between the data collection time information of the associated data in the public security incident data set and the time of occurrence of the corresponding security incident report data in the data source of the news network;

[0107] Count the distance difference between the data collection location information of the associated data in the public security incident data set and the time occurrence location of the corresponding security incident report data in the data source of the news network;

[0108] According to the spatiotemporal characteristics of public safety event data and the correlation analysis results of the data in the public safety event data set, the spatiotemporal correlation correlation evaluation values ​​of different public safety events are calculated;

[0109] According to the spatiotemporal correlation evaluation values ​​of different public safety events, the public safety events are grouped into high spatiotemporal correlation groups and low spatiotemporal correlation groups.

[0110] In one embodiment of the present invention, the spatiotemporal correlation evaluation values ​​of different public safety events are calculated by the following formula:

[0111]

[0112] Where K is the spatiotemporal correlation evaluation value of the data to be associated in the public security event dataset, E is the correlation evaluation value of the text data, image data, audio data or video data in the public security event dataset, E0 is the average correlation evaluation value of the text data, image data, audio data or video data in the public security event dataset, and E max is the maximum value of the association evaluation value of text data, image data, audio data or video data in the public security event data set, T is the time difference of the associated data in the public security event data set, T0 is the average time difference of all the associated data in the public security event data set, T max is the maximum value of the time difference of all the associated data in the statistical public safety event data set, L is the distance difference of the associated data in the statistical public safety event data set, L0 is the average value of the distance difference of all the associated data in the statistical public safety event data set, and L max It is the maximum value of the distance difference of all the associated data in the public safety event dataset.

[0113] In one embodiment of the present invention, according to the spatiotemporal correlation evaluation values ​​of different public safety events, public safety events are grouped into high spatiotemporal correlation groups and low spatiotemporal correlation groups, including the following steps:

[0114] According to the spatiotemporal correlation evaluation values ​​of different public safety events, the public safety events whose spatiotemporal correlation evaluation values ​​are greater than a preset threshold are grouped into a high spatiotemporal correlation group;

[0115] Public safety events whose spatiotemporal correlation evaluation values ​​are less than or equal to a preset threshold are grouped into low spatiotemporal correlation groups.

[0116] Specifically, in this embodiment, it is set to group the public security events whose spatiotemporal correlation evaluation value K of the data to be associated in the public security event data set is greater than 0.2 into a high spatiotemporal correlation group;

[0117] The public security events whose spatiotemporal correlation evaluation value K of the associated data in the public security event data set is less than or equal to 0.2 are grouped into a low spatiotemporal correlation group.

[0118] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A data mining method based on public safety events, characterized in that: The following steps are involved: Acquire multimodal data from different sources, including public area surveillance data, radar sensor data, and vehicle sensor data, and pre-process the multimodal data from different sources to form a public safety event dataset; By combining natural language processing and image processing technologies, an index structure is constructed to retrieve structured and unstructured data in public safety event datasets; By accessing the data source of the public news network, relevant data of public safety events are collected from the security events in the data source, and the popularity of public safety events is ranked; According to the ranking order of the popularity of security events, multimodal data retrieval is performed on structured data and unstructured data in the public security event dataset, and data on the same public security event in the public security event dataset is associated; By correlating the data in the public security event data set, and according to the security event characteristics, performing correlation analysis on the data in the correlated public security event data set; According to the spatiotemporal characteristics of public security event data and the correlation analysis results of the data in the public security event data set, security event correlation groups are established; Among them, according to the order of the popularity ranking of security events, multimodal data retrieval is performed on structured data and unstructured data in the public security event data set, and data on the same public security event in the public security event data set are associated, including the following steps: According to the order of the heat ranking of the security events in each group of public security events, according to the order of the heat ranking of the security events, extract the text data, image data, audio data or video data in the security event report data corresponding to each security event in the data source of the news network; Convert text information in image data and video data, and voice information in audio data, into text data; Extracting the keywords of the text data, the feature representation of the image data, the energy data and spectrum information of the spectrum band of the audio data, or the feature representation of the image information of the moving object of the video data from the security incident report data corresponding to each security incident in the data source of the news network; Retrieving text data, image data, audio data or video data of the security incident through the index structure of the corresponding type of data, and correlating the retrieved data as data about the same public security incident; According to the spatiotemporal characteristics of public security event data and the correlation analysis results of the data in the public security event data set, a security event correlation grouping is established, including the following steps: Count the time difference between the data collection time information of the associated data in the public security incident data set and the time of occurrence of the corresponding security incident report data in the data source of the news network; Count the distance difference between the data collection location information of the associated data in the public security incident data set and the time occurrence location of the corresponding security incident report data in the data source of the news network; According to the spatiotemporal characteristics of public safety event data and the correlation analysis results of the data in the public safety event data set, the spatiotemporal correlation correlation evaluation values ​​of different public safety events are calculated; According to the spatiotemporal correlation evaluation values ​​of different public safety events, the public safety events are grouped into high spatiotemporal correlation groups and low spatiotemporal correlation groups; The temporal and spatial correlation evaluation values ​​of different public safety events are calculated using the following formula: Where K is the spatiotemporal correlation evaluation value of the data to be associated in the public security event dataset, E is the correlation evaluation value of the text data, image data, audio data or video data in the public security event dataset, E0 is the average correlation evaluation value of the text data, image data, audio data or video data in the public security event dataset, and E max is the maximum value of the association evaluation value of text data, image data, audio data or video data in the public security event data set, T is the time difference of the associated data in the public security event data set, T0 is the average time difference of all the associated data in the public security event data set, T max is the maximum value of the time difference of all the associated data in the statistical public safety event data set, L is the distance difference of the associated data in the statistical public safety event data set, L0 is the average value of the distance difference of all the associated data in the statistical public safety event data set, and L max It is the maximum value of the distance difference of all the associated data in the public safety event dataset.

2. A data mining method based on public safety events according to claim 1, characterized in that: Preprocessing multimodal data from different sources to form a public safety incident dataset includes the following steps: Preprocess the multimodal data of public area monitoring data, radar sensor data, and vehicle sensor data to remove noise, outliers, and redundant information; Divide multimodal data into text data, image data, audio data, and video data, and convert each type of data into a unified format and structure; The time and location information of each text data, image data, audio data and video data collection is extracted as data labels.

3. A data mining method based on public safety events according to claim 2, characterized in that: By combining natural language processing and image processing technologies, an index structure is constructed to retrieve structured and unstructured data in the public safety incident dataset, including the following steps: The text information in the image data and video data is extracted from the image through OCR optical character recognition technology and converted into text data. At the same time, the voice information in the audio data is converted into text data through voice recognition tools; The text data obtained by converting the image data, audio data and video data is labeled according to the time and location information of the collection of the image data, audio data and video data, and added to the text data; For text data, the keyword extraction algorithm is used to extract keywords from each text data, and the extracted keywords are added to the data label as feature representation; For image data, the convolutional neural network model is used to apply convolution and pooling operations on the image to extract feature maps. The feature maps output by the convolutional layer are flattened into feature vectors through the fully connected layer of the convolutional neural network model. The feature vectors of the image are mapped to the output space of the image through the fully connected operation to obtain the feature representation of the image. The feature representation of the image is added to the data label. For audio data, the audio signal is decomposed into spectral bands through MFCC Mel-frequency cepstrum coefficients, and then the energy of the spectral bands is extracted; the audio signal is divided into short-time windows through short-time Fourier transform, and then each short-time window is transformed to obtain spectrum information; The energy data and spectrum information of the obtained spectrum band are added to the data label as feature representation; For video data, the Lucas-Kanade optical flow algorithm is used to capture the motion information between adjacent frames in the video and identify the image information of the moving object. The convolutional neural network model is used to apply convolution and pooling operations on the image information of the moving object to extract the feature map. The fully connected layer of the convolutional neural network model is used to flatten the feature map output by the convolutional layer into a feature vector. The feature vector of the image is mapped to the output space of the image through the fully connected operation to obtain the feature representation of the image information of the moving object, and the obtained feature representation is added to the data label. According to the data labels, index structures for text data, image data, audio data and video data are constructed respectively to retrieve the structured data and unstructured data in the public safety incident dataset.

4. A data mining method based on public safety events according to claim 1, characterized in that: Collecting relevant data of public security events from security events in the data source and ranking the popularity of public security events includes the following steps: Collect reports on security incidents from public news network data sources and perform data cleaning to remove irrelevant information and duplicate data; According to the time and location information of the security incident, all areas where public security incidents occurred are divided into blocks, the collected news reports are grouped according to security incidents, and security incidents that occurred in the same time period and the same block are grouped into the same security data group; Calculate the popularity of security events in each group based on the number of reports, exposure, and news reporting duration of security events; The security events in each group are ranked according to their heat values, and the ranking order of the heat values ​​is used as the heat ranking order.

5. The data mining method based on public safety events according to claim 3 is characterized in that: By correlating the data in the public security event data set, and according to the security event characteristics, performing correlation analysis on the data in the public security event data set to be correlated, the method includes the following steps: Obtain text data from the corresponding security incident report data in the data source of the news network, extract keywords from each text data through a keyword extraction algorithm, and count the frequency of occurrence of all keywords in the text data associated with the public security incident data set, and count the frequency of occurrence of all keywords in each text data through a weighted average method to obtain a correlation evaluation value of the text data; Obtaining the feature representation of the image of the image data or the feature representation of the image information of the moving object of the video data in the corresponding security incident report data in the data source of the news network, and the feature representation of the image data or video data associated with the corresponding public security incident data set; The correlation between image features is evaluated by training a Siamese neural network model, and the correlation evaluation value of the image data or video data associated in the public security event data set is evaluated by the trained Siamese neural network model; Obtain an audio spectrum graph of an image of audio data in the corresponding security incident report data in the news network data source, and an audio spectrum graph of the audio data associated in the corresponding public security incident data set; The correlation between audio spectrograms is evaluated by training the Siamese neural network model, and the correlation evaluation value of the audio data associated in the public safety event dataset is evaluated by the trained Siamese neural network model.

6. A data mining method based on public safety events according to claim 1, characterized in that: According to the spatiotemporal correlation evaluation values ​​of different public security events, the public security events are grouped into a high spatiotemporal correlation group and a low spatiotemporal correlation group, including the following steps: According to the spatiotemporal correlation evaluation values ​​of different public safety events, the public safety events whose spatiotemporal correlation evaluation values ​​are greater than a preset threshold are grouped into a high spatiotemporal correlation group; Public safety events whose spatiotemporal correlation evaluation values ​​are less than or equal to a preset threshold are grouped into low spatiotemporal correlation groups.

Images