Information collection and analysis method and system

Abstract

The invention belongs to the technical field of information, and provides an information collection and analysis method and system, and the method comprises the steps: S1, obtaining a data item; s2, processing the data obtained in the step S1, and deleting repeated and invalid data items; s3, classifying the data items obtained in the step S2; s4, performing classification prediction on the newly imported information data, and outputting a coping type; and S5, presenting an information data comprehensive thermodynamic diagram on the map according to the types, the number and the area distribution of the effective data items and the areas. According to the invention, the processing efficiency of clue data is effectively improved, and intelligent classification, early warning and distribution functions are realized. Through real-time data visualization, a decision maker can visually understand clue distribution and importance and make a response quickly.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and in particular relates to an information collection and analysis method and system. Background Art

[0002] In recent years, the popularity of the Internet and digital technology has enabled all kinds of information to flow more quickly and widely on the Internet, including a large amount of clues and data related to youth protection. However, most places still use traditional methods to handle these clues, such as relying entirely on manual data verification, classification and processing. This approach has several obvious problems.

[0003] First, data processing methods that rely entirely on manual labor are inefficient and often increase the time it takes to provide feedback to information providers, resulting in delayed responses. Second, most current platforms have not yet realized the functions of intelligent classification and intelligent distribution of clues, and still require manual tedious classification work. This situation is particularly evident when a large number of clues pour in, and it is difficult for staff to process these clues quickly and accurately, which may result in the loss of some key information.

[0004] To this end, we provide an information collection and analysis method and system. Summary of the invention

[0005] The present invention provides an information collection and analysis method and system, aiming to solve the problems raised by the above background technology.

[0006] The present invention is implemented in this way: an information collection and analysis method comprises the following steps: S1. Get data items; S2. Processing the data obtained in step S1 and deleting duplicate and invalid data items; S3. Classifying the data items obtained in step S2; S4. Classify and predict the newly imported information data and output the response type; S5. Based on the type, quantity and regional distribution of valid data items, a comprehensive heat map of information data is presented on the map according to the region.

[0007] Optionally, the clue acquisition in step S1 includes: S1a. Obtaining data items through active submission by users or by asking users; S1b. According to the user's description ability, it is divided into the first input section and the second input section, and information is collected and described in detail in the corresponding section; the first input section is configured to select the item type and enter a detailed description of the item; the second input section displays pictures and detailed descriptions of each item type, and users can choose the most suitable item type according to their own experience; S1c. Acquire security data related to the user, wherein the security data includes at least two data types.

[0008] Optionally, the step S1 further includes: S1d. Encode the image of the second input block into features, and decode the features into text to generate a text description of the natural image; extract text from the text image, and convert the image or video data into text data.

[0009] Optionally, step S3 includes S3a. The data items in step S2 are divided into first and second category data items; S3b. Classify the first category data items according to event types.

[0010] Optionally, step S4 includes S4a. The first data item after classification in step S3b is respectively labeled with different types of labels; S4b. Output corresponding response types according to different type labels.

[0011] Optionally, step S5 uses a heat map when visually displaying the event, wherein the number of data items in each region is indicated by color depth, wherein the darker the color, the more data items there are in the region, and vice versa, thereby intuitively presenting the distribution of each region and the regional characteristics of the protection type.

[0012] Optionally, the step S1 further includes S1e. collecting online posting content of users; It also includes S6. analyzing the content posted by users; A preliminary score is calculated based on a predetermined keyword library and a weight coefficient, and the analyzed published content is identified; the keyword library stores a plurality of keywords, and each of the keywords is assigned a specific weight, which represents the importance of the word in the published content; if the preliminary score exceeds a preliminary set threshold, a warning score is calculated for the user's published content; and it is determined whether the warning score exceeds a first threshold; S7. Correct the warning score that exceeds the first threshold. When the warning score obtained after correction exceeds the second threshold, the warning mechanism is triggered to conduct further manual review and intervention on the content posted by the user.

[0013] Optionally, the preliminary score formula is: The sentiment analysis scores are: The early warning score calculation formula is: in, represents the warning score; Indicates the total weight of keywords; Indicates the number of recent posts by the poster; The number of historical posts of the poster; Indicates the number of replies to the post; Indicates the number of views of the post by others; represents the sentiment analysis score; represents the sentiment score, and ; is a predefined weight coefficient; The step S7 comprises: S7a identifying and extracting key information from the published content, the key information includes name information, geographic location information and time information; S7b. When a specific name is clearly mentioned in the content, the warning score is increased according to the specificity of the situation, and when the name is a public figure or a well-known person, the warning score is reduced; S7c. When the content explicitly mentions specific geographic location information, increase the weight of the warning score; S7d. Correct the warning score according to the specificity of the time information mentioned in the content, and increase the warning score when specific time information is mentioned; S7e. When the content mentions specific name information, geographic location information, and time information at the same time, the warning score is increased.

[0014] An information collection and analysis system, the system is used to implement the information collection and analysis method; the system comprises: The data collection module is configured to: obtain data items and connect to a database to obtain data items related to a specific group of people; The data processing and screening module is configured to: receive data items from the data collection module and delete duplicate and invalid data items; The processor is configured to: receive valid data items from the data processing and screening module, and classify and analyze the data items to classify the data items into first-category data items and second-category data items; The prediction and classification module is configured to: receive data items from the processor; perform classification prediction on the newly imported data items and output corresponding response types; The visualization and display module is configured to: receive data items from the prediction and classification module, and present a comprehensive heat map of the data items on a map based on the type, quantity and regional distribution of the valid data items in units of regions.

[0015] The beneficial effects achieved by the present invention are that the information collection and analysis method greatly improves the efficiency and accuracy of data processing by automatically deleting duplicate and invalid clues. It provides accurate and clean data for subsequent analysis and ensures the reliability of the results. Newly imported information clues can be processed, classified and predicted by the system in real time, greatly reducing the actual incidence of potential risks.

[0016] The calculation of the warning score adopts a comprehensive method, integrating multiple important factors such as the weight of keywords, the user's posting activity, the degree of interaction of posts, and sentiment analysis scores. This calculation method not only takes into account the content of the post itself, but also involves the user behavior behind the post and the public feedback, making the warning score more objective and comprehensive. Since the warning score can accurately reflect the potential risks of a post, this greatly improves the timeliness and accuracy of the warning mechanism.

[0017] Moreover, the correction of warning scores adds a layer of depth and accuracy to the warning mechanism. Although the initial warning scores already reflect some potential risks, correction based on the specific content of the posts, such as identifying names of people, places and time information, can further determine the specificity and urgency of these risks. Through correction, false alarms caused by general content, such as discussions of public figures or simple mentions of major cities, can be reduced. This can avoid wasting resources on unnecessary warnings.

[0018] The present invention effectively improves the processing efficiency of clue data and realizes intelligent classification, early warning and distribution functions. Through real-time data visualization, decision makers can intuitively understand the distribution and importance of clues and respond quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 It is a flow chart of an information collection and analysis method provided by the present invention; Figure 2is a flow chart of step S1 of an information collection and analysis method provided by the present invention; Figure 3 It is a flow chart of event type classification of an information collection and analysis method provided by the present invention; Figure 4 It is a schematic diagram of a comprehensive heat map of clues obtained by an information collection and analysis method provided by the present invention; Figure 5 It is a flowchart of the processing of importing clues into the first input plate and the second input plate of an information collection and analysis method provided by the present invention; Figure 6 It is a flow chart of a first response type and a second protection type of an information collection and analysis method provided by the present invention; Figure 7 The present invention provides a flowchart of an information collection and analysis method for obtaining department classification results. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0022] The terms "first" and "second" in this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps, operations, components or modules is not limited to the listed steps, operations, components or modules, but may optionally include steps, operations, components or modules that are not listed, or may optionally include other steps, operations, components or modules that are inherent to these processes, methods, products or devices.

[0023] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0024] like Figures 1 to 5 As shown, an information collection and analysis method of an exemplary embodiment includes the following steps: S1. Get data items.

[0025] Specifically, users submit relevant information data items. The data items may include but are not limited to various information collected from channels such as schools, communities, network platforms, medical institutions, etc. In addition, through cooperation with relevant institutions, security incident data involving users in their hands can be obtained.

[0026] S2. Process the data obtained in step S1 and delete duplicate and invalid data items; The data information uploaded by users may contain invalid or duplicate data items. To ensure the accuracy and processing efficiency of subsequent analysis, this step optimizes data input by removing duplicate, erroneous or irrelevant data items. Specifically, this step applies predefined semantic filtering rules to identify and remove low-quality data, thereby improving the validity and relevance of the data and reducing the burden of subsequent processing.

[0027] Specifically, this step uses semantic filtering rules to identify and remove low-quality data items to improve the overall quality of the data and reduce the burden of subsequent processing. Predefined semantic filtering rules are used to automatically detect and remove data items that may be irrelevant or unuseful.

[0028] For example, in actual applications, some users may submit the same content repeatedly in order to increase attention; while some young users may enter information at random just for entertainment purposes. In these cases, the system will identify these data items as "many repeated values" or "many meaningless words" and remove them accordingly. In other words, this step deletes the same data items that appear multiple times and deduplicates the content to ensure that the data entering the subsequent analysis has higher validity and relevance.

[0029] S3. Classify the data items obtained in step S2.

[0030] Specifically, an intelligent comprehensive discrimination model based on multimodal data can achieve accurate classification of data items. Considering that data may come from different sources and have different formats (for example, text, pictures, videos, etc.), multimodal models can effectively process data in these different formats to achieve accurate and consistent classification.

[0031] The multimodal model integrates the characteristics of various data items during processing, ensuring that different types of input (such as text descriptions, image features, or video information) can be recognized and parsed. During classification, the model distinguishes the types of data items based on feature fusion and content analysis, thereby meeting subsequent processing requirements and achieving refined classification and management of data.

[0032] S4. Classify and predict the newly imported information data and output the response type.

[0033] For example, when a data item describes an abnormal behavior that a user encounters on the network, classification prediction classifies it as a "network" type for subsequent processing.

[0034] In the process of classification prediction, by extracting keywords and analyzing the context of the data content, the method can semantically parse the key information in the data item and determine the most suitable response type. For example, if the data item involves a description of a specific scene (such as a campus, home, or public place), the classification model will automatically identify and classify it as the corresponding type. In this way, the method outputs different response types based on the content characteristics of the data item, thereby achieving accurate classification prediction and facilitating subsequent analysis and response.

[0035] S5. Based on the type, quantity and regional distribution of valid data items, a comprehensive heat map of information data is presented on the map according to the region.

[0036] Based on the type, quantity and regional distribution of valid data items, step S5 intuitively presents the comprehensive distribution of information data on the map in the form of a heat map. This step aims to provide a clear view showing the density of data items in different regions to reveal which areas have greater potential risks. Heat maps can accurately mark high-risk areas, which is of great reference value for decision-making and resource allocation.

[0037] The information collection and analysis method not only relies on data items submitted by users, but also cooperates with relevant institutions to ensure the comprehensiveness and authenticity of the data. This cooperation helps to capture and record any potential risks that may involve users in a timely manner, and greatly improves the efficiency and accuracy of data processing by automatically removing duplicate or invalid data items. After this processing, the method obtains clean and accurate data, laying a reliable foundation for subsequent analysis.

[0038] Newly imported data items can be processed, classified and predicted in real time in this method, ensuring a quick response to potential risks and thus reducing the probability of actual risks. Through the visualization of heat maps, decision makers and relevant agencies can more intuitively identify areas with high concentrations of potential risks, thereby taking more targeted prevention and response measures.

[0039] like Figure 2 As shown, as an optional implementation, S1a. The data item is obtained by actively submitting or asking the user; In the implementation of this method, the input of data items depends on the model structure. Users can input their own data items into the system, which may include text data, image data, or video data. In addition to the data items actively provided by users, more content can be collected by asking users and encouraging them to provide more detailed information.

[0040] S1b. According to the user's ability to express, it is divided into the first input section and the second input section, and information is collected and described in detail in the corresponding section; the first input section is configured to select the item type and enter a detailed description of the item; the second input section displays pictures and detailed descriptions of each item type, and the user can choose the most suitable item type based on his or her own experience.

[0041] Different users may have different expression preferences. Some users may be better at text descriptions, while others may prefer to convey information through pictures or videos. More specifically: The first input section: For users who have the ability to express themselves in words, this section provides a text input interface, allowing users to describe their experiences or related matters in detail. This design takes into account that such users usually have good writing skills and tend to share their experiences or knowledge in detail.

[0042] Second input section: For users who prefer to express themselves through images or videos, this section provides image and video upload functions. For such users, they can share their experiences more intuitively by taking or selecting corresponding pictures and videos.

[0043] In actual use, in order to cover the diverse expressions of the user group, the input sections are divided to provide a more flexible submission method.

[0044] S1c. Acquire security data related to the user, wherein the security data includes at least two data types.

[0045] In order to obtain a comprehensive view of information, cooperation with relevant agencies is necessary. Through cooperation, relevant data from official records can be obtained and cross-validated with data items provided by users. By matching user submissions with these official data, the accuracy of data items can be verified and the accuracy of early warnings and risk assessments can be improved.

[0046] As an optional implementation, the step S1 further includes: S1d. Encode the image of the second input block into features, and decode the features into text to generate a text description of the natural image; extract text from the text image, and convert the image or video data into text data.

[0047] In order to unify clue data from different types of input (e.g., images, videos, and text) and facilitate subsequent processing, modern deep learning technology is used to convert images and videos into text descriptions. Video clues are processed into images by extracting frames. It is determined whether the image is a text image or an image image; for text images, OCR technology is used to extract text from the image, and for image images, Image Caption technology is used to generate text from the image.

[0048] Image Caption model architecture: First, we extract features from the image. By using a pre-trained convolutional neural network (CNN), we can convert the image into a feature vector, which represents the high-level features of the image. Next, we use a recurrent neural network (RNN) to model the sequence of these image feature vectors to generate the corresponding image description. Here, we often use an encoder-decoder model based on LSTM or GRU. In this model, both the encoder and decoder are based on RNN. The decoder outputs the probability distribution of the next word based on the intermediate state of the encoder and the previously generated word.

[0049] To optimize the performance of the model, we can use a dataset similar to that used for machine translation and language models for training. By optimizing the maximum likelihood function or other related loss functions, we can get a model that can generate the best image description.

[0050] Image Caption technology can generate text descriptions of natural images. This technology uses the im2txt model, which is an encoder-decoder model. In simple terms, it first converts the original image into features through encoding operations, and then decodes these features into text, thereby realizing the conversion from image to text. In this model, CNN is used in the encoding part because CNN performs well in image feature extraction. After the image is processed by CNN, we can get a vector of fixed length. In the decoding part, we use LSTM because it has a good memory ability for vector sequences and performs well in generating text. In this way, we can successfully decode the vector obtained earlier into a text description.

[0051] The word vector with the highest probability in the figure is the predicted description of the image.

[0052] Code implementation: Import dependent libraries and modules: pandas, numpy, CountVectorizer, MultinomialNB, train_test_split, nltk, re.

[0053] Define the load_data function to load the dataset.

[0054] Define the clean_text function to clean the text data.

[0055] Define the preprocess_data function to preprocess the data set: first define stopwords, then clean and segment each sentence, then remove stop words and stem the words, convert the processed sentences back into strings, and finally convert the text data into vector form.

[0056] Define the train_model function to train and evaluate the classification model: first, extract features from the image. Here, the ResNet network is used for feature extraction. The image is converted into an image feature vector. The input is the image and the output is the image feature vector. Finally, the TODO ResNet network is used for feature extraction, and the image feature vector is sequence modeled using a recurrent neural network to generate a description of the image (adding Embedding, LSTM layer, and fully connected layer in sequence). Then, the compilation model is built and the model parameters are optimized (the input is the model of the image description generated by the graph feature vector and the corresponding text sequence, and the output is the trained model of the generated image description).

[0057] OCR model architecture: First, the image is preprocessed, including image enhancement, denoising, binarization and other processing to improve the accuracy of character recognition.

[0058] The convolutional neural network is used to extract features from the preprocessed image to obtain a high-level semantic expression of the image.

[0059] The recurrent neural network is used to perform sequence modeling on the features extracted by the convolutional neural network. That is, the semantic expression of the image is used as the input of the sequence, and recurrent units such as LSTM or GRU are used in combination with the attention mechanism to perform character recognition and decoding, and finally the text information on the image is obtained.

[0060] Extract text from text images. This technology uses the third-party library pytesseract to implement the task of recognizing text in images. The pytesseract library consists of three main parts: image preparation preprocessing, text recognition, and post-processing.

[0061] In the image preparation and preprocessing stage, the CTPN algorithm is used to locate the text area. This algorithm is improved on the basis of Faster RCNN and combines CNN and LSTM deep networks. Through this combination, CTPN can accurately locate text lines in the convolution layer and can adapt to input images of various sizes. After the text is accurately located, text correction operations, including rotation and affine transformation, and text segmentation operations, including binarization and noise filtering, are also performed to improve the accuracy of subsequent text recognition.

[0062] In the text recognition stage, an algorithm based on visual attention is used, which combines the characteristics of CNN, RNN and attention model. First, by running the CNN model with a sliding window, key features are extracted on the input image. Then, the obtained feature sequence is input into the LSTM model superimposed on the top of CNN to encode the feature sequence. Finally, the attention model is used for decoding operation to convert the encoded feature sequence into an output label sequence, the final text recognition result.

[0063] In the post-processing stage, rules and language models are used to further process the recognition results. These post-processing methods can correct and optimize the recognition results to improve the overall recognition accuracy and consistency.

[0064] Code implementation: Import dependent libraries and modules: cv2, numpy, pytesseract, tensorflow, non_max_suppression.

[0065] Define the load_image function to load images.

[0066] Define the preprocess_image function to preprocess the image: first set the CTPN parameters min_confidence, width, height, padding, then build the CTPN network, then get the image size and scale it, after scaling the image, build a binary image, call the opencv DNN module for text detection and get the detection results, finally post-process the detection results to get the text box coordinates, use the non-maximum suppression method to get the region proposal box (RoI), perform text correction and segmentation operations on each RoI for subsequent OCR processing, here you need to calculate the width and height of the RoI for padding, and you need to construct the RoI image to prepare for OCR recognition.

[0067] Define the decode_predictions function to post-process the detection results and obtain the text box coordinates, that is, traverse the scores and extract the text box.

[0068] Define the preprocess_text function to preprocess the recognized text, that is, to process the text to extract key information.

[0069] Define the predict_text function to recognize and extract text from an image: first load the image, then preprocess the image and detect text, and finally preprocess the recognized text.

[0070] Specifically, the im2txt model is used to generate text descriptions of natural images by encoding the original image into features and decoding the features into text; the third-party pytesseract library is used to extract text from text images through image preparation preprocessing, text recognition and post-processing. The purpose of this operation is to convert data such as pictures and videos into text data.

[0071] For example, consider a scenario where a user uploads a picture of unusual behavior observed in a public place. The system automatically generates descriptive text, such as "an adult man talking to another person near a slide." In addition, if the picture contains text information (such as a license plate number or a store name), the method automatically recognizes and extracts the text content.

[0072] To further illustrate, assume that the input data includes 34 video files in mp4 format and 130 image files in jpg format. Each file is converted into a line of text description through processing, so the output result is 34 + 130 = 164 text files in txt format.

[0073] The final text data will be further sorted, summarized and analyzed as clue data to support subsequent analysis steps and prediction tasks.

[0074] Regarding the data item processing in step S2, specifically: enter the multimodal data item sorting module, process the data items obtained in step S1e, and delete duplicate and invalid data items. By deleting invalid data items, three tables of "processed answers of the first type of users", "processed answers of the second type of users" and "invalid data of the first type of users" are finally obtained for use in subsequent models.

[0075] As shown in FIG5 , the specific processing flow of importing data items into the first input block and the second input block is as follows: The first category of user data in the first input section: During the data cleaning process, delete all data items whose description information consists only of numbers, letters, or special characters; deduplicate the description information columns; delete the description information containing meaningless or empty rows; then, merge the original table with the same filtered data items and delete duplicate entries; finally, obtain the "processed data of the first category of users" for use in subsequent models.

[0076] The second type of user data in the second input section: During the data cleaning process, delete duplicate data items except batch ID; delete data with completely empty answer content; and finally obtain the "processed data of the second type of users" for use in subsequent models.

[0077] In addition, image and video data are processed separately by the multimodal model, and this model only processes table data. The attributes of each column in the table include: user ID, name, gender, age, organization, level, address, contact information, user type, batch ID, creation time, event type and description information.

[0078] like Figure 6 As shown, as an optional implementation, step S3 includes S3a. Divide the data items in step S2 into first category data items and second category data items.

[0079] Specifically, preliminary classification is carried out based on the content of the collected data to determine the nature of the data items, which are divided into first-category data items and second-category data items. The first-category data items refer to more important content, while the second-category data items refer to general content.

[0080] In the classification process, the classification model of the data items is applied. Given the large difference between the first and second category data items, the data items are firstly classified into two categories to improve the accuracy of the classification results. The two-category model uses the BERT classification algorithm. Through the BERT model, all data items are divided into the first and second categories. After the two-category classification is completed, it is convenient for the accurate execution of subsequent processing.

[0081] The datasets marked as the first and second categories are input into the BERT model, and the BERT-based binary classification model is trained to obtain the trained BERT binary classification model. Subsequently, the unclassified data items of various categories are input into the trained BERT binary classification model, and the data items are automatically classified by the model, and finally the results belonging to the first category and the second category are obtained. BERT is a bidirectional pre-trained language model based on the Transformer architecture released by Google in 2018, which is suitable for such natural language processing tasks.

[0082] S3b. Further classify the first category of data items according to specific event types.

[0083] In order to further optimize the processing flow and achieve rapid response, this step further classifies the first category of data items and divides them into different categories according to specific types of matters. The types of matters include but are not limited to: Category A matters, Category B matters, Category C matters, and Category D matters.

[0084] As shown in Figure 3, the data items that have been initially labeled are input into the untrained four-classification model based on naive Bayes. The model is then trained using the data set to complete the training of the four-classification model based on naive Bayes. Subsequently, the first type of data items that have not been refined are input into the trained four-classification model, which automatically classifies the data items and finally obtains data items of different types, namely, type A data items, type B data items, type C data items, and type D data items.

[0085] As shown in Figure 7, the TF-IDF algorithm is used to perform text mining on the labeled data set, and the keywords obtained by mining are combined to establish a keyword library. Then, the naive Bayesian category classification model is trained using the labeled data set to obtain a trained naive Bayesian category classification model. Next, the keyword library and the trained naive Bayesian category classification model are used to automatically classify the unclassified data items, and finally the classification results of the data items are obtained.

[0086] Among them, TF-IDF is a common technique used in information retrieval and text mining. TF-IDF is calculated by multiplying the TF (term frequency) of a word by its IDF (inverse document frequency). Through this calculation method, the relative importance of each word in the document can be determined in order to identify and extract key content.

[0087] As an optional implementation, step S4 includes S4a. Label the first category data items classified in step S3b with different types of labels.

[0088] Specifically, this method uses the BERT model to perform multi-label classification on data items, and refines the data items into multiple classification types to achieve comprehensive coverage and targeted processing. The classification labels include five types of labels, corresponding to different dimensions of user protection, to ensure the security, care and protection of information data in multiple scenarios.

[0089] In the classification process, the data with type labels are first input into the BERT-based classification model to obtain the trained BERT classification model. Subsequently, the unclassified data items are input into the trained BERT classification model for classification, and five different types of labels are finally obtained.

[0090] S4b. Output corresponding response types according to different types of labels.

[0091] As an optional implementation, in step S5, when visualizing the event, a heat map is used, and the number of data items in each area is marked with color depth. The darker the color, the more data items there are in the area, and vice versa, so as to intuitively present the distribution of each area and the regional characteristics of each type of security.

[0092] Specifically, as shown in Figure 4, a numerical value is calculated based on the number of relevant data items reported on each region. For example, if a street contains several data items, the value of the street is the number of data items. The numerical value is then converted to a specific color. A color gradient is set, usually from light to dark. Streets with a large number of data items will appear darker, and streets with a small number will appear lighter. Through the above color mapping method, each street is color-coded. When the entire map is presented, the distribution of data items in each region can be easily identified.

[0093] The visualization model includes the following functional modules: Home page: Displays the number of new data items in the past 15 days, the total number of data items, the number of valid data items, the number of important data items, and the number of analysis reports issued.

[0094] Data statistics: Display age distribution, gender distribution, event type, proportion of special data, distribution of data items in each street, and word cloud.

[0095] Heatmap: Based on Baidu Map API (Application Programming Interface), it displays street heatmaps. It adds point overlays to the streets and displays the number of data items. It also provides heatmap scale adjustment function and supports quick return to homepage function.

[0096] Event processing: includes data input functions such as uploading user basic information and description text, photos or videos. It also integrates the functions of data item integration, severity determination, type classification determination and summary report generation.

[0097] As an optional implementation, the step S1 further includes S1e: collecting online published content of users.

[0098] The information collection and analysis method further includes S6: analyzing the published content.

[0099] The analyzed published content is identified by calculating a preliminary score based on a predetermined keyword library and weight coefficient. The keyword library stores multiple keywords, each of which is assigned a specific weight, representing the importance of the word in the published content; if the preliminary score exceeds a preset threshold, a warning score is calculated for the user's published content.

[0100] Specifically, the predefined keyword library includes words related to potential risks. The weight coefficient of each word indicates the importance of the word or the degree of potential risk it indicates. The words in the keyword library are labeled with high and low weights according to semantics so that the degree of risk can be accurately assessed during the analysis process.

[0101] The purpose of judging whether the warning score exceeds the first threshold is to ensure that only high-risk content is further processed. By setting the first threshold, the published content that may require further intervention can be screened out to avoid generating a large number of unnecessary alerts.

[0102] S7. Correct the warning score that exceeds the first threshold. When the corrected warning score exceeds the second threshold, the warning mechanism is triggered to prompt relevant personnel or institutions to conduct further manual review and intervention on the user's published content.

[0103] In order to accurately assess potential risks, it is crucial to calibrate the initial warning score. The calibration is based on the specific information mentioned in the published content, such as key information such as names, locations, and times. For example, if the published content describes the time and location of the event in detail, the risk level may increase, so the warning score should be increased accordingly. If the corrected warning score exceeds the second threshold, the system will automatically trigger the warning mechanism, prompting relevant personnel or institutions to manually review and appropriately intervene in the published content. This mechanism ensures a rapid response in high-risk situations and provides users with the necessary support and protection.

[0104] As an optional implementation, the preliminary score formula is: The sentiment analysis scores are: The early warning score calculation formula is: in, represents the warning score; Indicates the total weight of keywords; Indicates the number of recent posts by the poster; The number of historical posts of the poster; Indicates the number of replies to the post; Indicates the number of views of the post by others; represents the sentiment analysis score; represents the sentiment score, and ; is a predefined weight coefficient; Indicates the activity of the posting user. An increase in this ratio means that the user has posted frequently recently. Indicates the level of interaction or attention a post receives.

[0105] As an implementation specific note: Suppose we set the threshold for the preliminary score to 5 and the threshold for the warning score to 10 based on past experience and data statistics.

[0106] A user posts a message on a social platform, the content of which contains certain emotional or other keywords that express his or her feelings.

[0107] Assume that the weight of one keyword is 4, and the weight of another keyword is 3. The corresponding weight coefficients is 1. Then the preliminary score is ; Exceeds the initially set threshold; enters the second stage of calculation.

[0108] Assumptions: The number of recent posts by the poster is 6 The number of posts by the poster is 10 The number of replies to this post is 30 The number of views of this post by others is 40 The sentiment analysis score is (-0.8) (here we assume the range is -1 to 1, with -1 representing completely negative and 1 representing completely positive).

[0109] Combining the above calculation method, the sentiment analysis score is 1.1 The coefficient is 2; The coefficient is 2; The coefficient is 1.

[0110] Then in this implementation mode, the warning score is "10.8", and the warning score "10.8" exceeds the first threshold value "10".

[0111] The step S7 comprises: S7a. Identify and extract key information from the published content, the key information including names of people, places and time information. When the content involves specific names of people, places and time information, it indicates that the authenticity of the event may be high.

[0112] S7b. When a specific person’s name is explicitly mentioned in the published content, the weight of the warning score is increased according to the specific situation; if the name is a public figure or a well-known person, the warning score is reduced. For example, if a specific person’s name is mentioned in the content, the warning score is increased by a certain amount; and when the name is a public figure or a well-known person, the warning score is reduced accordingly.

[0113] S7c. When the published content explicitly mentions a specific place name, increase the weight of the warning score.

[0114] S7d. The warning score is corrected based on the specificity of the time mentioned in the content. If a specific date or time is mentioned, the warning score is increased.

[0115] S7e. When the content contains specific names of people, places and time at the same time, the warning score should be increased appropriately.

[0116] In this embodiment, the weight value of the preset name is "0.1". If multiple specific names are mentioned in the published content, for example, three, "0.3" is added to the warning score. Assuming the initial warning score is "11.1", if a place name is mentioned, such as a township, "0.9" is added according to the preset administrative area level weight value, and the warning score is "12". For time parameters, the sum of the time parameters is calculated in order of date, hour, and minute. For example, the date corresponds to "0.1", the hour corresponds to "0.3", and the minute corresponds to "0.5". If a certain day, hour, and minute are mentioned in the content, and the sum of the time parameters is "0.9", the warning score increases from "12" to "12.9".

[0117] If the corrected warning score exceeds the preset second threshold (for example, "11.8"), the system will automatically trigger the warning mechanism and notify relevant personnel or institutions for further manual review and appropriate intervention.

[0118] As another specific implementation note: Assume that based on past experience and data statistics, the threshold of the preliminary score is set to 5 and the threshold of the warning score is set to 10. For example, a user posts content on a social platform that contains certain keywords in the keyword library. In this case, based on the number and weight distribution of keywords in the content, the warning score of the content can be calculated and preliminarily evaluated.

[0119] Among them, suppose that the weight of one keyword is 4, and the weight of another keyword is 2. The corresponding weight coefficient is 1. Then the preliminary score is ; Exceeds the initially set threshold; enters the second stage of calculation.

[0120] Assumptions: The number of recent posts by the poster is 4 The number of posts by the poster is 20 The number of replies to this post is 10 The number of views of this post by others is 20 The sentiment analysis score is (-0.8) (here we assume the range is -1 to 1, with -1 representing completely negative and 1 representing completely positive).

[0121] Combining the above calculation method, the sentiment analysis score is 1.1 The coefficient is 2; The coefficient is 2; The coefficient is 1.

[0122] Then the warning score in this implementation is "8.3", which does not exceed the first threshold value "10". Then the warning score is corrected, and of course the warning mechanism is not triggered.

[0123] The calculation of the warning score adopts a comprehensive method, integrating multiple important factors such as the weight of keywords, the activity of user content, the degree of interaction of posts, and sentiment analysis scores. This calculation method not only considers the content of the post, but also combines the behavioral characteristics of users and public feedback, making the evaluation of the warning score more objective and comprehensive. The warning score can accurately reflect the potential problems that may exist in the published content, thereby greatly improving the timeliness and accuracy of the warning mechanism.

[0124] When the warning score exceeds the set threshold, the system will automatically trigger the warning mechanism. This automated warning method greatly improves the response speed to potential risks and ensures that necessary measures can be taken in the shortest possible time. At the same time, for content identified as high-risk, the system can further arrange manual review and intervention to ensure that all potential risks are properly handled.

[0125] The correction of warning scores adds a layer of depth and precision to the warning mechanism. Although the initial warning score indicates potential risks, further correction of the specific content of the post (such as identifying the names of people, places and time information) can more accurately assess the specificity and urgency of the risk. The correction process can effectively reduce false positives caused by general content, such as discussions of public figures or simple mentions of a certain region. In this way, the system can avoid unnecessary warnings and optimize resource utilization.

[0126] Based on the corrected warning scores, relevant agencies can take more targeted actions. For example, after obtaining key information such as specific names, locations, and times, agencies can locate possible problems more quickly and accurately, and take corresponding measures in a timely manner. This corrected warning mechanism ensures a high level of attention to important content, improves the efficiency of intervention in potential problems, and provides more effective support for the handling of high-risk events.

[0127] The early warning mechanism is centered on automation and correction functions, ultimately achieving accurate calculation of early warning scores and timely response, thereby providing data support for subsequent analysis, review and intervention.

[0128] The present invention also provides an information collection and analysis system, which is used to implement the information collection and analysis method; the system comprises: The data collection module is configured to: obtain data items and connect to a database to obtain data items related to a specific group of people; The data processing and screening module is configured to: receive data items from the data collection module and delete duplicate and invalid data items; The processor is configured to: receive valid data items from the data processing and screening module, and classify and analyze the data items to classify the data items into first-category data items and second-category data items; The prediction and classification module is configured to: receive data items from the processor; perform classification prediction on the newly imported data items and output corresponding response types; The visualization and display module is configured to: receive data items from the prediction and classification module, and present a comprehensive heat map of the data items on a map based on the type, quantity and regional distribution of the valid data items in units of regions.

[0129] The system's processing flow is as follows: First, activate the data collection module. The data collection module is used to collect data information provided by users and connect to the database of a specific organization in real time to obtain the latest relevant event data. This real-time update ensures that the data processed by the system is always up-to-date and complete.

[0130] The collected data is then transferred to the data processing and screening module. Here, the data will undergo preliminary screening and cleaning, such as removing duplicates, null values ​​or meaningless data. This process is designed to ensure that only truly relevant and valuable data information is used for subsequent analysis and processing.

[0131] The cleaned and filtered data items are then passed to the processor, where they are further classified and analyzed, such as being broken down into specific categories such as "first-category data items" or "second-category data items."

[0132] Next, the prediction and classification module performs further prediction and classification operations on these classified data items, classifying the new information data into corresponding categories, such as the response type for a specific situation.

[0133] Finally, all processed data flows to the visualization and presentation module, which uses heat maps to comprehensively display the type, quantity and geographical distribution of data items, providing relevant personnel with a clear and intuitive regional overview.

[0134] The exemplary embodiments of the present application may be combined with each other, and the exemplary embodiments obtained by the combination also fall within the scope of the present application.

[0135] This application uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core idea of ​​this application. At the same time, for technical personnel in this field, according to the idea of ​​this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on this application.

Claims

1. A method for collecting and analyzing information, characterized in that: The steps include: S1. Get data items; S2. Processing the data obtained in step S1 and deleting duplicate and invalid data items; S3. Classifying the data items obtained in step S2; S4. Classify and predict the newly imported information data and output the response type; S5. Based on the type, quantity and regional distribution of valid data items, a comprehensive heat map of information data is presented on the map according to the region.

2. The information collection and analysis method according to claim 1, characterized in that: The clue acquisition in step S1 includes: S1a. Obtaining data items through active submission by users or by asking users; S1b. According to the user's description ability, it is divided into the first input section and the second input section, and information is collected and described in detail in the corresponding section; the first input section is configured to select the item type and enter a detailed description of the item; the second input section displays pictures and detailed descriptions of each item type, and users can choose the most suitable item type according to their own experience; S1c. Acquire security data related to the user, where the security data includes at least two data types.

3. The information collection and analysis method according to claim 2, characterized in that: The step S1 further comprises: S1d. Encode the image of the second input block into features, and decode the features into text to generate a text description of the natural image; extract text from the text image, and convert the image or video data into text data.

4. The information collection and analysis method according to claim 1, characterized in that: The step S3 includes S3a. The data items in step S2 are divided into first and second category data items; S3b. Classify the first category data items according to event types.

5. The information collection and analysis method according to claim 1, characterized in that: The step S4 comprises: S4a. The first data item after classification in step S3b is respectively labeled with different types of labels; S4b. Output corresponding response types according to different type labels.

6. The information collection and analysis method according to claim 1, characterized in that: When performing a visual display of the event in step S5, a heat map is used, in which the number of data items in each region is marked by color depth. The darker the color, the more data items there are in the region, and vice versa, so as to intuitively present the distribution of each region and the regional characteristics of the protection type.

7. The information collection and analysis method according to claim 1, characterized in that: The step S1 further includes S1e. collecting the user's online posting content; It also includes S6. analyzing the content posted by users; Calculating a preliminary score based on a predetermined keyword library and a weight coefficient to identify the analyzed published content; the keyword library stores a plurality of keywords, and each of the keywords is assigned a specific weight representing the importance of the word in the published content; If the preliminary score exceeds the preliminary set threshold, a warning score is calculated for the content posted by the user; and it is determined whether the warning score exceeds the first threshold; S7. Correct the warning score that exceeds the first threshold. When the warning score obtained after correction exceeds the second threshold, the warning mechanism is triggered to conduct further manual review and intervention on the content posted by the user.

8. The information collection and analysis method according to claim 8, characterized in that: The preliminary score formula is: The sentiment analysis scores are: The early warning score calculation formula is: in, represents the warning score; Indicates the total weight of keywords; Indicates the number of recent posts by the poster; The number of historical posts of the poster; Indicates the number of replies to the post; Indicates the number of views of the post by others; represents the sentiment analysis score; represents the sentiment score, and ; is a predefined weight coefficient; The step S7 comprises: S7a identifying and extracting key information from the published content, the key information includes name information, geographic location information and time information; S7b. When a specific name is clearly mentioned in the content, the warning score is increased according to the specificity of the situation, and when the name is a public figure or a well-known person, the warning score is reduced; S7c. When the content explicitly mentions specific geographic location information, increase the weight of the warning score; S7d. Correct the warning score according to the specificity of the time information mentioned in the content, and increase the warning score when specific time information is mentioned; S7e. When the content mentions specific name information, geographic location information, and time information at the same time, the warning score is increased.

9. An information collection and analysis system, characterized in that: The system is used to implement the information collection and analysis method according to any one of claims 1 to 8; the system comprises: The data collection module is configured to: obtain data items and connect to a database to obtain data items related to a specific group of people; The data processing and screening module is configured to: receive data items from the data collection module and delete duplicate and invalid data items; The processor is configured to: receive valid data items from the data processing and screening module, and classify and analyze the data items to classify the data items into first-category data items and second-category data items; The prediction and classification module is configured to: receive data items from the processor; perform classification prediction on the newly imported data items and output corresponding response types; The visualization and display module is configured to: receive data items from the prediction and classification module, and present a comprehensive heat map of the data items on a map based on the type, quantity and regional distribution of the valid data items in units of regions.

Images