Information file analysis response management system based on big data

Abstract

The invention discloses an information file analysis response management system based on big data, which belongs to the field of big data processing and government affair management and comprises a data acquisition module, a preprocessing module, an analysis module, a response module and a storage module. The data acquisition module is used for collecting information files by utilizing a crawler technology; the preprocessing module is used for processing the information file; the analysis module performs deep analysis on the information file by constructing a semantic model, so that the file processing efficiency is improved; the response module is used for generating a response file according to an analysis result of the analysis module and sending the response file; the storage module is used for storing files generated by all the modules, and a structured storage mode is adopted, so that the retrieval efficiency is improved.

Description

Technical Field

[0001] The invention relates to an information file parsing response management system based on big data, belonging to the field of big data processing and government affairs management. Background Art

[0002] With the rapid development of Internet technology, the "Internet +" model came into being and quickly penetrated into all areas of social life; especially in the field of government services, the new service model of "Internet + government services" is quietly emerging, saving labor costs while providing the people with more convenient and efficient government services; and with the deepening of departmental digital transformation, the number and complexity of information files are increasing, and traditional manual processing methods can no longer meet the needs of modern government management.

[0003] The invention patent with the existing announcement number CN112615855B is a data security transmission method based on the government affairs second approval platform, which belongs to the field of government affairs informationization. The data security transmission method pre-processes the request data of the client's government affairs second approval business, and determines them as data to be encrypted, sensitive data and ordinary data respectively; for ordinary data, the request data is preliminarily filtered, and when the data itself is safe, it is transmitted; for encrypted data, a dynamic key is generated according to the dynamic key request and fed back to the client to complete data encryption and communication encryption; for sensitive data, encryption rules are selected according to sensitive points for encryption; in communication encryption, an encryption path is generated, and the encryption path contains a certificate. After the data passes the certificate verification, it passes the checkpoint to complete the transmission and data archiving.

[0004] Although the existing technology has achieved data security and communication security during the transmission of necessary data, ensured data integrity, achieved rapid data circulation, improved the efficiency of second-approval business, and truly realized the approval of government affairs in seconds, the information files on different platforms or websites have complex file contents and are difficult to manage and process efficiently. Therefore, this application provides an information file parsing response management system based on big data, which deeply analyzes information files by building a semantic parsing model and automatically generates response strategies. For example, for coordination matters involving multiple departments, the system will assign responsible departments according to preset rules or directly push them to the responsible person's work interface, which greatly shortens the time for file processing and improves the response speed and processing efficiency of departmental work. Summary of the invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to provide an information file parsing and response management system based on big data, which realizes automatic parsing and rapid response of information files by constructing an automatic parsing and response system for files, thereby improving the file processing efficiency.

[0006] To achieve the above object, the present invention provides the following technical solutions: An information file parsing response management system based on big data, comprising: a data acquisition module, a preprocessing module, a parsing module, a response module and a storage module; The data acquisition module is used to collect information files using crawler technology and transmit the information files to the storage module in real time; The preprocessing module is used to perform preliminary processing on the collected original information files; The analysis module is used to perform in-depth analysis on the preprocessed information file, extract key information of the information file and mine potential related information to generate an analysis file; The parsing module is configured with a parsing strategy for parsing red-headed files; The parsing strategy includes: Constructing a semantic model, and using the semantic model to parse the red-headed files in the file type library to generate a parsing result; The parsing result is saved in the red-headed file of the file type library; Calculating the similarity between the red-headed file and a target file, where the target file is another red-headed file in the file type library; Set the similarity threshold to , determine whether the target file is an associated file of the red-headed file; if , the target file is not an associated file; if , the target file is an associated file; Based on the associated files of the red-headed file, a correlation simulation analysis is performed to generate a directional trend of the red-headed file, and the directional trend and the associated files are saved in a constructed analysis list; The response module is used to automatically generate a response file according to the parsed file, and send the response file to relevant institutions or personnel according to a preset response channel; The storage module is used to store files generated by each module, and adopts a structured storage method, as well as data encryption and access control technology.

[0007] Specifically, the data acquisition module is configured with a crawler strategy for acquiring information files; The crawler strategy includes: Obtain the target website's infrastructure and URL rules; use the browser's developer tools to view the target website's web page source code and obtain the HTML tags and attributes of the target file; Based on the structure of the target website, write Python crawler code; Configure the crawler network to avoid being blocked by the target website; Record the crawler's running log to monitor the crawler's running status and troubleshoot problems.

[0008] Specifically, the pre-processing module is configured with a classification strategy for preliminary processing of red-headed documents; The classification strategies include: Design a regular expression to match different types of document numbers in the red-headed document; Preprocessing the red-headed file; Using the regular expression to match the preprocessed red-headed file, extracting the document number information of the red-headed file; Rename the red-headed document in the format of document number + file name; Construct a file type library, and save the red-headed file into a subfolder in the file type library that uses partial information of the document number as a naming format.

[0009] Specifically, the steps of constructing the semantic model include: Collect historical official documents, clean the documents, and divide them into training set and verification set; Use the BERT tokenizer to convert the red-headed files in the validation set into word sequences ; use Transformer The encoder structure converts the word sequence into a word sequence vector ; Use RNN to extract the hidden sequence vector in the word sequence vector ; The emotion sequence vector in the definition file is , and calculate the average sentiment vector of the document ; After being processed by the sentiment attention mechanism, the feature vector of the file is obtained ; use Softmax Function to calculate the sentiment probability of a file p , the expression is as follows: , where is the weight matrix, is the bias matrix; Define two levels of sentiment, positive and negative, and set the threshold of sentiment probability as ,in ;like , the sentiment level is negative; if , the emotional level is positive; Output the feature vector of the red header file and emotional level; The semantic model is verified using the verification set; if the verification fails, the training set is expanded and the model is continuously trained; if the verification passes, the semantic model is successfully constructed.

[0010] Specifically, the processing steps of the emotional attention mechanism include: Based on the average sentiment vector, using Hyperbolic tangent function calculates hidden vector Sentiment score ,in, is the hidden sequence vector Middle hidden vectors, ; Based on the sentiment score, an exponential function is used to obtain the attention weight matrix of the hidden sequence vector ; Based on the attention weight matrix, a feature vector of the red-headed file is calculated.

[0011] Specifically, the similarity calculation step includes: Obtain the parsing results of the red-headed file and the target file, and extract the feature vector of the file and ; use Jaccard Similarity algorithm calculation and The similarity between them is expressed as follows: , where is the feature vector , The intersection feature quantity between is the feature vector , The union feature between .

[0012] Specifically, the steps of the association simulation analysis include: Extract sentiment probability sequences of red-headed files and related files , calculate the emotional probability mean sequence for each time period ; Calculate the TF-IDF value of each word in the red header file, and select the top ranked words according to the size of the TF-IDF value. words as keywords and construct keyword sequences ; Calculate the keyword probability sequence for each time period ; Use linear regression to fit the sentiment probability mean sequence and the keyword probability sequence The trend direction of the product and generate a trend line chart; An analysis list is constructed, and relevant information of the trend line graph and associated files is saved in the analysis list.

[0013] Specifically, the parsing strategy includes an update logic for accessing common information files in the target website and updating the analysis list in real time; The update logic includes: Get the keyword sequence of the red-headed file And the corresponding TF-IDF value ; Construct the keyword weight matrix of the red-headed file according to the TF-IDF value ; The scoring model is constructed using the keywords and keyword weight matrix of the red-headed document. The expression of the correlation degree in the scoring model is as follows: , where Keywords in general information files The matching score is represented by binary 0 or 1; if the general information file contains keywords , ; If it does not contain keywords , ; Get the web page structure and URL rules of the target website, use the requests library to send HTTP requests to the target website and get the web page content, use the Beautiful Soup library to parse the web page content and get the general information file; The scoring model is used to calculate the correlation between the general information file and the red-headed file, and a correlation threshold is set. ;like , the general information file is not an associated file; if , the general information file is an associated file; The associated file information in the analysis list is updated, and the analysis list is updated in real time using the associated simulation analysis.

[0014] Specifically, the response module includes a response generating unit and a sending unit; The response generation unit is used to generate a response file according to the parsing result and the analysis list; The sending unit is used to queue up according to the response file, the needs of the recipient and the urgency of the response file, select the carbon copy mode and send the response file.

[0015] Specifically, the storage module includes a data storage unit and a backup unit; The data storage unit is used to receive the files generated by each module in real time and construct a file data table, wherein the file data table is divided into a plurality of data sub-tables according to the CC recipient, and the data sub-tables are named after the recipient; The backup unit is used to automatically back up the file data table according to a set schedule and save the file data table to a corresponding position in the schedule; a data recovery function is set to restore the file data table from the backup when the file is lost or damaged.

[0016] Beneficial effects of the present invention: Crawler technology is used to capture red-headed documents on the Internet and extract key information from information files, reducing the probability of human errors. Information files are classified to match the corresponding response recipients. The extracted key information is analyzed and the files are deeply parsed to discover the connections and development trends between files, providing support for departmental decision-making. Response files are generated based on the parsing results to achieve automatic parsing and rapid response of information files, reducing the time and cost of manual processing and improving the efficiency of government processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a structural diagram of an information file parsing response management system based on big data; Figure 2 A crawler strategy flow chart for an information file parsing response management system based on big data; Figure 3 It is a flow chart of information extraction strategy for an information file parsing response management system based on big data; Figure 4 This is a flow chart of parsing strategy for an information file parsing response management system based on big data. DETAILED DESCRIPTION

[0018] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict. Example

[0019] refer to Figures 1 to 4 As shown, this embodiment introduces an information file parsing response management system based on big data, including: a data acquisition module, a preprocessing module, a parsing module, a response module and a storage module.

[0020] The data collection module is used to collect information files from the electronic service website or the internal system of the department using crawler technology, and transmit the collected information files to the storage module in real time; The preprocessing module is used to perform preliminary processing on the collected original information files to facilitate the analysis and processing of subsequent modules; The parsing module is used to perform in-depth analysis on the pre-processed information files, extract key information of the files using machine learning, and mine potential related information to generate parsed files; The response module is used to automatically generate a response file based on the parsed file and control the preset response channel to send the response file to the relevant organization or personnel; The storage module is used to store the files generated by each module. It adopts a structured storage method to improve the retrieval speed; and uses data encryption and access control technology to enhance the security of information files.

[0021] Specifically, the data collection module is configured with a crawler strategy for obtaining red-headed files; The specific steps of the crawler strategy include: Obtain the web page structure and URL rules of the target website, which includes the internal systems of departments at all levels or department data open platforms; Use Chrome's developer tools to view the target website's webpage source code in a tab, search for characteristic keywords related to red-header files (such as "red-header files") in the source code, and obtain related HTML tags (such as div, table) and tag attributes (such as the specific class name "red-header-table"); Based on the web page structure of the target website, write Python crawler code, use the requests library to send HTTP requests to the target website, and obtain the web page content; use the Beautiful Soup library to parse the web page content, and extract the download link of the red-headed file based on the obtained HTML tags and tag attributes; Configure the crawler network to avoid being blocked by the target website; for example, configure the proxy IP to hide the real IP address and send the request through a different IP address; set the request header, such as User-Agent, to simulate the access behavior of normal users to avoid being identified as a crawler by the target website; set the request interval according to the load of the target website, for example, based on the historical file crawling volume, set the crawling peak period and crawling trough period, the request interval during the peak period is half an hour, and the request interval during the trough period is one hour, to avoid causing too much pressure on the target website server; Use Python's standard library logging to record the crawler's running logs, configure the logger, and record the crawler's running status, abnormal errors, and crawling results in the crawled file to monitor the crawler's running status and troubleshoot problems.

[0022] Specifically, the pre-processing module is configured with a classification strategy for preliminary processing of red-headed documents; The specific steps of the classification strategy include: Design a set of regular expressions to match different types of document numbers in red-headed documents and identify the document number part in the red-headed documents. The document number includes the organization or type represented by letters and the serial number or year represented by numbers, such as Su Zhengfa

[2016] No. 1; Preprocess the header file, such as reading the content of the header file, using the strip() and replace() methods of the string to remove the extra spaces in the header file, and replace the line break at the end of each line with an empty string to remove the line break; Use the designed regular expression to match the pre-processed red-headed document content and extract the document number information of the red-headed document; Establish a naming rule for official documents, which is document number + file name, to facilitate subsequent indexing of files based on document numbers; Build a file type library to identify the letter information in the red-headed document number; if there is no subfolder named in the format of the letter information of the document number in the file type library, build a corresponding subfolder in the file type library and save the red-headed file in the subfolder; if there is a subfolder named in the format of the letter information of the document number, save the red-headed file directly in the subfolder.

[0023] Specifically, a parsing strategy for parsing red-headed files is configured in the parsing module; The specific steps of the parsing strategy include: Build a semantic model and use it to parse the red-headed files in the file type library to generate parsing results. The semantic model includes an input layer, a preprocessing layer, a hidden layer, and an output layer. The preprocessing layer is used to perform word segmentation on the file and split the file into multiple text sequences. The BERT pre-training model is used to obtain the dynamic word vector representation of the text. The hidden layer is used to capture the long-term dependencies and semantic information in the sequence and analyze the sentiment level of file processing. Update the file type library and save the parsing results to the corresponding red-headed files; Calculate the similarity between the header file and the target file , the target file is other red-headed files in the file type library; Setting similarity threshold , determine whether the target file is an associated file of the red-headed file; if , the target file is not an associated file; if , the target file is the associated file of the red-headed file; Based on the associated files of the red-headed file, an associated simulation analysis is performed to generate the pointing trend of the red-headed file, and the pointing trend and the associated files are saved in the constructed analysis list.

[0024] Specifically, the steps to build a semantic model include: Collect a large number of historical official documents, clean the documents, and divide the documents into training sets and validation sets; Use the BERT word segmenter to convert the red-headed files in the verification set into The word sequence ;in, For word sequence Middle words, ; use Transformer The encoder structure converts the word sequence Words in Transformed into word vectors containing contextual information and syntactic and semantic features , and generate word sequence vectors ;in, is the word sequence vector Middle word vectors, ; Use RNN to capture the dependencies between contexts and extract word vectors The corresponding hidden vector , and generate hidden sequence vector ;in, is the hidden sequence vector Middle hidden vectors, , , is the hidden state encoded by the forward GRU, and , is the hidden state encoded by the backward GRU, ; In the word sequence vector, different words or characters have different effects on the sentiment of the document, such as sentiment words, degree adverbs, and negative words. The sentiment attention mechanism is used to identify key information. The sentiment sequence vector in the document is defined as ; To avoid the situation where the same emotional characters appear repeatedly in the file and the weight is too large to affect the model, the emotional vector is averaged. The expression is as follows: , where is the sentiment sequence vector The sentiment vector; Based on the average sentiment vector, Hyperbolic tangent function calculates hidden vector Sentiment score , the expression is as follows: , where is the weight vector obtained through training with the training set, is the weight parameter matrix of the hidden vector, is the weight parameter matrix of the average sentiment vector, is the bias vector; Based on the sentiment score, the hidden vector is calculated using the exponential function Attention weight in red header files , and construct the attention weight matrix , the expression of attention weight is as follows: , where is the hidden sequence vector The hidden vectors; Based on the attention weight matrix, calculate the feature vector of the red-headed file , the expression is as follows: , where is the attention weight matrix The Attention weights; use Softmax Function and calculation of sentiment probability of documents , to obtain the emotional result of the red-headed file, so as to facilitate the subsequent generation of the emotional level of the red-headed file. The expression is as follows: , where is the weight matrix, is the bias matrix; Define two levels of sentiment, positive and negative, and set the threshold of sentiment probability as ,in ;like , the sentiment level is negative; if , the emotional level is positive; Output the feature vector of the red header file and emotional level; The semantic model is verified using the verification set; if the verification fails, the training set is expanded and the model is continued to be trained; if the verification passes, the semantic model is successfully constructed.

[0025] Specifically, the similarity calculation steps include: Get the parsing results of the red-head file and the target file, and extract the feature vector of the file and ;in, is the feature vector of the red-head file, is the feature vector of the target file; use Jaccard Similarity algorithm calculation and The similarity between them is expressed as follows: , where is the feature vector , The intersection feature quantity between is the feature vector , The union feature between .

[0026] Specifically, the specific steps of the association simulation analysis include: Extract sentiment probability sequences of red-headed files and related files , calculate the mean sequence of sentiment probabilities for each time period (e.g., weekly) ;in, is the emotional probability of the red-headed document, For the The sentiment probability of the associated file, Indicates The mean of the emotion probability of the week; Calculate the TF-IDF value of each word in the red header file, and select the top ranked words based on the TF-IDF value. words as keywords and construct keyword sequences ; The expression of TF-IDF value is as follows: , where Words The number of times it appears in the file, is the total number of words in the document; Calculate the keyword probability sequence for each time period ;in, Words In the The probability of keyword occurrence in the week, , Words In the The probability of keyword occurrence in the week, For the Total number of words in weekly documents; Fitting the mean of sentiment probability using linear regression Trends and keyword occurrence probability , and generate a trend line chart; Build an analysis list and save the trend line chart and related information of the associated files to the analysis list.

[0027] Specifically, the parsing strategy includes an update logic, which is used to access common information files in the target website and update the analysis list in real time; The specific steps of updating logic include: Get the keyword sequence of the red-headed file And the corresponding TF-IDF value ; Assign weights to keywords in red-headed files based on TF-IDF values ​​and construct a keyword weight matrix ;in, ; The scoring model is constructed using the keywords of the red-headed file and the keyword weight matrix. The scoring model is used to determine whether the general information file is an associated file of the red-headed file. The expression of the correlation degree in the scoring model is as follows: , where is the keyword weight matrix The weights, Keywords in general information files The matching score is represented by binary 0 or 1; if the general information file contains keywords , ; If it does not contain keywords , ; Get the web page structure and URL rules of the target website, use the requests library to send HTTP requests to the target website and get the web page content, use the Beautiful Soup library to parse the web page content and get the general information file; Use the scoring model to calculate the correlation between ordinary information files and red-headed files, and set the correlation threshold ;like , ordinary information files are not associated files; if , ordinary information files are associated files of red-headed files; Update the associated file information in the analysis list, and use the associated simulation analysis to update the analysis list in real time.

[0028] Specifically, the response module includes a response generating unit and a sending unit.

[0029] The response generation unit is used to obtain the parsing results and analysis list of the parsing module, and automatically generate a response file, matching different urgency levels for the response file according to the emotional tendency in the parsed file.

[0030] The sending unit is used to select the copy method and queue according to the urgency based on the purpose of the response document, the needs of the recipient and the urgency of the response document; the response document is sent to the relevant institutions or personnel based on the queue order, and the copy methods include: email, text message, system internal message push or specific website. If the response document involves urgent time, the response document will be queued and copied immediately.

[0031] Specifically, the storage module includes a data storage unit and a backup unit.

[0032] The data storage unit is used to receive the files generated by each module in real time, and store information files, parsed files and response files; construct a file data table, which is divided into multiple data sub-tables according to the CC recipients. The data sub-table is named after the recipient. The structure of the data sub-table includes: file number, original file name, processing time, information file, parsed file and response file. The files are stored in the table in the form of hyperlinks; the storage module is configured with a security policy, which uses encryption technology to encrypt the file data table, sets an identity authentication mechanism to limit user access rights, and reduces the risk of unauthorized access; assigns specific access rights to specific internal staff, through Enter the work number and face recognition to limit access; when accessing the file data table, first enter the password for identity authentication. The password is set to the work number of the internal staff. The storage module compares the entered work number with the permission data table to determine whether the user has the permission to access the file data table. The permission data table is used to store the work numbers with access rights; if the work number does not exist in the permission data table, access is directly denied; if the work number exists in the permission data table, preliminary password verification is performed; for users who pass the preliminary password verification, the identity of the operator is confirmed through face recognition; if the recognition is unsuccessful, an alarm is immediately issued, indicating illegal access; if the recognition is successful, access to the file data table is allowed.

[0033] The backup unit is used to automatically back up the file data table according to the set schedule and save the file data table to the corresponding position in the schedule, where the schedule is based on fixed time intervals or specific event setting times, such as the time intervals are set to 1:00, 9:00 and 15:00 every day, and the specific events are before the system is shut down and after the system is started; a data recovery function is set to allow the required file data table to be quickly restored from the backup when the file is lost or damaged, and a specific time in the schedule can be selected to restore the file data table at a specific time to avoid file loss.

[0034] In summary, the present invention constructs an automatic parsing and response system for files, deeply parses files by constructing a semantic parsing model, and generates a response file based on the parsing result, thereby realizing automatic parsing and rapid response of information files and improving the efficiency of government affairs processing.

[0035] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. An information file parsing response management system based on big data, characterized in that: include: Data acquisition module, preprocessing module, analysis module, response module and storage module; The data acquisition module is used to collect information files using crawler technology and transmit the information files to the storage module in real time; The preprocessing module is used to perform preliminary processing on the collected original information files; The analysis module is used to perform in-depth analysis on the preprocessed information file, extract key information of the information file and mine potential related information to generate an analysis file; The parsing module is configured with a parsing strategy for parsing red-headed files; The parsing strategy includes: Constructing a semantic model, and using the semantic model to parse the red-headed files in the file type library to generate a parsing result; The parsing result is saved in the red-headed file of the file type library; Calculating the similarity between the red-headed file and a target file, where the target file is another red-headed file in the file type library; Set the similarity threshold to , determine whether the target file is an associated file of the red-headed file; if , the target file is not an associated file; if , the target file is an associated file; Based on the associated files of the red-headed file, a correlation simulation analysis is performed to generate a directional trend of the red-headed file, and the directional trend and the associated files are saved in a constructed analysis list; The response module is used to automatically generate a response file according to the parsed file, and send the response file to relevant institutions or personnel according to a preset response channel; The storage module is used to store files generated by each module, and adopts a structured storage method, as well as data encryption and access control technology.

2. The information file parsing response management system based on big data according to claim 1, characterized in that: The data acquisition module is configured with a crawler strategy for acquiring information files; The crawler strategy includes: Obtain the target website's infrastructure and URL rules; use the browser's developer tools to view the target website's web page source code and obtain the HTML tags and attributes of the target file; Based on the structure of the target website, write Python crawler code; Configure the crawler network to avoid being blocked by the target website; Record the crawler's running log to monitor the crawler's running status and troubleshoot problems.

3. The information file parsing response management system based on big data according to claim 2 is characterized by: The pre-processing module is configured with a classification strategy for preliminary processing of red-headed documents; The classification strategies include: Design a regular expression to match different types of document numbers in the red-headed document; Preprocessing the red-headed file; Using the regular expression to match the preprocessed red-headed file, extracting the document number information of the red-headed file; Rename the red-headed document in the format of document number + file name; Construct a file type library, and save the red-headed file into a subfolder in the file type library that uses partial information of the document number as a naming format.

4. The information file parsing response management system based on big data according to claim 1 is characterized in that: The steps of constructing the semantic model include: Collect historical official documents, clean the documents, and divide them into training set and verification set; Use the BERT tokenizer to convert the red-headed files in the validation set into word sequences ; use Transformer The encoder structure converts the word sequence into a word sequence vector ; Use RNN to extract the hidden sequence vector in the word sequence vector ; The emotion sequence vector in the definition file is , and calculate the average sentiment vector of the document ; After being processed by the sentiment attention mechanism, the feature vector of the file is obtained ; use Softmax Function to calculate the sentiment probability of a file p , the expression is as follows: , where is the weight matrix, is the bias matrix; Define two levels of sentiment, positive and negative, and set the threshold of sentiment probability as ,in ;like , the sentiment level is negative; if , the emotional level is positive; Output the feature vector of the red header file and emotional level; The semantic model is verified using the verification set; if the verification fails, the training set is expanded and the model is continuously trained; if the verification passes, the semantic model is successfully constructed.

5. The information file parsing response management system based on big data according to claim 4 is characterized in that: The processing steps of the emotional attention mechanism include: Based on the average sentiment vector, using Hyperbolic tangent function calculates hidden vector Sentiment score ,in, is the hidden sequence vector Middle hidden vectors, ; Based on the sentiment score, an exponential function is used to obtain the attention weight matrix of the hidden sequence vector ; Based on the attention weight matrix, a feature vector of the red-headed file is calculated.

6. The information file parsing response management system based on big data according to claim 5, characterized in that: The similarity calculation step includes: Obtain the parsing results of the red-headed file and the target file, and extract the feature vector of the file and ; use Jaccard Similarity algorithm calculation and The similarity between them is expressed as follows: , where is the feature vector , The intersection feature quantity between is the feature vector , The union feature between .

7. The information file parsing response management system based on big data according to claim 6 is characterized in that: The steps of the association simulation analysis include: Extract sentiment probability sequences of red-headed files and related files , calculate the emotional probability mean sequence for each time period ; Calculate the TF-IDF value of each word in the red header file, and select the top ranked words according to the size of the TF-IDF value. words as keywords and construct keyword sequences ; Calculate the keyword probability sequence for each time period ; Use linear regression to fit the emotional probability mean sequence and the keyword probability sequence The trend direction of the product and generate a trend line chart; An analysis list is constructed, and the trend line chart and related information of the associated files are saved in the analysis class table.

8. The information file parsing response management system based on big data according to claim 7, characterized in that: The parsing strategy includes an update logic for accessing common information files in the target website and updating the analysis list in real time; The update logic includes: Get the keyword sequence of the red-headed file And the corresponding TF-IDF value ; Construct the keyword weight matrix of the red-headed file according to the TF-IDF value ; The scoring model is constructed using the keywords and keyword weight matrix of the red-headed document. The expression of the correlation degree in the scoring model is as follows: , where Keywords in general information files The matching score is represented by binary 0 or 1; if the general information file contains keywords , ; If it does not contain keywords , ; Get the web page structure and URL rules of the target website, use the requests library to send HTTP requests to the target website and get the web page content, use the Beautiful Soup library to parse the web page content and get the general information file; The scoring model is used to calculate the correlation between the general information file and the red-headed file, and a correlation threshold is set. ;like , the general information file is not an associated file; if , the general information file is an associated file; The associated file information in the analysis list is updated, and the analysis list is updated in real time using the associated simulation analysis.

9. The information file parsing response management system based on big data according to claim 8, characterized in that: The response module includes a response generating unit and a sending unit; The response generation unit is used to generate a response file according to the parsing result and the analysis list; The sending unit is used to queue up according to the response file, the needs of the recipient and the urgency of the response file, select the carbon copy mode and send the response file.

10. The information file parsing response management system based on big data according to claim 9, characterized in that: The storage module includes a data storage unit and a backup unit; The data storage unit is used to receive the files generated by each module in real time and construct a file data table, wherein the file data table is divided into a plurality of data sub-tables according to the CC recipient, and the data sub-tables are named after the recipient; The backup unit is used to automatically back up the file data table according to a set schedule, and save the file data table to a corresponding position in the schedule; Set up the data recovery function so that the file data table can be restored from the backup when the file is lost or damaged.

Images