Non-field data intelligent identification and disposal method and system

Abstract

The invention relates to the technical field of off-site data processing, and particularly discloses an off-site data intelligent identification and disposal method and system.The method comprises the steps that an access record is converted into a document set, and a document-topic matrix and a topic-word matrix are extracted on the basis of the document set; words with element values larger than a preset threshold value are selected from the theme-word matrix and fed back to the evaluation end, and theme names are determined; and feature topics are extracted from the document-topic matrix based on the topic names, the feature topics of the same account at different times are compared, and the identity anomaly degree is judged. According to the method, the access record of the important account is acquired in real time, the access record is converted into the word matrix, subject extraction is performed on the word matrix, the subject of the same account in the current use process is determined, the subject is compared with the subject in the historical stage, and therefore whether the phenomenon that the account is used instead of a person occurs or not is judged, and then management and control measures are adopted. And the safety is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of off-site data processing, and in particular to an off-site data intelligent identification and processing method and system. Background Art

[0002] For an organization or enterprise, especially an organization or enterprise with its own core technology, technical confidentiality is of paramount importance. However, in daily work, these confidentiality technologies need to be opened to certain accounts. The larger the scale, the more such accounts there are. The usage records of these accounts belong to off-site data indirectly obtained through computer equipment. The existing identification process for these off-site data is mostly found after use, that is, the management personnel regularly review the usage of the account and make abnormal judgments manually, which is not real-time. How to provide a real-time off-site data identification method to quickly locate account abnormalities and improve the security of confidentiality technology is a technical problem that the technical solution of the present invention wants to solve. Summary of the invention

[0003] The purpose of the present invention is to provide a method and system for intelligently identifying and processing off-site data to solve the problems raised in the above-mentioned background technology.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] A method for intelligently identifying and handling off-site data, the method comprising:

[0006] Receive the internal account table regularly updated by the management party, broadcast a backup request based on the internal account table to each user, and receive the backup permission fed back by the user; the internal account is an account provided by the management party and applied to a preset platform;

[0007] After receiving the backup permission from the user, the detection duration is determined according to the IP address of the account, and the access records backed up within the detection duration before the current moment are obtained with the current moment as the endpoint;

[0008] Convert the access records into a document set, and extract a document-topic matrix and a topic-word matrix based on the document set;

[0009] Select words with element values ​​greater than a preset threshold in the topic-word matrix, feed them back to the evaluation end, and determine the topic name;

[0010] Feature topics are extracted from the document-topic matrix based on topic names, and the feature topics of the same account at different times are compared to determine the degree of identity abnormality.

[0011] As a further solution of the present invention: after receiving the backup permission from the user, the detection duration is determined according to the IP address of the account, and the step of obtaining the access records backed up within the detection duration before the current moment with the current moment as the endpoint includes:

[0012] After receiving the backup permission from the user, the IP address of the user each time when logging into the account is obtained, and the address histogram is updated; the horizontal axis of the address histogram is used to represent the IP address, and the vertical axis represents the total number of logins corresponding to each IP address;

[0013] Querying the total number of login times of the current IP address in the address histogram, and determining the detection time according to the total number of login times; the detection time is proportional to the total number of login times;

[0014] With the current time as the endpoint, a connection channel with the backup database is established based on the backup permission to obtain access records backed up within the detection period before the current time.

[0015] As a further solution of the present invention: the step of converting the access record into a document set and extracting a document-topic matrix and a topic-word matrix based on the document set includes:

[0016] The access records are converted into a document set in chronological order; the document number of each document in the document set corresponds to the time one by one;

[0017] Perform word recognition on each document to obtain a word set for each document; wherein, when a word is recognized, determine whether the word belongs to a preset stop word set, and if the word belongs to the preset stop word set, remove the word;

[0018] Count the word sets of all documents to obtain the word space, calculate the TF-IDF value of each word in each word set, and build a document-word matrix;

[0019] Receive the number of topics input by the management party, and perform matrix decomposition on the document-word matrix based on the number of topics to obtain a document-topic matrix and a topic-word matrix;

[0020] Among them, the rows in the document-word matrix correspond to documents, the number of rows is the same as the total number of documents, the columns correspond to words, and the number of columns is the same as the total number of words; the rows in the document-topic matrix correspond to documents, the number of rows is the same as the total number of documents, the columns correspond to topics, and the number of columns is the same as the number of topics; the rows in the topic-word matrix correspond to topics, the number of rows is the same as the number of topics, the columns correspond to words, and the number of columns is the same as the total number of words.

[0021] As a further solution of the present invention: the calculation process of the TF-IDF value is:

[0022] TF-IDF(t,d)=TF(t,d)×IDF(t);

[0023]

[0024] Where TF-IDF(t,d) is the TF-IDF value of word t in document d, TF(t,d) is the word frequency of word t in document d, N(t) is the number of occurrences of word t, N(d) is the total number of words in document d; IDF(t) is the inverse document frequency of word t, w is the total number of documents, and w(t) is the number of documents containing word t;

[0025] The process of matrix decomposition of the document-term matrix is:

[0026] Let the document-word matrix be V, and the size of V is m×n;

[0027] Split V into W and H, where W represents the document-topic matrix with a size of m×k, and H represents the topic word matrix with a size of k×n;

[0028] The objective function in the decomposition process is:

[0029] Among them, ||x|| F is the Frobenius norm of x, which means the square root of the sum of the squares of the matrix elements;

[0030] The optimization process is:

[0031]

[0032] Among them, α and β are the rows and columns of the matrix H, and γ and δ are the rows and columns of the matrix W.

[0033] As a further solution of the present invention: the step of selecting words whose element values ​​are greater than a preset threshold in the topic-word matrix and feeding them back to the evaluation end to determine the topic name includes:

[0034] Read the topic-word matrix, traverse the element values ​​in the matrix, and compare them with the preset threshold;

[0035] When the element value is greater than a preset threshold, the word corresponding to the element is read as the characteristic word of the topic corresponding to the element;

[0036] Count the feature words, feed them back to the evaluation end, and receive the topic name sent by the evaluation end.

[0037] As a further solution of the present invention: the step of extracting characteristic topics in the document-topic matrix based on the topic name, comparing the characteristic topics of the same account at different times, and determining the degree of identity abnormality includes:

[0038] Insert topic names into the document-topic matrix;

[0039] Traverse each element in the document-topic matrix, and when the element value of an element meets the preset conditions, use the topic corresponding to the element as the characteristic topic of the document corresponding to the element;

[0040] Acquire characteristic topics of the same account at different times, and update a topic histogram; the horizontal axis of the topic histogram is used to represent the characteristic topics, and the vertical axis represents the frequency corresponding to each characteristic topic;

[0041] The frequency of the current feature topic is queried in the topic histogram, and the identity abnormality is determined according to the frequency; the identity abnormality is inversely proportional to the frequency.

[0042] The technical solution of the present invention also provides an off-site data intelligent identification and processing system, the system comprising:

[0043] A permission receiving module is used to receive the internal account table regularly updated by the management party, broadcast a backup request based on the internal account table to each user, and receive the backup permission fed back by the user; the internal account is an account provided by the management party and applied to a preset platform;

[0044] The record query module is used to determine the detection time according to the IP address of the account after receiving the backup permission fed back by the user, and obtain the access records backed up within the detection time before the current time with the current time as the endpoint;

[0045] A document set identification module, used for converting access records into document sets, and extracting document-topic matrix and topic-word matrix based on the document sets;

[0046] The topic name determination module is used to select words with element values ​​greater than a preset threshold in the topic-word matrix, feed them back to the evaluation end, and determine the topic name;

[0047] The abnormality determination module is used to extract characteristic topics in the document-topic matrix based on the topic name, compare the characteristic topics of the same account at different times, and determine the identity abnormality.

[0048] As a further solution of the present invention: the record query module includes:

[0049] The address recording unit is used to obtain the IP address of the user each time the user logs into the account after receiving the backup permission fed back by the user, and update the address histogram; the horizontal axis of the address histogram is used to represent the IP address, and the vertical axis represents the total number of login times corresponding to each IP address;

[0050] A duration determination unit, used to query the total number of login times of the current IP address in the address histogram, and determine the detection duration according to the total number of login times; the detection duration is proportional to the total number of login times;

[0051] The query execution unit is used to establish a connection channel with the backup database based on the backup permission with the current time as the endpoint, and obtain the access records of the backup within the detection time before the current time.

[0052] As a further solution of the present invention: the document set identification module includes:

[0053] A record conversion unit, used to convert the access record into a document set in chronological order; the document number of each document in the document set corresponds to the time one by one;

[0054] A word recognition unit is used to perform word recognition on each document to obtain a word set of each document; wherein, when a word is recognized, it is determined whether the word belongs to a preset stop word set, and when the word belongs to the preset stop word set, the word is removed;

[0055] The matrix construction unit is used to count the word sets of all documents, obtain the word space, calculate the TF-IDF value of each word in each word set, and construct the document-word matrix;

[0056] A matrix decomposition unit, used to receive the number of topics input by the management party, and perform matrix decomposition on the document-word matrix based on the number of topics to obtain a document-topic matrix and a topic-word matrix;

[0057] Among them, the rows in the document-word matrix correspond to documents, the number of rows is the same as the total number of documents, the columns correspond to words, and the number of columns is the same as the total number of words; the rows in the document-topic matrix correspond to documents, the number of rows is the same as the total number of documents, the columns correspond to topics, and the number of columns is the same as the number of topics; the rows in the topic-word matrix correspond to topics, the number of rows is the same as the number of topics, the columns correspond to words, and the number of columns is the same as the total number of words.

[0058] As a further solution of the present invention: the subject name determination module includes:

[0059] A comparison unit, used to read the topic-word matrix, traverse the element values ​​in the matrix, and compare them with a preset threshold;

[0060] A feature word generating unit, used for reading a word corresponding to the element as a feature word of the subject corresponding to the element when the element value is greater than a preset threshold;

[0061] The feedback statistics unit is used to count the feature words, feed back to the evaluation end, and receive the topic name sent by the evaluation end.

[0062] Compared with the prior art, the beneficial effects of the present invention are: the present invention obtains access records of important accounts in real time, converts the access records into a word matrix, extracts topics from the word matrix, determines the topic of the same account in the current use process, and compares it with the topic of the historical stage, so as to determine whether the account has been used by a different person, and then adopts control measures to greatly improve security. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0064] Figure 1 This is a flowchart of the method for intelligent identification and processing of off-site data.

[0065] Figure 2 This is the first sub-process flowchart of the off-site data intelligent identification and processing method.

[0066] Figure 3 This is the second sub-process flowchart of the off-site data intelligent identification and processing method.

[0067] Figure 4 This is the third sub-process flowchart of the off-site data intelligent identification and processing method.

[0068] Figure 5 This is the fourth sub-process flowchart of the off-site data intelligent identification and processing method.

[0069] Figure 6 This is a structural block diagram of the off-site data intelligent identification and processing system. DETAILED DESCRIPTION

[0070] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0071] Figure 1 The flowchart of the method for intelligently identifying and handling off-site data is as follows. In an embodiment of the present invention, a method for intelligently identifying and handling off-site data includes:

[0072] Step S100: receiving the internal account table regularly updated by the management party, broadcasting a backup request based on the internal account table to each user, and receiving the backup permission fed back by the user; the internal account is an account provided by the management party and applied to a preset platform;

[0073] The administrator creates internal account tables regularly. Internal account tables are accounts that need to be supervised. This must be clearly informed to users. When using such accounts, users must be aware that their usage records will be supervised. In actual applications, such accounts generally have extremely high permissions, and supervision is also a necessary step. The purpose of this application is to ensure the security of such accounts. For example, suppose there is an enterprise whose many confidential documents are only open to some internal accounts. These internal accounts are counted by an internal account table and then made public. At the same time, it is necessary to indicate that when using such accounts, the usage records will be retained by the administrator. When the staff needs to use it, since they know that the usage records will be retained by the administrator, when they use it, they are granted the recording permission by default, that is, the backup permission.

[0074] Step S200: after receiving the backup permission from the user, determine the detection time according to the IP address of the account, take the current time as the endpoint, and obtain the access records backed up within the detection time before the current time;

[0075] After receiving the backup permission from the user, the execution subject of this method obtains the usage record of the account in a limited way and obtains the IP address of the account, that is, the location at which the user uses the account, and determines the detection time according to the frequency of the location. If the location is a common location, the detection time will be longer, and the access records of a longer time period will be obtained. A longer time period means that even if there are some abnormal behaviors, they are allowed over the entire longer time period, that is, a longer time period means that there is a larger space for abnormalities. On the contrary, if the location is an uncommon location, at this time, it is necessary to reduce the detection time, identify the access records within a short period of time, and then judge the degree of abnormality.

[0076] It should be noted that the key point of the above solution is that the shorter the detection time, the easier it is to detect an abnormality.

[0077] Step S300: converting the access record into a document set, and extracting a document-topic matrix and a topic-word matrix based on the document set;

[0078] The access record itself is a collection of documents, such as a PDF file. The process of converting the access record into a document set is not complicated. At most, some text recognition algorithms are introduced, that is, a solution for recognizing text in a PDF file. The recognition process belongs to the existing technology.

[0079] The document set is identified and converted into a document-term matrix, and then a matrix decomposition scheme is applied to convert it into a main document-topic matrix and a topic-term matrix; each row in the main document-topic matrix represents the degree of association between a document and each topic, and each row in the topic-term matrix corresponds to a topic, and each column corresponds to a word, and its value represents the weight of the word on a specific topic.

[0080] Step S400: Select words whose element values ​​are greater than a preset threshold in the topic-word matrix, feed them back to the evaluation end, and determine the topic name;

[0081] During the matrix decomposition process, the document-word matrix can only be decomposed into the main document-topic matrix and the topic-word matrix with a preset number of topics. As for the topic name, a determination process needs to be introduced. This is determined by words. The topic name is determined based on the words with larger weights. The determination process is completed by the evaluation end. The evaluation end can use other bag-of-words models or manual evaluation. There is no specific limitation. The execution subject of this method only needs to establish a connection channel with the evaluation end and be able to transmit data. What is sent to the evaluation end is vocabulary, and what is received is the topic name fed back by the evaluation end.

[0082] Step S500: extracting characteristic topics in the document-topic matrix based on the topic name, comparing the characteristic topics of the same account at different times, and determining the degree of identity abnormality;

[0083] Select elements with larger weights in the document-topic matrix. On the basis that the topic names have been determined, the topic names corresponding to the main elements (elements with larger weights) in each document can be determined. At this time, these topic names are used as the characteristic topics of the documents. Compare the characteristic topics of the same account at different times, that is, the characteristic topics in the historical detection process, and determine the degree of identity abnormality based on whether they appear frequently or not.

[0084] Figure 2 The first sub-flow chart of the method for intelligently identifying and handling off-site data is as follows. After receiving the backup permission from the user, the detection time is determined according to the IP address of the account, and the access records backed up within the detection time before the current time are obtained with the current time as the endpoint. The steps include:

[0085] Step S201: after receiving the backup permission from the user, obtain the IP address of the user each time he logs into the account, and update the address histogram; the horizontal axis of the address histogram is used to represent the IP address, and the vertical axis represents the total number of logins corresponding to each IP address;

[0086] Step S202: querying the total number of login times of the current IP address in the address histogram, and determining the detection time according to the total number of login times; the detection time is proportional to the total number of login times;

[0087] Step S203: Taking the current time as the endpoint, a connection channel with the backup database is established based on the backup permission, and access records backed up within the detection period before the current time are obtained.

[0088] After receiving the backup permission from the user, obtain the IP address of each user when logging into the account, and make statistics in the form of a histogram, where the horizontal axis of each column represents an address and the vertical axis of each column represents the total number of logins; obtain the current IP address, query the total number of logins of the current IP address in the address histogram, and calculate the ratio of the total number of logins to all logins. The larger the ratio, the more common the current IP address is, and the longer the detection time is. On the contrary, the smaller the ratio, the rarer the current IP address is, and the shorter the detection time is; finally, with the current time as the endpoint, establish a connection channel with the backup database based on the backup permission, and obtain the access records backed up within the detection time before the current time.

[0089] It should be noted that the backup database is used to store access records of all accounts at all times.

[0090] Figure 3 The second sub-flow diagram of the method for intelligent identification and disposal of off-site data is as follows: the step of converting the access record into a document set and extracting a document-topic matrix and a topic-word matrix based on the document set includes:

[0091] Step S301: converting the access records into a document set in chronological order; the document number of each document in the document set corresponds to the time one by one;

[0092] Step S302: performing word recognition on each document to obtain a word set of each document; wherein, when a word is recognized, determining whether the word belongs to a preset stop word set, and if the word belongs to the preset stop word set, removing the word;

[0093] Step S303: Count the word sets of all documents to obtain a word space, calculate the TF-IDF value of each word in each word set, and construct a document-word matrix;

[0094] Step S304: receiving the number of topics input by the management party, and performing matrix decomposition on the document-word matrix based on the number of topics to obtain a document-topic matrix and a topic-word matrix;

[0095] Among them, the rows in the document-word matrix correspond to documents, the number of rows is the same as the total number of documents, the columns correspond to words, and the number of columns is the same as the total number of words; the rows in the document-topic matrix correspond to documents, the number of rows is the same as the total number of documents, the columns correspond to topics, and the number of columns is the same as the number of topics; the rows in the topic-word matrix correspond to topics, the number of rows is the same as the number of topics, the columns correspond to words, and the number of columns is the same as the total number of words.

[0096] In an example of the technical solution of the present invention, a certain document identification scheme is provided. First, the access records are converted into document sets in chronological order. During the conversion process, the document numbers are recorded, and the document numbers correspond to the time one by one. Then, word recognition is performed on each document, and a word set is obtained after the stop words are eliminated. Each document corresponds to a word set. The word sets of all documents are counted to obtain a word space, and the word space indicates which words are included in all documents. The TF-IDF value of each word in each word set is calculated, and the TF-IDF value is used as the matrix element value to construct a matrix to obtain a document-word matrix. Finally, the number of topics input by the management party is received, and the document-word matrix is ​​matrix decomposed based on the number of topics to obtain a document-topic matrix and a topic-word matrix.

[0097] In the process of matrix decomposition, assuming that the document-word matrix is ​​of size m×n and the number of topics is k, the split document-topic matrix is ​​of size m×k, and the split topic-word matrix is ​​of size k×n.

[0098] Specifically, the calculation process of TF-IDF value is:

[0099] TF-IDF(t,d)=TF(t,d)×IDF(t);

[0100]

[0101] Where TF-IDF(t,d) is the TF-IDF value of word t in document d, TF(t,d) is the word frequency of word t in document d, N(t) is the number of occurrences of word t, N(d) is the total number of words in document d; IDF(t) is the inverse document frequency of word t, w is the total number of documents, and w(t) is the number of documents containing word t.

[0102] A high TF-IDF value indicates that the word appears frequently in the current document but rarely in other documents, indicating that the word is of high importance to the current document. A low TF-IDF value indicates that the word appears frequently in the current document or other documents, or that the word rarely appears in the current document, indicating that the word contributes little to distinguishing the topic of the current document.

[0103] The process of matrix decomposition of the document-term matrix is:

[0104] Let the document-word matrix be V, and the size of V is m×n;

[0105] Split V into W and H, where W represents the document-topic matrix with a size of m×k, and H represents the topic word matrix with a size of k×n;

[0106] The objective function in the decomposition process is:

[0107] Among them, ||x|| F is the Frobenius norm of x, which means the square root of the sum of the squares of the matrix elements;

[0108] The optimization process is:

[0109]

[0110] Among them, α and β are the rows and columns of the matrix H, and γ and δ are the rows and columns of the matrix W.

[0111] The optimization process is a continuous iterative process, which continues until the convergence conditions are met. There are three commonly used convergence conditions: first, the update amount of W and H is lower than a certain threshold; second, the preset number of iterations is reached; third, the change of the objective function is less than the preset threshold; generally, the third convergence condition is adopted. Therefore, the above content limits the objective function.

[0112] Figure 4 The third sub-flow diagram of the method for intelligent identification and disposal of off-site data is shown in FIG. 1 , wherein the step of selecting words whose element values ​​are greater than a preset threshold in the subject-word matrix and feeding them back to the evaluation end to determine the subject name includes:

[0113] Step S401: read the topic-word matrix, traverse the element values ​​in the matrix, and compare them with the preset threshold;

[0114] Step S402: when the element value is greater than a preset threshold, the word corresponding to the element is read as a characteristic word of the topic corresponding to the element;

[0115] Step S403: Count the characteristic words, feed back to the evaluation end, and receive the topic name sent by the evaluation end.

[0116] In one example of the technical solution of the present invention, a topic-word matrix is ​​read, the element values ​​in the matrix are traversed, and compared with a preset threshold value. When the element value is greater than the preset threshold value, the word corresponding to the element (the word corresponding to the column) is read as the feature word of the topic corresponding to the element (the topic corresponding to the row); the feature words are counted and fed back to the evaluation end, and the topic name sent by the evaluation end is received.

[0117] This application does not limit the evaluation process on the evaluation side. Because the correspondence between words and topics can be evaluated using an existing bag-of-words model, or by professional evaluators, these are all feasible existing solutions. For this application, only data transmission is required.

[0118] Figure 5 This is a fourth sub-flow diagram of the method for intelligent identification and disposal of off-site data. The steps of extracting characteristic topics in the document-topic matrix based on the topic name, comparing the characteristic topics of the same account at different times, and determining the degree of identity abnormality include:

[0119] Step S501: insert the topic name into the document-topic matrix;

[0120] Step S502: traverse each element in the document-topic matrix, and when the element value of a certain element meets a preset condition, take the topic corresponding to the element as the characteristic topic of the document corresponding to the element;

[0121] Step S503: Acquire characteristic topics of the same account at different times, and update a topic histogram; the horizontal axis of the topic histogram is used to represent characteristic topics, and the vertical axis represents the frequency corresponding to each characteristic topic;

[0122] Step S504: query the frequency of the current feature topic in the topic histogram, and determine the identity abnormality according to the frequency; the identity abnormality is inversely proportional to the frequency.

[0123] After the topic name is determined, we can find out what topic each column in the document-topic matrix corresponds to. We traverse each element in the document-topic matrix, and when the element value of an element meets the preset conditions, the topic (column) corresponding to the element is used as the characteristic topic of the document (row) corresponding to the element.

[0124] Get the characteristic topics of the same account at different times, update the topic histogram, and query the frequency of the current characteristic topic in the topic histogram. The higher the frequency, the more regular it is and the smaller the identity abnormality. On the contrary, the smaller the frequency, the more unconventional it is and the greater the identity abnormality. At this time, some remote permission adjustment processes are required, such as locking the account or sending other verification requests to the account.

[0125] It is worth mentioning that step S501 to step S502 are similar to step S401 to step S402, and step S503 to step S504 are similar to step 201 to step S202.

[0126] In addition, after extracting the characteristic themes of the accounts, the present invention can also introduce a horizontal comparison process between accounts to determine whether there are differences between a certain account and other accounts at the current stage. If there are differences, it can also be regarded as a high degree of identity abnormality; in other words, obtaining the characteristic themes of the same account at different times belongs to the vertical comparison process.

[0127] Figure 6 1 is a structural block diagram of an off-site data intelligent identification and processing system. In an embodiment of the present invention, an off-site data intelligent identification and processing system, the system 10 includes:

[0128] The permission receiving module 11 is used to receive the internal account table regularly updated by the management party, broadcast a backup request based on the internal account table to each user, and receive the backup permission fed back by the user; the internal account is an account provided by the management party and applied to a preset platform;

[0129] The record query module 12 is used to determine the detection time according to the IP address of the account after receiving the backup permission fed back by the user, and obtain the access records backed up within the detection time before the current moment with the current moment as the endpoint;

[0130] A document set identification module 13, used to convert the access record into a document set, and extract a document-topic matrix and a topic-word matrix based on the document set;

[0131] The topic name determination module 14 is used to select words with element values ​​greater than a preset threshold in the topic-word matrix, feed them back to the evaluation end, and determine the topic name;

[0132] The abnormality determination module 15 is used to extract characteristic topics in the document-topic matrix based on the topic name, compare the characteristic topics of the same account at different times, and determine the abnormality of the identity.

[0133] Furthermore, the record query module 12 includes:

[0134] The address recording unit is used to obtain the IP address of the user each time the user logs into the account after receiving the backup permission fed back by the user, and update the address histogram; the horizontal axis of the address histogram is used to represent the IP address, and the vertical axis represents the total number of login times corresponding to each IP address;

[0135] A duration determination unit, used to query the total number of login times of the current IP address in the address histogram, and determine the detection duration according to the total number of login times; the detection duration is proportional to the total number of login times;

[0136] The query execution unit is used to establish a connection channel with the backup database based on the backup permission with the current time as the endpoint, and obtain the access records of the backup within the detection time before the current time.

[0137] Specifically, the document set identification module 13 includes:

[0138] A record conversion unit, used to convert the access record into a document set in chronological order; the document number of each document in the document set corresponds to the time one by one;

[0139] A word recognition unit is used to perform word recognition on each document to obtain a word set of each document; wherein, when a word is recognized, it is determined whether the word belongs to a preset stop word set, and when the word belongs to the preset stop word set, the word is removed;

[0140] The matrix construction unit is used to count the word sets of all documents, obtain the word space, calculate the TF-IDF value of each word in each word set, and construct the document-word matrix;

[0141] A matrix decomposition unit, used to receive the number of topics input by the management party, and perform matrix decomposition on the document-word matrix based on the number of topics to obtain a document-topic matrix and a topic-word matrix;

[0142] Among them, the rows in the document-word matrix correspond to documents, the number of rows is the same as the total number of documents, the columns correspond to words, and the number of columns is the same as the total number of words; the rows in the document-topic matrix correspond to documents, the number of rows is the same as the total number of documents, the columns correspond to topics, and the number of columns is the same as the number of topics; the rows in the topic-word matrix correspond to topics, the number of rows is the same as the number of topics, the columns correspond to words, and the number of columns is the same as the total number of words.

[0143] Furthermore, the subject name determination module 14 includes:

[0144] A comparison unit, used to read the topic-word matrix, traverse the element values ​​in the matrix, and compare them with a preset threshold;

[0145] A feature word generating unit, used for reading a word corresponding to the element as a feature word of the subject corresponding to the element when the element value is greater than a preset threshold;

[0146] The feedback statistics unit is used to count the feature words, feed back to the evaluation end, and receive the topic name sent by the evaluation end.

[0147] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for intelligent identification and processing of off-site data, characterized in that: The method comprises: Receive the internal account table updated regularly by the management party, broadcast a backup request based on the internal account table to each user, and receive the backup permission fed back by the user; the internal account is an account provided by the management party and applied to a preset platform; After receiving the backup permission from the user, the detection duration is determined according to the IP address of the account, and the access records backed up within the detection duration before the current moment are obtained with the current moment as the endpoint; Convert the access records into a document set, and extract a document-topic matrix and a topic-word matrix based on the document set; Select words with element values ​​greater than a preset threshold in the topic-word matrix, feed them back to the evaluation end, and determine the topic name; Feature topics are extracted from the document-topic matrix based on topic names, and the feature topics of the same account at different times are compared to determine the degree of identity abnormality.

2. The off-site data intelligent identification and processing method according to claim 1 is characterized in that: After receiving the backup permission from the user, the step of determining the detection time according to the IP address of the account, taking the current time as the endpoint, and obtaining the access records backed up within the detection time before the current time includes: After receiving the backup permission from the user, the IP address of the user each time when logging into the account is obtained, and the address histogram is updated; the horizontal axis of the address histogram is used to represent the IP address, and the vertical axis represents the total number of logins corresponding to each IP address; Querying the total number of login times of the current IP address in the address histogram, and determining the detection time according to the total number of login times; the detection time is proportional to the total number of login times; With the current time as the endpoint, a connection channel with the backup database is established based on the backup permission to obtain access records backed up within the detection period before the current time.

3. The off-site data intelligent identification and processing method according to claim 1 is characterized in that: The step of converting the access records into a document set and extracting a document-topic matrix and a topic-word matrix based on the document set comprises: The access records are converted into a document set in chronological order; the document number of each document in the document set corresponds to the time one by one; Perform word recognition on each document to obtain a word set for each document; wherein, when a word is recognized, determine whether the word belongs to a preset stop word set, and if the word belongs to the preset stop word set, remove the word; Count the word sets of all documents to obtain the word space, calculate the TF-IDF value of each word in each word set, and build a document-word matrix; Receive the number of topics input by the management party, and perform matrix decomposition on the document-word matrix based on the number of topics to obtain a document-topic matrix and a topic-word matrix; Among them, the rows in the document-word matrix correspond to documents, the number of rows is the same as the total number of documents, the columns correspond to words, and the number of columns is the same as the total number of words; the rows in the document-topic matrix correspond to documents, the number of rows is the same as the total number of documents, the columns correspond to topics, and the number of columns is the same as the number of topics; the rows in the topic-word matrix correspond to topics, the number of rows is the same as the number of topics, the columns correspond to words, and the number of columns is the same as the total number of words.

4. The off-site data intelligent identification and processing method according to claim 1 is characterized in that: The calculation process of TF-IDF value is: TF-IDF(t,d)=TF(t,d)×IDF(t); Where TF-IDF(t, d) is the TF-IDF value of word t in document d, TF(t, d) is the word frequency of word t in document d, N(t) is the number of occurrences of word t, N(d) is the total number of words in document d; IDF(t) is the inverse document frequency of word t, w is the total number of documents, and w(t) is the number of documents containing word t; The process of matrix decomposition of the document-term matrix is: Let the document-word matrix be V, and the size of V is m×n; Split V into W and H, where W represents the document-topic matrix with a size of m×k, and H represents the topic word matrix with a size of k×n; The objective function in the decomposition process is: Among them, ||x|| F is the Frobenius norm of x, which means the square root of the sum of the squares of the matrix elements; The optimization process is: Among them, α and β are the rows and columns of the matrix H, and γ and δ are the rows and columns of the matrix W.

5. The off-site data intelligent identification and processing method according to claim 1 is characterized in that: The step of selecting words whose element values ​​are greater than a preset threshold in the topic-word matrix and feeding them back to the evaluation end to determine the topic name includes: Read the topic-word matrix, traverse the element values ​​in the matrix, and compare them with the preset threshold; When the element value is greater than a preset threshold, the word corresponding to the element is read as the characteristic word of the topic corresponding to the element; Count the feature words, feed them back to the evaluation end, and receive the topic name sent by the evaluation end.

6. The off-site data intelligent identification and processing method according to claim 1 is characterized in that: The step of extracting characteristic topics in the document-topic matrix based on the topic name, comparing the characteristic topics of the same account at different times, and determining the degree of identity abnormality includes: Insert topic names into the document-topic matrix; Traverse each element in the document-topic matrix, and when the element value of an element meets the preset conditions, use the topic corresponding to the element as the characteristic topic of the document corresponding to the element; Acquire characteristic topics of the same account at different times, and update a topic histogram; the horizontal axis of the topic histogram is used to represent the characteristic topics, and the vertical axis represents the frequency corresponding to each characteristic topic; The frequency of the current feature topic is queried in the topic histogram, and the identity abnormality is determined according to the frequency; the identity abnormality is inversely proportional to the frequency.

7. An off-site data intelligent identification and processing system, characterized in that: The system comprises: A permission receiving module is used to receive the internal account table regularly updated by the management party, broadcast a backup request based on the internal account table to each user, and receive the backup permission fed back by the user; the internal account is an account provided by the management party and applied to a preset platform; The record query module is used to determine the detection time according to the IP address of the account after receiving the backup permission fed back by the user, and obtain the access records backed up within the detection time before the current time with the current time as the endpoint; A document set identification module, used for converting access records into document sets, and extracting document-topic matrix and topic-word matrix based on the document sets; The topic name determination module is used to select words with element values ​​greater than a preset threshold in the topic-word matrix, feed them back to the evaluation end, and determine the topic name; The abnormality determination module is used to extract characteristic topics in the document-topic matrix based on the topic name, compare the characteristic topics of the same account at different times, and determine the identity abnormality.

8. The off-site data intelligent identification and processing system according to claim 7 is characterized in that: The record query module includes: The address recording unit is used to obtain the IP address of the user each time the user logs into the account after receiving the backup permission fed back by the user, and update the address histogram; the horizontal axis of the address histogram is used to represent the IP address, and the vertical axis represents the total number of login times corresponding to each IP address; A duration determination unit, used to query the total number of login times of the current IP address in the address histogram, and determine the detection duration according to the total number of login times; the detection duration is proportional to the total number of login times; The query execution unit is used to establish a connection channel with the backup database based on the backup permission with the current time as the endpoint, and obtain the access records of the backup within the detection time before the current time.

9. The off-site data intelligent identification and processing system according to claim 7, characterized in that: The document set identification module includes: A record conversion unit, used to convert the access record into a document set in chronological order; the document number of each document in the document set corresponds to the time one by one; A word recognition unit is used to perform word recognition on each document to obtain a word set of each document; wherein, when a word is recognized, it is determined whether the word belongs to a preset stop word set, and when the word belongs to the preset stop word set, the word is removed; The matrix construction unit is used to count the word sets of all documents, obtain the word space, calculate the TF-IDF value of each word in each word set, and construct the document-word matrix; A matrix decomposition unit, used to receive the number of topics input by the management party, and perform matrix decomposition on the document-word matrix based on the number of topics to obtain a document-topic matrix and a topic-word matrix; Among them, the rows in the document-word matrix correspond to documents, the number of rows is the same as the total number of documents, the columns correspond to words, and the number of columns is the same as the total number of words; the rows in the document-topic matrix correspond to documents, the number of rows is the same as the total number of documents, the columns correspond to topics, and the number of columns is the same as the number of topics; the rows in the topic-word matrix correspond to topics, the number of rows is the same as the number of topics, the columns correspond to words, and the number of columns is the same as the total number of words.

10. The off-site data intelligent identification and processing system according to claim 7, characterized in that: The subject name determination module includes: A comparison unit, used to read the topic-word matrix, traverse the element values ​​in the matrix, and compare them with a preset threshold; A feature word generating unit, used for reading a word corresponding to the element as a feature word of the subject corresponding to the element when the element value is greater than a preset threshold; The feedback statistics unit is used to count the feature words, feed back to the evaluation end, and receive the topic name sent by the evaluation end.

Images