Intelligent Risk Perception Data Security Storage Method and System

Abstract

The present invention relates to the technical field of data security storage, and in particular, to a data security storage method and system with intelligent risk perception. The invention realizes the whole-process data monitoring of data storage through original data acquisition, data information extraction, data verification, data deduplication, data storage, data backup, and data access, and can detect various data transmission, storage, and access errors, including errors in the data content itself. Among them, the problem of errors in the source data itself is difficult to be detected by traditional methods. At the same time, for different modal data such as pictures, audio, and text, the present invention can realize fuzzy search according to the data content, improving the effectiveness of data access.

Description

Technical Field

[0001] The present invention relates to the technical field of data security storage, and particularly relates to a data security storage method and system with intelligent risk perception. Background Art

[0002] Data security verification is often involved in data storage. Its main function is to ensure the accuracy and reliability of data. Through data verification, a data security storage system can prevent illegal or insecure data from entering the system, improve the security and stability of the system. At the same time, by reducing the entry of invalid data into the business processing link, the system burden is reduced, the response speed is increased, and the system performance is optimized.

[0003] In practical applications, most traditional data security verifications only consider the risks during data transmission and ignore the errors at the data source. At the same time, traditional data transmissions usually have a single data type, so the data protocol is fixed. On the one hand, when the data is illegally intercepted, the difficulty coefficient of data cracking is low. On the other hand, when the transmitted data types are diverse and the transmission timing is uncertain, it is difficult for a traditional data security storage system to parse and securely store the data. Summary of the Invention

[0004] To solve the problems in the background art, the present invention provides a data security storage method with intelligent risk perception, including:

[0005] S101. Obtaining original data, obtaining source data from each data generation unit. The source data includes a data header and data content. The data header records brief information of the data, and the brief information at least includes: data protocol code, data volume size, data check code, data subject. The data content format at least includes: numbers, texts, images, audio;

[0006] S102. Extracting data information, extracting the data header of the source data according to the general data protocol and obtaining the local data protocol corresponding to the data content according to the data protocol code, and parsing the data content. When the data content cannot be parsed according to the data protocol specified in the data header, a data content error warning is issued;

[0007] S103. Data verification, calculating the data volume size in the data content and comparing it with the data volume size in the data header. The absolute value of the comparison difference is denoted as Δd1. Calculating the data content check code using the data verification method of the protocol and comparing it with the check code in the data header. The absolute value of the comparison difference is denoted as Δd2. When Δd1 + Δd2 > 0, a data transmission error warning is issued;

[0008] S104. Data deduplication: Compare the content similarity between the verified data and the data with the same theme stored in the database. When the content similarity is greater than the threshold, issue a data duplication warning. After manual confirmation that the data is duplicate data, delete the data.

[0009] S105. Data storage: Denote the data header of the verified and deduplicated data as K, and the data content as D. Save the data content D to a secure database, generate a unique address value V1 for data access, and save the data header K and the data content access address V1 in the form of key-value pairs to the search database.

[0010] S106. Data backup: When the computing tasks are few, automatically back up the data and map the backup data access address value V2 to V1. The mapping relationship table is saved in the search database.

[0011] Specifically, in S102 data information extraction, for image data, use a deep learning neural network to obtain the abstract information of the image and convert it into text. For audio data, use a natural language processing model to obtain the main information of the audio and convert it into text. When proceeding to step S103, the data verification also calculates the similarity value Tt between the image abstract text, audio content text, or original data text extracted from the information and the data theme text in the data header. When Tt is less than the set threshold, issue a data content error warning; when the original data content is a number, no similarity calculation is performed.

[0012] Furthermore, the data security storage method also includes the iterative optimization and update of the deep learning neural network model for image abstract generation and the natural language processing model for audio content recognition. The data source for iterative optimization is the data content saved in S105, which is obtained by addressing through the address V1, and the corresponding data label is the data theme text in the data header K. When a set magnitude of data is newly added to the database, the data security storage method automatically performs fine-tuning training on the original model and updates the model parameters.

[0013] Even further, the data security storage method also includes data security search. When a user needs to access data, the data security storage method verifies the user's identity. When the identity verification fails, the data is not accessible and an illegal access warning is issued. When the identity verification passes, the data access task can be executed.

[0014] Particularly, the verified user can perform fuzzy search through the theme information in the data header K to achieve extensive search of different types of data.

[0015] Particularly, when the accessed data cannot be correctly parsed according to the protocol, issue a data corruption warning. At this time, the data security storage method automatically restores the data according to the mapping between V1 and V2.

[0016] In particular, the present invention also provides a data security storage system with intelligent risk perception, including: a data acquisition module, a data risk perception module, a data storage module, and a data access module. The data acquisition module realizes the acquisition of the original data in S101; the data risk perception module realizes the extraction of data information, data verification, and data deduplication in S102 - 104. When corresponding problems occur in S102 - 104, the system issues corresponding warnings; the data storage module saves and backs up the data; the data access module realizes user verification. When the user verification passes, it realizes the matching of user access to data and the addressing of data.

[0017] Specifically, the data storage module includes a security database sub - library, a search database sub - library, a model database sub - library, and a backup database sub - library. Among them, the security database sub - library stores the original data content D after verification and deduplication, the search database sub - library saves the K, V1 key - value pair data, the V1 / V2 mapping table data, and different data protocols, the model database sub - library saves the deep - learning neural network model for generating image summaries and the natural language processing model for audio content recognition, and the backup database sub - library saves the backed - up data.

[0018] Advantages of the present invention:

[0019] 1. The present invention pays attention to the risk problems in the whole process of data generation, transmission, storage, and access, including: the risk of errors in the data content itself, the risk of data transmission errors, data duplication, illegal access risk, and data corruption risk. Among them, the risk of errors in the data content itself comes from the errors during data generation, and such errors are generally difficult to be discovered.

[0020] 2. Through local data protocol parsing, image summary generation, and audio content recognition, the present invention can identify the situation where the transmitted data is inconsistent with the determined theme during transmission. At the same time, since the local protocol code is transmitted during the data transmission process, the present invention can achieve the unified transmission of different - modality data and ensure the security of data storage.

[0021] 3. The deep - learning neural network for generating image summary information and the large - scale natural language processing model for audio recognition in the present invention have the ability of automatic iterative training, which can realize the continuous learning of the model and improve the data risk perception ability.

[0022] 4. In addition to being able to perform user identity verification for data access, the present invention also provides fuzzy search for data access, which can ensure the security of data storage and provide effective search for different - modality data at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the logical structure of the data security storage method with intelligent risk perception of the present invention;

[0024] Figure 2 Schematic diagram of database classification for the data security storage system with intelligent risk perception of the present invention. Detailed implementation manners

[0025] In order to make the technical methods, creative features, achieved purposes and effects realized by the present invention easy to understand, the present invention will be further described below in conjunction with the detailed implementation manners. Embodiment 1

[0026] As Figure 1 shown, the basic steps of the data security storage method with intelligent risk perception of the present invention are: S101 obtaining data; S102 extracting data; S103 verifying data; S104 data deduplication; S105 data storage; S106 data backup. Among them, since the present invention incorporates an image abstract generation network and a speech recognition network, in addition to receiving common digital and text data, the present invention can also receive image and audio data.

[0027] During application, the data sent by the data generation unit is packetized into two parts: a data header and data content. The data header records the basic information of the data content for verification, including but not limited to the protocol code for decoding the data content, the data volume size, the data checksum, and the data topic. The data protocol is selected according to the data type (such as numbers, audio, pictures, etc.) that the sending end wants to send, and the data topic is marked according to the data content that the sending end wants to send. When the data is received, the data receiving party first decodes the data header according to the general protocol to obtain the main information of the data, and then matches the correct decoder in the receiving-end server according to the data protocol code to decode the data content. When the corresponding decoder cannot correctly decode the data, it indicates that the data actually wanted to be sent by the data generation unit does not match the data actually sent. For example, the intention was to send a picture for storage, but actually, due to a misoperation, an audio file was sent. At this time, the data security storage method of the present invention can capture this type of error and issue a warning about the error in the sent data content. On the other hand, the method of the present invention further compares the sent data content. The comparison first determines the data content type according to the protocol. When the data is digital data, since it is difficult to obtain the semantic information of digital data, no verification of the data content is performed. When the data is text, image, or audio, the present invention first obtains the data topic text information in the data header, then generates a digest of the image data through a neural network model for image digest generation, recognizes the audio data through a speech recognition network and outputs the speech text, and finally compares the similarity between the data content text and the topic text in the data header through NLP natural language processing. One embodiment of NLP processing is to first perform word segmentation, then remove stop words, perform Word2Vec word vectorization, and finally calculate the similarity between texts using cosine similarity. The calculated text similarity is denoted as Tt. When Tt is less than the set threshold, it is considered that the data sent to the storage end does not match the data actually intended to be saved. For example, the intention was to send a picture of a person, but a picture of a beautiful landscape was sent, or the intention was to send an experimental report, but a paper was sent. At this time, the data security storage method of the present invention will capture this type of error and issue a warning about the error in the data content.

[0028] Of course, whether it is an error in data type mismatch or data content mismatch, the source of the error may also be due to incorrect selection of the data type or incorrect description of the data topic during data transmission. Therefore, when receiving a warning about an error in the data content, the operator needs to further confirm to clarify whether to modify the data header information and continue to save the current data or delete the data.

[0029] It should be noted that the reason why the data type mismatch is not automatically matched according to the file suffix at the data sending port or the data type is not verified at the sending end first is that for some data, due to confidentiality requirements, on the one hand, the suffix is not retained, and on the other hand, the data itself is in an encrypted mode, and the sending end cannot perform real data type verification locally.

[0030] It should be noted that if video data is received, the video data can be frame-sampled at intervals, image digests can be generated for the obtained frame data, and finally, all the obtained digests can be summarized using basic NLP natural language processing methods to generate the theme content of the video and obtain the video theme text.

[0031] It should be noted that as the stored data increases, models such as the image digest generation neural network and the speech recognition neural network will automatically perform iterative learning (fine-tuning on the basis of the original model). Among them, the data source is the data content stored in the database, and the data theme text in the data header stored in the database is the image digest and the corresponding text label of the speech. Embodiment 2

[0032] As Figure 1 shown, the data after information extraction needs to be subjected to S103 data verification and S104 data deduplication. Among them, for S103 data verification, the data volume size in the data content is calculated and compared with the data volume size in the data header, and the absolute value of the comparison difference is denoted as Δd1. The data content checksum is calculated using the data verification method of the protocol and compared with the checksum in the data header, and the absolute value of the comparison difference is denoted as Δd2. When Δd1 + Δd2 > 0, that is, when any one of the data volume size and the data verification fails to pass the verification, it is considered that problems such as data packet sticking, packet loss, and data errors may have occurred during data transmission at this time. Therefore, it will cause problems such as the data volume size at the receiving end being inconsistent with that at the sending end or the data checksum being inconsistent. The data security storage method of the present invention will issue a data transmission error warning. The data verification method can be parity check, longitudinal redundancy check (LRC), checksum check, cyclic redundancy check (CRC), etc.

[0033] The verified data needs to be de-duplicated. The method of the present invention obtains the data theme content in the data header. When the data theme of the currently stored data is the same as the data theme already stored in the database, at this time, the data content similarity between the current data and the data saved in the database is compared. When the similarity of the two pieces of data is greater than the preset threshold, it is considered that there may be duplicate data transmission, and a data duplication warning is issued. The operator confirms the data according to the prompt. When it is confirmed that the data is indeed sent repeatedly, the data is deleted; otherwise, the data is processed according to the set scheme (such as saving the data after marking). Compared with traditional data comparison, the method of the present invention does not need to compare all data, reducing the computing task volume. At the same time, since the data of the present invention stores data such as images and audios, the data comparison calculation efficiency is low for all data comparison.

[0034] The final data is subjected to S105 data storage and S106 data backup. Among them, data storage is divided into two parts. One part stores the data content D in the security database, and the other part stores the data header K corresponding to the data content D and the database address V1 where D is located in the form of a key-value pair K:V1 in the search database; when there are fewer computing tasks at night or other times, the method of the present invention automatically performs data backup and maps the backup data access address value V2 to V1, and the mapping relationship table is saved in the search database. When a user needs to access data, the method of the present invention first verifies the user's identity. When the verification fails, the method of the present invention issues an illegal access warning. When the verification passes, at this time, the user can search for data through the information in the data header K in the search database, so as to realize the search for unstructured data such as pictures and audios. Among them, it should be emphasized that since K contains rich voice information of the data content, therefore, the data security search of the present invention can realize fuzzy search of the data. For example, if you need to search for pictures containing people, the method of the present invention can use the NLP method to search for data with the theme information containing people and the data type of pictures in the data header K. Another example is to search for audio interviews of teenagers with a size greater than 10M. Then, the corresponding data can be obtained according to the data volume size and data theme, and the matched data is addressed in the security database through V1 corresponding to K.

[0035] S106 data backup is automatically performed during a period when the computer computing tasks are not intensive. Generally, it will be after the operator gets off work at night. The backup data will generate a data storage address, denoted as V2. V2 will be mapped to V1, and the mapping relationship table is saved in the search database. When the normal access to data cannot be correctly parsed according to the protocol, the present invention issues a data corruption warning. At this time, according to the V1, V2 mapping table, the corresponding backup data address V2 can be quickly found through V1, and the data can be restored. Embodiment 3

[0036] The present invention also provides an intelligent risk-aware data security storage system. This data security storage system applies the above-mentioned data security storage method for data storage and data access, including: a data acquisition module, a data risk perception module, a data storage module, and a data access module. Among them, the data acquisition module realizes the acquisition of the original data in S101; the data risk perception module realizes the data information extraction, data verification, and data deduplication in S102 - 104. When corresponding problems in S102 - 104 occur, the system issues corresponding warnings; the data storage module saves and backs up the data; the data access module realizes user verification. When the user verification passes, it realizes the matching of user access to the data and the addressing of the data to obtain. At the same time, in order to improve data storage security and take into account data retrieval efficiency, this data security storage system divides the data storage module into several database sub-libraries, such as Figure 2 As shown, the database sub-libraries include: a security database sub-library, a search database sub-library, a model database sub-library, and a backup database sub-library. Among them, the security database sub-library has a relatively high security level and is used to store the original data content D after verification and deduplication to ensure data security; the search database sub-library has a relatively high data access speed and is used to save key-value pair data of K and V1, V1 / V2 mapping table data, and different data protocols to provide fast retrieval of data access; the model database sub-library saves the deep learning neural network model for generating image summaries and the natural language processing model for audio content recognition. Since the model and corresponding parameters will be directly introduced into the cache during system operation, the requirement for storage access speed can be reduced; the backup database sub-library saves the backup data, and its security level is also relatively high.

[0037] In summary, the system and method of the present invention can identify risks in the whole process of data storage, including: the risk of errors in the data content itself, the risk of data transmission errors, data duplication, illegal access risks, and data corruption risks. Among them, the risk of errors in the data content itself comes from errors during data generation, and such errors are generally difficult to detect.

[0038] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art of this industry should understand that the present invention is not limited by the above embodiments. The above embodiments and the descriptions in the specification only illustrate the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the claims of the present invention. The scope of protection claimed by the present invention.

Claims

1. An intelligent risk-aware data security storage method, characterized by: The data security storage method comprises: S101. Original data acquisition: obtaining source data from each data generating unit. The source data includes a data header and data content. The data header records brief information about the data. The brief information includes at least: data protocol code, data size, data checksum, data subject, and the data content format includes at least: numbers, text, images, and audio. S102. Data information extraction, extracting the data header of the source data according to the universal data protocol and obtaining the local data protocol corresponding to the data content according to the data protocol code, parsing the data content, and issuing a data content error warning when the data content cannot be parsed according to the data protocol specified in the data header; S103. Data verification: Calculate the amount of data in the data content and compare it with the amount of data in the data header. The absolute value of the difference is recorded as Δd1. Use the data verification method of the protocol to calculate the data content check code and compare it with the check code in the data header. The absolute value of the difference is recorded as Δd2. When Δd1+Δd2>0, a data transmission error warning is issued; S104. Data deduplication: the verified data is compared with the data with the same subject stored in the database for content similarity. When the content similarity is greater than the threshold, a data duplication warning is issued. After the data is manually confirmed to be duplicate data, the data is deleted; S105. Data storage: record the data header after verification and deduplication as K, record the data content as D, save the data content D to the security database, generate a unique address value V1 for data access, and save the data header K and the data content access address V1 in the form of a key-value pair to the search database; S106. Data backup: when there are fewer computing tasks, the data is automatically backed up and the backed-up data access address value V2 is mapped to V1, and the mapping relationship table is saved in the search database.

2. The method for secure data storage based on intelligent risk perception according to claim 1, characterized in that: In the data information extraction of S102, the image data uses a deep learning neural network to obtain the summary information of the image and convert it into text. The audio data uses a natural language processing model to obtain the main information of the audio and convert it into text. When proceeding to step S103, the data verification also calculates the similarity value Tt between the image summary text, audio content text or original data text extracted from the information and the data subject text in the data header. When Tt is less than the set threshold, a data content error warning is issued; when the original data content is a number, no similarity calculation is performed.

3. The method for secure data storage based on intelligent risk perception according to claim 2 is characterized by: The data security storage method also includes iterative optimization and updating of the deep learning neural network model for image summary generation and the natural language processing model for audio content recognition, wherein the data source for iterative optimization is the data content stored in S105, the data is obtained by addressing address V1, and the corresponding data label is the data body text in the data header K. When a set amount of data is added to the database, the data security storage method automatically performs fine-tuning training on the basis of the original model and updates the model parameters.

4. The method for secure data storage based on intelligent risk perception according to claim 3 is characterized by: The data security storage method also includes data security search. When a user needs to access data, the data security storage method verifies the identity of the user. When the identity verification fails, the data is not accessed and an illegal access warning is issued. When the identity verification passes, the data access task can be executed.

5. The method for secure data storage based on intelligent risk perception according to claim 4 is characterized by: Users who have passed identity verification can perform fuzzy searches through the subject information in the data header K, thereby achieving extensive searches of different types of data.

6. The method for secure data storage based on intelligent risk perception according to claim 5 is characterized by: When the accessed data cannot be correctly parsed according to the protocol, a data corruption warning is issued. At this time, the data security storage method automatically restores the data according to the mapping between V1 and V2.

7. An intelligent risk-aware data security storage system, wherein the data security storage system adopts the intelligent risk-aware data security storage method according to any one of claims 1 to 6 for data storage, comprising: The data acquisition module, data risk perception module, data storage module and data access module are characterized in that: the data acquisition module realizes the acquisition of original data of S101; the data risk perception module realizes the data information extraction, data verification and data deduplication of S102-104, and when the corresponding problems in S102-104 occur, the system issues corresponding warnings; the data storage module saves and backs up the data; the data access module realizes user verification, and when the user verification passes, the matching of user access data is realized and the corresponding data is obtained by addressing according to the matched data address.

8. The intelligent risk-aware data security storage system according to claim 7 is characterized by: The data storage module includes a security database sub-library, a search database sub-library, a model database sub-library and a backup database sub-library. The security database sub-library stores the original data content D after verification and deduplication, the search database sub-library stores K, V1 key-value pair data, V1 / V2 mapping table data and different data protocols, the model database sub-library stores the deep learning neural network model for image summary generation and the natural language processing model for audio content recognition, and the backup database sub-library stores the backup data.

Images