A data storage method, apparatus, device, and storage medium

Abstract

The application discloses a data storage method, device and equipment and a storage medium. The method comprises the following steps: obtaining target data; splitting the target data to obtain parameter words; replacing the parameter words in the target data with placeholders to obtain parameterized data; and storing the target data if the parameterized data meets a preset condition. Through the technical scheme, the problem that storing a large amount of repeated data occupies too much storage space and data analysis and processing are very complicated is solved, the storage of data of the same key information is reduced, and the occupied storage space is reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data storage method, apparatus, device and storage medium. Background Technology

[0002] Programs typically generate various types of data during runtime, and this amount increases over time. Large amounts of data can consume significant storage space and become extremely tedious to analyze. Analysis revealed a high degree of repetition in the data. To minimize storage consumption and facilitate analysis, it's necessary to deduplicate the data generated by the program.

[0003] The general method for deduplicating data is to compare the newly generated data with the previously generated data. If the comparison results are completely equal, the newly generated data is retained, and the previously generated identical data is deleted.

[0004] However, even when using this deduplication method on the data generated by the program, there will still be a lot of duplicate data, resulting in excessive storage space consumption. Summary of the Invention

[0005] This invention provides a data storage method, apparatus, device, and storage medium that solves the problem of excessive storage space being occupied due to the storage of large amounts of duplicate data and the data analysis and processing being very cumbersome. It reduces the storage of data with the same key information, thereby reducing the storage space occupied.

[0006] According to one aspect of the present invention, a data storage method is provided, comprising:

[0007] Obtain the target data;

[0008] The target data is split to obtain parameter words;

[0009] Replace the parameter words in the target data with placeholders to obtain parameterized data;

[0010] If the parameterized data meets the preset conditions, the target data will be stored.

[0011] According to another aspect of the present invention, a data storage device is provided, the data storage device comprising:

[0012] The data acquisition module is used to acquire target data;

[0013] The splitting module is used to split the target data to obtain parameter words;

[0014] The replacement module is used to replace parameter words in the target data with placeholders to obtain parameterized data.

[0015] A storage module is used to store the target data if the parameterized data meets preset conditions.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data storage method described in any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the data storage method described in any embodiment of the present invention.

[0021] This invention solves the problem of excessive storage space consumption and cumbersome data analysis and processing caused by storing a large amount of duplicate data. It reduces the storage of data with the same key information, thereby reducing the storage space occupied.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a data storage method according to an embodiment of the present invention;

[0025] Figure 2 This is a flowchart of another data storage method in an embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram of the structure of a data storage device according to an embodiment of the present invention;

[0027] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] Example 1

[0031] Figure 1 This is a flowchart illustrating a data storage method provided in an embodiment of the present invention. This embodiment is applicable to data storage situations. The method can be executed by the data storage device in this embodiment, which can be implemented in software and / or hardware, such as... Figure 1 As shown, the method specifically includes the following steps:

[0032] S110, Obtain target data.

[0033] The target data can be an SQL statement, a piece of data in a log file, a sentence in a log file, a piece of data in an event file, or an event. This embodiment of the invention does not impose any restrictions on these.

[0034] S120, the target data is split to obtain parameter words.

[0035] The parameter word can be a word that includes special characters and numbers. The parameter word can also be a word that includes special characters and target characters. The target character can be a preset word or a preset character. For example, the parameter word can include: a string enclosed in single quotes, such as '192.168.100.164', or the parameter word can include: a numerical value enclosed in parentheses, such as (1620).

[0036] Specifically, the method for splitting the target data to obtain parameter words can be as follows: obtain the lexical format information of the parameter words; perform word segmentation on the target data according to the lexical format information of the parameter words to obtain key words and parameter words. For example, the strings enclosed in single quotes and the numerical values ​​enclosed in parentheses in the target data can be determined as parameter words, and the remaining words can be determined as key words.

[0037] Optionally, the target data can be split to obtain parameter words, including:

[0038] Obtain the lexical format information of the parameter word;

[0039] The target data is segmented based on the lexical format information of the parameter words to obtain key words and parameter words.

[0040] The lexical format information of the parameter word can be: special character + string + special character, for example, a string enclosed in single quotes, such as '192.168.100.164'. Alternatively, the lexical format information of the parameter word can be: special character + number + special character, for example, a number enclosed in parentheses, such as (1620).

[0041] The keywords are words other than the parameter words.

[0042] Specifically, the target data can be segmented based on the lexical format information of the parameter words to obtain key words and parameter words in the following ways: words in the target data that include special characters and numbers are identified as parameter words; words in the target data other than parameter words are identified as key words.

[0043] Optionally, the target data is segmented based on the lexical format information of the parameter words to obtain key words and parameter words, including:

[0044] Words containing special characters and numbers in the target data are identified as parameter words;

[0045] The words in the target data other than the parameter words are identified as keywords.

[0046] The numbers can be numerical or string values, and this embodiment of the invention does not impose any restrictions on them.

[0047] Specifically, the method for determining words in the target data that include special characters and numbers as parameter words can be as follows: determine words in the target data with the structure: special character + number + special character as parameter words.

[0048] In a specific example, the target data is: Can't connect to DM server on '192.168.100.164' port (1620) errno (111). Among them, '192.168.100.164', (1620) and (111) are parameter words, and the other words are keywords.

[0049] S130, replace the parameter words in the target data with placeholders to obtain parameterized data.

[0050] There can be multiple placeholders, and the placeholders can be determined in the following ways: different placeholders correspond to different types of parameter words; or the placeholders correspond to different special characters included in the parameter word. For example, the placeholder for a string parameter word could be 'String', and the placeholder for a numeric parameter word could be (Number).

[0051] Specifically, the parameterized data can be obtained by replacing the parameter words in the target data with placeholders by: obtaining the first placeholder corresponding to the string parameter words and the second placeholder corresponding to the numerical parameter words, wherein the first placeholder and the second placeholder are different; replacing the string parameter words in the target data with the first placeholder and replacing the numerical parameter words in the target data with the second placeholder to obtain the parameterized data.

[0052] In a specific example, the string parameter word in the target data is replaced with 'String', and the numeric parameter word in the target data is replaced with (Number).

[0053] Optionally, the parameter words include: string parameter words and numeric parameter words;

[0054] Replace the parameter words in the target data with placeholders to obtain parameterized data, including:

[0055] Obtain the first placeholder corresponding to the string parameter word and the second placeholder corresponding to the numeric parameter word, wherein the first placeholder and the second placeholder are different;

[0056] Replace the string parameter words in the target data with the first placeholder, and replace the numeric parameter words in the target data with the second placeholder to obtain parameterized data.

[0057] The string parameter word includes: a first special character and a first number. For example, the string parameter word can be a string enclosed in single quotes. The numeric parameter word includes: a second special character and a second number. For example, the numeric parameter word can be a numeric value enclosed in parentheses.

[0058] The first placeholder includes a first special character and a first word, wherein the first word is different from all words in the target data, the first word is not a keyword, and the first word is not a parameter word; for example, the first placeholder could be 'String'. The second placeholder includes a second special character and a second word, wherein the second word is different from all words in the target data, the second word is not a keyword, and the second word is not a parameter word; for example, the second placeholder could be (Number).

[0059] Optionally, the string parameter word includes: a first special character and a first number; the numeric parameter word includes: a second special character and a second number; the first placeholder includes: a first special character and a first word; the second placeholder includes: a second special character and a second word; the first word and the second word are not keywords, and the first word and the second word are different.

[0060] S140, if the parameterized data meets the preset conditions, the target data is stored.

[0061] The preset condition can be: there is no data in the hash table that is the same as the parameterized data; the preset condition can also be: there is no MD5 value in the hash table that is the same as the MD5 value of the parameterized data.

[0062] Specifically, if the parameterized data meets preset conditions, the target data can be stored as follows: if there is no data in the hash table that is the same as the parameterized data, then the parameterized data and the target data are stored in the hash table; if the parameterized data meets preset conditions, the target data can also be stored as follows: obtain the MD5 value of the parameterized data; if there is no MD5 value in the hash table that is the same as the MD5 value of the parameterized data, then the MD5 value of the parameterized data and the target data are stored in the hash table.

[0063] Optionally, if the parameterized data meets preset conditions, the target data is stored, including:

[0064] If no data identical to the parameterized data exists in the hash table, then the parameterized data and the target data are stored in the hash table.

[0065] The hash table stores parameterized data and the original data corresponding to the parameterized data.

[0066] Specifically, the parameterized data is compared with the parameterized data stored in the hash table. If no data matching the parameterized data exists in the hash table, then the parameterized data and the target data are stored in the hash table. Optionally, if the parameterized data meets a preset condition, then the target data is stored.

[0067] Get the MD5 value of parameterized data;

[0068] If there is no MD5 value in the hash table that is the same as the MD5 value of the parameterized data, then the MD5 value of the parameterized data and the target data are stored in the hash table.

[0069] The hash table stores MD5 values ​​and the original data corresponding to those MD5 values.

[0070] Specifically, the MD5 value of the parameterized data is compared with the MD5 value stored in the hash table. If there is no MD5 value in the hash table that is the same as the MD5 value of the parameterized data, then the MD5 value of the parameterized data and the target data are stored in the hash table.

[0071] It should be noted that if data identical to the parameterized data exists in the hash table, the parameterized data and the target data can be stored in the hash table as needed. For example, if the requirement is to retain the earliest generated data, the parameterized data and the target data will not be used to replace the existing data in the hash table; if the requirement is to retain the latest generated data, the parameterized data and the target data will be used to replace the existing data in the hash table.

[0072] If an MD5 value identical to the parameterized data exists in the hash table, then the MD5 value of the parameterized data and the target data are stored in the hash table as needed. For example, if the requirement is to retain the earliest generated data, then the MD5 value of the parameterized data and the target data are not used to replace the existing data in the hash table; if the requirement is to retain the latest generated data, then the MD5 value of the parameterized data and the target data are used to replace the existing data in the hash table.

[0073] In a specific example, the following data exists:

[0074] Can't connect to DM server on'192.168.100.164'port(1620)errno(111);

[0075] Can't connect to DM server on'192.168.100.163'port(1620)errno(111);

[0076] Can't connect to DM server on'192.168.100.163'port(1640)errno(111);

[0077] Can't connect to DM server on'192.168.100.164'port(1641)errno(111);

[0078] comm_inet_msg_recv_for_ecs got error,port Failure occurs in data_recv_inet_once,code(104)len(32892);

[0079] comm_inet_msg_recv_for_ecs got error,port Failure occurs in data_recv_inet_once,code(104)len(1020);

[0080] The six data entries above are runtime log data generated by the database program. It's clear that the first four entries are identical except for the specific string values ​​of the IP address, the numeric values ​​of the port number and the error number; the last two entries are also identical except for the numeric code number and length. Therefore, we consider the portion of the data excluding the specific parameter values ​​as the key information. The first four log entries both indicate a database connection creation failure, while the last two indicate a database message read failure.

[0081] The analysis revealed that data with identical key information can be considered duplicates. In this example, the first four data entries are duplicates, and the last two are also duplicates. The expected outcome of this example is to retain the most recently generated data after deduplication, specifically the fourth and sixth entries.

[0082] Conventional data deduplication methods using equality comparisons fail because the first four and last two data entries are unequal, making deduplication impossible and failing to achieve the desired result. Therefore, this invention provides a data deduplication method. First, the target data is split into keywords and parameter words. Placeholders are used to replace the parameter words, forming parameterized data. Then, the MD5 value of the parameterized data is calculated and compared with the key in the hash table to determine if it already exists. If it does, and the user requires the latest data to be stored in the table (requiring an update to the hash table), the data stored in the hash table is replaced according to the user's requirements. If it does not exist, the MD5 value and the corresponding original data are recorded in the hash table. Finally, the value stored in the hash table is the deduplicated data. This method effectively and quickly achieves data deduplication. This deduplication method can be applied to scenarios such as log deduplication, SQL deduplication, and event deduplication.

[0083] Step 1: Take a single piece of target data and split it into keywords and parameter words. Keywords: All non-parameter words are called keywords. Parameter words: Divided into string parameter words and numeric parameter words. String parameter words: Strings enclosed in single quotes, such as '192.168.100.164'. Numeric parameter words: Numeric values ​​enclosed in parentheses, such as (1620). After defining the lexical formats of keywords, string parameter words, and numeric parameter words, use a lexical analysis program to split a single piece of data into multiple words. The target data is as follows:

[0084] Can't connect to DM server on'192.168.100.164'port(1620)errno(111);

[0085] The data was split into words. Among them, '192.168.100.164' is the string parameter word, (1620) and (111) are the numerical parameter words, and the other words are keywords.

[0086] Step 2: After replacing the parameter words with placeholders, combine them with the keywords to form parameterized data, and calculate the MD5 value of the parameterized data. The placeholder for string parameter words is 'String'. The placeholder for numeric parameter words is (Number).

[0087] Use spaces to connect keywords and parameter placeholders to form parameterized data:

[0088] Can't connect to DM server on'String'port(Number)errno(Number)

[0089] Calculate the MD5 value of the parameterized data.

[0090] Generally, the MD5 value of a parameterized data string is shorter than the parameterized string itself, and comparing the MD5 values ​​of strings is faster than directly comparing strings.

[0091] Step 3: Use a hash table to store the deduplicated data and the MD5 value calculated from the parameterized data.

[0092] The keys of the hash table are the MD5 values ​​calculated from the parameterized data, and the values ​​are the original data corresponding to the MD5 values.

[0093] The hash table is traversed to find the keys, and the calculated MD5 value is compared to determine whether it already exists.

[0094] If the MD5 value calculated in step two already exists in the hash table, when the latest generated data is needed, the data corresponding to the MD5 value is replaced with the data retrieved in step one; when the earliest generated data is needed, no replacement is performed. This is not a limitation; this invention uses the requirement of the latest generated data as an example for illustration.

[0095] If it does not exist, add the MD5 value calculated in step two and the data retrieved in step one to the hash table.

[0096] Step 4: Repeat steps 1 to 3 until all data has been processed. The data in the hash table is the data that has been deduplicated and retained.

[0097] The data retained in the above example is:

[0098] Can't connect to DM server on'192.168.100.164'port(1641)errno(111)

[0099] comm_inet_msg_recv_for_ecs got error,port Failure occurs in data_recv_inet_once,code(104)len(1020)

[0100] In another specific example, such as Figure 2 As shown, the process checks if all data has been retrieved. If so, the data is stored in the hash table. If not, a single data entry is selected as the target data. Lexical analysis is performed on the target data to obtain a word list. Words are then extracted from this list. If the extracted word is a string parameter word, it is replaced with a string placeholder. If the extracted word is a numeric parameter word, it is replaced with a numeric placeholder. If all words in the word list have been replaced, parameterized data is obtained. The MD5 hash of the parameterized data is calculated. It is then checked if a matching MD5 hash exists in the hash table. If it does, the data corresponding to the MD5 hash in the hash table is replaced with the target data. If not, both the MD5 hash and the target data are stored in the hash table.

[0101] The technical solution of this embodiment obtains target data; splits the target data to obtain parameter words; replaces the parameter words in the target data with placeholders to obtain parameterized data; if the parameterized data meets preset conditions, the target data is stored. This solves the problem of storing a large amount of duplicate data, which leads to excessive storage space and cumbersome data analysis and processing. It also reduces the storage of data with the same key information, thereby reducing the storage space occupied.

[0102] Example 2

[0103] Figure 3 This is a schematic diagram of a data storage device provided in an embodiment of the present invention. This embodiment is applicable to data storage applications. The device can be implemented using software and / or hardware, and can be integrated into any device that provides data storage functionality, such as… Figure 3 As shown, the data storage device specifically includes: a data acquisition module 310, a splitting module 320, a replacement module 330, and a storage module 340.

[0104] The data acquisition module is used to acquire target data.

[0105] The splitting module is used to split the target data to obtain parameter words;

[0106] The replacement module is used to replace parameter words in the target data with placeholders to obtain parameterized data.

[0107] A storage module is used to store the target data if the parameterized data meets preset conditions.

[0108] The above-described products can perform the methods provided in any embodiment of the present invention, and have the corresponding functional modules and beneficial effects for performing the methods.

[0109] The technical solution of this embodiment obtains target data; splits the target data to obtain parameter words; replaces the parameter words in the target data with placeholders to obtain parameterized data; if the parameterized data meets preset conditions, the target data is stored. This solves the problem of storing a large amount of duplicate data, which leads to excessive storage space and cumbersome data analysis and processing. It also reduces the storage of data with the same key information, thereby reducing the storage space occupied.

[0110] Example 3

[0111] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0112] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0113] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0114] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as data storage methods.

[0115] In some embodiments, the data storage method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the data storage method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to execute the data storage method by any other suitable means (e.g., by means of firmware).

[0116] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0117] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0118] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0119] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0120] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0121] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0122] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0123] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data storage method, characterized by, The method comprises the following steps: acquiring target data; splitting the target data to obtain parameter words; wherein the parameter words comprise string parameter words and numerical value parameter words; acquiring a first placeholder corresponding to the string parameter words and a second placeholder corresponding to the numerical value parameter words, wherein the first placeholder and the second placeholder are different; replacing the string parameter words in the target data with the first placeholder and replacing the numerical value parameter words in the target data with the second placeholder to obtain parameterized data; storing the target data if the parameterized data meets a preset condition.

2. The method of claim 1, wherein, storing the target data if the parameterized data meets a preset condition, which comprises the following steps: storing the parameterized data and the target data in a hash table if the hash table does not contain data identical to the parameterized data.

3. The method of claim 1, wherein, storing the target data if the parameterized data meets a preset condition, which comprises the following steps: acquiring an MD5 value of the parameterized data; storing the MD5 value of the parameterized data and the target data in a hash table if the hash table does not contain an MD5 value identical to the MD5 value of the parameterized data.

4. The method of claim 1, wherein, splitting the target data to obtain parameter words, which comprises the following steps: acquiring the lexical format information of the parameter words; performing word segmentation processing on the target data according to the lexical format information of the parameter words to obtain key words and parameter words.

5. The method of claim 4, wherein, performing word segmentation processing on the target data according to the lexical format information of the parameter words to obtain key words and parameter words, which comprises the following steps: determining the words in the target data that contain special characters and numbers as parameter words; determining the words in the target data other than the parameter words as key words.

6. The method of claim 1, wherein, The string parameter words comprise first special characters and first numbers, the numerical value parameter words comprise second special characters and second numbers, the first placeholder comprises first special characters and a first word, the second placeholder comprises second special characters and a second word, the first word and the second word are both non-key words, and the first word and the second word are different.

7. A data storage device, characterized by The method comprises the following steps: a data acquisition module is configured to acquire target data; a splitting module is configured to split the target data to obtain parameter words; wherein the parameter words comprise string parameter words and numerical value parameter words; a replacing module is configured to replace the parameter words in the target data with placeholders to obtain parameterized data; a storage module is configured to store the target data if the parameterized data meets a preset condition; wherein the replacing module is specifically configured to acquire a first placeholder corresponding to the string parameter words and a second placeholder corresponding to the numerical value parameter words, wherein the first placeholder and the second placeholder are different; replace the string parameter words in the target data with the first placeholder and replace the numerical value parameter words in the target data with the second placeholder to obtain parameterized data.

8. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the data storage method in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to implement the data storage method in any one of claims 1-6 when executed.

Images