Method of storing data, method of querying data and apparatus

Abstract

The present disclosure provides a method for storing data, a method for querying data, an apparatus, a device, a storage medium and a program product, relating to the technical field of artificial intelligence, and particularly to the technical field of big data. The specific implementation scheme is: obtaining a plurality of to-be-stored identifier pair data, wherein the to-be-stored identifier pair data comprises an original identifier, a mapping identifier and a mapping identifier type; generating index data according to the original identifier and the mapping identifier type of each to-be-stored identifier pair data in the plurality of to-be-stored identifier pair data; dividing the to-be-stored identifier pair data corresponding to the same index data in the plurality of to-be-stored identifier pair data into an identifier data set to obtain at least one identifier data set; determining at least one identifier data according to the at least one identifier data set and the index data corresponding to the at least one identifier data set; and storing the at least one identifier data.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, in particular to the field of big data. BACKGROUND

[0002] With the rapid development of the Internet, the current has entered the era of big data. The attributes, behaviors and other data of users are scattered in different data sources and do not directly communicate with each other, forming data islands. ID-mapping (identity mapping) can be used to identify the identity of the same user in different data sources as the same subject, thereby linking the attributes, behaviors and other data of the user in different data sources, and eliminating data islands. SUMMARY

[0003] The present disclosure provides a method for storing data, a method for querying data, an apparatus, a device, a storage medium and a program product.

[0004] According to an aspect of the present disclosure, a method for storing data is provided, comprising: obtaining a plurality of to-be-stored identity pair data, wherein the to-be-stored identity pair data comprises an original identity, a mapping identity and a mapping identity type; generating index data according to the original identity and the mapping identity type of each to-be-stored identity pair data in the plurality of to-be-stored identity pair data; dividing the to-be-stored identity pair data corresponding to the same index data in the plurality of to-be-stored identity pair data into an identity data set, to obtain at least one identity data set; determining at least one identity data according to the at least one identity data set and the index data corresponding to the at least one identity data set; and storing the at least one identity data.

[0005] According to another aspect of the present disclosure, a method for querying data is provided, comprising: obtaining a reference identity and a to-be-queried identity type; generating target index data according to the reference identity and the to-be-queried identity type; querying candidate identity data corresponding to the target index data from stored identity data, wherein the identity data is stored according to the method of the present disclosure; in the case that the candidate identity data comprises one identity pair data, determining the identity pair data as the target identity pair data; in the case that the candidate identity data comprises a plurality of identity pair data, determining target identity pair data in the plurality of identity pair data according to the reference identity and the to-be-queried identity type; and outputting the mapping identity in the target identity pair data.

[0006] According to another aspect of the present disclosure, an apparatus for storing data is provided, comprising: a first obtaining module configured to obtain a plurality of to-be-stored identifier pair data, wherein the to-be-stored identifier pair data comprises an original identifier, a mapping identifier and a mapping identifier type; a first generating module configured to generate index data according to the original identifier and the mapping identifier type of each to-be-stored identifier pair data in the plurality of to-be-stored identifier pair data; a dividing module configured to divide to-be-stored identifier pair data corresponding to the same index data in the plurality of to-be-stored identifier pair data into an identifier data set, to obtain at least one identifier data set; a first determining module configured to determine at least one identifier data according to the at least one identifier data set and the index data corresponding to the at least one identifier data set; and a storing module configured to store the at least one identifier data.

[0007] According to another aspect of the present disclosure, an apparatus for querying data is provided, comprising: a second obtaining module configured to obtain a reference identifier and a to-be-queried identifier type; a second generating module configured to generate target index data according to the reference identifier and the to-be-queried identifier type; a querying module configured to query candidate identifier data corresponding to the target index data from stored identifier data, wherein the identifier data is stored according to the method of the embodiments of the present disclosure; a second determining module configured to determine an identifier pair data as the target identifier pair data in the case that the candidate identifier data comprises one identifier pair data; a third determining module configured to determine target identifier pair data in the case that the candidate identifier data comprises a plurality of identifier pair data according to the reference identifier and the to-be-queried identifier type; and an outputting module configured to output the mapping identifier in the target identifier pair data.

[0008] Another aspect of the present disclosure provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method shown in the embodiments of the present disclosure.

[0009] According to another aspect of the embodiments of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method shown in the embodiments of the present disclosure.

[0010] According to another aspect of the embodiments of the present disclosure, a computer program product is provided, comprising computer programs / instructions, characterized by that the computer programs / instructions are executed by a processor to implement the steps of the method shown in the embodiments of the present disclosure.

[0011] It is to be understood that the details set forth herein do not construe a limitation on the scope of embodiments of the present disclosure but merely provide description of potential specific embodiments. Furthermore, it is understood that the conjunction "or" as used herein is both the inclusive and exclusive sense unless otherwise stated. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings are included to provide a further understanding of the principles of the application and no limitation to the scope of the application is intended thereby. In drawings:

[0013] Figure 1 is a schematic diagram of an application scenario of a method for storing data, a method for querying data, and an apparatus according to embodiments of the present disclosure;

[0014] Figure 2 is a flowchart schematically illustrating a method for storing data according to embodiments of the present disclosure;

[0015] Figure 3 is a flowchart schematically illustrating a method for querying data according to embodiments of the present disclosure;

[0016] Figure 4 is a schematic diagram of a method for training a machine learning model according to embodiments of the present disclosure;

[0017] Figure 5 is a schematic diagram of a method for determining a target identified pair of data from a plurality of identified pairs of data according to embodiments of the present disclosure;

[0018] Figure 6 is a schematic diagram of a method for storing data according to another embodiment of the present disclosure;

[0019] Figure 7 is a schematic diagram of a method for querying data according to another embodiment of the present disclosure;

[0020] Figure 8 is a block diagram schematically illustrating an apparatus for storing data according to embodiments of the present disclosure;

[0021] Figure 9 is a block diagram schematically illustrating an apparatus for querying data according to embodiments of the present disclosure; and

[0022] Figure 10 is a block diagram schematically illustrating an example electronic device that can be used to implement embodiments of the present disclosure. DETAILED DESCRIPTION

[0023] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0024] The following will combine Figure 1 The application scenarios of the data storage method, data query method, and apparatus provided in this disclosure are described.

[0025] Figure 1 This is a schematic diagram illustrating an application scenario of the data storage method, data query method, and apparatus according to embodiments of this disclosure. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0026] like Figure 1 As shown, the application scenario 100 includes a database 110 and multiple identifier pairs 120.

[0027] According to embodiments of this disclosure, the identifier pair data 120 may include, for example, an original identifier, an original identifier type, a mapped identifier, and a mapped identifier type. The original identifier and the mapped identifier may, for example, include identity identifiers such as usernames and accounts, or hardware identifiers such as IP addresses, MAC addresses, and device identifiers. There is a mapping relationship between the original identifier and the mapped identifier. The original identifier type is the identifier type of the original identifier. The mapped identifier type is the identifier type of the mapped identifier. The identifier type may, for example, include username type, account type, IP address type, MAC address type, device identifier type, etc.

[0028] According to embodiments of this disclosure, multiple identifier pairs 120 can be stored in a database, i.e., database injection. For example, in this embodiment, multiple identifier pairs 120 can be obtained. Then, index data can be generated based on the original identifier and mapped identifier type of each identifier pair 120. Next, the identifier pairs 120 and their corresponding index data are stored in the database 110.

[0029] According to embodiments of this disclosure, different identifier pairs 120 may correspond to the same index data. When storing identifier pairs 120 with the same index data, an index data conflict can occur. Therefore, in this embodiment, the identifier pairs 120 to be stored that correspond to the same index data can be divided into an identifier data set, resulting in at least one identifier data set. Then, based on the at least one identifier data set and the index data corresponding to the at least one identifier data set, at least one identifier data is determined. This at least one identifier data is then stored. This avoids the problem of conflicting index data.

[0030] According to embodiments of this disclosure, after storing the identifier pair data 120 in the database 110, the identifier pair data in the database 110 can be retrieved according to a user's request. For example, in this embodiment, the user can input a reference identifier and a mapping identifier type 130. The reference identifier can be an identifier known to the user. The mapping identifier type can be the type of the target identifier that the user intends to find. There is a mapping relationship between the target identifier and the reference identifier.

[0031] Database 110 can obtain the reference identifier and mapping identifier type input by the user. Then, using the same index data generation method as when storing identifier pairs, it generates target index data based on the reference identifier and mapping identifier type. Next, it queries the candidate identifier data corresponding to the target index data from the identifier data already stored in database 110. If the candidate identifier data includes one identifier pair, that identifier pair is determined as the target identifier pair. If the candidate identifier data includes multiple identifier pairs, the target identifier pair is determined from among the multiple identifier pairs based on the reference identifier and the identifier type to be queried. Finally, it outputs the mapping identifier 140 from the target identifier pair.

[0032] The technical solutions disclosed herein involve identifiers, and the collection, storage, use, processing, transmission, provision, and disclosure of user personal information such as data, all of which comply with relevant laws and regulations and do not violate public order and good morals.

[0033] The following will combine Figure 2 The method for storing data provided in this disclosure is described.

[0034] Figure 2 A flowchart illustrating a method for storing data according to an embodiment of the present disclosure is shown schematically.

[0035] like Figure 2 As shown, the data storage method 200 includes operation S210, acquiring multiple pairs of identifiers to be stored.

[0036] The data to be stored includes the original identifier, the mapped identifier, and the mapped identifier type. The original identifier and the mapped identifier can include, for example, an identity identifier, a hardware identifier, etc. There is a mapping relationship between the original identifier and the mapped identifier. The original identifier type is the identifier type of the original identifier. The mapped identifier type is the identifier type of the mapped identifier. The identifier type can include, for example, an identity identifier type, a hardware identifier type, etc.

[0037] Then, in operation S220, index data is generated based on the original identifier and mapping identifier type of each identifier to be stored in the data.

[0038] According to embodiments of this disclosure, the index data may be, for example, a string generated based on the original identifier and the mapping identifier type.

[0039] In operation S230, the multiple pairs of identifiers to be stored that correspond to the same index data are divided into an identifier data set, resulting in at least one identifier data set.

[0040] According to embodiments of this disclosure, each identifier data set may include at least one identifier pair to be stored.

[0041] In operation S240, at least one identification data is determined based on at least one identification data set and index data corresponding to the at least one identification data set.

[0042] According to embodiments of this disclosure, for example, for each set of identifier data, the identifier data set and the index data corresponding to the identifier data set can be converted into data in a predetermined format, i.e., identifier data. The predetermined format may, for example, include protobuf.

[0043] In operation S250, at least one identification data is stored.

[0044] According to embodiments of this disclosure, when storing identifier pair data, index data is generated based on the original identifier and the mapping identifier type. Therefore, when a user queries, they only need to know the original identifier and the mapping identifier type to perform a query. Even when the user has no idea about the type of the original identifier, the desired mapping identifier can still be accurately output.

[0045] According to embodiments of this disclosure, for each pair of identifiers to be stored, the original identifier and the mapped identifier type in the pair of identifiers to be stored can be combined to obtain a combined string. Then, a hash calculation is performed on the combined string identifier to obtain index data.

[0046] For example, the original identifier in the data to be stored could be ID1, and the mapped identifier type could be type2. Therefore, the original identifier ID1 and the mapped identifier type type2 can be combined to obtain ID1-type2, where "-" is the separator. Then, a hash calculation is performed on ID1-type2 to obtain the index data.

[0047] According to embodiments of this disclosure, when storing identifiers, the stored identifiers are in a standard format. However, the identifiers entered by users during queries may be in a non-standard format, causing users to be unable to find the required identifiers. Therefore, in order to be able to query the corresponding identifiers even when users enter non-standard format identifiers, non-standard format identifiers can be stored along with standard format identifiers.

[0048] Based on this, for example, multiple first identifier pairs can be obtained, where each first identifier pair includes an original identifier, an original identifier type, a mapped identifier, and a mapped identifier type. Then, for each of the multiple first identifier pairs, a target format is determined based on the original identifier type within that first identifier pair. The target format can be a format that a user might use when inputting an identifier, such as all lowercase, all uppercase, or half uppercase and half lowercase. The original identifier in the first identifier pair can then be converted to the target format to obtain a second identifier. The original identifier in the first identifier pair is then replaced with the second identifier to obtain the second identifier pair. Finally, both the first and second identifier pairs are determined as identifier pairs to be stored.

[0049] The following will combine Figure 3 The method for querying data provided in this disclosure is described.

[0050] Figure 3 A flowchart illustrating a method for querying data according to an embodiment of the present disclosure is shown schematically.

[0051] like Figure 3 As shown, the method 300 for querying data includes operation S310, obtaining a reference identifier and the identifier type to be queried.

[0052] According to embodiments of this disclosure, the reference identifier may be, for example, an identifier known to the user. The mapping identifier type may be, for example, the type of the target identifier that the user intends to find. There is a mapping relationship between the target identifier and the reference identifier.

[0053] In operation S320, target index data is generated based on the reference identifier and the identifier type to be queried.

[0054] In operation S330, candidate identifier data corresponding to the target index data is queried from the stored identifier data.

[0055] According to embodiments of this disclosure, the identification data is stored using the data storage method described above. Each piece of identification data includes an identification data set and index data. Each identification data set includes at least one pair of identification data.

[0056] In operation S340, determine whether the candidate identifier data contains multiple identifier pairs. If the identifier data contains only one identifier pair, proceed to operation S350. If the candidate identifier data contains multiple identifier pairs, proceed to operation S360.

[0057] According to embodiments of this disclosure, if the candidate identifier data contains only one identifier pair, that identifier pair can be used as the target identifier pair. If the candidate identifier data contains multiple identifier pairs, it indicates that there are identifier pairs with index data conflicts, and it is necessary to further determine the target identifier pair among these identifier pairs.

[0058] In operation S350, the identifier pair data is determined as the target identifier pair data. Then operation S370 is executed.

[0059] In operation S360, the target identifier pair data in multiple identifier pair data is determined based on the reference identifier and the identifier type to be queried.

[0060] According to embodiments of this disclosure, when candidate identifier data contains multiple identifier pairs, the target identifier pair among the multiple identifier pairs can be further determined based on the reference identifier and the identifier type to be queried.

[0061] In operation S370, the target identifier is output as the mapping identifier in the data.

[0062] The data query method according to embodiments of this disclosure can query a mapping identifier based on a reference identifier and the type of the identifier to be queried. When a user performs a data query, they only need to know the original identifier and the type of the mapping identifier to perform the query. Even if the user has no idea about the type of the original identifier, the desired mapping identifier can still be accurately output.

[0063] According to embodiments of this disclosure, for example, a reference identifier and a mapping identifier type can be combined to obtain a combined string. Then, a hash calculation is performed on the combined string identifier to obtain the target index data.

[0064] According to the embodiments of this disclosure, the method for generating target index data is the same as the method for generating index data when storing data, as detailed above, and will not be repeated here.

[0065] According to embodiments of this disclosure, if only one candidate identifier is identified, that candidate identifier can be determined as the target identifier. If multiple candidate identifiers are identified, it is necessary to further determine the target identifier from among the multiple candidate identifiers.

[0066] For example, in this embodiment, when there are multiple candidate identifiers, a machine learning model can be used to determine the target identifier type corresponding to the reference identifier. Then, the identifier pairs containing the multiple identifier pairs whose original identifier type matches the target identifier type are determined as the target identifier pairs.

[0067] The following will combine Figure 4 The method for training machine learning models provided in this disclosure is described.

[0068] Figure 4 The illustration schematically depicts a method for training a machine learning model according to an embodiment of the present disclosure.

[0069] like Figure 4 As shown, the method for training the machine learning model includes obtaining a sample identifier 41 and a standard identifier type 42 corresponding to the sample identifier 41. Then, the machine learning model 410 is used to determine the type prediction result 43 corresponding to the sample identifier. Next, based on the type prediction result 43 and the standard identifier type 42 corresponding to the sample identifier 41, a loss value 44 is determined. Then, the parameters of the machine learning model 410 are adjusted based on the loss value 44. The above steps are repeated until the loss value 44 converges.

[0070] According to embodiments of this disclosure, a loss value corresponding to the type prediction result and the standard identifier type can be determined based on a loss function. The loss function can be selected according to actual needs, and this disclosure does not impose specific limitations on the loss function.

[0071] According to embodiments of this disclosure, identifiers can be pre-collected. Features are extracted from these identifiers, and then a machine learning model is trained to predict the most suitable type of the identifier. For example, data extraction is first performed, selecting features such as identifier length, whether the identifier character contains ":", the number of identifier characters containing ":", whether the identifier character contains "|", the number of identifier characters containing "|", whether the identifier character contains "-", the number of identifier characters containing "-", whether the identifier is purely numeric, and whether the identifier is half uppercase and half lowercase as the original dataset. In the embodiments, the concept of yes or no can be represented by the numbers 1 and 0, where 1 can represent yes and 0 can represent no. The identifier type is used as the classification label, and the data is processed into a format that the model can process. Then, a training set and a test set are split from the original dataset, with a ratio of 7:3. The training set is used for model training, and the test set is used for model evaluation.

[0072] According to embodiments of this disclosure, features of a reference identifier can be extracted, and then these features can be input into a machine learning model to obtain the probability that the reference identifier belongs to each identifier type. The identifier type with the highest probability is determined as the target identifier type corresponding to the reference identifier.

[0073] According to embodiments of this disclosure, machine learning models may include, for example, decision tree models, logistic regression models, and Bayesian classification models.

[0074] The following will combine Figure 5 The method for training machine learning models provided in this disclosure is described.

[0075] Figure 5 The illustration schematically depicts a method for determining a target identifier pair in a plurality of identifier pairs according to embodiments of the present disclosure.

[0076] like Figure 5 As shown, exemplarily, in this embodiment, the machine learning model may include three models: a decision tree model 510, a logistic regression model 520, and a Bayesian classification model 530. The original label 50 can be input into the decision tree model 510 to obtain a first prediction result 51. The original label 50 can be input into the logistic regression model 520 to obtain a second prediction result 52. The original label 50 can be input into the Bayesian classification model 530 to obtain a third prediction result 53. Then, based on the first prediction result 51, the second prediction result 52, and the third prediction result 53, the target label type 54 is determined.

[0077] According to embodiments of this disclosure, if the first prediction result, the second prediction result, and the third prediction result all indicate the same identifier type, then that identifier type will be determined as the target identifier type. If the identifier types indicated by the first prediction result, the second prediction result, and the third prediction result are different, then according to voting rules, the identifier type that accounts for the majority of the identifier types indicated by the first prediction result, the second prediction result, and the third prediction result will be determined as the target identifier type. For example, if the first prediction result indicates identifier type A, and the second and third prediction results indicate type B, then B can be determined as the target identifier type.

[0078] According to another embodiment of this disclosure, erroneous characters in the reference identifier can also be identified, for example. Correcting these erroneous characters can improve query accuracy.

[0079] For example, the identifier entered by the user might be non-standard. For instance, the suffix for type A identifier should be "|0", but the user enters "|O". Based on this, the incorrect suffix "|O" in the user-entered identifier can be identified and corrected to "|0". Then, the corrected identifier can be queried in the database, thereby improving query accuracy.

[0080] The following is for reference. Figures 6-7 The methods for storing and querying data described above will be further explained in conjunction with specific embodiments. Those skilled in the art will understand that the following example embodiments are for illustrative purposes only and are not limited thereto.

[0081] Figure 6 A schematic diagram illustrating a method for storing data according to another embodiment of the present disclosure is shown.

[0082] exist Figure 6 The diagram shows that in operation S601, multiple identifier pairs of data are acquired.

[0083] According to embodiments of this disclosure, for example, an identifier pair table can be obtained, wherein the identifier pair table includes multiple identifier pair data. Identifier pair data can be extracted from the identifier pair table.

[0084] According to embodiments of this disclosure, the identifier pair data includes the original identifier, the mapping identifier, the mapping identifier type (mapping_type), and the confidence level, etc.

[0085] In operation S602, the identifier pair data is parsed and loaded.

[0086] According to embodiments of this disclosure, for example, the identifier pair data can be loaded as an IdRecord. The IdRecord can be a predefined protobuf, including fields such as the original identifier, the mapped identifier, the mapped identifier type, and the confidence level.

[0087] In operation S603, index data is generated based on the original identifier and the mapped identifier type.

[0088] According to embodiments of this disclosure, for example, a hash calculation can be performed on the original identifier and the mapped identifier type to generate an index data key. Therefore, regardless of whether the user enters the wrong identifier type, or leaves it blank or enters random information, the identifier the user wants to find can still be retrieved.

[0089] In operation S604, a table is generated based on the index data and the loaded identifier pairs.

[0090] According to embodiments of this disclosure, to ensure confidentiality, IdRecord can be encrypted, for example, using the base64 encryption method base64encode() to encrypt IdRecord, resulting in base64encode(IdRecord). A table can then be generated based on the key and base64encode(IdRecord).

[0091] In operation S605, the identifier pairs corresponding to the same index data are divided into an identifier data set.

[0092] According to embodiments of this disclosure, for example, the value in the table, i.e., base64encode(IdRecord), can be obtained, and base64encode(IdRecord) can be decrypted to obtain IdRecord. Then, IdRecords corresponding to the same key are loaded into a MoonRecord, where MoonRecord is a predefined protobuf used to store IdRecords. Identifier data in protobuf format is generated based on the key and MoonRecord and stored at an intermediate path location.

[0093] When operating S606, store identification data.

[0094] According to embodiments of this disclosure, for example, the generated protobuf format identifier data can be converted into simpleDB format data and then stored in the database at the official path location, which is the path location that users can find.

[0095] Figure 7 A schematic diagram illustrating a method for querying data according to another embodiment of this disclosure is shown.

[0096] exist Figure 7 The diagram shows that in operation S701, the reference identifier and the identifier type to be queried are obtained.

[0097] According to embodiments of this disclosure, users can input a typeless reference identifier and the type of identifier they wish to query.

[0098] In operation S702, target index data is generated based on the reference identifier and the identifier type to be queried.

[0099] In operation S703, candidate identifier data corresponding to the target index data is queried from the database.

[0100] According to embodiments of this disclosure, a key can be generated based on a reference identifier and the type of the identifier to be queried, using the same key generation method as in the data storage stage. The database is then queried based on this key.

[0101] In operation S704, determine whether candidate identifier data is found. If found, proceed to operation S705; otherwise, proceed to operation S706.

[0102] According to embodiments of this disclosure, if the index data in a certain data entry in the database is the same as the key, it indicates that the data contains relevant information about the reference identifier and the identifier to be queried input by the user, and thus the data is determined to be candidate identifier data.

[0103] In operation S705, the candidate identifier data is traversed in order to determine the identifier to be queried.

[0104] According to embodiments of this disclosure, the candidate identifier data can be traversed until data of the same type as the identifier to be queried is found, and the output result is returned.

[0105] When operating S706, a prompt message is returned.

[0106] According to embodiments of this disclosure, if no record in the database contains the key, a message such as "not found" can be returned, indicating that no relevant record was found.

[0107] The following will combine Figure 8 The apparatus for storing data provided in this disclosure will be described.

[0108] Figure 8 A block diagram of a device for storing data according to an embodiment of the present disclosure is shown schematically.

[0109] like Figure 8 As shown, the data storage device 800 includes a first acquisition module 810, a first generation module 820, a partitioning module 830, a first determination module 840, and a storage module 850.

[0110] The first acquisition module 810 is used to acquire multiple identifier pairs to be stored, wherein the identifier pairs to be stored include the original identifier, the mapped identifier, and the mapped identifier type.

[0111] The first generation module 820 is used to generate index data based on the original identifier and mapping identifier type of each identifier to be stored in the data.

[0112] The partitioning module 830 is used to partition multiple pairs of identifier pairs that correspond to the same index data into an identifier data set, thereby obtaining at least one identifier data set.

[0113] The first determining module 840 is used to determine at least one identifier data based on at least one identifier data set and index data corresponding to at least one identifier data set.

[0114] Storage module 850 is used to store at least one identification data.

[0115] The following will combine Figure 9 The apparatus for querying data provided in this disclosure is described.

[0116] Figure 9 A block diagram of a device for querying data according to an embodiment of the present disclosure is shown schematically.

[0117] like Figure 9 As shown, the data query device 900 includes a second acquisition module 910, a second generation module 920, a query module 930, a second determination module 940, a third determination module 950, and an output module 960.

[0118] The second acquisition module 910 is used to acquire the reference identifier and the identifier type to be queried.

[0119] The second generation module 920 is used to generate target index data based on the reference identifier and the identifier type to be queried.

[0120] The query module 930 is used to query candidate identifier data corresponding to the target index data from the stored identifier data, wherein the identifier data is stored in accordance with the method of any one of claims 1-3.

[0121] The second determining module 940 is used to determine the identifier pair data as the target identifier pair data when the candidate identifier data includes an identifier pair data.

[0122] The third determining module 950 is used to determine the target identifier pair data among the multiple identifier pairs data when the candidate identifier data includes multiple identifier pairs data, based on the reference identifier and the identifier type to be queried.

[0123] Output module 960 is used to output the target identifier mapping identifier in the data.

[0124] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0125] Figure 10A block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is illustrated schematically. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0126] like Figure 10 As shown, device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1002 or a computer program loaded from storage unit 1008 into random access memory (RAM) 1003. The RAM 1003 may also store various programs and data required for the operation of device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0127] Multiple components in device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of monitors, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0128] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as methods for storing data and methods for retrieving data. For example, in some embodiments, the methods for storing data and methods for retrieving data may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the methods for storing data and methods for retrieving data described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured by any other suitable means (e.g., by means of firmware) to perform methods for storing data and methods for querying data.

[0129] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0130] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0131] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0132] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0133] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0134] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0135] A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system. It solves the shortcomings of traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS"), such as high management difficulty and weak business scalability. A server can also be a server for a distributed system, or a server that incorporates blockchain technology.

[0136] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0137] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for storing data, comprising: Acquire multiple pairs of identifiers to be stored, wherein the pairs of identifiers to be stored include original identifiers, mapping identifiers, and mapping identifier types, wherein the mapping identifier type is the identifier type of the mapping identifier, and there is a mapping relationship between the original identifier and the mapping identifier; Without needing to obtain the original identifier type, index data is generated based on the original identifier and mapped identifier type of each identifier to be stored in the multiple identifier pairs data, wherein the original identifier type is the identifier type of the original identifier; Divide the multiple pairs of identifiers to be stored that correspond to the same index data into an identifier data set to obtain at least one identifier data set. Based on the at least one set of identifier data and the index data corresponding to the at least one set of identifier data, at least one identifier data is determined; and Store the at least one identification data.

2. The method according to claim 1, wherein, The step of generating index data based on the original identifier and mapped identifier type of each identifier to be stored in the multiple identifier pairs without needing to obtain the original identifier type includes: For each pair of identifiers to be stored, The original identifier and the mapped identifier type in the identifier to be stored data are combined to obtain a combined string; and The combined string identifier is hashed to obtain the index data.

3. The method according to claim 1 or 2, wherein, The process of acquiring multiple pairs of identifiers to be stored includes: Obtain multiple first identifier pairs, wherein the first identifier pair data includes the original identifier, the original identifier type, the mapping identifier, and the mapping identifier type; For each of the plurality of first identifier pairs, Determine the target format based on the original identifier type in the data according to the first identifier; The original identifier in the first identifier pair data is converted into the target format to obtain the second identifier; Replace the original identifier in the first identifier pair data with the second identifier to obtain the second identifier pair data; and The first identifier pair data and the second identifier pair data are determined as the identifier pair data to be stored.

4. A method for querying data, comprising: Obtain the reference identifier and the identifier type to be queried; Based on the reference identifier and the identifier type to be queried, generate target index data; Query candidate identifier data corresponding to the target index data from the stored identifier data, wherein the identifier data is stored using the method according to any one of claims 1-3; If the candidate identifier data includes an identifier pair, then that identifier pair is determined as the target identifier pair. When the candidate identifier data includes multiple identifier pairs, the target identifier pair is determined from the multiple identifier pairs based on the reference identifier and the identifier type to be queried; and Output the target identifier mapping identifier in the data.

5. The method according to claim 4, wherein, The step of generating target index data based on the reference identifier and the identifier type to be queried includes: The reference identifier and the identifier type to be queried are combined to obtain a combined string; and The combined string identifier is hashed to obtain the target index data.

6. The method according to claim 4, wherein, Each of the aforementioned identifier pairs includes an original identifier, an original identifier type, a mapped identifier, and a mapped identifier type; The step of determining the target identifier pair data among the multiple identifier pair data based on the reference identifier and the identifier type to be queried includes: Using a machine learning model, the target identifier type corresponding to the reference identifier is determined; as well as Identify the target identifier pair data as the data containing the multiple identifier pairs whose original identifier type matches the target identifier type.

7. The method according to claim 6, wherein, The machine learning model is trained based on the following operations: Obtain the sample identifier and the standard identifier type corresponding to the sample identifier; The machine learning model is used to determine the type prediction result corresponding to the sample identifier; The loss value is determined based on the type prediction result and the standard identifier type corresponding to the sample identifier; as well as The parameters of the machine learning model are adjusted based on the loss value.

8. The method of claim 7, wherein the machine learning model comprises: Decision tree models, logistic regression models, and Bayesian classification models, wherein the use of machine learning models to determine the target identifier type corresponding to the reference identifier includes: The original identifiers are input into the decision tree model to obtain the first prediction result; The original identifiers are input into the logistic regression model to obtain the second prediction result; The original identifiers are input into the Bayesian classification model to obtain the third prediction result; and The target identifier type is determined based on the first prediction result, the second prediction result, and the third prediction result.

9. The method according to claim 4, further comprising: Identify erroneous characters in the reference identifier; as well as The erroneous characters are corrected.

10. A device for storing data, comprising: The first acquisition module is used to acquire multiple identifier pairs to be stored, wherein the identifier pairs to be stored include original identifiers, mapping identifiers, and mapping identifier types, the mapping identifier type is the identifier type of the mapping identifier, and there is a mapping relationship between the original identifier and the mapping identifier; The first generation module is used to generate index data based on the original identifier and mapping identifier type of each identifier pair to be stored in the plurality of identifier pairs to be stored data without needing to obtain the original identifier type; wherein the original identifier type is the identifier type of the original identifier. The partitioning module is used to divide the multiple pairs of identifiers to be stored that correspond to the same index data into an identifier data set, thereby obtaining at least one identifier data set. The first determining module is configured to determine at least one identifier data based on the at least one identifier data set and index data corresponding to the at least one identifier data set; and A storage module for storing the at least one identification data.

11. An apparatus for querying data, comprising: The second acquisition module is used to acquire the reference identifier and the identifier type to be queried; The second generation module is used to generate target index data based on the reference identifier and the identifier type to be queried; A query module is used to query candidate identifier data corresponding to the target index data from the stored identifier data, wherein the identifier data is stored using the method according to any one of claims 1-3; The second determining module is used to determine the identifier pair data as the target identifier pair data when the candidate identifier data includes an identifier pair data; The third determining module is configured to, when the candidate identifier data includes multiple identifier pairs, determine the target identifier pair among the multiple identifier pairs based on the reference identifier and the identifier type to be queried; and The output module is used to output the mapping identifier in the target identifier pair data.

12. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-9.

14. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1-9.

Images