A big data label storage method, analysis method and system

Abstract

The application relates to a big data label storage method, an analysis method and a system. The storage method comprises the following steps: obtaining a target label, wherein the target label comprises a target object and a label name; matching a target subject database from preset different specific subject databases according to the target object of the target label; judging whether the label name of the target label exists in the target subject database; and if not, storing the target label in the target subject database according to a set data storage structure. The application can realize quick classification storage of big data labels, improve data storage speed, and reduce data congestion.

Description

Technical Field

[0001] This application relates to the field of big data storage and analysis, and in particular to a big data tag storage method, analysis method and system. Background Technology

[0002] In our daily work, we need to use data tags with subject-specific characteristics to quickly search and analyze tag data for subjects such as people, objects, vehicles, and cases.

[0003] Currently, using relational databases to store tag data presents several challenges: the volume of target tag data has reached hundreds of millions, and relational databases cannot guarantee the speed of insertion, updates, and queries, failing to meet the demands of frequent, rapid updates and millisecond-level retrieval in tag data production. Furthermore, each subject's data tags are unique, and the data tag columns are dynamic. Using relational databases would only allow for an exhaustive search of all columns, resulting in a table with an excessive number of fields.

[0004] As the target business expands, the number of data tags continues to increase. Because relational database columns cannot be dynamically expanded, the only solution is to modify the original table structure and add new tag columns, resulting in high database maintenance costs. Searching for tags requires obtaining the tag's source information and the frequency of its occurrence for tag tracing. The existing tag data storage structure cannot accommodate adding new columns to store this information, preventing the tag tracing requirement from being fulfilled. Since many tag columns in a single data entry contain NULL values, and tag searches involve pre-fuzzy matching, the relational database index becomes ineffective during data retrieval, significantly slowing down search speeds. Because each data tag is a separate column, keyword searches require traversing hundreds of tag columns, making fast full-text searches impossible. Summary of the Invention

[0005] In view of this, this application provides a big data tag storage method, analysis method and system to solve the technical problem that existing databases store big data tags too slowly or inefficiently, which leads to obstacles in query business.

[0006] To address the aforementioned problems, firstly, this application provides a method for storing large data tags, the method comprising:

[0007] Obtain the target tag, which includes the target object and the tag name;

[0008] Based on the target object of the target tag, the target subject database is matched from different preset specific subject databases;

[0009] judging whether the label name of the target label exists in the target subject database, and if not, storing the target label in the target subject database according to a set data storage structure.

[0010] Optionally, the target label further comprises a label source; and the storing the target label in the target subject database according to the set data storage structure comprises:

[0011] storing the target object and the label name of the target label in an array form;

[0012] setting a label code for the label name of the target label;

[0013] creating a label field and a source field for the label code of the target label, and embedding the source field in the label field;

[0014] storing the label name and the label code of the target label in the label field, and storing the label source of the target label in the source field.

[0015] Optionally, the storing the target label in the target subject database according to the set data storage structure further comprises:

[0016] setting a retrieval mode, comprising: when the label name and the label code of the target label are retrieved simultaneously, adopting a segmented word index mode; and when the label name or the label code of the target label is retrieved individually, adopting a non-segmented word index mode;

[0017] setting the occurrence number of the label source of the target label to 1;

[0018] storing the occurrence number of the label source of the target label in the source field.

[0019] Optionally, after the storing the target label in the target subject database according to the set data storage structure, the method further comprises:

[0020] if a first deletion instruction about the target label is obtained, the first deletion instruction comprising a label source to be deleted, deleting the label source to be deleted corresponding to the target label from the target subject database;

[0021] judging whether the target label has other label sources after the deletion, if yes, retaining the label code and the label name of the target label, and if not, deleting the label code and the label name of the target label;

[0022] Optionally, after the storing the target label in the target subject database according to the set data storage structure, the method further comprises:

[0023] If a second deletion instruction about the target label is acquired, the second deletion instruction includes a label code to be deleted, and the target label corresponding label code to be deleted, the label name and the label source are deleted from the target subject database.

[0024] In a second aspect, the present application provides a big data label analysis method, comprising:

[0025] The big data label storage method is executed.

[0026] Acquiring data to be queried, the data to be queried including a plurality of label names and dimension parameters corresponding to a same object to be queried;

[0027] Determining a subject database to be queried corresponding to the object to be queried;

[0028] According to the plurality of label names and dimension parameters to be queried, a data analysis script is constructed;

[0029] Using the data analysis script to query in the subject database to be queried, a data analysis result is acquired.

[0030] Optionally, according to the plurality of label names and dimension parameters corresponding to the object to be queried, the data analysis script is constructed, comprising:

[0031] According to the hit requirements of the plurality of label names to be queried, a query logical relationship of the plurality of label names to be queried is determined;

[0032] According to the inner and outer logical order of the subject database to be queried, a data query syntax is constructed;

[0033] According to the dimension parameters, an aggregation syntax is constructed;

[0034] According to the query logical relationship of the plurality of label names to be queried, the data query syntax, the aggregation syntax and a preset calculation algorithm, the data analysis script is constructed.

[0035] Optionally, according to the hit requirements of the plurality of label names to be queried, the query logical relationship of the label names to be queried is determined, wherein the query logical relationship at least includes and and or logical relationship.

[0036] In a third aspect, the present application provides a big data label storage system, comprising:

[0037] An acquisition module is configured to acquire a target label, the target label including a target object and a label name;

[0038] A matching module is configured to match a target subject database from a plurality of different specific subject databases according to the target object of the target label;

[0039] a storage module configured to determine whether the label name of the target label exists in the target subject database, and if not, store the target label in the target subject database according to a set data storage structure.

[0040] In a fourth aspect, the present application provides a big data label analysis system, which comprises:

[0041] a data acquisition module configured to acquire to-be-queried data, the to-be-queried data comprising a plurality of to-be-queried label names and dimension parameters corresponding to a same to-be-queried object;

[0042] a database determination module configured to determine a to-be-queried subject database corresponding to the to-be-queried object;

[0043] a script construction module configured to construct a data analysis script according to the plurality of to-be-queried label names and dimension parameters;

[0044] an analysis module configured to query in the to-be-queried subject database by using the data analysis script to acquire a data analysis result.

[0045] The beneficial effects of the above embodiment are that the embodiment acquires a target label, and according to the label object of the target label, a target subject database can be matched, so as to facilitate storage of the target label into the target subject database; it is determined whether the label name of the target label exists in the target subject database, and if not, the target label is added to the target subject database according to a set storage structure, so as to store the target label into the corresponding target subject database, realize quick classification storage of big data labels, improve data storage speed, and reduce data congestion. BRIEF DESCRIPTION OF DRAWINGS

[0046] Fig. 1 is a method flowchart of an embodiment of a big data label storage method provided by the present application;

[0047] Figure 2 Fig. 2 is a method flowchart of an embodiment of step S103 of the big data label storage method provided by the present application;

[0048] Figure 3 Fig. 3 is a script schematic diagram of a data storage structure provided by the present application;

[0049] Figure 4 Fig. 4 is a method flowchart of another embodiment of step S103 of the big data label storage method provided by the present application;

[0050] Figure 5 Fig. 5 is a method flowchart of an embodiment of a big data label analysis method provided by the present application;

[0051] Figure 6A method flowchart of an embodiment of the big data tag analysis method provided in the present application is provided in step S503;

[0052] Figure 7 A data analysis script schematic diagram provided in the present application is provided in step S504;

[0053] Figure 8 A data analysis method flowchart provided in the present application is provided in step S505;

[0054] Figure 9 A principle block diagram of an embodiment of the big data tag storage system provided in the present application is provided in step S506;

[0055] Figure 10 A principle block diagram of an embodiment of the big data tag analysis system provided in the present application is provided in step S507. DETAILED DESCRIPTION

[0056] The preferred embodiments of the present application will be described in detail below with reference to the accompanying drawings, which form a part of this application. The accompanying drawings, together with the description, illustrate the principles of the application and, although not to be limited thereto, serve to explain the application.

[0057] In the description of the present application, the meaning of “a plurality of” is two or more, unless otherwise explicitly and specifically limited.

[0058] Reference to “an embodiment” herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase “in an embodiment” in various places in the specification are not necessarily referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is expressly understood that any of the embodiments described herein can be combined with any of the other embodiments.

[0059] Reference Figure 1 A method flowchart of an embodiment of the big data tag storage method provided in the present application is provided in step S501, and the big data tag storage method comprises the following steps:

[0060] S101, a target tag is acquired, the target tag comprising a target object and a tag name;

[0061] S102, according to the target object of the target tag, a target subject database is matched out from a pre-set different specific subject database;

[0062] S103, it is judged whether the tag name of the target tag exists in the target subject database, and if not, the target tag is stored in the target subject database according to a set data storage structure.

[0063] In the embodiment, the specific subject database is created by using an ElasticSearch database, different types of specific subject databases include a personnel information database, a vehicle information database, a case information database, an article information database, and the like; the target object refers to a data subject labeled, such as a person, a vehicle, a case, and the like; the label name refers to a label name for the target object, for example, for a certain person, the label name may be an involved person, a police-involved person, and an individual business, and the like; in the embodiment, a target label includes a target object, a label name, and a corresponding label source. In the embodiment, the ElasticSearch database is used to store data, which can solve the storage of an order of magnitude of data, and the storage by using a shard can be horizontally expanded, so that even if the data volume is large, the storage is also stress-free. A shard is an instance of Lucene, when searching data, each shard is searched and the results are summarized, so as to improve the search efficiency. When a large amount of data is written, the search speed can also be close to real-time, and reading and writing do not affect each other.

[0064] In the embodiment, the target label is obtained, according to the label object of the target label, the target subject database can be matched, so as to facilitate storage of the target label in the target subject database; it is judged whether the label name of the target label exists in the target subject database, if not, the target label is added to the target subject database according to the set storage structure, so as to store the target label in the corresponding target subject database, realize the rapid classification storage of big data labels, improve the data storage speed, and reduce data congestion.

[0065] In the embodiment, the data storage structure is created by using a table structure, an array, a field, a segmented word, and / or a non-segmented word retrieval mode; referring to Figure 2 , in step S103, the target label is stored in the target subject database according to the set data storage structure, including:

[0066] S201, the target object and the label name of the target label are stored by using an array form;

[0067] S202, the label name of the target label is set with a label code;

[0068] S203, the label field and the source field are created for the label code of the target label, and the source field is embedded in the label field;

[0069] S204, the label name and the label code of the target label are stored in the label field, and the label source of the target label is stored in the source field.

[0070] In a specific embodiment, referring to Figure 3The data storage structure script is shown, and one target object can have multiple target tags, all of which are stored in a tags field, which is an array, and each array element is a target tag, which can be stored by using a tag field and a source field, so that the index rule configured when the field is created does not need to be modified subsequently, and the maintenance cost of the table structure is greatly reduced.

[0071] Referring to Figure 4 In step S103, the target tag is stored in the target subject database according to the set data storage structure, and the method further includes:

[0072] S401, setting a retrieval mode, including: when the tag name and the tag code of the target tag are retrieved at the same time, using a segmented index mode; when the tag name or the tag code of the target tag is retrieved alone, using a non-segmented index mode;

[0073] S402, setting the number of occurrences of the tag source of the target tag to 1;

[0074] S403, storing the number of occurrences of the tag source of the target tag in the source field.

[0075] When storing the target tag, both the tag code and the tag name of the target tag are stored, the tag code is used as the basis field for insertion update, a non-segmented retrieval is used, and the ability of accurate search of the data tag and the data analysis parameter is provided.

[0076] The tag name and the tag code are stored together, and are set to general segmentation, so as to realize full-text search of the target tag. According to the user's search habit, the full-text search also conforms to the user's search habit, and the response speed of the full-text search is effectively improved, which only takes a few milliseconds.

[0077] In addition, the source field is embedded in the tag field, so that each target tag in each piece of data can find the corresponding source. The source of the same target tag can be multiple, so when storing the target tag, two sources need to be stored, and therefore the source information is also stored by using an array structure. The same tag of the same target object can appear multiple times, and the number of times is different according to the source of the target tag, so the number of times information is placed inside the tag source information.

[0078] In the embodiment, when the target tag is updated in the target subject database, a Painless script provided by ElasticSearch is used to realize the insertion update logic. The insertion update of the tag data in the Elasticsearch database is faster than that of the relational database, and several million tag data insertion updates can be completed in tens of minutes, and the performance of the search is not affected during the insertion update process, and the data search is almost real-time.

[0079] It should be noted that, because the data storage structure for the target label is in a nested format, the number of occurrences of the label needs to be calculated when storing. If the target object does not have the label name of the current output, the target object is directly updated to add new label information and corresponding source information, and the number of occurrences of the label is recorded as 1. If the data subject already has the label name of the current output, it is further determined whether there is a source of the current label name. If there is no same source information, a new source is added, and the number of occurrences of the label is recorded as 1. If the data subject already has the source information of the label name of the current output, the number of occurrences of the original source is increased by 1.

[0080] Optionally, the big data label storage method of the embodiment further comprises:

[0081] If a first deletion instruction about the target label is obtained, the first deletion instruction includes a label source to be deleted, the target label corresponding to the label source to be deleted is deleted from the target subject database;

[0082] It is determined whether the target label has other label sources after deletion. If yes, the label code and the label name of the target label are retained. If no, the label code and the label name of the target label are deleted.

[0083] Optionally, if a second deletion instruction about the target label is obtained, the second deletion instruction includes a label code to be deleted, the target label corresponding to the label code to be deleted, the label name and the label source are deleted from the target subject database.

[0084] In the embodiment, the label data clearing is also an update of the label field, and the Painless script is used to clear the label data.

[0085] It should be noted that the advantage of the label clearing method compared with the Update By Query method in Elasticsearch is that: the Update By Query method is similar to the "set…where…" statement of the relational database, and a large amount of data update will certainly time out to cause the update to be stopped. When the Update By Query is queried, snapshot information is obtained. When a plurality of label clearing operations are executed in parallel, version conflicts occur to cause the data update to fail, and the query failure also causes the data update to be stopped. The label clearing logic of the embodiment separates the query and the update, first queries the data primary key set of the label to be cleared according to the label code and the rule ID, and can be completed within a few hundred milliseconds. After the primary key set of the label to be cleared is obtained, the script for clearing the label is executed, which can support the rapid clearing of several ten million label data and is not easy to cause errors to stop in the middle, thereby avoiding the defects of the Update By Query.

[0086] Referring to Figure 5 The embodiment also discloses a big data label analysis method and a big data label storage method.

[0087] S501, obtaining to-be-queried data, the to-be-queried data including a plurality of to-be-queried label names and dimension parameters corresponding to a same to-be-queried object;

[0088] S502, determining a to-be-queried subject database corresponding to the to-be-queried object;

[0089] S503, constructing a data analysis script according to the plurality of to-be-queried label names and the dimension parameters;

[0090] S504, querying in the to-be-queried subject database by using the data analysis script to obtain a data analysis result.

[0091] In the embodiment, based on the data storage structure of the target label, label data analysis is much simpler. Regardless of the number of data analysis parameters or the combination of some label names, only the tag_code field of the label needs to be queried, the overall structure of the data analysis script does not need to be changed, and only different parameter values need to be transmitted.

[0092] In an embodiment, referring to Figure 6 In step S503, the data analysis script is constructed according to the plurality of to-be-queried label names and the dimension parameters, including:

[0093] S601, determining a query logical relationship of the plurality of to-be-queried label names according to a hit requirement of the plurality of to-be-queried label names; wherein the query logical relationship at least includes and and or logical relationships;

[0094] S602, constructing a data query syntax according to an inner and outer layer logical order of the to-be-queried subject database;

[0095] S603, constructing an aggregation syntax according to the dimension parameters;

[0096] S604, constructing the data analysis script according to the query logical relationship of the plurality of to-be-queried label names, the data query syntax, the aggregation syntax, and a preset calculation algorithm.

[0097] Referring to Figure 7 The script can analyze hundreds of millions of data labels and obtain an analysis result in a few hundred milliseconds, and the analysis speed is improved by several hundred times compared with using a relational database.

[0098] In a specific embodiment, the data analysis method is as shown in Figure 8As shown, the key of the data analysis method is to dynamically construct the logical relationship of the data analysis parameters and the dimension of the analysis. For example, to analyze the occupation distribution of different case types of personnel in the epidemic, a distribution graph of the infected population is obtained, and according to the infection rate of different personnel, it is analyzed which personnel are high-risk infected personnel.

[0099] In this example, there are five labels as the query tag names: "confirmed cases", "suspected cases", "positive detection", "other cases", and "close contact personnel". These five tag names need to be accurately hit, so the tag_code non-tokenized search should be used, and the logical relationship is set to "or" to take the union of the query conditions. The dimension parameter is the occupation tag_code, which is grouped and counted for more than a dozen types of occupations to obtain the number of each type of population that meets the query condition, and then the proportion is calculated.

[0100] Compared with the prior art, the embodiment optimizes the data storage structure, greatly reduces the maintenance cost of the data table, improves the writing performance and search performance of the target label, realizes full-text search and rapid data analysis of the target label, makes up for the lack of label source information record, provides a unified label data analysis method, saves a lot of development work, reduces the work cost, and meets the user's demand for full-text search, millisecond-level response, data analysis and traceability in the use of label name.

[0101] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0102] The embodiment also provides a big data label storage system, which corresponds to the big data label storage method in the above embodiment. As shown in the figure, Figure 9 The big data label storage system includes a data acquisition module 901, a matching module 902, and a storage module 903. The functions of each functional module are described as follows:

[0103] The data acquisition module 901 is used to acquire data, and is used to acquire a target label, the target label including a target object and a label name;

[0104] The matching module 902 is used to match a target subject database from a pre-set different specific subject database according to the target object of the target label;

[0105] The storage module 903 is used to determine whether the label name of the target label exists in the target subject database, and if not, the target label is stored in the target subject database according to the set data storage structure.

[0106] The specific definitions of the modules of the big data tag storage system can refer to the definitions of the big data tag storage method, which will not be repeated here. The modules of the big data tag storage system described above can be implemented by software, hardware, and combinations thereof, in whole or in part. The modules described above can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the above modules.

[0107] The embodiment also provides a big data tag analysis system, which corresponds to the big data tag analysis method in the above embodiment. As shown in the figure, the big data tag analysis system comprises an acquisition data module 1001, a database determination module 1002, a script construction module 1003, and an analysis module 1004. The detailed description of each functional module is as follows: Figure 10

[0108] The acquisition data module 1001 is configured to acquire to-be-queried data, wherein the to-be-queried data comprises a plurality of to-be-queried tag names and dimension parameters corresponding to a same to-be-queried object.

[0109] The database determination module 1002 is configured to determine a to-be-queried subject database corresponding to the to-be-queried object.

[0110] The script construction module 1003 is configured to construct a data analysis script according to the plurality of to-be-queried tag names and the dimension parameters.

[0111] The analysis module 1004 is configured to query in the to-be-queried subject database by using the data analysis script to acquire a data analysis result.

[0112] The specific definitions of the modules of the big data tag analysis system can refer to the definitions of the big data tag analysis method, which will not be repeated here. The modules of the big data tag analysis system described above can be implemented by software, hardware, and combinations thereof, in whole or in part. The modules described above can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the above modules.

[0113] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments.

[0114] ​Any reference to storage, memory, database or other medium herein includes non-volatile and / or volatile storage. Non-volatile storage can include read-only memory (ROM), programmable ROM (PROM), electronically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile storage can include random-access memory (RAM), or external cache memory. By way of illustration, and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM). The disclosure herein can be used in connection with any of the above or other types of RAM.

[0115] The foregoing is considered as illustrative only of the principles of the application. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the application to the exact construction and practice described. Accordingly, all such variations are intended to be included within the scope of the present application as defined in the claims below and their equivalents.

Claims

1. A method for storing big data tags, characterized in that, The method includes: Obtain the target tag, which includes the target object, tag name, tag source, and source occurrence count; each tag name corresponds to at least one tag source, and each tag source corresponds to one source occurrence count. Based on the target object of the target tag, a target subject database is matched from different preset specific subject databases; the specific subject database is created using an ElasticSearch database. Determine whether the tag name of the target tag exists in the target entity database. If not, store the target tag in the target entity database according to the set data storage structure. When updating the tag information of the target object in the target subject database, obtain the latest tag name of the currently input target object and the latest tag source corresponding to the latest tag name; If the target object does not have the latest tag name, then add the latest tag name and the latest tag source, and record the occurrence count of the latest tag source as 1; If the target tag contains the latest tag name but not the latest tag source, then the latest tag source is added, and the number of times the latest tag source appears is recorded as 1; If the target tag contains the latest tag name and the latest tag source, then increment the occurrence count of the latest tag source by 1; The search modes can be set as follows: when the tag name and tag code of the target tag are searched simultaneously, the word segmentation index mode is used; when the tag code of the target tag is searched alone, the non-word segmentation index mode is used. The process of storing the target tag in the target subject database according to the set data storage structure includes: Store the target object and tag name of the target tag in array form; Set the tag code for the tag name of the target tag; Create a tag field and a source field for the tag encoding of the target tag, wherein the source field is embedded within the tag field; The tag name and tag code of the target tag are stored in the tag field, and the tag source of the target tag is stored in the source field; The step of storing the target tag in the target subject database according to the set data storage structure further includes: Store the number of times the source of the target tag appears in the source field.

2. The big data tag storage method according to claim 1, characterized in that, After storing the target tag in the target subject database according to the set data storage structure, the method further includes: If a first deletion instruction for the target tag is obtained, and the first deletion instruction includes the source of the tag to be deleted, then the source of the tag to be deleted corresponding to the target tag is deleted from the target subject database; Determine whether the target tag has other tag sources after deletion. If so, retain the tag code and tag name of the target tag; otherwise, delete the tag code and tag name of the target tag.

3. The big data tag storage method according to claim 1, characterized in that, After storing the target tag in the target subject database according to the set data storage structure, the method further includes: If a second deletion instruction for the target tag is obtained, and the second deletion instruction includes the tag code to be deleted, then the tag code to be deleted, the tag name, and the tag source corresponding to the target tag are deleted from the target subject database.

4. A big data tag analysis method, characterized in that, include: Perform the big data tag storage method according to any one of claims 1-3; Obtain the data to be queried, which includes multiple tag names and dimension parameters corresponding to the same object to be queried; Determine the database of the entity to be queried corresponding to the object to be queried; Construct a data analysis script based on the multiple tag names and dimension parameters to be queried; The data analysis script is used to query the database of the subject to be queried, and the data analysis results are obtained.

5. The big data tag analysis method according to claim 4, characterized in that, Based on the multiple query tag names and dimension parameters corresponding to the query object, a data analysis script is constructed, including: Based on the hit requirements of multiple tag names to be queried, determine the query logic relationship of the multiple tag names to be queried; Construct the data query syntax based on the inner and outer logical order of the database to be queried; Based on the dimension parameters, construct the aggregation syntax; A data analysis script is constructed based on the query logic relationship of multiple tag names to be queried, the data query syntax, the aggregation syntax, and the preset statistical algorithm.

6. The big data tag analysis method according to claim 5, characterized in that, The step involves determining the query logic relationship of the tag names to be queried based on the hit requirements of multiple tag names to be queried, wherein the query logic relationship includes at least the AND and OR logical relationships.

7. A big data tag storage system, characterized in that, The system includes: The data acquisition module is used to acquire target tags, which include target objects, tag names, tag sources, and the number of times the source appears; each tag name corresponds to at least one tag source, and each tag source corresponds to one number of times the source appears. The matching module is used to match the target subject database from different preset specific subject databases based on the target object of the target tag; the specific subject database is created using an ElasticSearch database. The storage module is used to determine whether the tag name of the target tag exists in the target entity database. If not, the target tag is stored in the target entity database according to the set data storage structure. When updating the tag information of the target object in the target subject database, obtain the latest tag name of the currently input target object and the latest tag source corresponding to the latest tag name; If the target object does not have the latest tag name, then add the latest tag name and the latest tag source, and record the occurrence count of the latest tag source as 1; If the target tag contains the latest tag name but not the latest tag source, then the latest tag source is added, and the number of times the latest tag source appears is recorded as 1; If the target tag contains the latest tag name and the latest tag source, then increment the occurrence count of the latest tag source by 1; The search modes can be set as follows: when the tag name and tag code of the target tag are searched simultaneously, the word segmentation index mode is used; when the tag code of the target tag is searched alone, the non-word segmentation index mode is used. The process of storing the target tag in the target subject database according to the set data storage structure includes: Store the target object and tag name of the target tag in array form; Set the tag code for the tag name of the target tag; Create a tag field and a source field for the tag encoding of the target tag, wherein the source field is embedded within the tag field; The tag name and tag code of the target tag are stored in the tag field, and the tag source of the target tag is stored in the source field; The step of storing the target tag in the target subject database according to the set data storage structure further includes: Store the number of times the source of the target tag appears in the source field.

8. A big data tagging analysis system, characterized in that, The system is used to execute the big data tag storage method according to any one of claims 1-3, and the system comprises: The data acquisition module is used to acquire data to be queried, which includes multiple tag names and dimension parameters corresponding to the same object to be queried. The database determination module is used to determine the database of the subject to be queried corresponding to the object to be queried; The script construction module is used to construct a data analysis script based on the multiple tag names and dimension parameters to be queried; The analysis module is used to query the database of the subject to be queried using the data analysis script to obtain data analysis results.

Images