A method for generating buried point data and related equipment
By using columnar structure to store buried point data, the problems of low reading efficiency and poor field expansion capabilities in the existing technology are solved, and efficient data reading and flexible field expansion are realized, which is suitable for data storage and analysis needs of mobile Internet applications.
Patent Information
- Application Number
- CN202111566164.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-20
- Publication Date
- 2025-03-25
- Estimated Expiration
- 2041-12-20
AI Technical Summary
The existing buried point data storage has low reading efficiency and poor field expansion capabilities, making it difficult to meet the data storage and analysis needs of mobile Internet applications.
The columnar structure is used to store buried point data. By receiving relevant information of the target event, the target buried point data is generated and stored, including the primary key column, event name column, time stamp column, attribute column, etc., and supports secondary indexing to improve query efficiency.
It improves the reading efficiency of buried point data, reduces storage costs, and realizes flexible expansion of fields, which is suitable for the storage, reading and analysis processing of buried point data in mobile Internet.
Smart Images

Figure CN114238335B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing, and in particular to a method for generating buried point data and related equipment. Background Art
[0002] Tracking can also be called event tracking, which refers to the related technologies and implementation processes for capturing, processing and sending specific user behaviors or events. By adding tracking to the product, you can obtain the tracking data generated by users in the process of using the product, transfer the generated tracking data to a unified database, and analyze and process the tracking data in the database to optimize the product.
[0003] In order to continuously improve and enhance the quality of application services, mobile Internet developers often embed tracking codes in their applications to monitor and statistically analyze the operation and usage of applications, which will generate massive tracking data, and each tracking data has its own unique field attributes. When storing tracking data, the common row storage format and comma-separated values file format (Comma-Separated Values, csv), JS object notation JSON (JavaScript Object Notation, JSON) string format storage and reading efficiency are low, and the field expansion capability is poor. Summary of the invention
[0004] The purpose of the present invention is to provide a method for generating buried point data, aiming to solve the problems of low storage and reading efficiency of existing buried point data and poor field expansion capability. The present application provides:
[0005] A first aspect of an embodiment of the present application provides a method for generating buried point data, which is applied to a server and includes:
[0006] Receiving relevant information of a target event, wherein the relevant information of the target event is collected by the client according to a preset rule after the user triggers the target event;
[0007] Target burial point data is generated and stored based on the relevant information of the target event, and the target burial point data stores the relevant data of the target event in a column structure.
[0008] Based on the method for generating buried point data provided in the first aspect of the embodiment of the present application, optionally, the target buried point data includes: a primary key column and a target event column, the primary key is a string type, the target event column is used to store relevant data of the target event, and the primary key column and the target event column are stored in the target buried point data in a columnar structure.
[0009] Based on the method for generating buried point data provided in the first aspect of the embodiment of the present application, optionally, the target event column includes: an event name column, a timestamp column and an attribute column.
[0010] Based on the method for generating buried point data provided in the first aspect of the embodiment of the present application, optionally, the attribute column includes: an attribute name column and an attribute value column.
[0011] Based on the method for generating buried point data provided in the first aspect of the embodiment of the present application, optionally, the target buried point data also includes a secondary index area, and the secondary index area is used to store the correspondence between the secondary index key and the data address.
[0012] A second aspect of an embodiment of the present application provides a method for generating buried point data, which is applied to a client and includes:
[0013] Monitor whether the user triggers the target event;
[0014] If the user triggers the target event, relevant information of the target event is collected according to preset rules;
[0015] Send the relevant information of the target event to the server, so that the server generates and stores target burial point data based on the relevant information of the target event, and the target burial point data stores the relevant data of the target event in a column structure.
[0016] A third aspect of an embodiment of the present application provides a device for generating buried point data, including:
[0017] A receiving unit, the receiving unit is used to receive relevant information of a target event, wherein the relevant information of the target event is collected by the client according to a preset rule after the user triggers the target event;
[0018] A storage unit, wherein the storage unit is used to generate and store target buried point data based on the relevant information of the target event, and the target buried point data stores the relevant data of the target event in a column structure.
[0019] A fourth aspect of the present application provides a device for generating buried point data, comprising:
[0020] CPU, memory, input and output interfaces, wired or wireless network interfaces, and power supply;
[0021] The memory is a short-term storage memory or a persistent storage memory;
[0022] The central processing unit is configured to communicate with the memory and execute instruction operations in the memory on the device to perform the method described in any one of the first aspects of the embodiments of the present application.
[0023] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enable the computer to execute a method as described in any one of the first aspects of the embodiment of the present application.
[0024] A sixth aspect of the embodiments of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute a method as described in any one of the methods described in the first aspect of the embodiments of the present application.
[0025] It can be seen from the above technical scheme that the embodiments of the present application have the following advantages: the present application provides a method for generating buried point data, which is applied to a server, including: receiving relevant information of a target event, the relevant information of the target event is collected by a client according to preset rules after the user triggers the target event; generating and storing target buried point data according to the relevant information of the target event, the target buried point data stores the relevant data of the target event in a columnar structure. Based on the above technical scheme, it can be seen that the buried point data generated by the buried point data generation method provided by the present application stores the relevant data of the target event in a columnar structure. Through the above data storage method, while reducing the storage cost of the buried point data, the column fields can be read on demand, thereby improving the reading efficiency of the buried point data, and realizing flexible field expansion, which is suitable for the storage, reading and analysis and processing requirements of the mobile Internet buried point data. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work. It is understood that the drawings provided in this section are only used to better understand the present solution and do not constitute a limitation of the present application.
[0027] Figure 1 A flowchart of an embodiment of a method for generating buried point data provided by the present application;
[0028] Figure 2 A schematic diagram of the data structure of the buried point provided by this application;
[0029] Figure 3 Another flowchart of an embodiment of the method for generating buried point data provided by the present application;
[0030] Figure 4 A schematic diagram of a structure of an embodiment of a buried point data generating device provided in the present application;
[0031] Figure 5Another structural schematic diagram of an embodiment of the burial point data generating device provided in this application. DETAILED DESCRIPTION
[0032] The embodiment of the present application provides a method for generating buried point data, which is used to improve the reading efficiency of buried point data and realize flexible field expansion, and is suitable for the storage, reading, analysis and processing requirements of buried point data in mobile Internet.
[0033] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application are clearly and completely described below. Obviously, the described embodiments are only embodiments of a part of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should belong to the scope of protection of the present application. At the same time, for the sake of clarity and simplicity, the description of well-known functions and structures is omitted in the following description.
[0034] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0035] Tracking can also be called event tracking, which refers to the related technologies and implementation processes for capturing, processing and sending specific user behaviors or events. By adding tracking to the product, you can obtain the tracking data generated by users in the process of using the product, transfer the generated tracking data to a unified database, and analyze and process the tracking data in the database to optimize the product.
[0036] In order to continuously improve and enhance the quality of application services, mobile Internet developers often embed tracking codes in their applications to monitor and statistically analyze the operation and usage of applications. This will generate massive amounts of tracking data, and each tracking data has its own unique field attributes. When storing tracking data, the common row storage format and CSV and JSON string formats make it difficult to achieve both efficient storage and reading and flexible field expansion.
[0037] To solve the above problems, the present application provides a new method for generating buried point data. Figure 1 , an embodiment of the present application includes: step 101-step 102.
[0038] Step 101: Receive relevant information of a target event.
[0039] Specifically, the server receives relevant information about the target event. The relevant information about the target event is collected by the client according to preset rules after the user triggers the target event; the target event is the mobile phone condition data of the pre-set buried data, and the specific form of the target event can be determined according to the actual situation. The specific target event can be events such as: click, login, download, etc. When the user triggers the event, the buried data collection setting of the client will be triggered, and then the relevant information of the target event will be collected according to the preset rules, wherein the preset rules are set according to the specific situation of the target event, and different data collection rules can be set for different types of target events, and then different categories of information of the target event are collected. It can be determined according to the actual situation and is not limited here.
[0040] Step 102: Generate and store target burial point data based on the relevant information of the target event.
[0041] Specifically, the server generates and stores target tracking data based on the relevant information of the target event, and the target tracking data stores the relevant data of the target event in a column structure. Column storage means that the data in a column is stored continuously in the storage medium. For example, the target tracking data may store a primary key column and a timestamp column, and then record the relevant information of the target event. In this way, the tracking data is stored in a column form, which improves the query efficiency and expansion capability of the tracking data.
[0042] It can be seen from the above technical scheme that the embodiments of the present application have the following advantages: the present application provides a method for generating buried point data, which is applied to a server, including: receiving relevant information of a target event, the relevant information of the target event is collected by a client according to preset rules after the user triggers the target event; generating and storing target buried point data according to the relevant information of the target event, the target buried point data stores the relevant data of the target event in a columnar structure. Based on the above technical scheme, it can be seen that the buried point data generated by the buried point data generation method provided by the present application stores the relevant data of the target event in a columnar structure. Through the above data storage method, while reducing the storage cost of the buried point data, the column fields can be read on demand, thereby improving the reading efficiency of the buried point data, and realizing flexible field expansion, which is suitable for the storage, reading and analysis and processing requirements of the mobile Internet buried point data.
[0043] To facilitate understanding of this application, a schematic diagram of the data structure of the buried point is also provided. Figure 2 , Figure 2 This is a schematic diagram of the tracking data files stored on the server. The files include:
[0044] Header magic word: 8-byte special value, marking the beginning of the embedded column data structure file.
[0045] Metadata area: used to store the total length of stored data, the total number of entries, the number of blocks, the number of row groups, the primary key index offset address, the file tail area offset address, and the description information of the column-based embedded data structure.
[0046] Primary key index area: used to store primary key related information.
[0047] Block area: used to store the number of row groups, block length, and row group list.
[0048] Row group area: The row group area is used to store relevant information about the time of the buried point. Specifically, the target buried point data includes: a primary key column and a target event column. The primary key is a string type. Each buried point event will generate a unique identifier, such as a unique identifier, a session id, etc. The target event column is a set of buried point events, which are stored in the order of event occurrence. The target event column may include: an event name column, a timestamp column, and an attribute column. The event name is the name of the buried point event, which identifies the meaning of the buried point event, such as login, click, follow, download, etc. The timestamp is the time when the target event managed by the buried point data occurs, generally using millisecond accuracy. The attribute column is a list of attributes of the buried point event. The attribute is used to identify the current state of the event, such as the page, button, pit position, etc. of the event. The attribute column specifically includes an attribute name column and an attribute value column. The attribute name is used to identify the attribute meaning, and the attribute value is used to identify the specific value of the attribute. It can be understood that in the actual implementation process, the type and number of columns can be adjusted according to the actual situation, so that the relevant information of the buried point data can be better stored. It can be determined according to the actual situation and is not limited here.
[0049] Optionally, the file may also include a secondary index area, which is used to store the corresponding relationship between the secondary index key and the specific data address. The secondary index key can be set according to actual needs and is not specifically limited here.
[0050] Magic word at the end: 8-byte special value, marking the end of the embedded column data structure file.
[0051] During the actual implementation process, the metadata format description information of the buried data storage can be pre-designed. When the data sent by the client is received, the relevant data is parsed and processed according to the metadata format description information of the buried data storage, and the data storage is implemented according to the file format described in the above embodiment.
[0052] If reading is required, it can be implemented according to the file format described in this solution. During the reading process, a column-based embedded storage format is used for data, and the metadata index area and the secondary index area are pre-set so that the column fields can be read on demand during the data reading process, thereby improving the reading efficiency of the buried data. At the same time, the buried data columns can be added as needed, thereby realizing flexible expansion of the fields, better meeting the business usage requirements, and improving the feasibility of this method.
[0053] It can be seen from the above technical scheme that the embodiments of the present application have the following advantages: the present application provides a method for generating buried point data, which is applied to a server, including: receiving relevant information of a target event, the relevant information of the target event is collected by a client according to preset rules after the user triggers the target event; generating and storing target buried point data according to the relevant information of the target event, the target buried point data stores the relevant data of the target event in a columnar structure. Based on the above technical scheme, it can be seen that the buried point data generated by the buried point data generation method provided by the present application stores the relevant data of the target event in a columnar structure. Through the above data storage method, while reducing the storage cost of the buried point data, the column fields can be read on demand, thereby improving the reading efficiency of the buried point data, and realizing flexible field expansion, which is suitable for the storage, reading and analysis and processing requirements of the mobile Internet buried point data.
[0054] Based on the above embodiments, this application also provides a method for generating buried point data. Figure 3 ,include:
[0055] 301. Monitor whether the user triggers the target event.
[0056] Specifically, the target event is the data collection event, and the target event can be a login, download, or click event, etc., which can be determined according to the actual situation and is not limited here.
[0057] 302. If the user triggers the target event, collect relevant information of the target event according to a preset rule.
[0058] Specifically, if the user triggers the target event, relevant information of the target event is collected according to a preset rule. The type of information in the relevant information of the target event may be determined according to the preset rule, and the collected data category may include: current time, event name, and attributes, etc., which may be determined according to actual conditions and are not limited here.
[0059] 303. Send the relevant information of the target event to the server, so that the server generates and stores target burial point data based on the relevant information of the target event, and the target burial point data stores the relevant data of the target event in a column structure.
[0060] Specifically, the relevant information of the target event is sent to the server, so that the server generates and stores target burial point data based on the relevant information of the target event, and the target burial point data stores the relevant data of the target event in a column structure.
[0061] The metadata format description information of the tracking data storage includes:
[0062]
[0063] Format description:
[0064] Unique ID (primary key): Each tracking event will generate a unique ID, such as a unique ID, session ID, etc.
[0065] Event list: A collection of embedded events, stored in the order in which they occur.
[0066] Event name: The name of the buried event, which identifies the meaning of the buried event, such as login, click, follow, download, and other events.
[0067] Timestamp: The time when the tracking event occurs, usually with millisecond accuracy.
[0068] Attribute list: Attribute list of the buried event. Attributes are used to identify the current status of the event, such as the event page, button, slot, etc.
[0069] Attribute name: Identifies the meaning of the attribute.
[0070] Attribute value: identifies the specific value of the attribute.
[0071] It is understandable that the above format is only added to explain the purpose of this solution and does not limit the actual implementation process of this solution. In the actual implementation process, the data format and data type can be adjusted as needed, and no specific limitation is made here.
[0072] The generation and storage process of target tracking data in this step can refer to the above Figure 1 The contents described in the corresponding embodiments are not described in detail here.
[0073] It can be seen from the above technical scheme that the embodiments of the present application have the following advantages: the present application provides a method for generating buried point data, which is applied to the client, including: monitoring whether the user triggers the target event; if the user triggers the target event, collecting the relevant information of the target event according to the preset rules; sending the relevant information of the target event to the server, so that the server generates and stores the target buried point data according to the relevant information of the target event, and the target buried point data stores the relevant data of the target event in a column structure. Based on the above technical scheme, it can be known that the buried point data generated by the buried point data generation method provided by the present application stores the relevant data of the target event in a column structure. Through the above data storage method, while reducing the buried point data storage cost, the column fields can be read on demand, thereby improving the reading efficiency of the buried point data, and realizing flexible field expansion, which is suitable for the storage, reading and analysis and processing requirements of mobile Internet buried point data.
[0074] The above content describes the method for generating buried point data provided by the present application. To support the implementation of the above embodiment, the present application also provides a device for generating buried point data. Figure 4 , an embodiment of the device for generating buried point data provided by the present application includes:
[0075] A receiving unit 401, the receiving unit 401 is used to receive relevant information of a target event, the relevant information of the target event is collected by the client according to a preset rule after the user triggers the target event;
[0076] The storage unit 402 is used to generate and store target burial point data based on the relevant information of the target event, and the target burial point data stores the relevant data of the target event in a column structure.
[0077] In this embodiment, the process executed by each unit in the buried point data generation device is the same as the aforementioned Figure 1 The method flow described in the corresponding embodiment is similar and will not be repeated here.
[0078] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.
[0079] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0080] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0081] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as the buried point data generation method. For example, in some embodiments, it may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the buried point data generation method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the buried point data generation method in any other appropriate manner (e.g., by means of firmware).
[0082] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0083] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0084] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0085] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0086] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.
[0087] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.
[0088] The present application also provides a computer program product, which includes computer software instructions, which can be loaded by a processor to implement the above Figure 1 or Figure 3 The process of generating point data of any one of the above methods.
[0089] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the equivalent transformation of the circuit, the division of the unit, is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0090] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0091] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0092] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent substitution or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for generating buried point data, characterized in that: Applicable to servers, including: Receiving relevant information of a target event, wherein the relevant information of the target event is collected by the client according to a preset rule after the user triggers the target event; Generate and store target burying point data based on the relevant information of the target event, wherein the target burying point data stores the relevant data of the target event in a column structure; The target embedding data includes: Header magic word: 8 bytes of special value, marking the beginning of the embedded column data structure file; Metadata area: used to store the total length of stored data, the total number of entries, the number of blocks, the number of row groups, the primary key index offset address, the file tail area offset address, and the description information of the column embedded data structure; Primary key index area: used to store primary key related information, including primary key dictionary sorting sparse index table, used to store the mapping relationship between primary key and block address; Block area: used to store the number of row groups, block length, and row group list. The primary key index area includes a primary key dictionary sorted sparse index table. Row group area: The row group area is used to store relevant information of the buried event, including: a primary key column and a target event column. The primary key is a string type. Each buried event will generate a unique identifier. The target event column is a set of buried events, which are stored in the order of event occurrence. The target event column includes: an event name column, a timestamp column, and an attribute column. The event name is the buried event name, which identifies the meaning of the buried event. The timestamp is the time when the target event of the buried data management occurs. The attribute column is a list of attributes of the buried event. The attribute is used to identify the current state of the event. The attribute column specifically includes an attribute name column and an attribute value column. The attribute name is used to identify the attribute meaning, and the attribute value is used to identify the specific value of the attribute. Magic word at the end: 8-byte special value, marking the end of the embedded column data structure file.
2. The method for generating buried point data according to claim 1, characterized in that: The target buried point data also includes a secondary index area, and the secondary index area is used to store the corresponding relationship between the secondary index key and the data address.
3. A method for generating buried point data, characterized in that: Applied to the client, including: Monitor whether the user triggers the target event; If the user triggers the target event, relevant information of the target event is collected according to preset rules; Sending relevant information of the target event to a server, so that the server generates and stores target burying point data according to the relevant information of the target event, wherein the target burying point data stores relevant data of the target event in a column structure; The target embedding data includes: Header magic word: 8 bytes of special value, marking the beginning of the embedded column data structure file; Metadata area: used to store the total length of stored data, the total number of entries, the number of blocks, the number of row groups, the primary key index offset address, the file tail area offset address, and the description information of the column embedded data structure; Primary key index area: used to store primary key related information, including primary key dictionary sorting sparse index table, used to store the mapping relationship between primary key and block address; Block area: used to store the number of row groups, block length, and row group list; Row group area: The row group area is used to store relevant information of the buried event, including: a primary key column and a target event column. The primary key is a string type. Each buried event will generate a unique identifier. The target event column is a set of buried events, which are stored in the order of event occurrence. The target event column includes: an event name column, a timestamp column, and an attribute column. The event name is the buried event name, which identifies the meaning of the buried event. The timestamp is the time when the target event of the buried data management occurs. The attribute column is a list of attributes of the buried event. The attribute is used to identify the current state of the event. The attribute column specifically includes an attribute name column and an attribute value column. The attribute name is used to identify the attribute meaning, and the attribute value is used to identify the specific value of the attribute. Magic word at the end: 8-byte special value, indicating the end of the embedded column data structure file.
4. A device for generating buried point data, characterized in that: include: A receiving unit, the receiving unit is used to receive relevant information of a target event, wherein the relevant information of the target event is collected by the client according to a preset rule after the user triggers the target event; A storage unit, the storage unit is used to generate and store target buried point data according to the relevant information of the target event, and the target buried point data stores the relevant data of the target event in a column structure; The target embedding data includes: Header magic word: 8 bytes of special value, marking the beginning of the embedded column data structure file; Metadata area: used to store the total length of stored data, the total number of entries, the number of blocks, the number of row groups, the primary key index offset address, the file tail area offset address, and the description information of the column embedded data structure; Primary key index area: used to store primary key related information, including primary key dictionary sorting sparse index table, used to store the mapping relationship between primary key and block address; Block area: used to store the number of row groups, block length, and row group list; Row group area: The row group area is used to store relevant information of the buried event, including: a primary key column and a target event column. The primary key is a string type. Each buried event will generate a unique identifier. The target event column is a set of buried events, which are stored in the order of event occurrence. The target event column includes: an event name column, a timestamp column, and an attribute column. The event name is the buried event name, which identifies the meaning of the buried event. The timestamp is the time when the target event of the buried data management occurs. The attribute column is a list of attributes of the buried event. The attribute is used to identify the current state of the event. The attribute column specifically includes an attribute name column and an attribute value column. The attribute name is used to identify the attribute meaning, and the attribute value is used to identify the specific value of the attribute. Magic word at the end: 8-byte special value, indicating the end of the embedded column data structure file.
5. A device for generating buried point data, characterized in that: include: CPU, memory; The memory is a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instruction operations in the memory on the burial point data generating device to perform the method described in any one of claims 1 to 3.
6. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 3.
7. A computer program product comprising instructions, characterized in that When the method is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 3.
Citation Information
Patent Citations
Data storage and analysis method and device, equipment and readable medium
CN110647512A
Buried point data processing method, related device and storage medium
CN113239251A