Data query method and device, equipment and medium
By using multi-digit time, user information and business module information storage methods in the database table, and using binary tagging whether events have occurred, the problem of slow query under the large amount of data in a single table is solved, and the query performance is improved and storage is achieved Reduced costs.
Patent Information
- Application Number
- CN202411631689.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-15
- Publication Date
- 2025-02-11
AI Technical Summary
The problem of slow query caused by excessive single-table data volume, and the existing technology partition table strategy has high maintenance costs and performance overhead.
A database table for storing each event information using multi-digit time, user information and business module information, and marking whether an event occurred in binary mode, thereby quickly filtering the target event information.
Significantly improve query performance, reduce storage costs, and improve user experience.
Smart Images
Figure CN119415569A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of big data processing technology, and in particular to a data query method, device, equipment and medium. Background Art
[0002] With the rapid development of information technology, database systems have become an indispensable core component of modern information systems. However, with the continuous growth of data volume, especially when the data volume of a single table reaches millions, tens of millions or even higher, query performance often faces severe challenges. Problems such as slow query speed and extended response time not only affect the user experience, but may also have a serious impact on the stability and efficiency of the business system.
[0003] To address the slow query problem caused by excessive data volume in a single table, the following optimization solutions are mainly adopted in related technologies. First, partition table technology is used to divide a large table into multiple small tables according to certain rules (such as date, region, etc.), and each small table is stored and queried independently. However, although the amount of data in a single query can be reduced, the selection and maintenance costs of the partition strategy are high, and cross-partition query operations may bring additional performance overhead.
[0004] Therefore, how to optimize the query performance under large amounts of data in a single table has become a technical problem that needs to be solved urgently. Summary of the invention
[0005] The purpose of this application is to provide a data query method, device, equipment and medium, which can quickly query corresponding information from a database table, significantly improve query performance, greatly reduce storage costs, and improve user experience.
[0006] In a first aspect, a data query method is provided, comprising:
[0007] Acquire a query request carrying query configuration information sent by a user terminal device, wherein the query configuration information includes at least one of: a target time, target user information, and target service module information;
[0008] Obtain a database table; wherein the data in the database table stores each event information according to multi-digit time, user information and business module information, wherein the multi-digit time is in a multi-digit binary digital format, each binary digit corresponds to a unique time, and the binary digit is used to indicate the occurrence of the event within the corresponding time;
[0009] The target event information corresponding to the query configuration information is filtered from the database table to obtain feedback information, and the feedback information is sent to the user terminal device.
[0010] In a preferred example, the present application may be further configured as follows: before filtering the target event information corresponding to the query configuration information from the database table and obtaining the feedback information, the following may also be included:
[0011] Establish a database table, the database table fields of which include user information, business module information and multi-digit time;
[0012] Acquire multiple pieces of original data, each piece of original data includes: first user information, first business module information, and first time information;
[0013] For each piece of original data, query whether there is first event information in the database table according to the first user information and the first business module information;
[0014] If the first event information exists, the target binary digit number is determined from the multi-digit time in the first event information according to the first time information; if the first event information does not exist, the second event information is obtained according to the first user information and the first business module information recorded in the database table, and the target binary digit number is determined from the multi-digit time in the second event information according to the first time information;
[0015] adjusting the target binary digits to a target value, wherein the target value indicates that the event has occurred;
[0016] After the plurality of pieces of original data are stored in the database, a database table is obtained.
[0017] In a preferred example, the present application can be further configured as follows: the multiple pieces of original data include: incremental data and / or full historical data.
[0018] In a preferred example, the present application may be further configured as follows: querying the target event information corresponding to the query request from the database table to obtain feedback information includes:
[0019] When the query configuration information includes: target time and target user information, query the database table for a number of target event information corresponding to the target time and target user information to obtain first feedback information, wherein the first feedback information indicates relevant information of the business module information executed by the user corresponding to the target user information within the target time;
[0020] Or, when the query configuration information includes: target time and target business module information, several target event information corresponding to the target time and target business module information are queried from the database table to obtain second feedback information, and the second feedback information represents relevant information of user information corresponding to the target business module information executed within the target time.
[0021] In a preferred example, the present application can be further configured as follows: querying the database table for a number of target event information corresponding to the target time and user information to obtain first feedback information, including:
[0022] When the user information is parent user information, determining the child user information corresponding to the user information according to a preset corresponding relationship, wherein the preset corresponding relationship is a corresponding relationship between a plurality of parent user information and a plurality of child user information sets, each child user information set including a plurality of child user information;
[0023] Querying the target event information corresponding to the target time and the user information from the database table to obtain the first sub-feedback information;
[0024] Querying the target event information corresponding to the target time and the sub-level user information from the database table to obtain second sub-feedback information;
[0025] The first feedback information is determined according to the first sub-feedback information and the second sub-feedback information.
[0026] In a preferred example, the present application may be further configured as follows: each event information also includes a data dimension of access duration, and the method further includes at least one of the following:
[0027] Determine the access frequency and access duration of each business module information corresponding to the target user information according to the first feedback information; determine the preferred business module information according to the access frequency and access duration of each business module information; and establish a user profile corresponding to the target user information according to the preferred business module information;
[0028] Determine the overall access frequency of different user information corresponding to the target business module information, the access duration of each visit, and the number of users of different types of user information based on the second feedback information; determine the business tag information based on the overall access frequency of different user information, the access duration of each visit, and the number of users of different types of user information; and establish a business profile corresponding to the target business module information based on the business tag information.
[0029] In a preferred example, the present application may be further configured as follows: the service tag information includes: an activity status tag, an active user type tag;
[0030] Determine business tag information based on the overall access frequency of different user information, the access duration of each access of different user information, and the number of users of different types of user information, including:
[0031] Determining a first access value according to the overall access frequency of different user information;
[0032] Determine a second access value indicating the number of valid accesses according to the access duration of each access of different user information and a preset minimum access duration;
[0033] Determine an activity status label corresponding to the target business module information according to the first access value and the second access value;
[0034] According to the number of users of different types of user information, determine the active user type label corresponding to the target business module information.
[0035] In a second aspect, a data query device is provided, comprising:
[0036] A first acquisition module, used to acquire a query request carrying query configuration information sent by a user terminal device, wherein the query configuration information includes at least one of: a target time, target user information, and target service module information;
[0037] A second acquisition module is used to acquire a database table; wherein the data in the database table stores each event information according to a multi-digit time, user information and business module information, wherein the multi-digit time is in a multi-digit binary digital format, each binary digit corresponds to a unique time, and the binary digit is used to indicate the occurrence of the event within the corresponding time;
[0038] The query and feedback module is used to filter the target event information corresponding to the query configuration information from the database table, obtain feedback information, and send the feedback information to the user terminal device.
[0039] In a third aspect, an electronic device is provided, including:
[0040] one or more processors;
[0041] Memory;
[0042] One or more applications, wherein one or more applications are stored in a memory and configured to be executed by one or more processors, and one or more programs are configured to: perform operations corresponding to the method shown in any possible implementation of the first aspect.
[0043] In a fourth aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded by a processor and executes the steps of the method shown in any possible implementation of the first aspect.
[0044] In a fifth aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, operations corresponding to the method shown in any possible implementation manner in the first aspect are implemented.
[0045] The present application provides a data query method, including at least one of the following beneficial technical effects: using data in a database table to store each event information according to multi-digit time, user information and business module information, the multi-digit time is in a multi-digit binary digital format, each binary digit corresponds to a unique time, and the binary digit is used to represent the occurrence of the event within the corresponding time; further, when a query request carrying query configuration information is obtained, since a binary method is used to mark whether an event has occurred, the corresponding information can be quickly queried from the database table, the query performance is significantly improved, and the storage cost is greatly reduced, and the user experience is improved.
[0046] The present application also provides a data query device, equipment and medium, all of which have the above-mentioned beneficial technical effects. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions of the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0048] Figure 1 A schematic diagram of a scenario for applying a data query method provided in an embodiment of the present application;
[0049] Figure 2 It is a flowchart of a data query method provided in an embodiment of the present application;
[0050] Figure 3 It is a structural schematic diagram of a data query device provided in an embodiment of the present application;
[0051] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0052] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, a person skilled in the art may make non-creative modifications to the present embodiment as needed, but such modifications are protected by the patent law as long as they are within the scope of the present application.
[0053] It should be noted that in the optional embodiments of the present application, the object information and other related data involved, when the embodiments in the present application are applied to specific products or technologies, need to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if the embodiments of the present application involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information needs to obtain the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained. The embodiments also need to be implemented with the authorization and consent of the object.
[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0055] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.
[0056] As an open source relational database management system, MySQL has been widely used in various application scenarios due to its high performance, high reliability and ease of use. Among them, the amount of data in a single MySQL table is too large, and the query is slow, so performance needs to be optimized.
[0057] Based on the above problems, this application proposes a new method: using binary method to mark whether an event occurs, by recording data in the time dimension and storing it in the database, finally realizing multi-dimensional data statistics within a period of time in one table, achieving the effect of data compression, and greatly reducing the workload of data storage and use; further, after receiving the query request, it is possible to quickly filter the target event information corresponding to the query configuration information from the database table, obtain feedback information, and improve the query efficiency.
[0058] To better understand the solution provided by the embodiments of this application, please refer to Figure 1 , Figure 1 A schematic diagram of an application scenario of a data query method provided in an embodiment of the present application. The data query method can be applied to a data query system.
[0059] In some embodiments, the data query system includes an electronic device and a user terminal device. The user terminal device includes but is not limited to: mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), and fixed terminals such as digital TVs, desktop computers, etc. The electronic device can be a server or a terminal device, and the server can be an independent server or a server cluster or a distributed system composed of multiple servers, or a cloud server that provides cloud computing services. The user sends a query request carrying query configuration information through the user terminal device; the electronic device obtains the query request carrying query configuration information sent by the user terminal device, and the query configuration information includes: at least one of the target time, target user information, and target business module information; obtains a database table; wherein the data in the database table stores each event information according to multi-digit time, user information, and business module information, and the multi-digit time is in a multi-digit binary digital format, each binary digit corresponds to a unique time, and the binary digit is used to indicate the occurrence of the event within the corresponding time; the target event information corresponding to the query configuration information is filtered from the database table to obtain feedback information, and the feedback information is sent to the user terminal device.
[0060] It is understandable that the above is only an example and is not limited to this embodiment.
[0061] The present application relates to the field of data organization and storage technology. Specifically, the present application embodiment provides a data query method, such as Figure 2 As shown, the method provided in the embodiment of the present application can be executed by an electronic device, and the method includes:
[0062] S101, obtaining a query request carrying query configuration information sent by a user terminal device, where the query configuration information includes: at least one of target time, target user information, and target service module information;
[0063] The user sends a query request to the electronic device through the user terminal device to request to obtain data information. The query request carries the query configuration information so that the system can return the corresponding result according to the query configuration information.
[0064] This application is applicable to a wide range of scenarios, including but not limited to counting users visiting web pages corresponding to specific business modules, generally for technical personnel to maintain web pages.
[0065] Alternatively, one method is that the user inputs query configuration information through a graphical user interface (GUI) of a user terminal device. For example, in a search engine, the user can enter keywords in a search box, select relevant filter conditions (such as target time, target user information, and target business module information), and then click a search button to initiate a query request.
[0066] Alternatively, another method is for the user to input the query configuration information through a voice assistant or voice input function of the user terminal device. For example, on a smartphone, the user speaks their query requirements, and the voice assistant converts these requirements into query configuration information and automatically initiates a query request.
[0067] It is understandable that other methods may be used to obtain the query request carrying the query configuration information sent by the user terminal device, such as inputting the query configuration information in the QR code through the scanning function of the user terminal device, etc., which is not limited here.
[0068] In the embodiment of the present application, the query configuration information may be: target time, or target user information, or target service module information, or target time and target user information, or target time and target service module information, or target user information and target service module information, or target time, target user information and target service module information. Users may set the information according to actual needs.
[0069] S102, obtaining a database table; wherein the data in the database table stores each event information according to multi-digit time, user information and business module information, the multi-digit time is in the form of multi-digit binary digital format, each binary digit corresponds to a unique time, and the binary digit is used to indicate the occurrence of the event within the corresponding time;
[0070] A database table is maintained in the electronic device, and the fields of the database table include user information (such as user name, user ID), business module information (specifically event, which is the name of a business module) and multi-digit time information (such as year and date), so that each event information has the function of showing what a user did and when. Among them, the time field is set in a multi-digit binary digital format, using 0 and 1 to mark whether an event occurs. The number of digits can be customized by the user based on his own statistics, which can be 1 year, 365 digits are set according to 365 days; it can be 1 month, 31 digits are set according to 31 days; it can be 1 day, 24 digits are set according to 24 hours; of course, it can also be other setting rules, which are no longer limited in the embodiment of the present application, and users can set according to actual needs.
[0071] For example, the preset 365 bits represent the statistics of the event occurrence for 365 days from January 1 to December 31 of a certain year, and each day is 0 or 1. If the second bit is 0, it means that the event did not occur the next day. Specifically, with user, event, and year as the main data dimensions, the event that occurred on a certain day is marked as 1, and the event that did not occur is marked as 0; the default raw data for each day is 0; so that the table has the function of summarizing raw data according to time, user, and event.
[0072] For example, when the time is one year, according to 365 bits, referring to Table 1, it means that user 1 does not access module 1 on the first day, accesses module 1 on the second day, and accesses module 1 on the third day.
[0073]
[0074] For another example, when the time is one day, according to 24 bits, referring to Table 2, it means that user 2 accesses module 2 at the 1st hour, the 8th hour, the 9th hour, the 15th hour, and the 16th hour.
[0075]
[0076] S103: Filter the target event information corresponding to the query configuration information from the database table, obtain feedback information, and send the feedback information to the user terminal device.
[0077] In an embodiment of the present application, after filtering the target event information corresponding to the query configuration information from the database table, the feedback information may be the target event information, and / or information obtained after analyzing the target event information. The specific analysis method may be set according to actual needs.
[0078] When the query configuration information includes only one dimension of information, the corresponding target event information can be directly queried from the database table based on the single dimension information to obtain feedback information;
[0079] When the query configuration information includes information of at least two dimensions, the corresponding initial event information is first queried from the database table based on a certain dimension information, and then the corresponding event information is filtered out from the initial event information based on the information of the next dimension, until the event information filtered by the information of all dimensions is obtained, and the feedback information is obtained; the specific order of dimensions can be set by the user according to actual needs, or customized.
[0080] It can be seen that in the embodiment of the present application, the data in the database table is used to store each event information according to the multi-digit time, user information and business module information. The multi-digit time is in the form of a multi-digit binary number format. Each binary digit corresponds to a unique time, and the binary number is used to represent the occurrence of the event within the corresponding time; further, when a query request carrying query configuration information is obtained, since a binary method is used to mark whether an event has occurred, the corresponding information can be quickly queried from the database table, and the query performance is significantly improved. At the same time, the storage cost is greatly reduced, and the user experience is improved.
[0081] A possible implementation of the embodiment of the present application includes, before filtering the target event information corresponding to the query configuration information from the database table and obtaining the feedback information, further comprising:
[0082] Establish a database table, the database table fields of which include user information, business module information and multi-digit time;
[0083] Acquire multiple pieces of original data, each piece of original data includes: first user information, first business module information, and first time information;
[0084] For each piece of original data, query whether there is first event information in the database table according to the first user information and the first business module information;
[0085] If the first event information exists, the target binary digit number is determined from the multi-digit time in the first event information according to the first time information; if the first event information does not exist, the second event information is obtained according to the first user information and the first business module information recorded in the database table, and the target binary digit number is determined from the multi-digit time in the second event information according to the first time information;
[0086] The target binary digits are adjusted to the target value, and the target value indicates that the event has occurred;
[0087] After completing the storage of multiple pieces of original data, a database table is obtained.
[0088] In the embodiments of the present application, the original standard database table is used to store data and query data. When the amount of data in a single table reaches millions, tens of millions or even higher, problems such as slow query speed and prolonged response time not only affect the user experience, but may also have a serious impact on the stability and efficiency of the business system.
[0089] Therefore, the embodiment of the present application constructs a simple database table that uses binary method to mark whether an event occurs. By recording and storing data in the time dimension, multi-dimensional data statistics within a period of time are finally realized in one table, achieving the effect of data compression and greatly reducing the workload of data storage and use.
[0090] After building the database tables, technicians develop data processing programs;
[0091] In addition, data needs to be filled into the database table, where the filled data can be incremental data or historical full data, or both incremental data and historical full data.
[0092] For incremental data (usually newly added original data containing user information, business module information, i.e. events, and dates), configure and execute incremental data tasks. Based on the user information and business module information in the original data, query whether there is a corresponding record in the database table. If there is a record, find the record and fill the corresponding date into the date digit corresponding to the time field of the table; if there is no record, add a new record based on the user and event information, and fill the corresponding date into the date digit corresponding to the time information field of the record (the default value for dates that have not been counted before is 0).
[0093] For the full data (usually historical original data containing user information, business module information, i.e. events, and dates), configure and execute the full data task. Based on the user and business module information in the original data, query whether there is a corresponding record in the database table. If there is a record, find the record and fill the corresponding date into the date digit corresponding to the time field of the table; if there is no record, add a new record based on the user and event information, and fill the corresponding date into the date digit corresponding to the time field of the record, and sort out all the historical data and store it in the database.
[0094] In some specific embodiments, all historical data can be sorted and stored in the database by importing it once, and incremental data tasks can be subsequently configured to perform data entry regularly to achieve tabular statistics of all data.
[0095] After completing the storage of multiple pieces of original data, a database table is obtained.
[0096] A possible implementation of the embodiment of the present application is to query the target event information corresponding to the query request from the database table to obtain feedback information, including:
[0097] When the query configuration information includes: target time and target user information, the target time and target user information are queried from the database table to obtain the first feedback information, which indicates the relevant information of the business module information executed by the user corresponding to the target user information within the target time; the target time can be a time represented by a specific digit or a time period composed of multiple digits. The user can query based on one or more of the user name, specific date or a time period, and the time name to obtain whether the user has executed a certain business / or which businesses have been executed within a period of time or a certain day.
[0098] Among them, the first feedback information can be several target event information corresponding to the business module information executed by the user corresponding to the target user information within the target time, and can also be analyzed in combination with several target event information to obtain: the overall access frequency of different business module information corresponding to the target user information; of course, if the event information in the database table also includes the dimension of access duration, the first feedback information can also include the access duration of each visit. It can be understood that the number of digits in the access duration dimension is the same as that of the time information. 0-9 can be used to represent the access duration, and each digit represents the duration itself or a duration range, which can be customized by the user.
[0099] It can be seen that in the embodiment of the present application, when the query configuration information includes: target time and target user information, relevant data can be filtered out from the database according to the target time and target user information to obtain the first feedback information.
[0100] Specifically, when a user accesses a module, a parent account or a child account may be used. Therefore, if the target user information is the parent user information, such as the company account information, in order to make the result accurate, not only the parent user information but also the child user information corresponding to the parent user information should be considered. Therefore, in the implementation of this application, several target event information corresponding to the target time and user information is queried from the database table to obtain the first feedback information, including:
[0101] When the user information is parent user information, determining the child user information corresponding to the user information according to a preset corresponding relationship, the preset corresponding relationship is a corresponding relationship between multiple parent user information and multiple child user information sets, each child user information set includes multiple child user information;
[0102] Querying a number of target event information corresponding to the target time and user information from a database table to obtain first sub-feedback information;
[0103] Querying a number of target event information corresponding to the target time and the sub-level user information from the database table to obtain the second sub-feedback information;
[0104] The first feedback information is determined according to the first sub-feedback information and the second sub-feedback information.
[0105] Among them, the parent user information can be company user information, and the child user information can be employee user information; or, the parent user information can be parent company user information, and the child user information can be subsidiary company user information; it can be adjusted according to actual conditions, and the embodiments of the present application are no longer limited.
[0106] It can be seen that in the embodiment of the present application, if the target user information is the parent user information, since in actual operation, it may be accessed by the parent user or the child user, only considering the parent user information may lead to certain deviations in the results. Therefore, the access situations of the parent user information and the child user information are considered at the same time, and the first feedback information is comprehensively determined to make the result more accurate.
[0107] Based on the above embodiment, each event information also includes a data dimension of access duration, and the method further includes:
[0108] Determine the access frequency and access duration of each business module information corresponding to the target user information according to the first feedback information;
[0109] Determine the preferred business module information based on the access frequency of each business module information and the access duration of each access;
[0110] Create a user profile corresponding to the target user information based on the preferred business module information.
[0111] Specifically, the frequency assignment is determined according to the order of the access frequency of each business module information, and the greater the frequency, the higher the assignment; the total duration corresponding to each business module information is determined according to the access duration of each access, and the duration assignment is determined according to the order of the total duration, and the greater the duration, the higher the assignment; then, a weighted calculation is performed based on the frequency assignment and the duration assignment to obtain a comprehensive value, and at least one business module information with the highest value is selected as the preferred business module information according to the size of the comprehensive value. Furthermore, the user profile is constructed by combining the basic information corresponding to the target user information, such as the user type, etc.
[0112] It is understandable that the portrait constructed at this time is only the portrait corresponding to the target time information, which indicates the user's preferences at the target time information. Of course, it is also possible to construct a user portrait within the entire multi-bit time range. After constructing the initial user portrait, after a set period, the user portrait is adjusted with more incremental event information.
[0113] It can be seen that in the embodiment of the present application, the target user's access frequency to each business module and the duration of each visit are obtained based on the first feedback information, so as to determine the user's preferred business module information and build a user portrait, which can be used as research and development reference information for technical personnel, or combined with the user portrait to provide personalized recommendations or personalized displays for users.
[0114] A possible implementation method of an embodiment of the present application is that when the query configuration information includes: target time and target business module information, several target event information corresponding to the target time and target business module information are queried from the database table to obtain second feedback information, and the second feedback information indicates relevant information of user information corresponding to the target business module information executed within the target time.
[0115] Among them, the second feedback information can be several target event information of user information corresponding to the target business module information executed within the target time, and can also be combined with several target event information for analysis to obtain the overall access frequency of different user information and the number of users of different types of user information; of course, if the event information in the database table also includes the dimension of access duration, the second feedback information can also include the access duration of each visit.
[0116] It can be seen that in the embodiment of the present application, when the query configuration information includes: target time and target business module information, relevant data can be filtered out from the database according to the target time and target business module information to obtain the second feedback information.
[0117] Based on the above embodiment, each event information also includes a data dimension of access duration, and the method further includes:
[0118] Determine the overall access frequency of different user information corresponding to the target business module information, the access duration of each access, and the number of users of different types of user information according to the second feedback information;
[0119] Determine business tag information based on the overall access frequency of different user information, the access duration of each access, and the number of users of different types of user information;
[0120] Create a business profile corresponding to the target business module information based on the business tag information.
[0121] The frequency of each user accessing the target business module within the target time is determined according to the second feedback information, and the average value of the frequency is taken as the overall access frequency. In addition, each user has a category division, which can be divided according to rules such as attributes or regions to obtain multiple category attributes.
[0122] Furthermore, based on the overall access frequency of different user information, the duration of each access, and the number of users of different types of user information, the business tag information is determined to label the target business module information and obtain a business portrait.
[0123] It can be seen that in the embodiment of the present application, the overall access frequency of different user information corresponding to the target business module information, the access duration of each access, and the number of users of different types of user information are obtained according to the second feedback information, so as to determine the business tag information and construct a business portrait, which can be used as research and development reference information for technical personnel.
[0124] Specifically, the business tag information includes: activity status tag, active user type tag;
[0125] Determine business tag information based on the overall access frequency of different user information, the access duration of each access of different user information, and the number of users of different types of user information, including:
[0126] Determining a first access value according to the overall access frequency of different user information;
[0127] Determine a second access value indicating the number of valid accesses according to the access duration of each access of different user information and a preset minimum access duration;
[0128] Determine an activity status label corresponding to the target business module information according to the first access value and the second access value;
[0129] According to the number of users of different types of user information, determine the active user type label corresponding to the target business module information.
[0130] Among them, the overall access frequency of different user information is used as the first access value; for each user information, when the access duration is not greater than the preset minimum access duration, it is determined that this is an invalid access, based on which the corresponding invalid access times and valid access times are obtained, and the second access value is obtained by the valid access times / (valid access times + invalid access times).
[0131] Furthermore, in one possible scenario, a weighted calculation can be performed based on the first access value and the second access value to obtain a comprehensive value. If the comprehensive value exceeds the first preset value, it is represented as an active business module, otherwise it is a general business module. In another possible scenario, a total value can be obtained based on the first access value * the second access value. If the comprehensive value exceeds the second preset value, it is represented as an active business module, otherwise it is a general business module. Among them, the first preset value and the second preset value can be set by technicians according to actual conditions.
[0132] Based on any of the above embodiments, an embodiment of the present application provides a method for data compression, including steps Sa1 to Sa4, wherein:
[0133] Sa1: Create a database table, the fields of which include user name, event and time (such as year), so that it can display what a user did and when. The time field is set in a multi-bit binary format, using 0 and 1 to mark whether an event has occurred. The number of digits can be customized by the user based on the number of days they want to count.
[0134] Sa2: Develop a data processing program: For the original data containing users, events, and dates, query whether there is a corresponding record in the table based on the user and event information in the original data. If there is a record, find the record and fill the corresponding date into the date digit corresponding to the time field of the table; if there is no record, add a new record based on the user and event information, and fill the corresponding date into the date digit corresponding to the time field of the record.
[0135] Sa3: Fill data into the table, where the filled data can be incremental data or full historical data.
[0136] All historical data can be sorted and stored in the database after being imported once. By configuring incremental data tasks to enter data regularly, tabular statistics of all data can be realized.
[0137] Sa4: Query table data. Users can query based on one or more of the user name, specific date, time period, and time name to find out whether the user has executed a certain business / or which businesses have been executed in a certain period of time or on a certain day. Users can also query which users have executed or accessed a certain business in a certain period of time or on a certain day.
[0138] Specifically, according to the technical solution, by counting the number of users visiting the web pages corresponding to a specific business module, the UV (Unique Visitor) corresponding to different business modules refers to the number of unique visitors, that is, the number of different users visiting the website. UV is used to measure the number of unique visitors to the website. A user will only be counted once no matter how many times he visits in a day.
[0139] For example, the specific query script implementation is shown in the following code:
[0140] select company_id,platform_code,count(distinct user_id) as user_count
[0141] from company_user_data
[0142] where company_id = 100
[0143] and event_year = 2024
[0144] and jan_visit & b'11111111111111111111111111111110' != b'00000000000000000000000000000000'
[0145] group by company_id,platform_code
[0146] It can be seen that compared with the original data storage method, the data storage implemented by the embodiment actually reduces the amount of data entered by dozens of times (theoretically, it can be reduced by hundreds of times), and the query performance is improved by 60%, which greatly reduces the storage cost and improves the user experience.
[0147] The following is an introduction to a device provided in an embodiment of the present application. The device described below and the method described above can be referred to each other. The device 200 of this embodiment is set in an electronic device. Figure 3 , Figure 3 It is a structural block diagram of a device of one embodiment of the present application, comprising:
[0148] The first acquisition module 210 is used to acquire a query request carrying query configuration information sent by a user terminal device, where the query configuration information includes at least one of: target time, target user information, and target service module information;
[0149] The second acquisition module 220 is used to acquire a database table; wherein the data in the database table stores each event information according to a multi-digit time, user information and business module information, the multi-digit time is in a multi-digit binary digital format, each binary digit corresponds to a unique time, and the binary digit is used to represent the occurrence of the event within the corresponding time;
[0150] The query and feedback module 230 is used to filter the target event information corresponding to the query configuration information from the database table, obtain feedback information, and send the feedback information to the user terminal device.
[0151] In a preferred example, the present application can be further configured as: the device 200 further includes:
[0152] A creation module is used to create a database table, the database table fields of which include user information, business module information and multi-digit time;
[0153] The data storage module is used to obtain multiple pieces of original data, each piece of original data includes: first user information, first business module information, and first time information;
[0154] For each piece of original data, query whether there is first event information in the database table according to the first user information and the first business module information;
[0155] If the first event information exists, the target binary digit number is determined from the multi-digit time in the first event information according to the first time information; if the first event information does not exist, the second event information is obtained according to the first user information and the first business module information recorded in the database table, and the target binary digit number is determined from the multi-digit time in the second event information according to the first time information;
[0156] The target binary digits are adjusted to the target value, and the target value indicates that the event has occurred;
[0157] After completing the storage of multiple pieces of original data, a database table is obtained.
[0158] In a preferred example, the present application can be further configured as follows: the multiple pieces of original data include: incremental data and / or full historical data.
[0159] In a preferred example, the present application can be further configured as follows: the query and feedback module 230 is also used to:
[0160] When the query configuration information includes: target time and target user information, query the target time and target user information corresponding to a number of target event information from the database table to obtain first feedback information, the first feedback information indicating relevant information of the business module information executed by the user corresponding to the target user information within the target time;
[0161] Or, when the query configuration information includes: target time and target business module information, several target event information corresponding to the target time and target business module information are queried from the database table to obtain second feedback information, and the second feedback information indicates relevant information of user information corresponding to the target business module information executed within the target time.
[0162] In a preferred example, the present application can be further configured as follows: the query and feedback module 230 is also used to:
[0163] When the user information is parent user information, determining the child user information corresponding to the user information according to a preset corresponding relationship, the preset corresponding relationship is a corresponding relationship between multiple parent user information and multiple child user information sets, each child user information set includes multiple child user information;
[0164] Querying a number of target event information corresponding to the target time and user information from a database table to obtain first sub-feedback information;
[0165] Querying a number of target event information corresponding to the target time and the sub-level user information from the database table to obtain the second sub-feedback information;
[0166] The first feedback information is determined according to the first sub-feedback information and the second sub-feedback information.
[0167] In a preferred example, the present application can be further configured as follows: the device 200 further includes:
[0168] The first portrait building module is used to determine the access frequency of each business module information corresponding to the target user information and the access duration of each access according to the first feedback information; determine the preferred business module information according to the access frequency of each business module information and the access duration of each access; and establish a user portrait corresponding to the target user information according to the preferred business module information;
[0169] The second portrait construction module is used to determine the overall access frequency of different user information corresponding to the target business module information, the access duration of each visit, and the number of users of different types of user information based on the second feedback information; determine the business tag information based on the overall access frequency of different user information, the access duration of each visit, and the number of users of different types of user information; and establish a business portrait corresponding to the target business module information based on the business tag information.
[0170] In a preferred example, the present application may be further configured as follows: the business tag information includes: an activity status tag, an active user type tag;
[0171] The second portrait building module is used to:
[0172] Determining a first access value according to the overall access frequency of different user information;
[0173] Determine a second access value indicating the number of valid accesses according to the access duration of each access of different user information and a preset minimum access duration;
[0174] Determine an activity status label corresponding to the target business module information according to the first access value and the second access value;
[0175] According to the number of users of different types of user information, determine the active user type label corresponding to the target business module information.
[0176] An electronic device is provided in an embodiment of the present application, such as Figure 4 As shown, Figure 4 The electronic device 300 shown includes: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the electronic device 300 may also include a transceiver 304. It should be noted that in actual applications, the transceiver 304 is not limited to one, and the structure of the electronic device 300 does not constitute a limitation on the embodiments of the present application.
[0177] The processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0178] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0179] The memory 303 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.
[0180] The memory 303 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the contents shown in the above method embodiment.
[0181] Figure 4 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0182] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding content in the aforementioned method embodiment.
[0183] An embodiment of the present application provides a computer program product, including a computer program, which implements the corresponding contents of the aforementioned method embodiment when the computer program is executed by a processor.
[0184] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.
[0185] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A data query method, characterized in that: include: Acquire a query request carrying query configuration information sent by a user terminal device, wherein the query configuration information includes at least one of: a target time, target user information, and target service module information; Obtain a database table; wherein the data in the database table stores each event information according to multi-digit time, user information and business module information, wherein the multi-digit time is in a multi-digit binary digital format, each binary digit corresponds to a unique time, and the binary digit is used to indicate the occurrence of the event within the corresponding time; The target event information corresponding to the query configuration information is filtered from the database table to obtain feedback information, and the feedback information is sent to the user terminal device.
2. The data query method according to claim 1, characterized in that: Before filtering the target event information corresponding to the query configuration information from the database table and obtaining the feedback information, the method further includes: Establish a database table, the database table fields of which include user information, business module information and multi-digit time; Acquire multiple pieces of original data, each piece of original data includes: first user information, first business module information, and first time information; For each piece of original data, query whether there is first event information in the database table according to the first user information and the first business module information; If the first event information exists, the target binary digit number is determined from the multi-digit time in the first event information according to the first time information; if the first event information does not exist, the second event information is obtained according to the first user information and the first business module information recorded in the database table, and the target binary digit number is determined from the multi-digit time in the second event information according to the first time information; adjusting the target binary digits to a target value, wherein the target value indicates that the event has occurred; After the plurality of pieces of original data are stored in the database, a database table is obtained.
3. The data query method according to claim 2, characterized in that: The multiple pieces of original data include: incremental data and / or full historical data.
4. The data query method according to any one of claims 1 to 3, characterized in that: The step of querying the target event information corresponding to the query request from the database table to obtain feedback information includes: When the query configuration information includes: target time and target user information, query the database table for a number of target event information corresponding to the target time and target user information to obtain first feedback information, wherein the first feedback information indicates relevant information of the business module information executed by the user corresponding to the target user information within the target time; Or, when the query configuration information includes: target time and target business module information, several target event information corresponding to the target time and target business module information are queried from the database table to obtain second feedback information, and the second feedback information represents relevant information of user information corresponding to the target business module information executed within the target time.
5. The data query method according to claim 4, characterized in that: Querying the database table for a number of target event information corresponding to the target time and the user information to obtain first feedback information includes: When the user information is parent user information, determining the child user information corresponding to the user information according to a preset corresponding relationship, wherein the preset corresponding relationship is a corresponding relationship between a plurality of parent user information and a plurality of child user information sets, each child user information set including a plurality of child user information; Querying the target event information corresponding to the target time and the user information from the database table to obtain the first sub-feedback information; Querying the target event information corresponding to the target time and the sub-level user information from the database table to obtain second sub-feedback information; The first feedback information is determined according to the first sub-feedback information and the second sub-feedback information.
6. The data query method according to claim 4, characterized in that: Each event information also includes a data dimension of access duration, and the method further includes at least one of the following: Determine the access frequency of each business module information corresponding to the target user information and the access duration of each access according to the first feedback information; determine the preferred business module information according to the access frequency of each business module information and the access duration of each access; And establish a user portrait corresponding to the target user information according to the preferred business module information; Determine the overall access frequency of different user information corresponding to the target business module information, the access duration of each visit, and the number of users of different types of user information based on the second feedback information; determine the business tag information based on the overall access frequency of different user information, the access duration of each visit, and the number of users of different types of user information; and establish a business profile corresponding to the target business module information based on the business tag information.
7. The data query method according to claim 6, characterized in that: The service tag information includes: activity status tag and active user type tag; Determine business tag information based on the overall access frequency of different user information, the access duration of each access of different user information, and the number of users of different types of user information, including: Determining a first access value according to the overall access frequency of different user information; Determine a second access value indicating the number of valid accesses according to the access duration of each access of different user information and a preset minimum access duration; Determine an activity status label corresponding to the target business module information according to the first access value and the second access value; According to the number of users of different types of user information, determine the active user type label corresponding to the target business module information.
8. A data query device, characterized in that: include: A first acquisition module, used to acquire a query request carrying query configuration information sent by a user terminal device, wherein the query configuration information includes at least one of: a target time, target user information, and target service module information; A second acquisition module is used to acquire a database table; wherein the data in the database table stores each event information according to a multi-digit time, user information and business module information, wherein the multi-digit time is in a multi-digit binary digital format, each binary digit corresponds to a unique time, and the binary digit is used to indicate the occurrence of the event within the corresponding time; The query and feedback module is used to filter the target event information corresponding to the query configuration information from the database table, obtain feedback information, and send the feedback information to the user terminal device.
9. An electronic device, characterized in that: include: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to: execute the steps of the data query method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded by the processor and executes the steps of the data query method according to any one of claims 1 to 7.