Access data statistics method and apparatus, computer device, and storage medium

Abstract

The application relates to a data access statistical method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining access data obtained by accessing a target resource; determining an access amount corresponding to the target resource based on the access data; when the access amount meets a mass access condition, converting each access data into corresponding hash data; the hash data comprises a hash number and a hash value; distributing each hash value to a bucket space associated with the corresponding hash number; performing a base estimation process according to the hash values included in each bucket space to obtain an access statistical result corresponding to the target resource. The method can reduce the consumption of processing resources for data statistics and improve the processing efficiency.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer device, and storage medium for accessing and statistically analyzing data. Background Technology

[0002] With the rapid development of computer technology, computer devices need to process massive amounts of data every moment, such as object request data, browsing data, and validation data. Therefore, statistical techniques for handling massive amounts of data have emerged. Through data statistical techniques, the distribution of massive amounts of data in various aspects can be statistically analyzed, enabling subsequent data retrieval or querying.

[0003] Traditional data statistics methods require determining the numerical range of the massive dataset to be analyzed. Based on the upper limit of the range and a pre-defined statistical error, the class interval of the statistical interval where the upper limit falls is determined. Then, based on the class interval and statistical error, the numerical range is divided into at least two statistical intervals according to a pre-defined class interval strategy. Finally, the number of histogram groups used for each statistical interval is determined, and each massive data point is statistically analyzed within the histogram of its corresponding statistical interval. However, this data statistics method requires enormous computing resources and has low processing efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide a data access statistics method, apparatus, computer equipment, and storage medium that can improve processing efficiency in response to the above-mentioned technical problems.

[0005] A method for accessing data statistics, the method comprising:

[0006] Obtain access data obtained from accessing the target resource;

[0007] The access volume corresponding to the target resource is determined based on the access data;

[0008] When the access volume meets the massive access condition, each access data is converted into corresponding hash data; the hash data includes a hash number and a hash value;

[0009] Each hash value is assigned to a bucket space associated with its corresponding hash number;

[0010] Based on the hash values ​​included in each bucket space, cardinality estimation is performed to obtain access statistics results corresponding to the target resource.

[0011] An access data statistics device, the device comprising:

[0012] The acquisition module is used to acquire access data obtained from accessing the target resource;

[0013] The determination module is used to determine the access volume corresponding to the target resource based on the access data;

[0014] The conversion module is used to convert each access data into corresponding hash data when the access volume meets the massive access condition; the hash data includes a hash number and a hash value;

[0015] An allocation module is used to allocate each hash value to a bucket space associated with the corresponding hash number;

[0016] The processing module is used to perform cardinality estimation processing based on the hash values ​​included in each bucket space to obtain access statistics results corresponding to the target resource.

[0017] In one embodiment, the acquisition module is further configured to acquire log data corresponding to the target resource within a preset time period; and to acquire from the log data the object identifier that accesses the target resource, and the access identifier corresponding to the access request sent to the target resource.

[0018] In one embodiment, the determining module is further configured to generate an object set and an access set on the aggregation operator; write each object identifier into the object set and the access identifier into the access set; after each writing of the object identifier and the access identifier, based on the object identifier in the object set, count the number of objects accessing the target resource, and based on the access identifier in the access set, count the number of times the target resource has been accessed.

[0019] In one embodiment, the device further includes a creation module; the creation module is configured to create an array of a preset length and divide the array into multiple bucket spaces when the access volume meets the massive access condition;

[0020] The allocation module is further configured to number the plurality of bucket spaces using the hash number in each of the hash data, and allocate each hash value to the bucket space corresponding to the corresponding hash number.

[0021] In one embodiment, the allocation module is further configured to use a preset number of digits as a hash number, starting from the first digit of the hash data, and use the digits after the preset number of digits as hash values; and to number the plurality of bucket spaces using each hash number.

[0022] In one embodiment, the processing module is further configured to determine the position of the first occurrence of a preset value in each hash value of each bucket space, and to determine the access statistics result corresponding to the target resource based on each of the preset value positions.

[0023] In one embodiment, the processing module is further configured to: determine the first numerical position of the preset value of the first occurrence of each hash value in each bucket space corresponding to the object identifier within the corresponding hash value; determine the average value of objects accessing the target resource based on each of the first numerical positions; determine the second numerical position of the preset value of the first occurrence of each hash value in each bucket space corresponding to the access identifier within the corresponding hash value; determine the average access value of the target resource based on each of the second numerical positions; and determine the access statistics result corresponding to the target resource based on the average value of objects and the average access value.

[0024] In one embodiment, the processing module is further configured to determine the object cardinality for accessing the target resource based on the object mean; the object cardinality is used to characterize the number of objects accessing the target resource; determine the access cardinality for the target resource being accessed based on the access mean; the access cardinality is used to characterize the number of times the target resource is accessed; and use the object cardinality and the access cardinality as the access statistics result corresponding to the target resource.

[0025] In one embodiment, the target resource is promotional information, and the access statistics result includes the object base and the access base; the device further includes a push module; the push module is used to filter out target promotional information that meets the push conditions based on at least one of the object base and the access base corresponding to each promotional information; and push the target promotional information to different promotional channels.

[0026] In one embodiment, the apparatus further includes a prediction module; the prediction module is configured to acquire object attribute features corresponding to the target resource and object attribute features corresponding to the object identifier; and predict the conversion rate of the target resource within a target time period based on the object attribute features, the object attribute features and the access statistics results.

[0027] In one embodiment, the device further includes a push module; the push module is configured to filter target object identifiers that meet the conversion conditions based on the conversion rate of each object identifier to the target resource within the target time period; and push information containing the target resource to the target object identifiers.

[0028] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0029] Obtain access data obtained from accessing the target resource;

[0030] The access volume corresponding to the target resource is determined based on the access data;

[0031] When the access volume meets the massive access condition, each access data is converted into corresponding hash data; the hash data includes a hash number and a hash value;

[0032] Each hash value is assigned to a bucket space associated with its corresponding hash number;

[0033] Based on the hash values ​​included in each bucket space, cardinality estimation is performed to obtain access statistics results corresponding to the target resource.

[0034] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0035] Obtain access data obtained from accessing the target resource;

[0036] The access volume corresponding to the target resource is determined based on the access data;

[0037] When the access volume meets the massive access condition, each access data is converted into corresponding hash data; the hash data includes a hash number and a hash value;

[0038] Each hash value is assigned to a bucket space associated with its corresponding hash number;

[0039] Based on the hash values ​​included in each bucket space, cardinality estimation is performed to obtain access statistics results corresponding to the target resource.

[0040] A computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0041] Obtain access data obtained from accessing the target resource;

[0042] The access volume corresponding to the target resource is determined based on the access data;

[0043] When the access volume meets the massive access condition, each access data is converted into corresponding hash data; the hash data includes a hash number and a hash value;

[0044] Each hash value is assigned to a bucket space associated with its corresponding hash number;

[0045] Based on the hash values ​​included in each bucket space, cardinality estimation is performed to obtain access statistics results corresponding to the target resource.

[0046] The aforementioned access data statistics method, apparatus, computer equipment, computer-readable storage medium, and computer program product acquire access data obtained from accessing a target resource, determine the access volume corresponding to the target resource based on the access data, and thus determine the magnitude of the access data. When the access volume meets the condition of massive access, indicating that the magnitude of the access data is relatively large, each access data is converted into corresponding hash data, which includes a hash number and a hash value. Each hash value is allocated to a bucket space associated with the corresponding hash number. Cardinality estimation is performed based on the hash values ​​included in each bucket space. Cardinality estimation can quickly and accurately obtain access statistics results corresponding to the target resource by statistically analyzing massive amounts of access data. Furthermore, applying cardinality estimation to access data statistics scenarios with large data volumes can save storage space while reducing the consumption of computing resources, thereby effectively improving processing efficiency. Attached Figure Description

[0047] Figure 1 This is a diagram illustrating the application environment of a data statistics access method in one embodiment.

[0048] Figure 2 This is a flowchart illustrating a data statistics access method in one embodiment;

[0049] Figure 3 This is a schematic diagram of a process for determining the access volume corresponding to a target resource based on access data in one embodiment.

[0050] Figure 4 This is a flowchart illustrating the process of determining the position of the first occurrence of a preset value in each hash value of each bucket space, and determining the access statistics result corresponding to the target resource based on each position of the preset value in one embodiment.

[0051] Figure 5 This is a flowchart illustrating the process of determining the access statistics corresponding to a target resource based on the object mean and the access mean in one embodiment.

[0052] Figure 6 This is a flowchart illustrating the data statistics access method in another embodiment;

[0053] Figure 7 This is a flowchart illustrating the data statistics method in one embodiment;

[0054] Figure 8 This is a structural block diagram of an access data statistics device in one embodiment;

[0055] Figure 9 Here is a structural block diagram of the data statistics device in another embodiment;

[0056] Figure 10This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] This application relates to the field of Artificial Intelligence (AI) technology. AI is the theory, methods, technology, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine capable of reacting in a manner similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities. The solutions provided in this application relate to methods for statistical analysis of access data in AI, which are specifically illustrated through the following embodiments.

[0059] The access data statistics method provided in this application can be applied to, for example... Figure 1 The access data statistics system shown. Figure 1 As shown, the access data statistics system includes a terminal 110 and a server 120. In one embodiment, both the terminal 110 and the server 120 can independently execute the access data statistics method provided in this embodiment. The terminal 110 and the server 120 can also work together to execute the access data statistics method provided in this embodiment. When the terminal 110 and the server 120 work together to execute the access data statistics method provided in this embodiment, the terminal 110 obtains access data obtained from accessing the target resource and sends the access data to the server 120. The server 120 determines the access volume corresponding to the target resource based on the access data. When the access volume meets the massive access condition, each access data is converted into corresponding hash data, which includes a hash number and a hash value. The server 120 allocates each hash value to a bucket space associated with the corresponding hash number, performs cardinality estimation processing based on the hash values ​​included in each bucket space, and obtains the access statistics result corresponding to the target resource. The server 120 returns the access statistics result of the target resource to the terminal 110.

[0060] The server 120 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services. The terminal 110 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, vehicle terminal, smart TV, etc., but is not limited to these. The terminal 110 and the server 120 can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0061] In one embodiment, multiple servers can form a blockchain, with the servers acting as nodes on the blockchain.

[0062] In one embodiment, data related to the access data statistics method may be stored on a blockchain, such as access data, access volume, massive access conditions, hash data, and access statistics results, but not limited to these.

[0063] In one embodiment, such as Figure 2 As shown, a method for accessing data statistics is provided, which is applied to a computer device (the computer device may specifically be...) Figure 1 Taking a terminal or server as an example, the following steps are included:

[0064] Step S202: Obtain the access data obtained from accessing the target resource.

[0065] The target resource can be at least one of various applications, websites, web pages, promotional information, and various items. Applications can include a parent application and child applications running on the parent application. A parent application is an application that can run independently. A child application is an application that cannot run independently and requires the parent application to function. Both parent and child applications can include, but are not limited to, instant messaging applications, SNS (Social Network Sites) applications, short video applications, long video applications, game applications, music sharing applications, shopping and sales applications, UGC (User Generated Content) applications, and various intelligent recognition applications.

[0066] A website is a collection of web pages created using tools such as HTML (HTML) according to certain rules on the Internet to display specific content. Simply put, a website is a communication tool; users can use websites to publish information they want to share or to provide related online services. Websites include, but are not limited to, social networking sites, e-commerce websites, educational websites, and search engines.

[0067] Information can be in various forms, including but not limited to text, emoticons, images, audio, video, files, or links. Items can be physical or virtual. Physical items include various physical products, specifically electronic devices such as mobile phones, computers, laptops, and watches, as well as clothing and shoes; no further restrictions are imposed.

[0068] Virtual items include, but are not limited to, insurance products, financial products, virtual gift resources, virtual scenes, virtual characters, and virtual props. Virtual scenes can specifically be game scenes on gaming devices, virtual reality simulation scenes, etc.; virtual characters can specifically be various characters in games; and virtual props can specifically be various props in games.

[0069] Access data refers to the data obtained from accessing a target resource. It may include at least one of an object identifier and an access identifier, and may also include content identifiers for different parts of the target resource. The object identifier can be a user identifier, and the access identifier is the identifier corresponding to a single access request, that is, the identifier corresponding to a single access request sent by the object identifier. Different access requests correspond to different access identifiers.

[0070] In one embodiment, each access request may correspond to an access identifier, and one object identifier may send multiple access requests. A single access request may correspond to multiple access contents, each with its own content identifier; therefore, one access identifier may correspond to multiple content identifiers. For example, if the target resource is an e-commerce website, which includes various products, each with a corresponding product identifier, then the access data includes the object identifier accessing the e-commerce website, the access identifiers corresponding to the access requests sent by each object identifier, and the product identifiers corresponding to the products accessed on the e-commerce website.

[0071] Specifically, a computer can obtain access data obtained when an object identifier accesses a target resource. For example, when the target resource is a website, the computer device can obtain access data generated when a user accesses the website. When the target resource is an application, the computer device can obtain access data generated when a user accesses the application.

[0072] Step S204: Determine the access volume corresponding to the target resource based on the access data.

[0073] In this context, access volume refers to the number of times a target resource is accessed, including at least one of the number of object identifiers accessing the target resource and the number of access identifiers corresponding to access requests. Access volume may also include the number of content identifiers corresponding to content accessed within the target resource.

[0074] Specifically, the computer device can determine the number of object identifiers accessing the target resource based on the access data, and determine the access identifier corresponding to the access request for accessing the target resource based on the access data, and determine the number of access identifiers. The computer device can use at least one of the number of object identifiers and the number of access identifiers as the access quantity corresponding to the target resource.

[0075] In one embodiment, the computer device may also determine the content accessed by the object identifier in the target resource based on the access data, and determine the content identifier corresponding to the accessed content, and take at least one of the number of object identifiers, the number of access identifiers, and the number of content identifiers as the access volume corresponding to the target resource.

[0076] Step S206: When the access volume meets the massive access condition, each access data is converted into corresponding hash data; the hash data includes a hash number and a hash value.

[0077] In this context, hash data refers to transforming an input of arbitrary length into a fixed-length output using a hash algorithm; this output is the hash value. Meeting the massive volume condition means that the number of accesses exceeds a certain threshold.

[0078] Specifically, the computer device acquires a large number of access conditions and determines whether the access volume of the target resource meets the massive access conditions. When the access volume of the target resource meets the massive access conditions, the computer device can use a hash function to convert each access data into corresponding hash data. This hash data may include a hash number and a hash value.

[0079] In one embodiment, the access data includes an object identifier and an access identifier. When the access volume meets the conditions for massive access, the computer device can convert each object identifier and each access identifier into corresponding hash data using a hash function.

[0080] In one embodiment, access data includes object identifiers and access identifiers, and access volume includes object quantity and accessed quantity. Object quantity refers to the number of object identifiers accessing the target resource, and accessed quantity refers to the number of access identifiers corresponding to access requests for the target resource. When at least one of the object quantity and accessed quantity does not meet the massive access condition, the object quantity and accessed quantity are used as the access statistics results for the target resource.

[0081] Step S208: Allocate each hash value to a bucket space associated with the corresponding hash number.

[0082] Bucket space refers to the space allocated in memory, specifically a hash bucket.

[0083] Specifically, a computer device can divide memory into multiple bucket spaces and associate each bucket space with a hash number in the hash data. The computer device can then allocate each hash value to a bucket space associated with its corresponding hash number.

[0084] Step S210: Perform cardinality estimation based on the hash values ​​included in each bucket space to obtain access statistics results corresponding to the target resource.

[0085] Cardinality estimation, also known as cardinality counting, is the statistical calculation of the number of unique elements in a repeatable set. In practical applications, it can be used as the cardinality for calculating various metrics. For example, in e-commerce, it can be used to calculate the number of unique visitors to a website and the number of product categories viewed on a given day.

[0086] Access statistics refer to the results obtained by statistically analyzing access data, and may include at least one of object cardinality and access cardinality. Object cardinality is used to characterize the number of objects accessing the target resource; further, object cardinality is used to characterize the number of unique object identifiers accessing the target resource. Access cardinality is used to characterize the number of times the target resource is accessed.

[0087] Specifically, for each bucket space, the computer device performs cardinality estimation based on the hash values ​​included in the bucket space to obtain access statistics corresponding to the target resource. Further, the computer device can perform cardinality estimation based on the hash values ​​included in the bucket space to obtain the object cardinality for accessing the target resource and the access cardinality of the target resource being accessed, and use at least one of the object cardinality and the access cardinality as the access statistics corresponding to the target resource.

[0088] In the aforementioned access data statistics method, access data obtained from accessing the target resource is acquired, and the access volume corresponding to the target resource is determined based on the access data to determine the magnitude of the access data. When the access volume meets the condition of massive access, indicating that the magnitude of the access data is relatively large, each access data is converted into corresponding hash data, which includes a hash number and a hash value. Each hash value is allocated to a bucket space associated with the corresponding hash number. Cardinality estimation is performed based on the hash values ​​included in each bucket space. Cardinality estimation can quickly and accurately obtain access statistics results corresponding to the target resource by statistically analyzing massive amounts of access data. Furthermore, applying cardinality estimation to access data statistics scenarios with large data volumes can save storage space while reducing the consumption of computing resources, thereby effectively improving processing efficiency.

[0089] In one embodiment, the access data includes an object identifier and an access identifier; obtaining access data obtained by accessing the target resource includes:

[0090] Obtain log data corresponding to the target resource within a preset time period; extract the object identifier that accessed the target resource and the access identifier corresponding to the access request sent to the target resource from the log data.

[0091] Specifically, the computer device acquires log data within a preset time period and filters out the log data corresponding to the target resource. The computer device extracts the object identifiers that accessed the target resource from the log data, thus obtaining the object identifiers that accessed the target resource within the preset time period. Furthermore, the computer device determines the access request sent to the target resource by each object identifier from the log data and obtains the access identifier corresponding to each access request. The computer device uses each object identifier and each access identifier as the access data corresponding to the target resource.

[0092] In one embodiment, the computer device can determine the content accessed by an object identifier in a target resource from log data, and obtain the content identifier of the accessed content. The computer device can use the object identifier, access identifier, and content identifier as access data corresponding to the target resource.

[0093] In this embodiment, log data corresponding to the target resource within a preset time period is obtained. The object identifier that accesses the target resource and the access identifier corresponding to the access request sent to the target resource are obtained from the log data. Only the object identifier and the access identifier are used as the access data of the target resource, so that the access data can be statistically analyzed based on the object identifier and the access identifier. The data used is simple, which can effectively reduce the resource consumption of statistical processing, save storage space, and improve processing efficiency.

[0094] In one embodiment, such as Figure 3 As shown, the number of visits corresponding to the target resource is determined based on the access data, including:

[0095] Step S302: Generate an object set and an access set on the aggregation operator.

[0096] Aggregation operators can combine multiple inputs into a single output, and an aggregation operator can be a reduce operator. The reduce operator operates on data streams grouped by the same key, accepting two inputs and generating one output, that is, performing a summarization operation by combining two inputs into one, generating a new data of the same type.

[0097] Object collections are used to collect object identifiers and automatically count the number of object identifiers collected. Access collections collect access identifiers and automatically count the number of access identifiers collected.

[0098] Specifically, after acquiring the access data, the computer device sends the access data to the aggregation operator. The computer device then constructs an object set and an access set on the aggregation operator.

[0099] Step S304: Write each object identifier into the object set and write the access identifier into the access set.

[0100] Specifically, the aggregation operator writes each received object identifier to the object set and each received access identifier to the access set. The aggregation operator can receive at least one object identifier each time, and similarly, the aggregation operator can receive at least one access identifier each time.

[0101] When an aggregation operator receives multiple object identifiers, it can write all object identifiers to the object collection at once, or it can write each object identifier to the object collection in multiple parts. The number of object identifiers written each time can be set according to requirements, and there are no restrictions here. Similarly, when an aggregation operator receives multiple access identifiers, it can write all access identifiers to the access collection at once, or it can write each access identifier to the access collection in multiple parts. The number of access identifiers written each time can be set according to requirements, and there are no restrictions here.

[0102] Step S306: After each writing of object identifier and access identifier, count the number of objects accessing the target resource based on the object identifier in the object set, and count the number of times the target resource has been accessed based on the access identifier in the access set.

[0103] Specifically, the aggregation operator writes the received object identifiers into the object set, and after each write, it counts the size of the current object set. The size of the current object set represents the number of object identifiers in the current object set, i.e., the number of objects accessing the target resource. Furthermore, the size of the current object set represents the number of unique object identifiers in the current object set.

[0104] The aggregation operator writes the received access identifiers into the access set, and after each write, it counts the size of the current access set. The size of the current access set represents the number of access identifiers in the current access set, i.e., the access frequency of the target resource. Furthermore, the size of the current access set represents the number of unique access identifiers in the current access set.

[0105] In this embodiment, an object set and an access set are generated on the aggregation operator. This allows for the automatic counting of the number of object identifiers in the object set after each object identifier is written to it, thus enabling a rapid count of the number of objects accessing the target resource. Similarly, the automatic counting of the number of access identifiers in the access set after each access identifier is written to it allows for a rapid count of the number of times the target resource has been accessed.

[0106] In one embodiment, after determining the access volume corresponding to the target resource based on access data, the method further includes: when the access volume meets the massive access condition, creating an array of a preset length and dividing the array into multiple bucket spaces.

[0107] Assigning each hash value to a bucket space associated with its corresponding hash number includes: numbering multiple bucket spaces using the hash number in each hash data, and assigning each hash value to the bucket space corresponding to its corresponding hash number.

[0108] Specifically, after determining the access volume corresponding to the target resource based on the access data, the computer device obtains the massive access conditions and determines whether the access volume meets the massive access conditions. When the access volume meets the massive access conditions, it indicates that the amount of accessed data is relatively large. In this case, the computer device can create an array of a preset length and divide the array into multiple bucket spaces.

[0109] When the access volume reaches a massive level, the computer device converts each accessed data into corresponding hash data, which includes a hash number and a hash value. Multiple hash data may have the same hash number but different hash values, or different hash numbers but the same hash value, or both hash number and hash value may be different.

[0110] The computer device assigns unique hash numbers to each bucket space. The number of bucket spaces can be the same as the number of hash numbers. The computer device allocates hash values ​​to the bucket spaces numbered by the corresponding hash numbers. Following the same processing method, each hash value can be allocated to a corresponding bucket space.

[0111] For example, hash data A has a hash number of 0101 and a hash value of 11101, hash data B has a hash number of 0101 and a hash value of 001101, and hash data C has a hash number of 1001 and a hash value of 110110. The bucket spaces are then numbered using hash numbers 0101 and 1001, resulting in bucket space numbered 0101 and bucket space numbered 1001. The computer can allocate the hash values ​​001101 of hash data A and 001101 of hash data B to bucket space numbered 0101, and allocate the hash value 110110 of hash data C to bucket space numbered 1001.

[0112] In one embodiment, the access data includes an object identifier and an access identifier; when at least one of the object quantity corresponding to the object identifier and the access quantity corresponding to the access identifier meets the massive access condition, an object array and an access array of preset length are created, and the object array and the access data are divided into multiple bucket spaces respectively.

[0113] When at least one of the object quantity corresponding to the object identifier and the access quantity corresponding to the access identifier meets the massive access condition, each object identifier is converted into corresponding hash data, and each access identifier is converted into corresponding hash data. The hash data includes a hash number and a hash value.

[0114] The process of assigning each hash value to a bucket space associated with its corresponding hash number includes: for multiple bucket spaces under an object array, numbering the multiple bucket spaces by the hash number in the hash data corresponding to each object identifier, and assigning the hash value in the hash data corresponding to each object identifier to the bucket space corresponding to its corresponding hash number; for multiple bucket spaces under an access array, numbering the multiple bucket spaces by the hash number in the hash data corresponding to each access identifier, and assigning the hash value in the hash data corresponding to each access identifier to the bucket space corresponding to its corresponding hash number.

[0115] In this embodiment, when the access volume meets the massive access condition, an array of preset length is created and divided into multiple bucket spaces to distribute the massive access data to each bucket space for separate processing. The multiple bucket spaces are numbered using the hash number in each hash data, allowing each hash value to be allocated to its corresponding bucket space based on the hash number. This enables simultaneous cardinality estimation processing of the hash values ​​in each bucket space, achieving parallel cardinality estimation and effectively improving data processing efficiency.

[0116] In one embodiment, multiple bucket spaces are numbered using hash numbers in each hash data, including:

[0117] Starting from the first digit of the hash data, a value with a preset number of digits is used as the hash number, and the value after the preset number of digits is used as the hash value; multiple bucket spaces are numbered using each hash number.

[0118] Specifically, after the computer device converts the accessed data into corresponding hash data, it selects a preset number of digits from the first digit of the hash data as the hash number, and uses all the digits after the preset number of digits as the hash value. For example, if the hash data is 0101 1110101, then the first 4 digits are taken as the hash number, and all the digits after the 4th digit constitute the hash value, that is, the hash number is 0101 and the hash value is 1110101.

[0119] The computer equipment can determine all hash numbers, filter out the unique hash numbers, number multiple bucket spaces using the unique hash numbers, and number one bucket space for each unique hash number.

[0120] In this embodiment, starting from the first digit of the hash data, a preset number of digits is used as the hash number, and the digits after the preset number of digits are used as the hash value. Multiple bucket spaces are numbered using each hash number so that each hash value can be allocated in the future. This facilitates the parallel cardinality estimation processing of the hash values ​​in each bucket space and improves processing efficiency.

[0121] In one embodiment, cardinality estimation is performed based on the hash values ​​included in each bucket space to obtain access statistics corresponding to the target resource, including:

[0122] Determine the position of the first occurrence of a preset value in each hash value of each bucket space, and determine the access statistics corresponding to the target resource based on each value position.

[0123] The position of the preset value refers to the position of the preset value in the hash value. The preset value can be 1 or 0.

[0124] Specifically, for each hash value in each bucket space, the computer device can determine the position of the first preset value in the hash value, thus obtaining the position of the preset value corresponding to each hash value.

[0125] For example, with a preset value of 1, the first 1 in hash value 11101 and hash value 001101 appears in the 1st and 3rd positions, respectively.

[0126] The computer device determines the access statistics corresponding to the target resource based on the numerical positions in each bucket space. Further, for each bucket space, the computer device calculates the average position value of each numerical position in the bucket space, obtaining the average position value corresponding to each bucket space. Based on the average position value corresponding to each bucket space, the access statistics corresponding to the target resource are determined.

[0127] In one embodiment, the computer device determines the object cardinality for accessing the target resource and the access cardinality for the target resource based on the average location value corresponding to each bucket space, and uses at least one of the object cardinality and the access cardinality as the access statistics result corresponding to the target resource.

[0128] In a set of values, the greater the cardinality, the higher the probability of consecutive preset values ​​appearing. In this embodiment, the overall cardinality is estimated by the position of the first occurrence of the preset data in the hash value, which can effectively count massive amounts of access data. Furthermore, distributing the hash value into various bucket spaces to estimate the cardinality of each bucket space separately can eliminate errors caused by randomness and improve the accuracy of the estimation. Then, based on the estimated cardinality values ​​of each bucket, the overall cardinality of the massive access data is further obtained, reducing the overall error and improving the accuracy of the statistics.

[0129] In one embodiment, such as Figure 4 As shown, the access data includes object identifiers and access identifiers; the position of the first occurrence of a preset value in each hash value of each bucket space is determined, and the access statistics corresponding to the target resource are determined based on each value position, including:

[0130] Step S402: Determine the first value position of the preset value of the first occurrence of each hash value in each bucket space corresponding to the object identifier.

[0131] Specifically, when at least one of the object quantity corresponding to an object identifier and the access quantity corresponding to an access identifier meets the massive access condition, the computer device creates an object array and an access array of preset lengths, and divides the object array and access data into multiple bucket spaces respectively. When at least one of the object quantity corresponding to an object identifier and the access quantity corresponding to an access identifier meets the massive access condition, each object identifier is converted into corresponding hash data, and each access identifier is converted into corresponding hash data. The hash data includes a hash number and a hash value. For the multiple bucket spaces under the object array, the multiple bucket spaces are numbered using the hash number in the hash data corresponding to each object identifier, and the hash value in the hash data corresponding to each object identifier is allocated to the bucket space corresponding to the corresponding hash number. The multiple bucket spaces under the object array are the bucket spaces corresponding to the object identifiers. For the multiple bucket spaces under the access array, the multiple bucket spaces are numbered using the hash number in the hash data corresponding to each access identifier, and the hash value in the hash data corresponding to each access identifier is allocated to the bucket space corresponding to the corresponding hash number. The multiple bucket spaces under the access array are the bucket spaces corresponding to the access identifiers.

[0132] For a bucket space within an object array, the computer device can determine a preset value for the first occurrence of each hash value in the bucket space, and determine the first value position of the preset value within the corresponding hash value, thus obtaining each first value position corresponding to that bucket space. Following the same processing method, the first value positions corresponding to each bucket space within the object array can be obtained.

[0133] Step S404: Determine the average value of objects accessing the target resource based on each first numerical position.

[0134] Here, the object mean refers to the mean value corresponding to the first value position in each bucket space under the object array.

[0135] Specifically, for each bucket space in the object array, the computer device sums the positions of the first values ​​corresponding to each bucket space and then calculates the average to obtain the average position value for that bucket space. Following the same processing method, the average position value corresponding to each bucket space in the object array can be obtained.

[0136] For example, a computer device can calculate the positional mean using the following formula:

[0137]

[0138] Where k is the positional mean, M i Let be the position of the i-th value in the bucket space.

[0139] The computer device determines the average value of the objects accessing the target resource based on the average value of the location corresponding to each bucket space under the object array.

[0140] In one embodiment, the computer device sums the average position values ​​corresponding to each bucket space under the object array to obtain the average value of objects accessing the target resource.

[0141] For example, a computer device can calculate the mean of an object using the following formula:

[0142]

[0143] Where, k user_id Let m be the average value of the objects, m be the number of buckets corresponding to each object identifier, and j be the j-th bucket corresponding to each object identifier. This represents the i-th numerical position in the bucket space corresponding to the object identifier.

[0144] Step S406: Determine the position of the second value in the corresponding hash value where the preset value of the first occurrence of each hash value in each bucket space corresponding to the access identifier is located.

[0145] Specifically, for each bucket space in the access array, the computer device can determine the preset value of the first occurrence of each hash value in the bucket space, and determine the second value position of the preset value in the corresponding hash value, thus obtaining each second value position corresponding to that bucket space. Following the same processing method, the second value positions corresponding to each bucket space in the access array can be obtained.

[0146] Step S408: Determine the average access value of the target resource based on each second numerical position.

[0147] The average access value refers to the average value corresponding to the second numerical position of each bucket space in the access array.

[0148] Specifically, for each bucket space in the access array, the computer device sums the values ​​of the second numerical positions corresponding to each bucket space and then calculates the average value to obtain the average position value for that bucket space. Following the same processing method, the average position value corresponding to each bucket space in the access array can be obtained. Based on the average position value corresponding to each bucket space in the access array, the computer device determines the average access value for the target resource.

[0149] In one embodiment, the computer device sums the average of the locations corresponding to each bucket space in the access array to obtain the average access value of the target resource.

[0150] For example, computer devices can calculate the average access value using the following formula:

[0151]

[0152] Where, k request_id The average number of visits is denoted by m, where m is the number of buckets corresponding to the access identifier, and j represents the j-th bucket corresponding to the access identifier. This indicates the position of the i-th value in the bucket space corresponding to the access identifier.

[0153] Step S410: Determine the access statistics corresponding to the target resource based on the object average and access average.

[0154] Specifically, the computer device estimates the number of objects accessing the target resource based on the object average, and estimates the number of accesses to the target resource based on the access average. The object average and the access average are used as the access statistics for the target resource.

[0155] In this embodiment, hash values ​​are allocated to each bucket space to estimate the cardinality of each bucket space, thus eliminating errors caused by randomness. A preset value for the first occurrence of each hash value in each bucket space corresponding to an object identifier is determined as the first numerical position within the corresponding hash value. Based on each first numerical position, the average number of objects accessing the target resource is estimated, effectively reducing errors in object quantity estimation. A preset value for the first occurrence of each hash value in each bucket space corresponding to an access identifier is determined as the second numerical position within the corresponding hash value. Based on each second numerical position, the average number of access identifiers corresponding to access requests accessing the target resource is determined, effectively reducing errors in access quantity estimation. Based on the object average and access average, access statistics corresponding to the target resource are determined. This allows for further estimation of the overall cardinality of massive access data based on local averages, reducing overall error and improving estimation accuracy.

[0156] In one embodiment, such as Figure 5 As shown, based on the object mean and access mean, the access statistics corresponding to the target resource are determined, including:

[0157] Step S502: Determine the object cardinality for accessing the target resource based on the object mean; the object cardinality is used to characterize the number of objects accessing the target resource.

[0158] Specifically, object cardinality is used to characterize the number of unique object identifiers accessing a target resource. A computer device obtains an exponential function for cardinality estimation, substitutes the object mean into this exponential function, and calculates the object cardinality accessing the target resource.

[0159] For example, a computer device can calculate the cardinality n of an object using the following formula. user_id :

[0160]

[0161] Step S504: Determine the access base number of the target resource based on the access average; the access base number is used to characterize the number of times the target resource is accessed.

[0162] Specifically, the access cardinality represents the number of times a target resource is accessed. The computer device substitutes the average number of accesses into an exponential function used for cardinality estimation to calculate the access cardinality of the target resource.

[0163] For example, a computer device can calculate the access cardinality n using the following formula. request_id :

[0164]

[0165] Step S506: Use the object cardinality and access cardinality as the access statistics results corresponding to the target resource.

[0166] Specifically, computer devices can use the object cardinality and access cardinality as access statistics for the target resource.

[0167] In one embodiment, the computer device can calculate the object cardinality and access cardinality corresponding to each target resource according to the above processing method, thereby obtaining the access statistics results corresponding to each target resource.

[0168] In one embodiment, the computer device can calculate the target resource's target audience and access volume across different promotional channels using the processing described above. It can also calculate the target resource's target audience and access volume over different time periods.

[0169] In one embodiment, the computer device can generate visualization charts based on the object base and access base for different time periods, and use the object base, access base, and visualization charts as the access statistics results corresponding to the target resource. The visualization charts can be line graphs, histograms, pie charts, etc., but are not limited to these.

[0170] In one embodiment, a computer device can draw visualization charts based on the target resource's target audience and access volume in different promotion channels, and use the target audience, access volume, and visualization charts as the access statistics results corresponding to the target resource.

[0171] In this embodiment, the object cardinality for accessing the target resource is determined based on the object mean, and the access cardinality for the target resource is determined based on the access mean. The object cardinality can represent the number of objects accessing the target resource, and the access cardinality can represent the number of times the target resource is accessed. This allows for effective statistical analysis of massive access data using both object cardinality and access cardinality. Furthermore, statistical analysis of massive access data using object cardinality and access cardinality significantly saves storage space while keeping errors within a certain range.

[0172] like Figure 6 The diagram shown is a flowchart illustrating a data statistics access method in one embodiment, applied to a computer device, and includes:

[0173] Step S602: Collect data and report it to the server:

[0174] The software development kit (SDK) collects user behavior and item data related to the target resource across different systems and reports it to the backend server. The backend server then sends the data to HDFS. HDFS is a highly fault-tolerant, scalable distributed file system and an important component of the Hadoop system.

[0175] Step S604: Read and parse the log data.

[0176] Spark is used to read log data stored in HDFS, and the map operator is used to parse the request_id (i.e., access identifier) ​​and user_id (i.e., user identifier). Each data entry is output in the format (request_id, 1) and (user_id, 1), and the output data is transmitted to downstream aggregation operators, which can be reduce operators. Aggregation is performed based on either request_id or user_id; entries with the same request_id and user_id will be transmitted to the same reduce operator.

[0177] Step S606: The aggregation operator counts the number of visits corresponding to the target resource.

[0178] Construct a hashset(u) for user_id and a hashset(req) for request_id on the memory of the reduce operator. Write each request_id data into hashset(req) and each user_id data into hashset(u). Then, calculate the current size of hashset(u) n_user and the current size of hashset(req) n_request.

[0179] Step S608: Determine if the access volume meets the massive access condition:

[0180] Set a user threshold t_user and a request threshold t_request respectively. If n_user is less than or equal to t_user or n_request is less than or equal to t_request, then directly return n_user and n_request as the final access statistics result.

[0181] Step S610: If the massive access condition is not met, then perform a base number estimation process:

[0182] When n_user > t_user or n_request > t_request, it indicates a large volume of accessed data. For example, if both t_user and t_request are 1000, after each write of user_id to hashset(u), the corresponding n_user is counted; after each write of request_id to hashset(req), the corresponding n_request is counted. When n_user is greater than 1000 or n_request is greater than 1000, it indicates a large access volume, making it difficult to use hashset to count the results. In this case, hashset(u) and hashset(req) are cleared. A cardinality prediction algorithm is used on the reduce operator to deduplicate request_id and user_id. The algorithm uses an improved LogLog algorithm to quickly calculate the number of unique user and access identifiers. Specifically, this includes:

[0183] Construct a hash function h(x), where the hash value is a binary string of fixed length L. The hash result should ideally follow a uniform distribution with minimal collision probability.

[0184] In the reduce operator's memory, create an array of length n and divide it into buckets, with m buckets of length n / m. For each user identifier and access identifier, construct a hash function h(x) to generate hash data. The hash value has length L, with the first i bits serving as the hash number and the last Li bits as the hash value. Use these hash numbers to number the buckets.

[0185] Hash values ​​with the same hash number are assigned to the same bucket, and the position M of the first 1 in each hash value within each bucket is calculated. i For example, if the first 1 in 00010101 appears at position 4, then position M i =4.

[0186] The average value of the m buckets is calculated, and the positional mean is...

[0187] Calculate the average of request_id and user_id for each reduce operator using the method described above:

[0188] Average number of visits:

[0189] User mean is the same as object mean:

[0190] Calculate the access cardinality and user cardinality on the reduce operator respectively.

[0191] Access base:

[0192] User base is the same as object base:

[0193] Step S612, access the cardinality n request_id and user base n user_id The final access statistics are written into the downstream database.

[0194] In one embodiment, the target resource is promotional information, and the access statistics include the object cardinality and the access cardinality; the method further includes:

[0195] Based on at least one of the target audience and the number of visits corresponding to each promotional message, target promotional messages that meet the push conditions are selected; and the target promotional messages are pushed to different promotional channels.

[0196] In this context, "promotional information" refers to information created by the promotion requester for content, products, or services that need to be promoted. Specifically, it can be an ad title, ad link, thumbnail, content summary, part of the ad content, or all of the ad content. The conditions for push notifications are met if the target audience for the promotional information is greater than a first threshold, or the number of visits to the promotional information is greater than a second threshold, or both the target audience and the number of visits are greater than the first and second thresholds. The second threshold can be different from the first threshold, or it can be the same as the first threshold.

[0197] Promotion channels refer to the channels through which information is promoted, including but not limited to different applications, different web pages, and different websites.

[0198] Specifically, the computer device calculates the target audience and access volume for each promotional message to obtain the push criteria. The computer device can then filter out target promotional messages that meet the push criteria from among all promotional messages based on the target audience; or based on the access volume; or based on both the target audience and access volume for each promotional message.

[0199] In one embodiment, the computer device filters out promotional information whose target audience is greater than a first threshold based on the target audience size corresponding to each promotional message. Alternatively, the computer device filters out promotional information whose access size is greater than a second threshold based on the access size corresponding to each promotional message. Alternatively, the computer device filters out promotional information whose target audience size is greater than both the first threshold and the second threshold based on both the access size and the target audience size corresponding to each promotional message.

[0200] Computer devices can distribute targeted promotional information across different promotional channels to push targeted promotional information.

[0201] In this embodiment, based on at least one of the target base and access base corresponding to each promotional information, target promotional information that meets the push conditions is selected, thereby pushing the target promotional information to different promotional channels, which is conducive to the dissemination and promotion of information and can effectively improve the dissemination rate of information.

[0202] In one embodiment, the method further includes:

[0203] Obtain the object attribute features corresponding to the target resource, as well as the object attribute features corresponding to the object identifier; based on the object attribute features, object attribute features and access statistics, predict the conversion rate of the target resource within the target time period.

[0204] Among them, object attribute features are attribute feature data related to the target resource, specifically including at least one of the following: the category to which the target resource belongs, the platform to which it belongs, and related descriptions. Object attribute features are attribute feature data related to the object itself, specifically including at least one of the following: the object's name, gender, age, hobbies, occupation, and city of residence.

[0205] Conversion rate represents the probability that a target resource will be converted into an object within a certain period of time. Specifically, the conversion rate of a target resource can be the probability that the target resource is clicked or downloaded by an object within a certain period of time.

[0206] In one embodiment, the computer device obtains at least one of the following information when accessing the target resource: object name, gender, age, hobbies, job, and city. The computer device uses at least one of the following information as object attribute features: object name, gender, age, hobbies, job, and city.

[0207] Specifically, the computer device determines the attributes associated with the target resource, i.e., object attribute features, and obtains the object attribute features corresponding to the object identifier. Based on the object attribute features, the object attribute features, and access statistics, the computer device can predict the conversion rate of the target resource within a target time period. Furthermore, the computer device can predict the conversion rate of the target resource within a target time period based on at least one of the object cardinality and access cardinality of the target resource, as well as the object attribute features and the sum of the object attribute features.

[0208] In one embodiment, the computer device fuses object attribute features, object attribute features, and access statistics to obtain the conversion rate of the target resource within a target time period. The computer device can input the object attribute features, object attribute features, and access statistics into a trained conversion rate prediction model. The trained conversion rate prediction model concatenates the object attribute features, object attribute features, and access statistics, and then fuses the weight matrix of the conversion rate prediction model with the concatenated features to output the conversion rate of the target resource within the target time period. Further, the computer device performs a dot product between the concatenated features and the weight matrix of the conversion rate prediction model to obtain the conversion rate of the target resource within the target time period.

[0209] In this embodiment, the object attribute features corresponding to the target resource and the object attribute features corresponding to the object identifier are obtained. Based on the object attribute features, the object attribute features and the access statistics results, the conversion rate of the target resource within the target time period can be accurately predicted.

[0210] In one embodiment, the method further includes:

[0211] Based on the conversion rate of each object identifier to the target resource within the target time period, target object identifiers that meet the conversion criteria are selected; information containing the target resource is pushed to the target object identifiers.

[0212] Among them, meeting the conversion conditions means that the conversion rate of the target resource is greater than the conversion rate threshold, or the difference between the conversion rate of the target resource and the conversion rate threshold is within a preset range.

[0213] Specifically, the computer device acquires the object attribute characteristics corresponding to the target resource, and acquires the object attribute characteristics corresponding to each object identifier. Based on the object attribute characteristics, each object attribute characteristic, and access statistics, the computer device predicts the conversion rate of each object identifier to the target resource within the target time period.

[0214] The computer equipment can obtain conversion conditions and, based on the conversion rate of each object identifier to the target resource within the target time period, filter out object identifiers with a conversion rate greater than the conversion rate threshold as target object identifiers.

[0215] In one embodiment, the computer device can calculate the difference between the conversion rate and the conversion rate threshold corresponding to each object identifier, and filter out the object identifiers whose difference is within a preset range as target object identifiers.

[0216] The computer device acquires information containing the target resource and pushes the information containing the target resource to the target object identifier.

[0217] In this embodiment, based on the conversion rate of each object identifier to the target resource within the target time period, target object identifiers that meet the conversion conditions are selected, and information containing the target resource is pushed to the target object identifier. This can effectively realize personalized push of target resources, which is conducive to the effective dissemination and promotion of target resources.

[0218] In one embodiment, such as Figure 7 As shown, a method for accessing data statistics is provided, applied to computer devices, including:

[0219] Step S702: Obtain log data corresponding to the target resource within a preset time period.

[0220] Step S704: Obtain the object identifier that accessed the target resource and the access identifier corresponding to the access request sent to the target resource from the log data.

[0221] Step S706: Generate an object set and an access set on the aggregation operator; write the object identifier to the object set and the access identifier to the access set.

[0222] Step S708: After each writing of object identifier and access identifier, count the number of objects accessing the target resource based on the object identifier in the object set, and count the number of times the target resource has been accessed based on the access identifier in the access set.

[0223] Step S710: Determine whether the conditions for massive access are met.

[0224] Step S712: When at least one of the object quantity and the number of accessed items does not meet the massive access condition, the object quantity and the number of accessed items are used as the access statistics results of the target resource.

[0225] Step S714: When the access volume meets the massive access condition, convert each object identifier and each access identifier into corresponding hash data; starting from the first digit of the hash data, use the preset number of digits as the hash number, and use the digits after the preset number of digits as the hash value.

[0226] Step S716: Create an object array and an access array of preset length, and divide the object array and access data into multiple bucket spaces respectively; number the multiple bucket spaces of the object array according to the hash number corresponding to each object identifier, and allocate the hash value corresponding to each object identifier to the bucket space corresponding to the corresponding hash number; number the multiple bucket spaces of the access array according to the hash number corresponding to each access identifier, and allocate the hash value corresponding to each access identifier to the bucket space corresponding to the corresponding hash number.

[0227] Step S718: Determine the first value position of the preset value of the first occurrence of each hash value in each bucket space of the object array; determine the average value of objects accessing the target resource based on each first value position; determine the cardinality of objects accessing the target resource based on the average value of objects; the object cardinality is used to characterize the number of objects accessing the target resource.

[0228] Step S720: Determine the second value position of the preset value of the first occurrence of each hash value in each bucket space of the access array; based on each second value position, determine the average access value of the target resource; based on the average access value, determine the access cardinality of the target resource; the access cardinality is used to characterize the number of times the target resource is accessed.

[0229] Step S722: Use the object cardinality and access cardinality as the access statistics results corresponding to the target resource.

[0230] In this embodiment, the object identifier that accesses the target resource and the access identifier corresponding to the access request sent to the target resource are obtained from the log data corresponding to the target resource within a preset time period. Only the object identifier and the access identifier are used as the access data of the target resource, so that the access data can be statistically analyzed based on the object identifier and the access identifier. The data used is simple and can effectively reduce the resource consumption of statistical processing.

[0231] An object set and an access set are generated on the aggregation operator. This allows for the automatic counting of the number of object identifiers in the object set after each object identifier is written to it, thus enabling a quick count of the number of objects accessing the target resource. Similarly, the automatic counting of the number of access identifiers in the access set after each access identifier is written to it allows for a quick count of the number of times the target resource has been accessed.

[0232] When at least one of the number of objects and the number of accessed objects meets the massive access condition, indicating a large volume of accessed data, each object identifier and each access identifier is converted into corresponding hash data. Furthermore, an object array and an access array of preset length are created, and the object array and access data are respectively divided into multiple bucket spaces.

[0233] Starting from the first digit of the hash data, a value with a preset number of digits is used as the hash number, and the value after the preset number of digits is used as the hash value. Multiple bucket spaces in the object array are numbered using the hash number corresponding to each object identifier, and multiple bucket spaces in the access array are numbered using the hash number corresponding to each access identifier. This facilitates the subsequent allocation of each hash value and enables parallel cardinality estimation processing of the hash values ​​in each bucket space, effectively improving data processing efficiency.

[0234] By determining the first occurrence of a preset value of each hash value in each bucket space corresponding to an object identifier, and placing it at the first position within the corresponding hash value, the average number of objects accessing the target resource can be estimated based on each first position, effectively reducing the error in object quantity estimation. Similarly, by determining the second position of the preset value of each hash value in each bucket space corresponding to an access identifier, and placing it at the second position, the average number of access identifiers corresponding to access requests accessing the target resource can be determined, effectively reducing the error in access quantity estimation. Based on the object average and access average, the access statistics corresponding to the target resource are determined, allowing for further estimation of the overall cardinality of massive access data based on local averages, reducing overall error and improving estimation accuracy. Furthermore, applying cardinality estimation to access data statistics scenarios with large data volumes can save storage space while reducing computational resource consumption, thereby effectively improving processing efficiency.

[0235] It is understood that the object information and related data involved in each embodiment are all information and data collected after authorization by the object or after full authorization by all parties. Object information includes, but is not limited to, object device information and object personal information, such as object identifiers; related data includes, but is not limited to, data used for display and analysis, such as access data, object attribute characteristics, and object attribute features. Furthermore, the object may choose not to authorize the object information and related data, or may choose not to accept the push of related information, etc.

[0236] It should be understood that, although Figures 2-7 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 2-7 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0237] In one embodiment, such as Figure 8 As shown, an access data statistics device 800 is provided. This device can be a software module, a hardware module, or a combination of both as part of a computer device. Specifically, the device includes: an acquisition module 802, a determination module 804, a conversion module 806, an allocation module 808, and a processing module 810, wherein:

[0238] The acquisition module 802 is used to acquire access data obtained from accessing the target resource.

[0239] The determination module 804 is used to determine the access volume corresponding to the target resource based on the access data.

[0240] The conversion module 806 is used to convert each access data into corresponding hash data when the access volume meets the massive access condition; the hash data includes hash number and hash value.

[0241] The allocation module 808 is used to allocate each hash value to a bucket space associated with the corresponding hash number.

[0242] The processing module 810 is used to perform cardinality estimation based on the hash values ​​included in each bucket space to obtain access statistics results corresponding to the target resource.

[0243] In this embodiment, by acquiring access data obtained from accessing the target resource, the access volume corresponding to the target resource is determined based on the access data, thereby determining the magnitude of the access data. When the access volume meets the massive access condition, indicating a large magnitude of access data, each access data is converted into corresponding hash data, which includes a hash number and a hash value. Each hash value is allocated to a bucket space associated with the corresponding hash number. Cardinality estimation is performed based on the hash values ​​included in each bucket space. Cardinality estimation enables rapid statistical analysis of massive access data, accurately obtaining access statistics results corresponding to the target resource. Furthermore, applying cardinality estimation to access data statistics scenarios with large data volumes can save storage space while reducing computational resource consumption, thereby effectively improving processing efficiency.

[0244] In one embodiment, the acquisition module 802 is further configured to acquire log data corresponding to the target resource within a preset time period; and to acquire from the log data the object identifier that accesses the target resource, and the access identifier corresponding to the access request sent to the target resource.

[0245] In this embodiment, log data corresponding to the target resource within a preset time period is obtained. The object identifier that accesses the target resource and the access identifier corresponding to the access request sent to the target resource are obtained from the log data. Only the object identifier and the access identifier are used as the access data of the target resource, so that the access data can be statistically analyzed based on the object identifier and the access identifier. The data used is simple, which can effectively reduce the resource consumption of statistical processing, save storage space, and improve processing efficiency.

[0246] In one embodiment, the determining module 804 is further configured to generate an object set and an access set on the aggregation operator; write each object identifier into the object set and write the access identifier into the access set; after each writing of the object identifier and the access identifier, based on the object identifier in the object set, count the number of objects accessing the target resource, and based on the access identifier in the access set, count the number of times the target resource has been accessed.

[0247] In this embodiment, an object set and an access set are generated on the aggregation operator. This allows for the automatic counting of the number of object identifiers in the object set after each object identifier is written to it, thus enabling a rapid count of the number of objects accessing the target resource. Similarly, the automatic counting of the number of access identifiers in the access set after each access identifier is written to it allows for a rapid count of the number of times the target resource has been accessed.

[0248] In one embodiment, the device further includes a creation module; the creation module is used to create an array of a preset length and divide the array into multiple bucket spaces when the access volume meets the massive access condition;

[0249] The allocation module 808 is also used to number multiple bucket spaces by the hash number in each hash data, and to allocate each hash value to the bucket space corresponding to the corresponding hash number.

[0250] In this embodiment, when the access volume meets the massive access condition, an array of preset length is created and divided into multiple bucket spaces to distribute the massive access data to each bucket space for separate processing. The multiple bucket spaces are numbered using the hash number in each hash data, allowing each hash value to be allocated to its corresponding bucket space based on the hash number. This enables simultaneous cardinality estimation processing of the hash values ​​in each bucket space, achieving parallel cardinality estimation and effectively improving data processing efficiency.

[0251] In one embodiment, the allocation module 808 is further configured to use a preset number of digits as a hash number starting from the first digit of the hash data, and use the digits after the preset number of digits as the hash value; and number multiple bucket spaces using each hash number.

[0252] In this embodiment, starting from the first digit of the hash data, a preset number of digits is used as the hash number, and the digits after the preset number of digits are used as the hash value. Multiple bucket spaces are numbered using each hash number so that each hash value can be allocated in the future. This facilitates the parallel cardinality estimation processing of the hash values ​​in each bucket space and improves processing efficiency.

[0253] In one embodiment, the processing module 810 is further configured to determine the position of the first occurrence of a preset value in each hash value of each bucket space, and to determine the access statistics result corresponding to the target resource based on each value position.

[0254] In this embodiment, the overall cardinality is estimated by using the position of the first occurrence of a preset data value in the hash value, which effectively enables statistical analysis of massive amounts of access data. Furthermore, distributing the hash value into various bucket spaces to estimate the cardinality of each bucket space separately eliminates errors caused by randomness and improves the accuracy of the estimation. Finally, based on the estimated cardinality values ​​of each bucket, the overall cardinality of the massive access data is obtained, reducing the overall error and improving the accuracy of the statistics.

[0255] In one embodiment, the processing module 810 is further configured to determine the first numerical position of the preset value of the first occurrence of each hash value in each bucket space corresponding to the object identifier in the corresponding hash value; determine the average value of objects accessing the target resource based on each first numerical position; determine the second numerical position of the preset value of the first occurrence of each hash value in each bucket space corresponding to the access identifier in the corresponding hash value; determine the average access value of the target resource based on each second numerical position; and determine the access statistics result corresponding to the target resource based on the average value of objects and the average access value.

[0256] In this embodiment, hash values ​​are allocated to each bucket space to estimate the cardinality of each bucket space, thus eliminating errors caused by randomness. A preset value for the first occurrence of each hash value in each bucket space corresponding to an object identifier is determined as the first numerical position within the corresponding hash value. Based on each first numerical position, the average number of objects accessing the target resource is estimated, effectively reducing errors in object quantity estimation. A preset value for the first occurrence of each hash value in each bucket space corresponding to an access identifier is determined as the second numerical position within the corresponding hash value. Based on each second numerical position, the average number of access identifiers corresponding to access requests accessing the target resource is determined, effectively reducing errors in access quantity estimation. Based on the object average and access average, access statistics corresponding to the target resource are determined. This allows for further estimation of the overall cardinality of massive access data based on local averages, reducing overall error and improving estimation accuracy.

[0257] In one embodiment, the processing module 810 is further configured to determine the object cardinality of the target resource based on the object mean; the object cardinality is used to characterize the number of objects accessing the target resource; determine the access cardinality of the target resource being accessed based on the access mean; the access cardinality is used to characterize the number of times the target resource is accessed; and use the object cardinality and the access cardinality as the access statistics results corresponding to the target resource.

[0258] In this embodiment, the object cardinality for accessing the target resource is determined based on the object mean, and the access cardinality for the target resource is determined based on the access mean. The object cardinality can represent the number of objects accessing the target resource, and the access cardinality can represent the number of times the target resource is accessed. This allows for effective statistical analysis of massive access data using both object cardinality and access cardinality. Furthermore, statistical analysis of massive access data using object cardinality and access cardinality significantly saves storage space while keeping errors within a certain range.

[0259] In one embodiment, the target resource is promotional information, and the access statistics result includes the object base and the access base; the device also includes a push module; the push module is used to filter out target promotional information that meets the push conditions based on at least one of the object base and the access base corresponding to each promotional information; and push the target promotional information to different promotional channels.

[0260] In this embodiment, based on at least one of the target base and access base corresponding to each promotional information, target promotional information that meets the push conditions is selected, thereby pushing the target promotional information to different promotional channels, which is conducive to the dissemination and promotion of information and can effectively improve the dissemination rate of information.

[0261] In one embodiment, such as Figure 9 As shown, an access data statistics device 800 is provided. This device specifically includes: an acquisition module 802, a determination module 804, a conversion module 806, an allocation module 808, a processing module 810, and a prediction module 812. The processing of the acquisition module 802, determination module 804, conversion module 806, allocation module 808, and processing module 810 can be referred to... Figure 8 Each module in

[0262] The prediction module 812 is used to obtain the object attribute features corresponding to the target resource and the object attribute features corresponding to the object identifier; based on the object attribute features, the object attribute features and the access statistics results, the conversion rate of the target resource within the target time period is predicted.

[0263] In this embodiment, the object attribute features corresponding to the target resource and the object attribute features corresponding to the object identifier are obtained. Based on the object attribute features, the object attribute features and the access statistics results, the conversion rate of the target resource within the target time period can be accurately predicted.

[0264] In one embodiment, the device further includes a push module; the push module is used to filter out target object identifiers that meet the conversion conditions based on the conversion rate of each object identifier to the target resource within a target time period; and push information containing the target resource to the target object identifier.

[0265] In this embodiment, based on the conversion rate of each object identifier to the target resource within the target time period, target object identifiers that meet the conversion conditions are selected, and information containing the target resource is pushed to the target object identifier. This can effectively realize personalized push of target resources, which is conducive to the effective dissemination and promotion of target resources.

[0266] Specific limitations regarding the access data statistics device can be found in the limitations of the access data statistics method described above, and will not be repeated here. Each module in the aforementioned access data statistics device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0267] In one embodiment, a computer device is provided, which may be a terminal or a server. This embodiment takes a server as an example, and its internal structure diagram can be as follows. Figure 10 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores statistical data on accessed data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a method for statistical analysis of accessed data.

[0268] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0269] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0270] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0271] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0272] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0273] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0274] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for statistical analysis of access data, characterized in that, The method includes: Obtain access data obtained from accessing a target resource; the access data includes an object identifier and an access identifier; the target resource is promotional information; wherein, one object identifier sends multiple access requests, and each access request corresponds to one access identifier; The access volume corresponding to the target resource is determined based on the access data; When the access volume meets the massive access condition, each access data is converted into corresponding hash data; the hash data includes a hash number and a hash value; Each hash value is assigned to a bucket space associated with its corresponding hash number; Determine the position of the first value of the preset value that first appears in each hash value in each bucket space corresponding to the object identifier; Based on each of the first numerical positions, determine the average number of objects accessing the target resource; Determine the position of the second value in the corresponding hash value of each hash value in each bucket space corresponding to the access identifier, based on the preset value of the first occurrence of the hash value. Based on each of the second numerical positions, the average access value of the target resource is determined; The cardinality of objects accessing the target resource is determined based on the mean value of the objects; the object cardinality is used to characterize the number of objects accessing the target resource. The access base number for the target resource is determined based on the access average; the access base number is used to characterize the number of times the target resource is accessed. Based on the target audience and access volume of each promotional message, select the target promotional messages that meet the push conditions; Push the target promotional information.

2. The method according to claim 1, characterized in that, The acquisition of access data obtained from accessing the target resource includes: Retrieve log data corresponding to the target resource within a preset time period; Obtain from the log data the object identifier that accessed the target resource, and the access identifier corresponding to the access request sent to the target resource.

3. The method according to claim 2, characterized in that, Determining the access volume corresponding to the target resource based on the access data includes: Generate object collections and access collections on aggregate operators; Write each of the object identifiers into the object set, and write the access identifier into the access set; After each writing of object identifier and access identifier, based on the object identifier in the object set, the number of objects accessing the target resource is counted, and based on the access identifier in the access set, the number of times the target resource is accessed is counted.

4. The method according to claim 1, characterized in that, After determining the access volume corresponding to the target resource based on the access data, the method further includes: When the access volume meets the massive access condition, an array of preset length is created and the array is divided into multiple bucket spaces; The step of allocating each hash value to a bucket space associated with its corresponding hash number includes: The plurality of bucket spaces are numbered using the hash numbers in each of the hash data, and each hash value is assigned to the bucket space corresponding to the corresponding hash number.

5. The method according to claim 4, characterized in that, The step of numbering the multiple bucket spaces using the hash numbers in each of the hash data includes: Starting from the first digit of the hash data, a preset number of digits is used as the hash number, and the digits after the preset number of digits are used as the hash value. The plurality of bucket spaces are numbered using the hash numbers described above.

6. The method according to claim 1, characterized in that, The method further includes: The object cardinality and the access cardinality are used as the access statistics results corresponding to the target resource.

7. The method according to any one of claims 1 to 6, characterized in that, The push of the target promotional information includes: The target promotional information is pushed through different promotional channels.

8. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Obtain the object attribute features corresponding to the target resource, and the object attribute features corresponding to the object identifier; Based on the object attribute features corresponding to the target resource, the object attribute features corresponding to the object identifier, the object cardinality, and the access cardinality, the conversion rate of the target resource within the target time period is predicted.

9. The method according to claim 8, characterized in that, The method further includes: Based on the conversion rate of each object identifier to the target resource within the target time period, target object identifiers that meet the conversion conditions are selected. Information containing the target resource is pushed to the target object identifier.

10. An access data statistics device, characterized in that, The device includes: The acquisition module is used to acquire access data obtained from accessing the target resource; the access data includes an object identifier and an access identifier; the target resource is promotional information; wherein, one object identifier sends multiple access requests, and each access request corresponds to one access identifier; The determination module is used to determine the access volume corresponding to the target resource based on the access data; The conversion module is used to convert each access data into corresponding hash data when the access volume meets the massive access condition; the hash data includes a hash number and a hash value; An allocation module is used to allocate each hash value to a bucket space associated with the corresponding hash number; The processing module is configured to: determine the first numerical position of the preset value of the first occurrence of each hash value in each bucket space corresponding to the object identifier within the corresponding hash value; determine the average number of objects accessing the target resource based on each of the first numerical positions; determine the second numerical position of the preset value of the first occurrence of each hash value in each bucket space corresponding to the access identifier within the corresponding hash value; determine the average access value of the target resource based on each of the second numerical positions; determine the cardinality of objects accessing the target resource based on the average object value; the cardinality of objects accessing the target resource is used to characterize the number of objects accessing the target resource; and determine the access cardinality of the target resource based on the average access value; the access cardinality is used to characterize the number of times the target resource is accessed. The filtering module is used to filter out target promotional information that meets the push conditions based on the number of objects and the number of visits corresponding to each promotional information; The push module is used to push the target promotional information.

11. The apparatus according to claim 10, characterized in that, The acquisition module is further configured to acquire log data corresponding to the target resource within a preset time period; and to acquire from the log data the object identifier that accesses the target resource, as well as the access identifier corresponding to the access request sent to the target resource.

12. The apparatus according to claim 11, characterized in that, The determining module is further configured to generate an object set and an access set on the aggregation operator; write each object identifier into the object set, and write the access identifier into the access set; After each writing of object identifier and access identifier, based on the object identifier in the object set, the number of objects accessing the target resource is counted, and based on the access identifier in the access set, the number of times the target resource is accessed is counted.

13. The apparatus according to claim 10, characterized in that, The device also includes a creation module, which is used to create an array of a preset length and divide the array into multiple bucket spaces when the access volume meets the massive access conditions; The allocation module is further configured to number the plurality of bucket spaces using the hash number in each of the hash data, and allocate each hash value to the bucket space corresponding to the corresponding hash number.

14. The apparatus according to claim 13, characterized in that, The allocation module is also used to take a number of preset digits as the hash number starting from the first digit of the hash data, and take the number of digits after the preset digits as the hash value. The plurality of bucket spaces are numbered using the hash numbers described above.

15. The apparatus according to claim 10, characterized in that, The processing module is also used to use the object cardinality and the access cardinality as the access statistics result corresponding to the target resource.

16. The apparatus according to any one of claims 10 to 15, characterized in that, The push module is also used to push the target promotional information to different promotional channels.

17. The apparatus according to any one of claims 10 to 15, characterized in that, The device further includes a prediction module, which is used to acquire object attribute features corresponding to the target resource and object attribute features corresponding to the object identifier; and predict the conversion rate of the target resource within a target time period based on the object attribute features corresponding to the target resource, the object attribute features corresponding to the object identifier, the object cardinality, and the access cardinality.

18. The apparatus according to claim 17, characterized in that, The filtering module is also used to filter out target object identifiers that meet the conversion conditions based on the conversion rate of each object identifier to the target resource within the target time period. The push module is also used to: push information containing the target resource to the target object identifier.

19. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 9.

20. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 9.

21. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the method of any one of claims 1 to 9.

Images