User behavior data writing method, device, computer equipment and storage medium

Abstract

The present application relates to a method, device, computer equipment and storage medium for writing user behavior data. The method includes: obtaining multi-dimensional user behavior data, the user behavior data carries a data source, creating a target Hudi table that matches the data source, aggregating the user behavior data according to a preset window size, obtaining target business data corresponding to the target dimension, and synchronizing the target business data to the target Hudi table according to the target submission frequency, and the target submission frequency is the same as the preset window size. This method can be used to write user behavior data into the Hudi table. Based on Hudi storage, it solves the problem of insufficient memory caused by Flink's intermediate state storage of large windows or large data volumes, thereby avoiding the introduction of other components as storage media.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, computer device and storage medium for writing user behavior data. Background Art

[0002] At present, when the industry performs real-time window analysis in various scenarios, it usually uses Flink's built-in rolling window or sliding window. After setting the window size and sliding step size, the desired results can be counted in real time according to the established dimensions. However, such statistics require the statistical dimensions and window size and step size to be set in advance, which is inflexible. Among them, since Flink is based on memory computing, the window size and state result set should not be too large. If the amount of data is too large, it needs to rely on third-party components for storage. Introducing other components will not only make the architecture complicated, but also greatly reduce performance. Summary of the invention

[0003] Based on this, it is necessary to provide a method, device, computer equipment and storage medium for writing user behavior data to the Hudi table in response to the above technical problems. Based on Hudi storage, the problem of insufficient memory caused by Flink's intermediate state storage when storing large windows or large amounts of data is solved, thereby avoiding the introduction of other components as storage media.

[0004] A method for writing user behavior data, the method comprising:

[0005] Obtain multi-dimensional user behavior data, which carries the data source;

[0006] Create a target Hudi table that matches the data source;

[0007] Aggregate user behavior data according to the preset window size to obtain target business data corresponding to the target dimension;

[0008] The target business data is synchronized to the target Hudi table according to the target submission frequency. The target submission frequency is the same as the preset window size.

[0009] In one of the embodiments, multi-dimensional user behavior data is obtained, and the user behavior data carries a data source, including: obtaining the multi-dimensional user behavior data from a log file corresponding to a business system, and the user behavior data is obtained by the business system through dimensional screening from the collected original data.

[0010] In one of the embodiments, creating a target Hudi table that matches a data source includes: creating a MySql table corresponding to the data source, and creating a corresponding target Hudi table based on the MySql table.

[0011] In one of the embodiments, after creating a target Hudi table that matches the data source, it also includes: obtaining an initial submission frequency corresponding to the target Hudi table, setting the initial submission frequency to a preset window size, and obtaining a target submission frequency.

[0012] In one of the embodiments, user behavior data is aggregated according to a preset window size to obtain target business data corresponding to the target dimension, including: obtaining the preset window size, and when the current window size reaches the preset window size, filtering the intermediate user behavior data corresponding to the target dimension from the multi-dimensional user behavior data, and aggregating the intermediate user behavior data corresponding to the target dimension to obtain the target business data corresponding to the aggregated target dimension.

[0013] In one of the embodiments, the target business data is synchronized to the target Hudi table according to the target submission frequency, and the target submission frequency is the same as the preset window size, including: obtaining the last submission time, determining the target submission time according to the last submission time and the target submission frequency, and when the current submission time reaches the target submission time, synchronizing the target business data and the corresponding target submission time to the target Hudi table.

[0014] In one of the embodiments, the method for writing user behavior data also includes: sending a query request to a target Hudi table, the query request carrying dimension information to be queried and submission time information to be queried, and searching the target Hudi table for business data that matches the dimension information to be queried and the submission time information to be queried according to the query request.

[0015] A user behavior data writing device, the device comprising:

[0016] The acquisition module is used to obtain multi-dimensional user behavior data, which carries the data source;

[0017] Create a module to create a target Hudi table that matches the data source;

[0018] Aggregation module, used to aggregate user behavior data according to a preset window size to obtain target business data corresponding to the target dimension;

[0019] The synchronization module is used to synchronize the target business data to the target Hudi table according to the target submission frequency. The target submission frequency is the same as the preset window size.

[0020] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0021] Obtain multi-dimensional user behavior data, which carries the data source;

[0022] Create a target Hudi table that matches the data source;

[0023] Aggregate user behavior data according to the preset window size to obtain target business data corresponding to the target dimension;

[0024] The target business data is synchronized to the target Hudi table according to the target submission frequency. The target submission frequency is the same as the preset window size.

[0025] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0026] Obtain multi-dimensional user behavior data, which carries the data source;

[0027] Create a target Hudi table that matches the data source;

[0028] Aggregate user behavior data according to the preset window size to obtain target business data corresponding to the target dimension;

[0029] The target business data is synchronized to the target Hudi table according to the target submission frequency. The target submission frequency is the same as the preset window size.

[0030] The above-mentioned user behavior data writing method, device, computer equipment and storage medium obtain multi-dimensional user behavior data, the user behavior data carries the data source, creates a target Hudi table that matches the data source, aggregates the user behavior data according to the preset window size, obtains the target business data corresponding to the target dimension, and synchronizes the target business data to the target Hudi table according to the target submission frequency, and the target submission frequency is the same as the preset window size. Therefore, user behavior data can be written from the business system to the Hudi table. Based on Hudi storage, the problem of insufficient memory caused by Flink's intermediate state storage of large windows or large data volumes is solved, thereby avoiding the introduction of other components as storage media. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 An application environment diagram of a method for writing user behavior data in one embodiment;

[0032] Figure 2 A flowchart of a method for writing user behavior data in one embodiment;

[0033] Figure 3 It is a structural block diagram of a user behavior data writing device in one embodiment;

[0034] Figure 4 is an internal structure diagram of a computer device in one embodiment;

[0035] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0037] The user behavior data writing method provided in this application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 through a network. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0038] Specifically, the terminal 102 collects multi-dimensional user behavior data that carries the data source and sends it to the server 104 through network communication. The server 104 obtains multi-dimensional user behavior data that carries the data source, creates a target Hudi table that matches the data source, aggregates the user behavior data according to a preset window size, obtains the target business data corresponding to the target dimension, and synchronizes the target business data to the target Hudi table according to the target submission frequency, and the target submission frequency is the same as the preset window size.

[0039] In another embodiment, the server 104 obtains multi-dimensional user behavior data that carries a data source, creates a target Hudi table that matches the data source, aggregates the user behavior data according to a preset window size, obtains target business data corresponding to the target dimension, and synchronizes the target business data to the target Hudi table according to a target submission frequency, and the target submission frequency is the same as the preset window size.

[0040] In one embodiment, Figure 2 As shown, a method for writing user behavior data is provided, and the method is applied to Figure 1 The terminal or server in the example is used to illustrate, including the following steps:

[0041] Step 202, obtaining multi-dimensional user behavior data, where the user behavior data carries a data source.

[0042] Among them, multi-dimensional user behavior data is user behavior data of multiple dimensions. User behavior data is data related to user behavior in each dimension. Each dimension can be determined based on actual business application scenarios, actual business needs or actual product needs, such as user behavior data in a certain category of goods, user behavior data in a certain region, and user behavior data purchased through a certain channel.

[0043] The user behavior data carries a data source, where the data source is the source of the user behavior data. For example, if the user behavior data is originally stored in a MySql table, the data source is the MySql table.

[0044] In one embodiment, step 202 includes: obtaining multi-dimensional user behavior data from a log file corresponding to the business system, where the user behavior data is obtained by the business system through dimensional screening of the collected raw data.

[0045] Among them, the business system here is a system related to the actual business, which can be used to store user behavior data of various dimensions, and the user behavior data of various dimensions can be collected from the business system. Specifically, the business system can include a MySql business library, and the MySql business library can include a MySql table to store user behavior data of various dimensions, or the business system records user behavior data of various dimensions through business logs, obtains all data recorded in the log file from the log file of the business system, determines it as raw data, and then filters the raw data to obtain user behavior data under multiple dimensions. Among them, filtering the raw data collected by the business system can be to obtain the raw data corresponding to each dimension, perform data type analysis on the raw data, filter out the data related to user behavior, and obtain user behavior data corresponding to each dimension.

[0046] Step 204, create a target Hudi table that matches the data source.

[0047] The target Hudi table here is a Hudi table that matches the data source, which is an Apache Hudi table. After obtaining multi-dimensional user behavior data, a target Hudi table that matches the data source is created. At this time, the created target Hudi is temporarily an empty table without any data written. In one embodiment, step 204 includes: creating a MySql table corresponding to the data source, and creating a corresponding target Hudi table according to the MySql table.

[0048] The creation of the target Hudi table can specifically be to create a MySql table that matches the data source. At this time, the MySql table has not written any data, and then the corresponding target Hudi table is created according to the created MySql table. The creation of the MySql table can be created through a Sql statement, and the creation of the target Hudi table can be created according to the MySql table.

[0049] In one embodiment, after step 204, the process further includes: obtaining an initial submission frequency corresponding to the target Hudi table, setting the initial submission frequency to a preset window size, and obtaining a target submission frequency.

[0050] Specifically, after creating the target Hudi table, you can set the initial submission frequency of the target Hudi table to the preset window size. The preset window size here is the statistical window size set in advance. It is the window size for statistical analysis of user behavior data. It can be a time rolling window size, etc. Keeping the submission frequency of the target Hudi table consistent with the preset window size can ensure the smoothness of writing user behavior data.

[0051] Step 206, aggregate the user behavior data according to the preset window size to obtain target business data corresponding to the target dimension.

[0052] Specifically, before writing to the target Hudi table, a lightweight aggregation operation can be performed on the user behavior data according to the preset window size. The aggregation operation can be a statistical calculation based on the user behavior data to obtain the target business data under the target dimension. For example, based on the data recorded in the user behavior data, the order volume of goods under the target channel is calculated.

[0053] In one embodiment, step 206 includes: obtaining a preset window size, and when the current window size reaches the preset window size, filtering the intermediate user behavior data corresponding to the target dimension from the multi-dimensional user behavior data, aggregating the intermediate user behavior data corresponding to the target dimension, and obtaining the target business data corresponding to the aggregated target dimension.

[0054] Specifically, a lightweight aggregation operation is performed on the user behavior data according to a preset window size, which may be, obtaining a pre-set window size, which may be set according to actual business needs, actual product needs or actual application scenarios, and may be a time rolling window size, and then obtaining the current window size, comparing whether the current window size reaches the preset window size, and when the current window size reaches the preset window size, filtering out the intermediate user behavior data corresponding to the target dimension from the user behavior data of multiple dimensions, the target dimension is filtered out from multiple dimensions, and the determination of the target dimension may be obtained by filtering according to actual business needs, actual product needs or actual application scenarios.

[0055] Furthermore, the intermediate user behavior data under the selected target dimension is statistically calculated to obtain the target business data under the aggregated target dimension. Specifically, data related to the target business in the user behavior data under the target dimension is obtained, and statistical calculations are performed on the data to finally obtain the target business data under the target dimension.

[0056] Step 208: synchronize the target business data to the target Hudi table according to the target submission frequency, where the target submission frequency is the same as the preset window size.

[0057] Specifically, after setting the target submission frequency of the target Hudi table to be the same as the preset window size, the target business data can be synchronously written to the target Hudi table according to the target submission frequency, so that the consistency of statistics and writing can be ensured. After the target business data is counted, it can be written to the target Hudi table in a timely manner. For example, if the preset window size is the time rolling window size of 5s, then the target submission frequency is also 5s. When the target submission frequency is reached, the counted target business data can be written to the target Hudi table.

[0058] In one embodiment, step 208 includes: obtaining the last submission time, determining the target submission time according to the last submission time and the target submission frequency, and synchronizing the target business data and the corresponding target submission time to the target Hudi table when the current submission time reaches the target submission time.

[0059] Specifically, the last submission time of the target Hudi table can be obtained from the configuration file corresponding to the target Hudi table, and then the next submission time, that is, the target submission time, is calculated based on the last submission time and the target submission frequency. Compare whether the current submission time reaches the target submission time. If the current submission time reaches the target submission time, the target business data and the corresponding target submission time can be written into the target Hudi table.

[0060] In one embodiment, step 208 also includes: sending a query request to the target Hudi table, the query request carries the dimension information to be queried and the submission time information to be queried, and searching the target Hudi table for business data that matches the dimension information to be queried and the submission time information to be queried according to the query request.

[0061] Specifically, after the target business data in the user behavior data is written into the target Hudi table, it can be queried from the target Hudi table. Specifically, a query request can be sent to the target Hudi table. The query request can carry the dimension information to be queried and the submission information to be queried. That is to say, when querying, as long as different dimension information and submission time information are locked, matching business data can be found in the target Hudi table. In this way, as long as different dimension information and submission time information are locked, matching business data can be found from the target Hudi table for subsequent statistical analysis.

[0062] In the above-mentioned user behavior data writing method, multi-dimensional user behavior data is obtained, the user behavior data carries the data source, a target Hudi table matching the data source is created, the user behavior data is aggregated according to the preset window size, the target business data corresponding to the target dimension is obtained, and the target business data is synchronized to the target Hudi table according to the target submission frequency, and the target submission frequency is the same as the preset window size. Therefore, writing user behavior data into the Hudi table, based on Hudi storage, solves the problem of insufficient memory caused by Flink's intermediate state storage of large windows or large data volumes, thereby avoiding the introduction of other components as storage media.

[0063] Moreover, in the prior art, if the user wants to analyze based on different dimensions, multiple tasks need to be created and multiple status result sets need to be counted, which obviously consumes a lot of computing and storage resources. However, this application only needs to lock different dimension information and submission time information, and then the matching business data can be found in the target Hudi table, without creating multiple tasks, which leads to the problem of occupying a lot of computing resources and storage resources.

[0064] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0065] In one embodiment, Figure 3 As shown, a user behavior data writing device 300 is provided, comprising: an acquisition module 302, a creation module 304, an aggregation module 306 and a synchronization module 308, wherein:

[0066] The acquisition module 302 is used to acquire multi-dimensional user behavior data, where the user behavior data carries a data source.

[0067] A creation module 304 is used to create a target Hudi table that matches the data source;

[0068] Aggregation module 306 is used to aggregate user behavior data according to a preset window size to obtain target business data corresponding to the target dimension.

[0069] The synchronization module 308 is used to synchronize the target business data to the target Hudi table according to the target submission frequency, and the target submission frequency is the same as the preset window size.

[0070] In one embodiment, the acquisition module 302 acquires multi-dimensional user behavior data from the log files corresponding to the business system. The user behavior data is obtained by the business system through dimensional screening of the collected raw data.

[0071] In one embodiment, the creation module 304 creates a MySql table corresponding to the data source, and creates a corresponding target Hudi table based on the MySql table.

[0072] In one embodiment, the user behavior data writing device 300 obtains the initial submission frequency corresponding to the target Hudi table, sets the initial submission frequency to a preset window size, and obtains the target submission frequency.

[0073] In one embodiment, the aggregation module 306 obtains a preset window size. When the current window size reaches the preset window size, it filters the intermediate user behavior data corresponding to the target dimension from the multi-dimensional user behavior data, aggregates the intermediate user behavior data corresponding to the target dimension, and obtains the target business data corresponding to the aggregated target dimension.

[0074] In one embodiment, the synchronization module 308 obtains the last submission time, determines the target submission time according to the last submission time and the target submission frequency, and when the current submission time reaches the target submission time, synchronizes the target business data and the corresponding target submission time to the target Hudi table.

[0075] In one embodiment, the user behavior data writing device 300 sends a query request to the target Hudi table, the query request carries the dimension information to be queried and the submission time information to be queried, and searches the target Hudi table for business data that matches the dimension information to be queried and the submission time information to be queried according to the query request.

[0076] For the specific definition of the user behavior data writing device, please refer to the definition of the user behavior data writing method above, which will not be repeated here. Each module in the above-mentioned user behavior data writing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0077] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store the target Hudi table. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for writing user behavior data is implemented.

[0078] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for writing user behavior data is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0079] Those skilled in the art will understand that Figure 4 or Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0080] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented: obtaining multi-dimensional user behavior data, the user behavior data carrying a data source, creating a target Hudi table matching the data source, aggregating the user behavior data according to a preset window size, obtaining target business data corresponding to the target dimension, and synchronizing the target business data to the target Hudi table according to a target submission frequency, the target submission frequency being the same as the preset window size.

[0081] In one embodiment, when the processor executes the computer program, the following steps are also implemented: multi-dimensional user behavior data is obtained from the log file corresponding to the business system, and the user behavior data is obtained by the business system through dimensional screening from the collected original data.

[0082] In one embodiment, when the processor executes the computer program, the following steps are also implemented: creating a MySql table corresponding to the data source, and creating a corresponding target Hudi table according to the MySql table.

[0083] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining an initial submission frequency corresponding to the target Hudi table, setting the initial submission frequency to a preset window size, and obtaining a target submission frequency.

[0084] In one embodiment, when the processor executes the computer program, it also implements the following steps: obtaining a preset window size, and when the current window size reaches the preset window size, filtering the intermediate user behavior data corresponding to the target dimension from the multi-dimensional user behavior data, aggregating the intermediate user behavior data corresponding to the target dimension, and obtaining the target business data corresponding to the aggregated target dimension.

[0085] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining the last submission time, determining the target submission time based on the last submission time and the target submission frequency, and when the current submission time reaches the target submission time, synchronizing the target business data and the corresponding target submission time to the target Hudi table.

[0086] In one embodiment, when the processor executes the computer program, the following steps are also implemented: a query request is sent to the target Hudi table, the query request carries the dimension information to be queried and the submission time information to be queried, and business data matching the dimension information to be queried and the submission time information to be queried are searched in the target Hudi table according to the query request.

[0087] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: obtaining multi-dimensional user behavior data, the user behavior data carries a data source, creating a target Hudi table matching the data source, aggregating the user behavior data according to a preset window size, obtaining target business data corresponding to the target dimension, and synchronizing the target business data to the target Hudi table according to a target submission frequency, and the target submission frequency is the same as the preset window size.

[0088] In one embodiment, when the processor executes the computer program, the following steps are also implemented: multi-dimensional user behavior data is obtained from the log file corresponding to the business system, and the user behavior data is obtained by the business system through dimensional screening from the collected original data.

[0089] In one embodiment, when the processor executes the computer program, the following steps are also implemented: creating a MySql table corresponding to the data source, and creating a corresponding target Hudi table according to the MySql table.

[0090] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining an initial submission frequency corresponding to the target Hudi table, setting the initial submission frequency to a preset window size, and obtaining a target submission frequency.

[0091] In one embodiment, when the processor executes the computer program, it also implements the following steps: obtaining a preset window size, and when the current window size reaches the preset window size, filtering the intermediate user behavior data corresponding to the target dimension from the multi-dimensional user behavior data, aggregating the intermediate user behavior data corresponding to the target dimension, and obtaining the target business data corresponding to the aggregated target dimension.

[0092] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining the last submission time, determining the target submission time based on the last submission time and the target submission frequency, and when the current submission time reaches the target submission time, synchronizing the target business data and the corresponding target submission time to the target Hudi table.

[0093] In one embodiment, when the processor executes the computer program, the following steps are also implemented: a query request is sent to the target Hudi table, the query request carries the dimension information to be queried and the submission time information to be queried, and business data matching the dimension information to be queried and the submission time information to be queried are searched in the target Hudi table according to the query request.

[0094] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0095] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0096] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for writing user behavior data, characterized in that: The method comprises: Acquire multi-dimensional user behavior data, wherein the user behavior data carries a data source; Create a target Hudi table that matches the data source; Aggregating the user behavior data according to a preset window size to obtain target business data corresponding to the target dimension; Synchronize the target business data to the target Hudi table according to the target submission frequency, where the target submission frequency is the same as the preset window size; The creating a target Hudi table matching the data source includes: Create a MySql table corresponding to the data source; Create the corresponding target Hudi table according to the MySql table; After creating the target Hudi table matching the data source, it also includes: Get the initial submission frequency corresponding to the target Hudi table; The initial submission frequency is set to a preset window size to obtain a target submission frequency.

2. The method according to claim 1, characterized in that: The acquiring of multi-dimensional user behavior data, wherein the user behavior data carries a data source, includes: The multi-dimensional user behavior data is obtained from the log files corresponding to the business system. The user behavior data is obtained by the business system through dimensional screening from the collected original data.

3. The method according to claim 1, characterized in that The aggregating the user behavior data according to the preset window size to obtain target business data corresponding to the target dimension includes: Get the preset window size; When the current window size reaches the preset window size, filtering the user behavior data of the multi-dimensional dimensions to obtain the intermediate user behavior data corresponding to the target dimension; The intermediate user behavior data corresponding to the target dimension are aggregated to obtain target business data corresponding to the aggregated target dimension.

4. The method according to claim 1, characterized in that: The synchronizing the target business data to the target Hudi table according to the target submission frequency, wherein the target submission frequency is the same as the preset window size, includes: Get the last submission time; Determining a target submission time according to the last submission time and the target submission frequency; When the current submission time reaches the target submission time, the target business data and the corresponding target submission time are synchronized to the target Hudi table.

5. The method according to claim 4, characterized in that The method further comprises: Send a query request to the target Hudi table, the query request carrying the dimension information to be queried and the submission time information to be queried; According to the query request, the business data matching the dimension information to be queried and the submission time information to be queried is searched in the target Hudi table.

6. A user behavior data writing device, characterized in that: The device comprises: An acquisition module, used to acquire multi-dimensional user behavior data, wherein the user behavior data carries a data source; A creation module is used to create a target Hudi table that matches the data source; An aggregation module, used to aggregate the user behavior data according to a preset window size to obtain target business data corresponding to a target dimension; A synchronization module, used to synchronize the target business data to the target Hudi table according to a target submission frequency, wherein the target submission frequency is the same as the preset window size; The creation module is also used to create a MySql table corresponding to the data source; Create the corresponding target Hudi table according to the MySql table; After creating the target Hudi table matching the data source, the aggregation module is further used to obtain the initial submission frequency corresponding to the target Hudi table; The initial submission frequency is set to a preset window size to obtain a target submission frequency.

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images