Data processing method, device and equipment based on user portrait, medium and program

Abstract

The application provides a data processing method and device based on user portrait, equipment, medium and program, comprising: acquiring a plurality of user data and the generation time of the last user data in the plurality of user data; if a first user data is read from the plurality of user data, the user feature corresponding to the first user data is generated; if the generation time of the first user data is consistent with the generation time of the last user data, the user feature corresponding to the first user data is stored in the target database; if the generation time of the first user data is inconsistent with the generation time of the last user data, the user feature corresponding to the first user data is not stored in the target database; and the user portrait is generated based on the user feature in the target database. The accuracy of user feature query can be improved, and the accuracy of generating user portrait is improved, thereby improving the efficiency and accuracy of user portrait application.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method, apparatus, device, medium and program based on user profiles. Background Technology

[0002] User profiling is a tool for describing users, connecting user needs with product design direction, and is applied in product design, precision marketing, and other fields. Servers can determine user characteristics such as behavioral preferences based on user data including gender, age, page visit history, and transaction history, thereby generating user profiles. These profiles, representing one or more user characteristics, can then be used to uncover user needs and provide more efficient and targeted services.

[0003] Kappa architecture is a data processing approach that not only enables real-time data processing but also allows for data replay capabilities based on its message queue data retention function, thereby facilitating offline data analysis or recalculation. For example, when a server recalculates user characteristics to generate a user profile, it can leverage Kappa's data replay capability to recalculate multiple user data points stored in the message queue. During this recalculation, the server can sequentially read each user data point. When the first user data point is read, the server can calculate and generate corresponding user characteristics, storing these characteristics in a data table. When the second user data point is read, the server can calculate and generate corresponding user characteristics, then update the user characteristics of the first user data point in the data table using these characteristics. This process continues, allowing the server to recalculate multiple user data points to obtain user characteristics, ultimately generating a user profile based on these characteristics.

[0004] However, the data storage method in the above calculation process can lead to inaccurate user feature queries, resulting in inaccurate user profiles generated based on the queried user features. For example, if user features are queried during the recalculation process, the server has obviously not yet completed the recalculation of the above multiple user data. Therefore, the queried user features will not be the user features calculated based on the above multiple user data, leading to inaccurate query results. This will have a certain impact on the generation and application of user profiles. Summary of the Invention

[0005] This application provides a data processing method, apparatus, device, medium, and program based on user profiles to solve the problem in the prior art where inaccurate user feature queries are caused by data storage, resulting in inaccurate user profiles generated based on the queried user features. This can improve the accuracy of user feature queries, thereby improving the accuracy of user profile generation, and thus improving the efficiency and accuracy of user profile applications.

[0006] Firstly, this application provides a data processing method based on user profiles. The method includes: acquiring multiple user data sets and the generation time of the last user data set among the multiple user data sets; generating user features corresponding to each first user data set read from the multiple user data sets; storing the user features corresponding to the first user data set in a target database if the generation time of the first user data set is the same as the generation time of the last user data set; not storing the user features corresponding to the first user data set in the target database if the generation time of the first user data set is different from the generation time of the last user data set; and generating a user profile based on the user features in the target database.

[0007] Secondly, this application provides a data processing apparatus based on user profiles, comprising: a first acquisition module, a generation module, a processing module, and a second generation module. The first acquisition module is used to acquire multiple user data sets and the generation time of the last user data set among the multiple user data sets. The generation module is used to generate user features corresponding to each first user data set read from the multiple user data sets. The processing module is used to store the user features corresponding to the first user data set in a target database if the generation time of the first user data set is the same as the generation time of the last user data set; otherwise, it does not store the user features corresponding to the first user data set in the target database. The second generation module is used to generate user profiles based on the user features in the target database.

[0008] Thirdly, an electronic device is provided, comprising: a processor and a memory for storing a computer program, the processor for calling and running the computer program stored in the memory, and performing the methods as described in the first aspect or its various implementations.

[0009] Fourthly, a computer-readable storage medium is provided for storing a computer program that causes a computer to perform the methods described in the first aspect or its various implementations.

[0010] Fifthly, a computer program product is provided, including computer program instructions that cause a computer to perform the methods as described in the first aspect or its various implementations.

[0011] Sixthly, a computer program is provided that causes a computer to perform the methods described in the first aspect or its various implementations.

[0012] According to the technical solution of this application, the server can first obtain multiple user data and the generation time of the last user data among the multiple user data. If a first user data is read from the multiple user data, the server can generate the user features corresponding to the first user data. If the generation time of the first user data is the same as the generation time of the last user data, the server can store the user features corresponding to the first user data in the target database. If the generation time of the first user data is different from the generation time of the last user data, the server can choose not to store the user features corresponding to the first user data in the target database. Finally, the server can generate a user profile based on the user features in the target database. In the above process, the server can determine whether the currently read user data is the last user data by checking if its generation time matches the generation time of the last user data. This allows the server to store only the user features corresponding to the last user data in the target database, while storing the user features corresponding to other user data. Therefore, when querying user features, such as when the server is reading user data, the query result will not be the user features corresponding to other user data, but only the last original user feature—the final user feature calculated from all user data. The server can then generate an accurate user profile based on this final user feature. This solves the problem in existing technologies where inaccurate user feature queries due to data storage lead to inaccurate user profiles generated from the queried user features. It improves the accuracy of user feature queries, thereby improving the accuracy of user profile generation and ultimately enhancing the efficiency and precision of user profile applications. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 An application scenario diagram provided for an embodiment of this application;

[0015] Figure 2 A flowchart illustrating a data processing method based on user profiles provided in this application embodiment;

[0016] Figure 3This application provides a schematic diagram of data processing based on user profiles as an embodiment of the present application.

[0017] Figure 4 This is a schematic diagram illustrating another user profile-based data processing method provided in an embodiment of this application.

[0018] Figure 5 This is a schematic diagram illustrating another user profile-based data processing method provided in an embodiment of this application.

[0019] Figure 6 This is a schematic diagram illustrating another user profile-based data processing method provided in an embodiment of this application.

[0020] Figure 7 This is a schematic diagram illustrating another user profile-based data processing method provided in an embodiment of this application.

[0021] Figure 8 This is a schematic diagram illustrating another user profile-based data processing method provided in an embodiment of this application.

[0022] Figure 9 This is a schematic diagram illustrating another user profile-based data processing method provided in an embodiment of this application.

[0023] Figure 10 A schematic diagram of a data processing device 1000 based on user profiles provided in an embodiment of this application;

[0024] Figure 11 This is a schematic block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0027] As described above, when the server recalculates user features to generate user profiles, it can leverage the data replay capability of the Kappa architecture to recalculate multiple user data stored in its message queue. During this recalculation, the server can sequentially read each user data point. When the first user data is read, the server can calculate and generate corresponding user features, storing these features in a data table. Similarly, when the second user data is read, the server can calculate and generate corresponding user features, updating the user features of the first user data in the data table. This process continues until all user data is recalculated. However, this data storage method can lead to inaccurate user feature queries, resulting in inaccurate user profiles generated based on these features. For example, if a user feature query is performed during the recalculation process, the server has not yet completed the recalculation of all user data. Therefore, the retrieved user features will not be those calculated from the previously obtained data, leading to inaccurate query results and impacting both the generation and application of user profiles.

[0028] To address the aforementioned technical issues, the server can first acquire multiple user data sets and the generation time of the last user data set. If a first user data set is read from the multiple user data sets, the server can generate user features corresponding to that first user data set. If the generation time of the first user data set is the same as the generation time of the last user data set, the server can store the user features corresponding to the first user data set in the target database. If the generation time of the first user data set is different from the generation time of the last user data set, the server can choose not to store the user features corresponding to the first user data set in the target database. Finally, the server can generate a user profile based on the user features in the target database. In the above process, the server can determine whether the currently read user data is the last user data by checking if its generation time matches the generation time of the last user data. This allows the server to store only the user features corresponding to the last user data in the target database, while storing the user features corresponding to other user data. Therefore, when querying user features, such as when the server is reading user data, the query result will not be the user features corresponding to other user data, but only the last original user feature—the final user feature calculated from all user data. The server can then generate an accurate user profile based on this final user feature. This solves the problem in existing technologies where inaccurate user feature queries due to data storage lead to inaccurate user profiles generated from the queried user features. It improves the accuracy of user feature queries, thereby improving the accuracy of user profile generation and ultimately enhancing the efficiency and precision of user profile applications.

[0029] It should be understood that the technical solution of this application can be applied to the following scenarios, but is not limited to:

[0030] In some possible ways, Figure 1 An application scenario diagram provided for an embodiment of this application, such as... Figure 1 As shown, this application scenario may include terminal 110 and server 120. Terminal 110 and server 120 can communicate with each other.

[0031] For example, server 120 can recalculate multiple user data stored in its message queue based on the data replay capability of the Kappa architecture to obtain user features, and store the user features in a database. This allows for the generation of user profiles based on the user features in the database. This database can be an internal database of server 120, or it can be a database within server 120 itself; this application does not impose any restrictions on this. For instance, server 120 can be a data platform. When it is necessary to rebuild the user profile, such as when the calculation method for user data like stock account opening changes, the data platform can re-obtain the user data from the message queue, determine the new user features, and then regenerate the user profile. A user feature query client can be installed on terminal 110. Users can access the user feature query client to query the aforementioned user features based on natural language. Alternatively, the user feature query client may not be installed on terminal 110. Users can query the aforementioned user features based on natural language through a browser. During the query, server 120 can convert the aforementioned natural language into the corresponding Structured Query Language (SQL) and query the user features stored in the aforementioned database based on the SQL corresponding to the natural language, and return the query results to terminal 110.

[0032] In some possible implementations, terminal 110 can be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, in-vehicle device, ultra-mobile personal computer (UMPC), netbook, cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device; this application does not limit this. Server 120 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services; this application embodiment does not limit this.

[0033] It should be understood that Figure 1 The number of terminals and servers shown is merely illustrative. In reality, any number of terminals and servers can be set up as needed, and this application does not impose any restrictions on this.

[0034] After introducing the application scenarios of the embodiments of this application, the technical solution of this application will be described in detail below:

[0035] Figure 2A flowchart illustrating a data processing method based on user profiles provided in this application embodiment. This method can be performed by, for example... Figure 1 The server 120 shown is executed; this application does not impose any restrictions on this. Figure 2 As shown, the method may include the following steps:

[0036] S210: Obtain multiple user data and the generation time of the last user data among the multiple user data;

[0037] S220: If a first user data is read from multiple user data, then the user feature corresponding to the first user data is generated;

[0038] S230: Determine whether the generation time of the first user data is the same as the generation time of the last user data. If the generation time of the first user data is the same as the generation time of the last user data, then execute S240. If the generation time of the first user data is different from the generation time of the last user data, then execute S250.

[0039] S240: Store the user characteristics corresponding to the first user data into the target database;

[0040] S250: Do not store the user features corresponding to the first raw data in the target database;

[0041] S260: Generate user profiles based on user features in the target database.

[0042] It should be understood that the above steps are executed by the server during data replay. For example, these steps might be performed when the server recalculates multiple user data sets based on the data replay capability of the Kappa architecture to generate user profiles. It should be noted that the Kappa architecture includes message queues, stream processing clusters, and data tables. The message queue has data retention capabilities and can store user data. During data replay, the stream processing cluster can read user data from the message queue. Since Kafka is a messaging system with historical data storage and replay capabilities, it can be chosen as the message queue. The stream processing cluster can perform calculations on the user data read from the message queue to obtain the user characteristics corresponding to the user data. Since Flink is a computing framework that supports both stream and batch processing, it can be chosen as the stream processing cluster. The data table can be used to store the user characteristics corresponding to the user data calculated by the stream processing cluster.

[0043] For example, this application will first briefly introduce the above steps using the server's data replay capability based on the Kappa architecture to recalculate multiple user data, taking the generation of user profiles as an example. First, the server can obtain multiple user data from other databases and store these multiple user data in a message queue. When the server needs to recalculate the multiple user data stored in its message queue based on the Kappa architecture's data replay capability, the server can obtain the generation time of the last user data among the multiple user data. If the data version or generation time is smaller than the stored version, the data in this process can be filtered or deleted to ensure that the data is written correctly. Specifically, the server can sequentially read multiple user data entries stored in the message queue based on the stream processing cluster. When the first user data entry is read, the stream processing cluster can perform calculations on the first user data entry to generate corresponding user features. If the generation time of the first user data entry is inconsistent with the generation time of the last user data entry, the server will not store the user features corresponding to the first user data entry in the data table (i.e., the target database mentioned above). Next, the server can read the second user data entry stored in the message queue based on the stream processing cluster. The stream processing cluster can perform calculations on the second user data entry to generate corresponding user features. It will then determine if the generation time of the second user data entry is inconsistent with the generation time of the last user data entry. If they are consistent, the server can determine that the second user data entry is the last user data entry, and thus the server can determine that the user features corresponding to the second user data entry are the same as those corresponding to the last user data entry, and the server can store these user features in the data table. If they are inconsistent, the server can determine that the second user data entry is not the last user data entry, and the server can choose not to store the user features corresponding to the second user data entry in the data table. Similarly, for other user data, the server can perform similar steps as with the second user data entry. In this way, when a terminal queries user features, the query result will only be the user features corresponding to the last user data, i.e., the latest user data, and not the user features corresponding to other user data. Therefore, the query result is accurate, which can solve the problem of inaccurate user feature query caused by data storage in the existing technology, which leads to inaccurate user profiles generated based on the queried user features. This can improve the accuracy of user feature query, and thus improve the efficiency and accuracy of user profile application.

[0044] It should be noted that, as Figure 3As shown, when building user profiles, the server can first determine user characteristics such as behavioral preferences based on user data such as gender, age, page visits, and product transaction history. This allows for the construction of user profiles, which can then be used to uncover user needs and provide more efficient and targeted services. These user characteristics can be stored in a target database. When uncovering user needs based on these characteristics, the corresponding user characteristics can be queried through the user profile interface service. The retrieved user characteristics can then be applied to downstream business processes, such as product design and precision marketing.

[0045] The following embodiments will use a server as a data platform to illustrate the technical solution of this application. It should be noted that in the following embodiments, user data is acquired and calculated based on data compliance, and the acquired user data is authorized by the user. At the same time, user data and user characteristics are encrypted and protected.

[0046] In some feasible implementations, suppose the user characteristic that the data platform needs to determine is the number of stock account opening users, and the user data required to determine this user characteristic is the user's stock account opening status. The data platform can perform calculations on the above user data based on the Kappa architecture to determine the user characteristic. In this architecture, the message queue is Kafka, the stream processing cluster is Flink, and the target database for storing user characteristics is a data table.When a data platform needs to recalculate user characteristics based on user data—for example, previously when determining the number of stock account opening users, the calculation method was: counting the number of users whose account opening status was "opened"—but now, the calculation method has changed to: counting the number of users whose account opening status is "opened" and "under review"—the data platform needs to redetermine the user characteristic of "stock account opening users." The specific process is as follows: First, the data platform can obtain multiple user data points related to user stock account opening status from other databases, such as business source databases that record user stock account opening information, and store them in Kafka. Assume the data platform obtains... Three sets of user data regarding stock account opening status were obtained. The first set of user data shows that User 1's account opening status for Stock 1 was "opened" at 13:22 on June 30, 2022. The second set of user data shows that User 2's account opening status for Stock 1 was "under review" at 13:23 on June 30, 2022. The third set of user data shows that User 3's account opening status for Stock 2 was "not opened" at 13:30 on June 30, 2022. The data platform can then obtain the generation time of the last set of user data, i.e., the generation time of the third set of user data: 13:30 on June 30, 2022. Finally, the data platform can use Flink to read these three sets of user data from Kafka. When the first user data is read, Flink can determine that the user characteristic corresponding to the first user data, i.e., the number of stock account opening users, is 1. It can also determine that the generation time of the first user data is June 30, 2022, at 13:22. This is inconsistent with the generation time of the last user data, June 30, 2022, at 13:30. Therefore, the data platform will not store the user characteristic corresponding to the first user data in the data table. Next, the data platform can read the second user data from Kafka based on Flink. Flink can determine that the user characteristic corresponding to the second user data is 2, and it can determine that the generation time of the second user data is June 30, 2022, at 13:23. This is inconsistent with the generation time of the last user data, June 30, 2022, at 13:23. Since the generation time of the second user data is inconsistent with 13:30 on June 30, 2022, the data platform will not store the user characteristics corresponding to the second user data in the data table. Finally, the data platform can read the third user data from Kafka using Flink. Flink can determine that the user characteristic corresponding to the third user data is 2, and it can determine that the generation time of the third user data is 13:30 on June 30, 2022, which is consistent with the generation time of the last user data, 13:30 on June 30, 2022. Therefore, the data platform can store the user characteristics corresponding to the third user data in the data table, and thus the data platform can determine the user characteristic: the number of stock account opening users is 2.Therefore, for the query request for querying user features received during the process of recalculating user features, the returned query result will only be the user feature calculated based on the last user data, i.e., the number of stock account opening users is 2. Since the user features calculated based on other user data are not stored in the data table, the query result will not be the user feature calculated based on other user data, such as the number of stock account opening users being 1. Therefore, the accuracy of user feature query can be improved, which in turn can improve the accuracy and efficiency of generating user profiles.

[0047] In some possible implementations, such as Figure 4As shown, when building user profiles, the data platform can first obtain user data from other databases, such as business data sources, through methods like Binlog synchronization or Agent reporting. This user data can include information such as user stock account opening status and user login status. The obtained user data is then placed in the Operation Data Store (ODS) layer. The ODS layer is the layer in the data model closest to the original data in the data source, i.e., the user data, and can generally be directly integrated from the original data source. Binlog synchronization is one way the data platform obtains user data from other databases and stores it in the ODS layer. Agent reporting is one way for other databases to actively send user data to the ODS layer. Additionally, Binlog synchronization includes Changelog (update log). The data platform can capture changes in other databases in real time through the changelog and synchronize these changes to the ODS layer. For example, if another database records user data: User 2's stock account opening status is "under review," the data platform can obtain this data from the other database through Binlog synchronization. The user data is obtained and stored in the ODS layer. At this point, the user data stored in the ODS layer shows that User 2's account opening status for Stock 1 is "under review." Later, when the user data recorded in other databases changes to "Account Opened," the data platform can capture this change in real-time through update logs. The data platform can then synchronize this change to the ODS layer, changing the "under review" status of User 2's account opening status for Stock 1 in the ODS layer to "Account Opened." This reduces data transmission overhead and improves transmission efficiency. Next, the data platform can perform calculations on the user data in the ODS layer at the computation layer to determine user characteristics. The calculation method can be either stream computing or batch computing. The specific calculation process will be described in detail in the following embodiments and will not be repeated here. Then, the data platform can store the user characteristics in the Data Warehouse Service (DWS) layer to generate user profiles. The DWS layer can be used to store calculation results, such as the calculation results for the user data in the ODS layer. It is understandable that in the above data platform's calculation of the user data based on the Kappa architecture, Kafka can serve as the ODS layer, and the data table can serve as the DWS layer.

[0048] In some implementations, when the server reads a first user data point from multiple user data sets and generates the corresponding user features, it can first select a method for generating the user features based on the calculation criteria and application scenario. This method can be either a stream generation method or a batch generation method. Then, the server can generate the corresponding user features based on this method. The data platform can implement the stream generation method (i.e., stream computation) based on the Kappa architecture, or it can trigger the batch generation method (i.e., batch computation) periodically via Airflow. Alternatively, other methods can be used to trigger the batch generation method periodically; this application does not impose any restrictions on this. Airflow is a task scheduling tool that allows setting the trigger time and execution duration of computation tasks, such as batch computation tasks. It should be noted that the server can implement the stream generation method based on either the Kappa or Lambda architecture; this application does not impose any restrictions on this. The Lambda architecture is a data processing architecture that includes two modules: real-time processing (stream computing) and offline processing (batch computing). Therefore, it has a high maintenance cost. The Kappa architecture, on the other hand, does not have an offline processing module (batch computing). Therefore, using the Kappa architecture to complete the stream generation method can reduce the maintenance cost.

[0049] It should be understood that batch generation, or batch computation, is a batch-based, high-latency, proactively initiated computation method. Batch computation requires defining the computation job logic beforehand and submitting it to the streaming computation system; this logic cannot be changed during the entire execution process. The data for batch computation must be pre-loaded into the computation system before the system performs calculations. Unlike batch computation, stream generation, or stream computation, places greater emphasis on the computational data flow and low latency. Stream computation can distribute large amounts of data across various time points, continuously transmitting small batches of data. Data flows continuously, and data is discarded after computation. The results of stream computation can be immediately delivered to the online system for real-time display.

[0050] For example, suppose the user feature 1 to be determined is: the number of times user 1 viewed page 1 in the 10 days prior to 13:30 on May 30, 2022; the application scenario 1 of user feature 1 is: predicting the number of times user 1 viewed page 1 in the 10 days after 13:30 on May 30, 2022 based on user feature 1, in order to determine whether the actual number of times user 1 viewed page 1 in the 10 days after 13:30 on May 30, 2022 is consistent with the predicted number of views; the calculation method 1 for determining user feature 1 is: obtaining the browsing data of all pages by user 1 in the 10 days prior to 13:30 on May 30, 2022, and then statistically analyzing the data... Based on the application scenario 1 and calculation method 1 corresponding to user feature 1, it can be determined that the real-time requirement for determining user feature 1 is not high. Therefore, the data platform can choose a batch generation method, i.e., a batch calculation method, to generate user feature 1. For example, the data platform can set the start time for generating user feature 1 through Airflow, such as 13:30 on June 30, 2022. Then, at that time, it can obtain the browsing data of all pages by user 1 in the 10 days before 13:30 on May 30, 2022. Then, it can use a batch calculation method to count the number of times page 1 is viewed in the data, thereby determining user feature 1.

[0051] For example, suppose the user feature 2 that needs to be determined is: whether user 2 has logged into application 1 in the past three days; the application scenario 2 of user feature 2 is: to determine whether user 2 is an active user based on user feature 2. If user 2 is determined to be an active user, then message 1 is pushed to user 2 in real time; if user 2 is determined not to be an active user, then message 1 is not pushed to user 2; the calculation method 2 of user feature 2 is: to obtain the data of user 2's most recent login to application 1, and to determine whether the generation time of the data is within the past three days. If it is within the past three days, then user feature 2 is determined to be "1"; if it is not within the past three days, then user feature 2 is determined to be "0". Based on the application scenario 2 and calculation method 2 corresponding to user feature 2, it can be determined that the real-time requirement for determining user feature 2 is high. Therefore, the data platform can choose the stream generation method, i.e., the stream computing method, as the generation method for user feature 2. For example, suppose the data platform needs to determine whether to push message 1 to user 2 at 13:30 on June 30, 2022. The data platform can obtain the data of user 2's most recent login to application 1 in real time: user 2 logged into application 1 at 12:30 on June 30, 2022. It can be determined that the data was generated within three days. Therefore, the user characteristic can be determined as "1", that is, user 2 can be determined to be an active user, and message 1 can be pushed to user 2 in real time.

[0052] In some possible implementations, such as Figure 4As shown, the target database in this embodiment may include, but is not limited to, a Remote Dictionary Server (Redis) database, an ElasticSearch database, and a cloud database. When storing user features in the DWS layer (target database), the aforementioned data platform can store user features in a Redis database, an ElasticSearch database, or a cloud server database, etc.

[0053] It should be understood that Redis databases use a key-value storage method, meaning each record contains only a key for querying data and its corresponding value for storing data. Therefore, query interfaces with high real-time requirements and small data volumes, such as Online Transaction Processing (OLTP), generally choose Redis as their query engine. Elasticsearch databases can perform high-performance complex aggregation queries, so query interfaces with lower real-time requirements and large data volumes, such as Online Analytical Processing (OLAP), generally choose Elasticsearch as their query engine. Cloud storage is suitable for data with large volumes, broad coverage, and high real-time requirements. Specifically, when determining the storage method for a data packet, this embodiment first obtains the data parameters of the data packet. These parameters may include the query frequency Dat_fre corresponding to the data name, the data volume Dat_voe corresponding to the data packet, and the data priority Dat_pro corresponding to the data name. Then, based on the data parameters, the attribute parameter Dat_pre corresponding to the data packet is determined as follows:

[0054]

[0055] Where α, γ, and ∈ represent attribute factors obtained through training based on historical data, and Dat_mon represents a preset frequency threshold. The frequency threshold measures the query frequency to determine the appropriate attribute parameter determination method. In this embodiment, the query frequency, priority, and data volume are considered in the calculation of attribute parameters to measure the data packet storage method. Then, the storage location of the data packet is determined based on the attribute parameters. Specifically, preset parameter thresholds for each storage method can be used to determine the storage location of the data packet. This method ensures personalized data storage, thereby improving the efficiency of data storage and retrieval, and reducing data storage costs.

[0056] After storing user characteristics in Redis, Elasticsearch databases, and cloud servers, the data platform can select and determine the corresponding query interface based on the query request. This ensures that different query interfaces, when querying based on appropriate databases, all have databases storing user characteristics, thereby improving query efficiency. The specific implementation of how the data platform selects and determines the corresponding query interface based on the query request will be described in detail in the following embodiments, and will not be repeated here.

[0057] In some possible implementations, such as Figure 5 As shown, the terminal can send a first user feature query request to the server. Upon receiving the request, the server can respond by searching the target database for the presence of the target user feature. If the target database contains the target user feature, the server can send it to the terminal. If the target database does not contain the target user feature, and the target user feature consists of multiple sub-user features from the database, the server can decompose the target user feature into these sub-user features and perform a data query based on them. Finally, the server can determine the target user feature based on the retrieved sub-user features and send it to the terminal. In other words, the server can also use multiple user features for combined judgment. This eliminates the need for the server to generate the target user feature; the target database only needs to store the sub-user features, meaning it can store only atomic features. When the server receives the first query request, it can determine the target user feature based on the sub-user features, thus reducing server computational costs, reducing target database storage costs, and reducing redundancy and repetitive development costs associated with user profile data. Furthermore, in the above process, the server can perform combined judgment of user features based on queries of multiple sub-user features. That is, the server can perform combined judgment using multiple user features, which improves the server's processing power and enhances the user experience.

[0058] For example, suppose the target user characteristic that the terminal needs to query is: a user who has logged into application 1 within the past three days and whose account status for stock 1 is "opened". The target database does not store the above target user characteristic, but stores sub-user characteristic 1 and sub-user characteristic 2, which are: a user who has logged into application 1 within the past three days and a user whose account status for stock 1 is "opened". After the server receives the first user characteristic query request sent by the terminal, the server can search for the above target user characteristic in the target database and determine that the target user characteristic is not stored in the target database. The server can determine that the target user characteristic can be composed of the intersection of sub-user characteristic 1 and sub-user characteristic 2. Then the server can search for sub-user characteristic 1 and sub-user characteristic 2 in the target database respectively, and determine that sub-user characteristic 1 is "user 1, user 2, user 3" and sub-user characteristic 2 is "user 1, user 2". Thus, the server can determine that the target user characteristic is "user 1, user 2". Then, the server can send the determined target user characteristic to the terminal. In this process, after determining that the target user features are not stored in the target database and that the target user features can be composed of multiple sub-user features stored in the target database, the server can query the multiple sub-user features in the target database based on the AST tree and determine the target user features based on the multiple sub-user features. The AST tree is converted from the code about the multiple sub-user features and the compositional relationship when the multiple sub-user features constitute the target user features.

[0059] In some implementations, after receiving a first user feature query request from a terminal, the server can respond by converting it into a second user feature query request with the same meaning. Then, in response to the second user feature query request, the server can search the target database to see if the target user feature is stored. In this way, if the server does not find the user feature corresponding to the first user feature query request in the target database, it can search for the corresponding user feature based on the second user feature query request, which has the same meaning. If the target database contains the user feature corresponding to the second user feature query request, the server can send that user feature to the terminal, thereby improving query efficiency and the efficiency of user profile generation.

[0060] For example, assuming the target database does not store user feature 1: whether user 1 has logged into application 1 in the past three days, and the target database stores user feature 2: user 1's most recent login to application 1 was within the past three days, the terminal sends a first user feature query request to the server to search for user feature 1 in the target database. After receiving the first user feature query request, the server can respond by converting the first user feature query request into a second user feature query request with the same meaning. This second user feature query request is used to search for user feature 2 in the target database. Then, the server can respond by searching for user feature 2 in the target database and sending the found user feature 2 to the terminal. The server can pre-store the correspondence between the second user feature query request and the first user feature query request with the same meaning. When converting the first user feature query request into the second user feature query request, the conversion can be performed according to the stored correspondence. This application does not impose any restrictions on this. Of course, after receiving the first user feature query request, the server can respond to the first user feature query request by querying the target database for user feature 1 corresponding to the first user feature query request. If user feature 1 is not found, the first user feature query request can be converted into a second user feature query request. Then, in response to the second user feature query request, the server can search the target database to see if user feature 2 is stored. This application does not impose any restrictions on this.

[0061] In some implementations, before the server responds to the first user feature query request and searches the target database to see if the target user feature is stored, it can also perform permission verification on the sender of the first user feature query request. If the permission verification on the sender is successful, the server can respond to the first user feature query request and search the target database to see if the target user feature is stored, thereby improving the security of data query and the security of generating user profiles.

[0062] For example, before the server responds to the first user feature query request and searches the target database for stored target user features, the server can obtain the sender's identifier. Then, the server can determine the sender's permission scope based on the sender's identifier. If the sender's permission scope includes the permission to query target user features, the server can determine that the permission verification for the sender has passed; if the sender's permission scope does not include the permission to query target user features, the server can determine that the permission verification for the sender has failed. The server can pre-store the correspondence between the sender's identifier and the sender's permission scope, where the sender's permission scope includes the user features that the sender can search for in the target database. For example: Suppose the first user feature query request is used to query user feature 1 in the target database. The server pre-stores the correspondence between the identifier of business party 1 and the permission scope 1 of business party 1. Suppose the permission scope 1 includes: user feature 1 and user feature 2. After receiving the first user feature query request sent by the terminal, the server can first determine that the identifier of the sender of the first user feature query request is sender 1. Then the server can find in the pre-stored correspondence that the permission scope of sender 1 includes user feature 1. Then the server can determine that the permission verification of sender 1 has passed.

[0063] In some implementations, before the server responds to the first user feature query request and searches the target database for whether the target user feature is stored, it can determine the query interface corresponding to the first user feature query request. Then, based on the query interface corresponding to the first user feature query request, the server can respond to the first user feature query request and search the target database for whether the target user feature is stored. For example, when determining the query interface corresponding to the first user feature query request, the server can first determine whether the first user feature query request includes a user identifier. If the first user feature query request includes a user identifier, the query interface corresponding to the first user feature query request can be determined to be an OLTP interface; if the first user feature query request does not include a user identifier, the query interface corresponding to the first user feature query request can be determined to be an OLAP interface. It is understandable that when the first user feature query request includes a user identifier, it can generally be determined that the first user feature query request is used to query user features related to the user corresponding to that user identifier. Therefore, the amount of data included in the query result is relatively small. Combining the above description of OLTP and OLAP interfaces, the query interface can be selected as the OLTP interface. In this embodiment, the OLTP interface can support second-level query of user feature values, aggregated counts of feature values, and second-level metadata query and return. Similarly, when the first user feature query request does not include a user identifier, it can generally be determined that the first user feature query request is used to query more complex user features, such as all users who have logged into application 1 within the last three days. In this case, the query interface can be selected as the OLAP interface. In this way, the server can select an appropriate query interface based on the user feature query request to improve data query efficiency and reliability, thereby improving the reliability and efficiency of generating user profiles.

[0064] For example, suppose the first user feature query request is used to query user feature 1: user 1's age in the target data. The first user feature query request includes the identifier "user 1". After receiving the first user feature query request, the server can determine that it includes the user identifier "user 1". Therefore, the server can choose the query interface corresponding to this first user feature query request as an OLTP interface. This method supports anchoring multiple types of user data through user profiles, enabling diverse targeting needs, such as setting real-time user tags, searching routine or static user groups, etc. It also supports more flexible user anchoring methods, allowing the gateway layer to translate more types of SQL statements. For example, it supports real-time data queries with dynamic ranges, allowing businesses to more flexibly use the underlying feature data of user profiles on the front end.

[0065] For example, the server can use an OLTP interface to determine whether a user meets certain conditions, and then send corresponding pop-up ads to users who meet those conditions. For instance, suppose a first user feature query request is used to query user feature 2 in the target data: whether user 2 has opened an account. The first user feature query request includes the identifier user 2. After receiving the first user feature query request, the server can determine that the first user feature query request includes the user identifier user 2. The server can then choose the query interface corresponding to the first user feature query request as the OLTP interface. Furthermore, the server can return true to the terminal when it finds that user feature 2 indicates user 2 has opened an account, and return false to the terminal when it finds that user feature 2 indicates user 2 has not opened an account. Additionally, the server can recommend an account opening pop-up ad to user 2 when it confirms that user feature 2 indicates user 2 has not opened an account.

[0066] For example, when a new stock opens, this embodiment queries and matches the user's authorized watchlist or watchlist information. For customers whose watchlist contains the stock, it pulls the entire user group of "user watchlist = xxxx" and pushes the new stock opening information.

[0067] For example, a server can use an OLAP interface to identify users who meet certain criteria and push corresponding messages to those users. For instance, suppose a first user feature query request is used to query user feature 3: active users (users who have logged in within the last three days) in the target data. Upon receiving this first user feature query request, the server can determine that it does not include a user identifier. Therefore, the server can select the OLAP interface as the query interface corresponding to this first user feature query request, and then use the OLAP interface to identify all active users and push messages to them.

[0068] For example, the user features in this embodiment may also include a user-blocked user list, a blacklist of users, self-selected information, a list of stocks to watch, etc. When a user enters NiuNiu Circle, the system can filter recommended content based on the first two features and use the latter two features to recommend relevant posts to the user.

[0069] For example, a server can query multiple user characteristics of a user through an OLTP interface, and then combine these characteristics to analyze the user's interests and preferences, recommending relevant articles to the user. These user characteristics can include information about followed stocks, holdings, etc. For instance, suppose a first user characteristic query request 1 is used to query user characteristic 4 in the target data: whether user 4 follows or holds stock 1, and a first user characteristic query request 2 is used to query user characteristic 5: whether user 4 follows stock 2. After receiving both first user characteristic query requests 1 and 2, the server can determine that both include the user identifier: user 4. The server can then choose OLTP interfaces for both first user characteristic query requests. The server can then query that user characteristic 4 indicates user 4 follows stock 1, and user characteristic 5 indicates user 4 does not follow stock 2. Therefore, the server can analyze user 4's interests and preferences as: liking stock 1 and disliking stock 2. The server can then recommend articles, announcements, related posts, news, and forum content related to stock 1 to user 4.

[0070] For example, the server can use an OLTP interface to determine whether a user meets a certain condition, and then issue a corresponding reward to the user who meets the condition. For instance, suppose the first user feature query request is used to query user feature 6 in the target data: whether user 6 has deposited money. The first user feature query request includes the identifier user 6. After receiving the first user feature query request, the server can determine that the first user feature query request includes the user identifier user 6. Then the server can choose the query interface corresponding to the first user feature query request as the OLTP interface, and the server can issue a deposit reward to user 6 when it finds that user feature 6 indicates that user 6 has deposited money.

[0071] For example, different users have different stock market access permissions. High-asset users have higher access to stock market data, while users without an account only have access to specific data. Therefore, the server can control a user's stock market data access permissions by determining whether their assets meet target criteria. For instance, suppose a first user characteristic query request is used to query whether user 7's assets are 10,000. Upon receiving this request, the server can determine that it includes user identifier user 7. The server can then choose the OLTP interface for this query request, and if it finds that user 7's assets have reached 10,000, it can grant user 7 access to stock market data.

[0072] In some implementation methods, the server can determine whether there are abnormal user characteristics in the target database. If abnormal user characteristics are found in the target database, the server can generate a prompt message and push the prompt message to notify the user that there are abnormal user characteristics in the target database. If there are no abnormal user characteristics in the target database, the server can choose not to generate a prompt message. This can ensure the accuracy of the user characteristics stored in the target database, improve the accuracy of data query results, and thus improve the accuracy and efficiency of generating user profiles.

[0073] For example, the server can establish a user profile monitoring module to build feature models for different user characteristics or user profiles to determine whether there are abnormal user characteristics in the target database, i.e., to issue monitoring alerts. For instance, when determining whether there are abnormal user characteristics in the target database, the server can obtain a first user characteristic stored in the target data at any given time and at least one second user characteristic within a preset time period before that time. Then, the server can statistically analyze the at least one second user characteristic to obtain the distribution range of the first user characteristic. If the first user characteristic is not within the distribution range, the server can determine that there are abnormal user characteristics in the target database; if the first user characteristic is within the distribution range, the server can determine that there are no abnormal user characteristics in the target database. For example: The server can retrieve the first user characteristic stored in the target database at 24:00 on June 30, 2022: the number of users who logged into application 1 on June 30, 2022, which is denoted as 'a'. It can then retrieve two second user characteristics from the two days prior to 24:00 on June 30, 2022: second user characteristic 1 and second user characteristic 2, which are respectively the number of users who logged into application 1 on June 29, 2022, and the number of users who logged into application 1 on June 28, 2022, which are denoted as 'b' and 'c'. Then, the server can calculate the... The average of user feature 1 and user feature 2 is (b+c) / 2 = d. Then, the server can determine (de, d+e) as the distribution range of the first user feature. If the first user feature a is within the distribution range (de, d+e), the first user feature can be determined to be normal data, and it can be determined that there are no abnormal user features in the target database. If the first user feature a is not within the distribution range (de, d+e), the first user feature can be determined to be abnormal data, and it can be determined that there are abnormal user features in the target database. Here, a, b, c, d, and e are positive integers.

[0074] Understandably, the user profile production process is lengthy, and data migration or changes in data sources can easily lead to inaccurate user data, resulting in inaccurate user profiles. Therefore, monitoring and alerting on user features or user profiles are necessary. For example, the user data mentioned above might be obtained by the server from other databases, such as business source databases recording user stock account opening information or login information. If the user data stored in these other databases changes, such as a change in the storage location, the user data obtained by the server will be inaccurate, leading to inaccurate user features and consequently inaccurate data query results. Therefore, by judging whether there are abnormal user features in the target database, the accuracy of user features stored in the target database can be guaranteed, optimizing the speed of feature data calculation changes and enabling faster real-time stream computing based on business data. Currently, over 90% of features support second-level real-time updates, improving the accuracy of data query results and thus enhancing the accuracy and efficiency of user profile generation.

[0075] In some possible implementations, such as Figure 1 As shown, assuming application 1 is installed on terminal 110, a user accesses application 1 and queries for target user characteristics based on natural language. After the user inputs the natural language used to query the target user characteristics in application 1, the server can convert the natural language into SQL and query the target user characteristics in the target database based on the SQL. The server can pre-store the correspondence between natural language and SQL, and then convert the natural language into SQL based on this correspondence. For example, as shown in Table 1, assuming the server pre-stores a partial correspondence between natural language and SQL, as shown in Table 1, the natural language "age" corresponds to the SQL "SQL1".

[0076] Table 1

[0077] Natural Language SQL Natural Language SQL age SQL1 and SQL4 ＝ SQL2 Social days SQL5 20 SQL3 30 SQL6

[0078] For example, such as Figure 6 As shown, assuming the user inputs "age = 20" in the natural language 1 on application 1, the terminal can send the natural language 1 to the server. After receiving the natural language 1, the server can convert the natural language 1 into the corresponding SQL "SQL1, SQL2, SQL3" based on the above table 1.

[0079] For example, such as Figure 7 As shown, there is a plus sign button after Natural Language 1 "Age = 20". When the user clicks the plus sign button, they can continue to enter Natural Language 2 "Social Days = 20". The relationship between Natural Language 2 and Natural Language 1 is as follows: Figure 4The "and" shown indicates that the user wants to query "users whose age = 20 and social days = 20". Then, the server can convert the user's input in natural language into the corresponding SQL "SQL1, SQL2, SQL3, SQL4, SQL5, SQL2, SQL3".

[0080] It should be understood that the correspondence between natural language and SQL in this application is merely illustrative, and the conversion of the natural language into the corresponding SQL is also illustrative.

[0081] In some possible implementations, such as Figure 8 As shown, the server may include: an authorization verification module, a profile service module, and a target database. The terminal can send a first user feature query request to the server to query the target user features. After receiving the first user feature query request, the server can first verify the authorization of the sender of the first user feature query request based on the authorization verification module. If the authorization verification of the sender is successful, the server can respond to the first user feature query request, determine the query interface corresponding to the first user feature query request based on the profile service module, and then, based on the query interface corresponding to the first user feature query request, search for the target user features in the target database. If the target user features are stored in the target database, the server can send the target user features to the terminal.

[0082] In some possible implementations, such as Figure 9 As shown, the server may include: a parameter validation module, an SQL parsing module, a routing module, and a target database. The terminal can send a first user feature query request to the server to query target user features. The first user feature query request can be in natural language. After receiving the first user feature query request, the server can first perform permission verification on the sender of the first user feature query request based on the parameter validation module. If the permission verification is successful, the server can respond to the first user feature query request by parsing the natural language form of the first user feature query request into an SQL form based on the SQL parsing module. Then, the server can determine the query interface corresponding to the first user feature query request based on the routing module. Finally, the server can respond to the first user feature query request by checking whether the target user features are stored in the target database based on the query interface corresponding to the first user feature query request. If the target user features are stored in the target database, the server can send the target user features to the terminal.

[0083] In summary, the technical solutions provided by the above embodiments bring at least the following beneficial effects: Through the technical solution of this application, the server can first obtain multiple user data and the generation time of the last user data among the multiple user data. If each first user data is read from the multiple user data, the server can generate user features corresponding to the first user data. If the generation time of the first user data is the same as the generation time of the last user data, the server can store the user features corresponding to the first user data in the target database. If the generation time of the first user data is different from the generation time of the last user data, the server can choose not to store the user features corresponding to the first user data in the target database. Finally, the server can generate user profiles based on the user features in the target database. In the above process, the server can determine whether the currently read user data is the last user data by checking if its generation time matches the generation time of the last user data. This allows the server to store only the user features corresponding to the last user data in the target database, while storing the user features corresponding to other user data. Therefore, when querying user features, such as when the server is reading user data, the query result will not be the user features corresponding to other user data, but only the last original corresponding user feature—the final user feature calculated from all user data. The server can then generate an accurate user profile based on this final user feature. This solves the problem in existing technologies where inaccurate user feature queries due to data storage lead to inaccurate user profiles generated from the queried user features. This improves the accuracy of user feature queries, thereby enhancing the efficiency and precision of user profile applications.

[0084] Furthermore, upon receiving the first user feature query request, the server can respond by checking the target database to see if the target user feature is stored. If the target database does not store the target user feature, and the target user feature consists of multiple sub-user features from the target database, the server can decompose the target user feature into these sub-user features and perform data queries based on them. In this way, the server does not need to generate the target user feature; the target database only needs to store the sub-user features. When the server receives the first query request, it can determine the target user feature based on the sub-user features, thereby reducing the server's computational cost and the target database's storage cost.

[0085] Furthermore, after receiving a first user feature query request from a terminal, the server can respond by converting it into a second user feature query request with the same meaning. Then, in response to the second user feature query request, the server can search the target database to see if the target user feature is stored. In this way, if the server does not find the user feature corresponding to the first user feature query request in the target database, it can search for the corresponding user feature based on the second user feature query request, which has the same meaning. If the target database contains the user feature corresponding to the second user feature query request, the server can send that user feature to the terminal, thereby improving query efficiency and the efficiency of generating user profiles.

[0086] Furthermore, before responding to the first user feature query request and searching the target database for whether the target user feature is stored, the server can also perform permission verification on the sender of the first user feature query request. If the permission verification on the sender is successful, the server can respond to the first user feature query request and search the target database for whether the target user feature is stored, thereby improving the security of data query and the security of generating user profiles.

[0087] Furthermore, before responding to the aforementioned first user feature query request and searching the target database for whether the target user feature is stored, the server can determine the query interface corresponding to the first user feature query request. Then, based on the query interface corresponding to the first user feature query request, the server can respond to the first user feature query request and search the target database for whether the target user feature is stored. In this way, the server can select an appropriate query interface according to the user feature query request, thereby improving data query efficiency and reliability, and improving the efficiency and reliability of generating user profiles.

[0088] Furthermore, the server can determine whether there are abnormal user characteristics in the target database. If abnormal user characteristics are found in the target database, the server can generate a prompt message and push the prompt message to notify the user that abnormal user characteristics exist in the target database. If there are no abnormal user characteristics in the target database, the server can choose not to generate a prompt message, thereby ensuring the accuracy of the user characteristics stored in the target database, improving the accuracy of data query results, and improving the accuracy of generating user profiles.

[0089] Figure 10 A schematic diagram of a data processing device 1000 based on user profiles provided in this application embodiment is shown below. Figure 10 As shown, the device 1000 includes:

[0090] The first acquisition module 1001 is used to acquire multiple user data and the generation time of the last user data among the multiple user data;

[0091] The first generation module 1002 is used to generate user features corresponding to each first user data if each first user data is read from multiple user data.

[0092] The processing module 1003 is used to: if the generation time of the first user data is the same as the generation time of the last user data, then store the user features corresponding to the first user data in the target database; if the generation time of the first user data is different from the generation time of the last user data, then do not store the user features corresponding to the first user data in the target database.

[0093] The second generation module 1004 is used to generate user profiles based on user features in the target database.

[0094] In some implementations, the device 1000 further includes: a second acquisition module 1005, a search module 1006, a decomposition module 1007, and a query module 1008, wherein the second acquisition module 1005 is used to acquire a first user feature query request; the search module 1006 is used to search in the target database for whether the target user feature is stored in response to the first user feature query request; the decomposition module 1007 is used to decompose the target user feature into multiple sub-user features if the target user feature is not stored in the target database and the target user feature is composed of multiple sub-user features in the target database; and the query module 1008 is used to perform data query based on the multiple sub-user features.

[0095] In some implementations, the lookup module 1006 is specifically used to, in response to the first user feature query request, convert the first user feature query request into a second user feature query request with the same meaning; and in response to the second user feature query request, search in the target database to see if the target user feature is stored.

[0096] In some implementations, the device 1000 further includes: a verification module 1009, wherein the verification module 1009 is used to perform permission verification on the sender of the first user feature query request; and the search module 1006 is specifically used to, in response to the first user feature query request, search in the target database whether the target user features are stored when the permission verification on the sender is passed.

[0097] In some implementations, the verification module 1009 is specifically used to obtain the sender's identifier; determine the sender's permission scope based on the sender's identifier; if the sender's permission scope includes the permission to query the target user's characteristics, then the permission verification of the sender is determined to be successful; if the sender's permission scope does not include the permission to query the target user's characteristics, then the permission verification of the sender is determined to be unsuccessful.

[0098] In some implementations, the apparatus 1000 further includes: a determining module 1010, wherein the determining module 1010 is used to determine the query interface corresponding to the first user feature query request; and the searching module 1006 is specifically used to search in the target database for whether the target user features are stored, based on the query interface corresponding to the first user feature query request and in response to the first user feature query request.

[0099] In some implementations, the determining module 1010 is specifically used to determine that the query interface corresponding to the first user feature query request is an OLTP interface if the first user feature query request includes a user identifier; and to determine that the query interface corresponding to the first user feature query request is an OLAP interface if the first user feature query request does not include a user identifier.

[0100] In some implementations, the device 1000 further includes: a judgment module 1011, a third generation module 1012, and a push module 1013, wherein the judgment module 1011 is used to judge whether there are abnormal user features in the target database; the third generation module 1012 is used to generate a prompt message if there are abnormal user features in the target database; and the push module 1013 is used to push the prompt message to notify the user that there are abnormal user features in the target database.

[0101] In some implementation methods, the judgment module 1011 is specifically used to obtain a first user feature stored in the target database at any time and at least one second user feature within a preset time period before any time; to perform statistics on the at least one second user feature to obtain the distribution range of the first user feature; if the first user feature is not within the distribution range, it is determined that there is an abnormal user feature in the target database; if the first user feature is within the distribution range, it is determined that there is no abnormal user feature in the target database.

[0102] In some possible implementations, the first generation module 1003 is specifically used to select a method for generating user features corresponding to the first user data according to the calculation method and application scenario for each first user data read from multiple user data. The generation method is either a stream generation method or a batch generation method; and to generate user features corresponding to the first user data based on the method for generating user features corresponding to the first user data.

[0103] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details will not be provided here. Specifically, Figure 10 The apparatus 1000 shown can execute the above-described method embodiments, and the aforementioned and other operations and / or functions of each module in the apparatus 1000 are respectively for implementing the corresponding processes in the above-described methods. For the sake of brevity, they will not be described in detail here.

[0104] The apparatus 1000 of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0105] Figure 11 This is a schematic block diagram of the electronic device provided in the embodiments of this application.

[0106] like Figure 11 As shown, the electronic device may include:

[0107] The system includes a memory 1110 and a processor 1120. The memory 1110 stores computer programs and transfers the program code to the processor 1120. In other words, the processor 1120 can retrieve and run the computer program from the memory 1110 to implement the methods described in the embodiments of this application.

[0108] For example, the processor 1120 can be used to execute the above-described method embodiments according to instructions in the computer program.

[0109] In some embodiments of this application, the processor 1120 may include, but is not limited to:

[0110] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0111] In some embodiments of this application, the memory 1110 includes, but is not limited to:

[0112] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0113] In some embodiments of this application, the computer program may be divided into one or more modules, which are stored in the memory 1110 and executed by the processor 1120 to complete the method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the electronic device.

[0114] like Figure 11 As shown, the electronic device may also include:

[0115] Transceiver 1130, which can be connected to processor 1120 or memory 1110.

[0116] The processor 1120 can control the transceiver 1130 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 1130 may include a transmitter and a receiver. The transceiver 1130 may further include antennas, and the number of antennas may be one or more.

[0117] It should be understood that the various components in the electronic device are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.

[0118] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.

[0119] When implemented using software, it can be implemented entirely or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0120] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0122] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0123] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method based on user profiling, characterized in that, include: Obtain multiple user data sets and the generation time of the last user data set among the multiple user data sets; If a first user data is read from each of the multiple user data, then the user feature corresponding to the first user data is generated; If the generation time of the first user data is the same as the generation time of the last user data, then the user features corresponding to the first user data are stored in the target database. If the generation time of the first user data is inconsistent with the generation time of the last user data, then the user features corresponding to the first user data will not be stored in the target database. User profiles are generated based on user characteristics in the target database.

2. The method of claim 1, wherein, Also includes: Obtain the first user feature query request; In response to the first user feature query request, the system searches the target database to see if the target user features are stored. If the target user feature is not stored in the target database, and the target user feature is composed of multiple sub-user features in the target database, then the target user feature is decomposed into the multiple sub-user features; User feature query is performed based on the multiple sub-user features.

3. The method of claim 2, wherein, The step of responding to the first user feature query request by searching the target database for whether the target user features are stored includes: In response to the first user feature query request, the first user feature query request is converted into a second user feature query request with the same meaning; In response to the second user feature query request, the system searches the target database to see if the target user feature is stored.

4. The method according to claim 2 or 3, characterized in that, Before searching the target database for whether the target user features are stored in response to the first user feature query request, the method further includes: Perform permission verification on the sender of the first user feature query request; The step of responding to the first user feature query request by searching the target database for whether the target user features are stored includes: When the sender's permissions are verified, in response to the first user feature query request, the system searches the target database to see if the target user features are stored.

5. The method of claim 4, wherein, The permission verification of the sender of the first user feature query request includes: Obtain the sender's identifier; The sender's permission scope is determined based on the sender's identifier; If the sender's permission scope includes the permission to query the target user's characteristics, then the permission verification of the sender is deemed successful. If the sender's permission scope does not include the permission to query the target user's characteristics, then the permission verification for the sender is determined to have failed.

6. The method according to claim 2 or 3, characterized in that, Before searching the target database for whether the target user features are stored in response to the first user feature query request, the method further includes: Determine the query interface corresponding to the first user feature query request; The step of responding to the first user feature query request by searching the target database for whether the target user features are stored includes: Based on the query interface corresponding to the first user feature query request, in response to the first user feature query request, the system searches the target database to see if the target user features are stored.

7. The method of claim 6, wherein, The step of determining the query interface corresponding to the first user feature query request includes: If the first user feature query request includes a user identifier, then the query interface corresponding to the first user feature query request is determined to be the Online Transaction Processing (OLTP) interface. If the first user feature query request does not include a user identifier, then the query interface corresponding to the first user feature query request is determined to be the Online Analytical Processing (OLAP) interface.

8. The method according to any one of claims 1 to 3, characterized in that, Also includes: Determine whether there are any abnormal user characteristics in the target database; If abnormal user characteristics are found in the target database, a prompt message will be generated; The notification message is pushed to inform the user that there are abnormal user characteristics in the target database.

9. The method of claim 8, wherein, The determination of whether there are abnormal user characteristics in the target database includes: Obtain a first user feature stored in the target database at any given time and at least one second user feature within a preset time period prior to any given time. Statistical analysis is performed on the at least one second user feature to obtain the distribution range of the first user feature; If the first user feature is not within the distribution range, then it is determined that there are abnormal user features in the target database; If the first user characteristic is within the distribution range, then it is determined that there are no abnormal user characteristics in the target database.

10. The method according to any one of claims 1 to 3, characterized in that, The step of generating user features corresponding to each piece of first user data read from the plurality of user data includes: If each first user data is read from the plurality of user data, then according to the calculation method and application scenario, the method for generating user features corresponding to the first user data is selected, wherein the generation method is a stream generation method or a batch generation method; The method for generating user features corresponding to the first user data generates user features corresponding to the first user data.

11. A data processing device based on user profiles, characterized in that, include: The first acquisition module is used to acquire multiple user data and the generation time of the last user data among the multiple user data; The first generation module is used to generate user features corresponding to each first user data read from the plurality of user data. Processing module, used for: If the generation time of the first user data is the same as the generation time of the last user data, then the user features corresponding to the first user data are stored in the target database. If the generation time of the first user data is inconsistent with the generation time of the last user data, then the user features corresponding to the first user data will not be stored in the target database. The second generation module is used to generate user profiles based on user features in the target database.

12. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store a computer program, the processor being used to invoke and run the computer program stored in the memory to perform the method of any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method as described in any one of claims 1-10.

14. A computer program product containing instructions, characterized in that, When the computer program product is run on an electronic device, the electronic device causes the electronic device to perform the method of any one of claims 1-10.

Images