Data processing method, device, server and storage medium
By receiving and aggregating user data and performing correlation processing, the problem of user data isolation is solved, and the deep connection and value mining of data are realized.
Patent Information
- Application Number
- CN202210476141.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-29
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-04-29
AI Technical Summary
In the prior art, the user data collected by various business parties are isolated from each other, resulting in the data value being unable to be effectively mined.
By receiving the target user account and the target acquisition time, dynamic attribute data are obtained and aggregated, and dynamic attribute data of multiple data categories are correlated according to the target acquisition time, their correspondence relationship is determined, data correlation is performed, and target association data is generated.
Make the data of different business parties interconnected, and the target related data processed through correlation can represent deeper connections, which is conducive to effectively mining the value of data.
Smart Images

Figure CN114996308B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method, device, server and storage medium. Background Art
[0002] When each business party obtains user authorization to collect user positioning data, in most cases they can only obtain single user data, for example, only the base station data where the user is located, or only the user's location data. There is a problem of mutual isolation between these data, which makes it impossible for the application party to effectively apply these data, so that a large amount of data value cannot be effectively mined. Summary of the invention
[0003] The present disclosure provides a data processing method, device, server and storage medium to at least solve the problem that data are isolated from each other and the value of data cannot be effectively mined. The technical solution of the present disclosure is as follows:
[0004] According to a first aspect of an embodiment of the present disclosure, there is provided a data processing method, which is applied to a first server and includes:
[0005] receiving a target user account and a target collection time sent by a second server; the target user account and the target collection time are extracted by the second server from a first database; the first database stores user data collected by various business parties, the user data including user account, collection time and dynamic attribute data, the dynamic attribute data being data whose attribute value changes according to time, and the dynamic attribute data including multiple data categories;
[0006] Acquire aggregated dynamic attribute data of the target user account within a target time period from the first database; the target time period is determined based on the target collection time, and the aggregated dynamic attribute data is the dynamic attribute data collected by each business party within the target time period;
[0007] The dynamic attribute data of multiple data categories in the aggregated dynamic attribute data are associated with each other according to the target collection time to obtain target associated data.
[0008] Optionally, the associating the dynamic attribute data of multiple data categories in the aggregated dynamic attribute data according to the target collection time includes:
[0009] Determining a correspondence between dynamic attribute data of multiple data categories in the aggregated dynamic attribute data according to the target collection time;
[0010] The dynamic attribute data of the multiple data categories are associated according to the corresponding relationship to obtain target associated data.
[0011] Optionally, the aggregated dynamic attribute data includes two data categories, namely, location data and base station data, wherein the location data includes the geographical location of the target user account in the target time period, and the base station data includes the identifier of the base station to which the target user account is connected in the target time period. The corresponding relationship between the dynamic attribute data of multiple data categories in the aggregated dynamic attribute data is determined according to the target collection time, including:
[0012] Determining a correspondence between the base station identifier and the geographical location;
[0013] The step of associating the dynamic attribute data of the plurality of data categories according to the corresponding relationship to obtain target associated data includes:
[0014] Determine a geographical location list corresponding to the base station identifier according to the corresponding relationship;
[0015] The base station identifier and the geographical location list are associated to obtain target association data.
[0016] Optionally, before using the aggregated dynamic attribute data to perform association processing on the dynamic attribute data of the multiple data categories, the method further includes:
[0017] Get the number of attribute values of the aggregated dynamic attribute data corresponding to each data category;
[0018] If there are target data categories with two or more attribute values, the dynamic attribute data of the target data categories are calibrated.
[0019] Optionally, the calibrating the dynamic attribute data of the target data category includes:
[0020] Clustering the multiple attribute values to obtain a clustering result;
[0021] Determine the number of cluster center points in the clustering result;
[0022] If the number of the cluster center point is one, the attribute value corresponding to the cluster center point is used as the calibration result of the dynamic attribute data;
[0023] If there are multiple cluster center points, multiple attribute values corresponding to the multiple cluster center points are used as calibration results of the dynamic attribute data.
[0024] Optionally, before obtaining the aggregated dynamic attribute data of the target user account within a target time period from the first database, the method further includes:
[0025] Acquire a splicing time window corresponding to the dynamic attribute data from a configuration database;
[0026] A target time period is determined according to the target acquisition time and the splicing time window.
[0027] Optionally, after obtaining the target associated data, the following is further included:
[0028] The target associated data is stored in a second database, and the second database is used for a third server to query the associated data.
[0029] According to a second aspect of an embodiment of the present disclosure, there is provided a data processing device, applied to a first server, comprising:
[0030] The receiving module is configured to receive a target user account and a target collection time sent by the second server; the target user account and the target collection time are extracted by the second server from the first database; the first database stores user data collected by various business parties, the user data includes the user account, the collection time and dynamic attribute data, the dynamic attribute data is data whose attribute value changes according to time, and the dynamic attribute data includes multiple data categories;
[0031] an acquisition module configured to acquire aggregated dynamic attribute data of the target user account within a target time period from the first database; the target time period is determined based on the target collection time, and the aggregated dynamic attribute data is the dynamic attribute data collected by each business party within the target time period;
[0032] The association processing module is configured to perform association processing on the dynamic attribute data of multiple data categories in the aggregated dynamic attribute data according to the target collection time to obtain target association data.
[0033] Optionally, the association processing module is specifically configured to execute:
[0034] Determining a correspondence between dynamic attribute data of multiple data categories in the aggregated dynamic attribute data according to the target collection time;
[0035] The dynamic attribute data of the multiple data categories are associated according to the corresponding relationship to obtain target associated data.
[0036] Optionally, the aggregated dynamic attribute data includes two data categories: location data and base station data, the location data includes the geographic location of the target user account in the target time period, and the base station data includes the base station identifier to which the target user account is connected in the target time period, and the association processing module is further configured to execute:
[0037] Determining a correspondence between the base station identifier and the geographical location;
[0038] The association processing module is further configured to execute:
[0039] Determine a geographical location list corresponding to the base station identifier according to the corresponding relationship;
[0040] The base station identifier and the geographical location list are associated to obtain target association data.
[0041] Optionally, the device further comprises:
[0042] A number acquisition module is configured to execute acquisition of the number of attribute values of the aggregated dynamic attribute data corresponding to each data category;
[0043] The calibration module is configured to calibrate the dynamic attribute data of the target data category if there are two or more target data categories with attribute values.
[0044] Optionally, the calibration module is specifically configured to perform:
[0045] Clustering the multiple attribute values to obtain a clustering result;
[0046] Determine the number of cluster center points in the clustering result;
[0047] If the number of the cluster center point is one, the attribute value corresponding to the cluster center point is used as the calibration result of the dynamic attribute data;
[0048] If there are multiple cluster center points, multiple attribute values corresponding to the multiple cluster center points are used as calibration results of the dynamic attribute data.
[0049] Optionally, the device further comprises:
[0050] A time window acquisition module is configured to acquire a splicing time window corresponding to the dynamic attribute data from a configuration database;
[0051] The target time period determination module is configured to determine the target time period according to the target acquisition time and the splicing time window.
[0052] Optionally, the device further comprises:
[0053] The first storage module is configured to store the target associated data in a second database, where the second database is used for a third server to query the associated data.
[0054] According to a third aspect of an embodiment of the present disclosure, a server is provided, including:
[0055] processor;
[0056] a memory for storing instructions executable by the processor;
[0057] The processor is configured to execute the instructions to implement the data processing method as described in the first aspect.
[0058] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided. When instructions in the storage medium are executed by a processor of a server, the server can execute the data processing method as described in the first aspect.
[0059] According to a fifth aspect of an embodiment of the present disclosure, there is provided a computer program product, comprising a computer program / instruction, wherein the computer program / instruction implements the data processing method as described in the first aspect when executed by a processor.
[0060] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:
[0061] In an embodiment of the present invention, a target user account and a target collection time sent by a second server are received; the target user account and the target collection time are extracted by the second server from a first database; the first database stores user data collected by each business party, the user data includes a user account, a collection time, and dynamic attribute data, the dynamic attribute data is data whose attribute value changes according to time, and the dynamic attribute data includes multiple data categories; the aggregated dynamic attribute data of the target user account within a target time period is obtained from the first database; the target time period is determined based on the target collection time, and the aggregated dynamic attribute data is the dynamic attribute data collected by each business party within the target time period; the dynamic attribute data of multiple data categories in the aggregated dynamic attribute data are associated according to the target collection time to obtain target associated data. In this way, all dynamic attribute data collected by each business party within the target time period are aggregated and associated according to data categories, so that data from different business parties can be connected to each other, and the target associated data after the associated processing can represent a deeper connection between the data, which is conducive to effectively mining the value of data. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.
[0063] Figure 1 is a flowchart of steps of a first data processing method according to an exemplary embodiment;
[0064] Figure 2 is a flowchart of steps of a second data processing method according to an exemplary embodiment;
[0065] Figure 3 is a schematic diagram of a data processing system according to an exemplary embodiment;
[0066] Figure 4 is a structural block diagram of a data processing device according to an exemplary embodiment;
[0067] Figure 5 The invention is a block diagram of a server for data processing according to an exemplary embodiment. DETAILED DESCRIPTION
[0068] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.
[0069] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0070] It should be noted that the user information involved in the embodiments of the present disclosure (including but not limited to user device information, user personal information, etc.) is all information authorized by the user or fully authorized by all parties.
[0071] Figure 1 is a flowchart of a first data processing method according to an exemplary embodiment, wherein the method is applied to a first server, such as Figure 1 As shown, the data processing method includes the following steps.
[0072] In step S11, the target user account and target collection time sent by the second server are received; the target user account and the target collection time are extracted by the second server from the first database; the first database stores user data collected by each business party respectively, the user data includes user account, collection time and dynamic attribute data, the dynamic attribute data is data whose attribute value changes according to time, and the dynamic attribute data includes multiple data categories.
[0073] In an embodiment of the present disclosure, a data processing system is provided, which includes multiple business parties, a first server, a second server, a first database, and a second database. The multiple business parties may be server terminals of multiple different applications, or may be different service departments in the same application. For example, the multiple business parties may be a user service department, a risk control department, a local city department, etc. in the same application.
[0074] After obtaining the user's authorization, the business party collects the user data and reports the collected user data to the second server.
[0075] The user data collected by the business party includes the user account, collection time and dynamic attribute data, where dynamic attribute data refers to data that changes according to time, including multiple data categories. For example, the data categories of dynamic attribute data may include: the base station data where the user is located, the wifi (Wireless Fidelity) hotspot data to which the user is connected, the user's IP (Internet Protocol) address, the user's location data, etc. Among them, the base station data can be a base station identifier, the wifi hotspot data can be a wifi router identifier, and the location data can be latitude and longitude information.
[0076] Generally, dynamic attribute data of two or more data categories are required to perform correctness verification and effective application. However, due to various reasons, it is difficult for the business party to collect dynamic attribute data of two or more data categories at the same time. In this way, the first server can only obtain a single data category, such as a single base station data, or a single location data.
[0077] Therefore, the embodiment of the present disclosure adds nodes such as a first database, a second server, and a second database to perform association processing on mutually isolated user data collected by various business parties.
[0078] First, each business party sends the collected user data to the second server, and the second server processes the received user data, mainly including legality verification, data filtering, and converting the data format into a unified format. After processing, the second server stores the processed data in the first database.
[0079] Then, the second server extracts the latest stored user data from the first database as the target user account and the target collection time, and sends the target user account and the target collection time to the first server. In this way, the first server can always obtain the latest collected user data, so as to efficiently perform association processing on the latest collected user data, thereby ensuring the timeliness of the target association data, so that the target association data can be effectively applied as soon as possible.
[0080] Correspondingly, the first server receives the target user account and the target collection time to perform association processing on the relevant user data according to the target user account and the target collection time.
[0081] In step S12, the aggregated dynamic attribute data of the target user account within the target time period is obtained from the first database; the target time period is determined based on the target collection time, and the aggregated dynamic attribute data is the dynamic attribute data collected by each business party within the target time period.
[0082] First, the first server determines a target time period based on the target collection time, where the target time period is a time period within a target time window based on the target collection time. Specifically, the target time period is a time period from a target time window before the target collection time to a target time window after the target collection time. The target time window can be obtained by the first server in real time or can be preset in advance.
[0083] Then, the first server uses the target user account and the target time period as keywords to obtain dynamic attribute data related to the target user account and the target time period from the first database. The dynamic attribute data is a collection of user data of the target user account in the target time period collected by various business parties, which can be called aggregated dynamic attribute data.
[0084] In step S13, the dynamic attribute data of multiple data categories in the aggregated dynamic attribute data are associated with each other according to the target collection time to obtain target associated data.
[0085] The aggregated dynamic attribute data of the target user account within the target time period may include location data collected by the first business party and base station data collected by the second business party. In this way, both the location data and base station data of the target user account within the target time period are available.
[0086] The first server can associate the dynamic attribute data collected by different business parties through the target user account and the target time period, so that the target user account has two or more types of dynamic attribute data at the same time within the target time period. In this way, the data of different business parties can be connected to each other, the total amount of data can be increased, the correctness can be verified, and the application party can effectively mine the data value.
[0087] In summary, the disclosed embodiment receives the target user account and target collection time sent by the second server; the target user account and the target collection time are extracted by the second server from the first database; the first database stores the user data collected by each business party, the user data includes the user account, the collection time and the dynamic attribute data, the dynamic attribute data is the data whose attribute value changes according to time, and the dynamic attribute data includes multiple data categories; the aggregated dynamic attribute data of the target user account within the target time period is obtained from the first database; the target time period is determined based on the target collection time, and the aggregated dynamic attribute data is the dynamic attribute data collected by each business party within the target time period; the dynamic attribute data of multiple data categories in the aggregated dynamic attribute data are associated according to the target collection time to obtain the target associated data. In this way, all the dynamic attribute data collected by each business party within the target time period are aggregated and associated according to the data category, so that the data of different business parties can be connected with each other, and the target associated data after the associated processing can represent the deeper connection of the data, which is conducive to effectively mining the value of data.
[0088] Figure 2 is a flowchart of the steps of a second data processing method according to an exemplary embodiment, which is applied to a first server, such as Figure 2 As shown, the data processing method includes the following steps.
[0089] In step S21, the target user account and target collection time sent by the second server are received; the target user account and the target collection time are extracted by the second server from the first database; the first database stores user data collected by each business party respectively, the user data includes user account, collection time and dynamic attribute data, the dynamic attribute data is data whose attribute value changes according to time, and the dynamic attribute data includes multiple data categories.
[0090] In the embodiment of the present disclosure, step S21 may refer to step S11 and will not be described in detail here.
[0091] In step S22, the splicing time window corresponding to the dynamic attribute data is obtained from the configuration database.
[0092] Before acquiring the dynamic attribute data from the first database, the first server needs to pull the splicing time window from the configuration database to calculate the target time period based on the target collection time.
[0093] Since users use applications at different times, the amount of user data collected by the business party is also different. The configuration database is used to configure the time of the splicing time window T. Specifically, the configuration database estimates the amount of data in a unit time period according to the target collection time, and adjusts the window of T according to the estimated amount of data, so that the amount of dynamic attribute data obtained by the first server is kept within a suitable range, so as to facilitate the first server to process data.
[0094] Specifically, since the amount of data in similar time periods is similar, the size of similar time periods T is also similar. In order to simplify the operating process and improve processing efficiency, the first server does not need to pull the splicing time window from the first database every time before obtaining dynamic attribute data. The first server can pull the splicing time window from the configuration database at certain intervals, for example, every one minute.
[0095] In step S23, a target time period is determined according to the target acquisition time and the splicing time window.
[0096] Specifically, if the target acquisition time is t1, and the splicing time window obtained from the configuration database is T, then the target time period is (t1-T, t1+T).
[0097] In addition, the target time period may also include the target acquisition time, that is, the lengths of the splicing time windows before and after the target acquisition time may be the same or different.
[0098] Exemplarily, if the front splicing time window obtained from the configuration database is T1, and the back splicing time window is T2 (T1≠T2), then the target time period is (t1-T1, t1+T2).
[0099] In step S24, the aggregated dynamic attribute data of the target user account within the target time period is obtained from the first database; the target time period is determined based on the target collection time, and the aggregated dynamic attribute data is the dynamic attribute data collected by each business party within the target time period.
[0100] In the embodiment of the present disclosure, step S24 may refer to step S12 and will not be described in detail here.
[0101] In step S25, the number of attribute values of the aggregated dynamic attribute data corresponding to each data category is obtained.
[0102] Exemplarily, if the data category is location data, the attribute value represents the location content of the location data. If the location data obtained from the first business party is ** area 1, the location data obtained from the second business party is ** area 1, the location data obtained from the third business party is ** area 1, and the location data obtained from the fourth business party is ** area 3, where ** area 1 and ** area 3 respectively represent the attribute values of the location data, then the number of the location data attribute values is two.
[0103] In step S26 , if there are target data categories with two or more attribute values, the dynamic attribute data of the target data category are calibrated.
[0104] If the number of attribute values is two or more, it means that the target user account has multiple attribute values in the target time period. The multiple attribute values may be caused by normal user activities, or there may be abnormal values or erroneous values.
[0105] For dynamic attribute data of the same data category, the attribute values collected by various business parties should be the same or similar. If there are attribute values that differ greatly, the attribute value may be an abnormal value or an erroneous value. Such values are not suitable for direct use and need to be calibrated in advance.
[0106] For example, if the actual distance between the first zone and the third zone is greater than the distance threshold, the third zone is considered to be an outlier and needs to be calibrated to the first zone.
[0107] In addition, if the attribute has only one value, no calibration is required.
[0108] In one implementation, the calibrating the dynamic attribute data of the target data category includes the following steps S261-S264:
[0109] Step S261: clustering the multiple attribute values to obtain a clustering result;
[0110] Step S262, determining the number of cluster center points in the clustering result;
[0111] Step S263: If the number of the cluster center point is one, the attribute value corresponding to the cluster center point is used as the calibration result of the dynamic attribute data;
[0112] Step S264: If there are multiple cluster center points, then multiple attribute values corresponding to the multiple cluster center points are used as calibration results of the dynamic attribute data.
[0113] In step S261-step S264, the attribute values can be clustered according to the clustering algorithm to extract outliers for calibration. The clustering process is the process of identifying and eliminating outliers. If the attribute values are very similar, they can be clustered to obtain a cluster center point; if most of the attribute values are approximate values, there are a small number of other values, and the other values are not equal, then the other values are marginalized in the clustering process, so that the cluster center point obtained by clustering is one; if the number of multiple attribute values is similar, multiple cluster centers can be clustered.
[0114] Specifically, the number of cluster center points in the clustering result can be obtained. If the number of cluster center points is one, it means that the attribute values are similar, or there are individual abnormal attribute values. The attribute value corresponding to the cluster center point can be used as the calibration result.
[0115] If there are multiple cluster center points, it means that the number of multiple attribute values is similar, and the attribute values corresponding to the cluster center points can all be used as the calibration results of the dynamic attribute data.
[0116] Steps S261 to S264 use a clustering algorithm to calibrate the attribute values, so that abnormal attribute values are eliminated and normal attribute values are clustered, which facilitates the second server to use the correct attribute values for subsequent association processing and improves data accuracy.
[0117] In step S27, the correspondence between the dynamic attribute data of multiple data categories in the aggregated dynamic attribute data is determined according to the target collection time.
[0118] The data categories of dynamic attribute data may include base station data, location data, IP address, wifi hotspot, etc. There is a corresponding relationship between different data categories at the same target collection time. For example, if the target user account has three location data of location 1, location 2, and location 3 at the same target collection time, and there is a base station data 001, then there is a corresponding relationship between base station 001 and location 1, location 2, and location 3; if there are two IP addresses, IP address 1 and IP address 2, then IP address 1 and IP address 2 have a corresponding relationship with base station 001.
[0119] In step S28, the dynamic attribute data of the plurality of data categories are associated according to the corresponding relationship to obtain target associated data.
[0120] Dynamic attribute data of different data categories with corresponding relationships can be associated. For example, base station 001 is associated with location 1, location 2, and location 3, base station 001 is associated with IP address 1 and IP address 2, base station 002 is associated with location 4, location 5, location 6, and location 7, and location 1 is associated with wifi hotspot 1, wifi hotspot 2, and wifi hotspot 3.
[0121] In this way, data from different business parties can be connected to each other, solving the problem of data isolation between unconnected data sources. By associating dynamic attribute data, accurate association of massive data can be achieved at an extremely low cost, thereby further exploring deeper connections between data and improving data utilization.
[0122] In one embodiment, the aggregated dynamic attribute data includes two data categories, namely, location data and base station data, wherein the location data includes the geographical location of the target user account in the target time period, and the base station data includes the base station identifier to which the target user account is connected in the target time period, and the determining of the correspondence between the dynamic attribute data of multiple data categories in the aggregated dynamic attribute data according to the target collection time includes: determining the correspondence between the base station identifier and the geographical location;
[0123] The method of associating the dynamic attribute data of the multiple data categories according to the corresponding relationship to obtain target associated data includes: determining a geographical location list corresponding to the base station identifier according to the corresponding relationship; and associating the base station identifier, the geographical location list, and the collection time to obtain target associated data.
[0124] In the disclosed embodiment, the aggregated dynamic attribute data may include two data categories: location data and base station data. The location data includes the geographic location of the user account in the target time period, and the base station data includes the base station identifier to which the user account is connected in the target time period. For example, for user account 00001, its location data in the target time period is location 1, location 2, location 3, location 4, location 5, and its base station data in the target time period is base station 001 and base station 002.
[0125] According to the collection time information, the corresponding relationship between the base station identifier and the geographical location can be obtained. For example, base station 001 corresponds to location 1, location 2, location 3, and base station 002 corresponds to location 4 and location 5.
[0126] According to the corresponding relationship, the geographical location list corresponding to the base station identifier is determined, and the following geographical location list can be obtained:
[0127] Base Station 001 Base Station 002 Position 1 Position 4 Position 2 Location 5 Location 3
[0128] Then, the base station identifier and the geographical location list are associated to obtain target associated data, that is, the target associated data is: base station 001 is associated with location 1, location 2, and location 3, and base station 002 is associated with location 4 and location 5.
[0129] According to the target association data, it can be known that user account 00001 was successively within the coverage range of base station 1 and base station 2 during the target time period, and when within the coverage range of base station 1, the places visited were location 1, location 2, and location 3; when within the coverage range of base station 2, the places visited were location 4 and location 5.
[0130] In one implementation, if the data category of the aggregated dynamic attribute data is one, no association processing is performed on the aggregated dynamic attribute data.
[0131] Because there is only one data category, different data categories cannot be associated and processed, so the current aggregation dynamic attribute data processing flow ends.
[0132] In step S29, the target associated data is stored in a second database, and the second database is used for a third server to query the associated data.
[0133] The first server stores the target associated data in the second database, which is convenient for the third server to query. The third server can provide query services to the outside, for example, according to the base station information connected by the user, the user's location data can be queried.
[0134] Target-related data can describe the user's location and activity range in detail and accurately, which is conducive to exploring the user's interests and needs and providing users with more accurate services.
[0135] Figure 3 It is a schematic diagram of a data processing system according to an exemplary embodiment.
[0136] The data processing system includes multiple business parties, a first server, a first database, a second server, a third server, a second database and a configuration database, wherein the multiple business parties include the first business party, the second business party and the third business party in the figure.
[0137] like Figure 3 As shown, the workflow of the system is as follows:
[0138] The first business party, the second business party, and the third business party respectively send the collected user ID, collection time, and dynamic attribute data to the second server.
[0139] The second server, on the one hand, stores these data in the first database, and on the other hand, sends the target user ID and target collection time t1 data to the first server.
[0140] The first server periodically pulls the splicing time window T from the configuration database to obtain the target time period (t1-T, t1+T). Then it queries the first database for all dynamic attribute data of the target user ID and the target time period (t1-T, t1+T). The first server performs association processing on these dynamic attribute data to obtain target association data, and stores the target association data in the second database.
[0141] The third server queries the target associated data from the second database.
[0142] In summary, Figure 2 and Figure 3 The data processing method in Figure 1 In addition to the beneficial effects of the data processing method, a splicing time window corresponding to the dynamic attribute data is obtained from the configuration database so that the amount of dynamic attribute data obtained by the second server is kept within a suitable range to facilitate data processing by the second server.
[0143] In addition, the attribute values of the dynamic attribute data are calibrated using a clustering algorithm, so that abnormal attribute values are eliminated and normal attribute values are clustered, which facilitates the second server to use the correct attribute values for subsequent association processing, thereby improving data accuracy.
[0144] Furthermore, target-related data can connect data from different business parties to each other, solving the problem of data isolation between unconnected data sources. By processing the association of dynamic attribute data, accurate association of massive data can be achieved at an extremely low cost, thereby further exploring deeper connections between data and improving data utilization.
[0145] In addition, target-associated data including base station data and location data can describe the user's location and activity range in detail and accurately, which is conducive to exploring the user's interests and needs and providing users with more accurate services.
[0146] Figure 4 A structural block diagram of a data processing device is shown according to an exemplary embodiment.
[0147] The data processing device is applied to a first server, such as Figure 4 As shown, the data processing device 40 includes:
[0148] The receiving module 41 is configured to receive a target user account and a target collection time sent by the second server; the target user account and the target collection time are extracted by the second server from the first database; the first database stores user data collected by various business parties, the user data includes a user account, a collection time and dynamic attribute data, the dynamic attribute data is data whose attribute value changes according to time, and the dynamic attribute data includes multiple data categories;
[0149] The acquisition module 42 is configured to acquire aggregated dynamic attribute data of the target user account within a target time period from the first database; the target time period is determined based on the target collection time, and the aggregated dynamic attribute data is the dynamic attribute data collected by each business party within the target time period;
[0150] The association processing module 43 is configured to perform association processing on the dynamic attribute data of multiple data categories in the aggregated dynamic attribute data according to the target collection time to obtain target association data.
[0151] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0152] Figure 5 is a block diagram of a server for data processing according to an exemplary embodiment, and its internal structure diagram can be as follows Figure 5 As shown. The server includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the server is used to provide computing and control capabilities. The memory of the server includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the server is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method of data processing is implemented.
[0153] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a partial structure related to the scheme of the present disclosure, and does not constitute a limitation on the server to which the scheme of the present disclosure is applied. The specific server may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0154] In an exemplary embodiment, a computer-readable storage medium is also provided, and when the instructions in the computer-readable storage medium are executed by a processor of a server, the server is enabled to perform the data processing method in the embodiment of the present disclosure. The computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0155] In an exemplary embodiment, a computer program product containing instructions is also provided. When the computer program product is run on a computer, the computer is caused to execute the data processing method in the embodiment of the present disclosure.
[0156] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0157] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0158] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A data processing method, characterized in that: Applied to the first server, comprising: receiving a target user account and a target collection time sent by a second server; the target user account and the target collection time are extracted by the second server from a first database; the first database stores user data collected by various business parties, the user data including user account, collection time and dynamic attribute data, the dynamic attribute data being data whose attribute value changes according to time, and the dynamic attribute data including multiple data categories; Acquire aggregated dynamic attribute data of the target user account within a target time period from the first database; the target time period is determined based on the target collection time, and the aggregated dynamic attribute data is the dynamic attribute data collected by each business party within the target time period; The dynamic attribute data of multiple data categories in the aggregated dynamic attribute data are associated with each other according to the target collection time to obtain target associated data.
2. The method according to claim 1, characterized in that The associating the dynamic attribute data of multiple data categories in the aggregated dynamic attribute data according to the target collection time includes: Determining a correspondence between dynamic attribute data of multiple data categories in the aggregated dynamic attribute data according to the target collection time; The dynamic attribute data of the multiple data categories are associated according to the corresponding relationship to obtain target associated data.
3. The method according to claim 2, characterized in that The aggregated dynamic attribute data includes two data categories: location data and base station data. The location data includes the geographical location of the user account in the target time period. The base station data includes the base station identifier to which the user account is connected in the target time period. The corresponding relationship between the dynamic attribute data of multiple data categories in the aggregated dynamic attribute data is determined according to the target collection time, including: Determining a correspondence between the base station identifier and the geographical location; The step of associating the dynamic attribute data of the plurality of data categories according to the corresponding relationship to obtain target associated data includes: Determine a geographical location list corresponding to the base station identifier according to the corresponding relationship; The base station identifier and the geographical location list are associated to obtain target association data.
4. The method according to claim 1, characterized in that Before using the aggregated dynamic attribute data to perform association processing on the dynamic attribute data of the multiple data categories, the method further includes: Get the number of attribute values of the aggregated dynamic attribute data corresponding to each data category; If there are target data categories with two or more attribute values, the dynamic attribute data of the target data categories are calibrated.
5. The method according to claim 4, characterized in that The calibrating the dynamic attribute data of the target data category includes: Clustering the plurality of attribute values to obtain a clustering result; Determine the number of cluster center points in the clustering result; If the number of the cluster center point is one, the attribute value corresponding to the cluster center point is used as the calibration result of the dynamic attribute data; If there are multiple cluster center points, multiple attribute values corresponding to the multiple cluster center points are used as calibration results of the dynamic attribute data.
6. The method according to claim 1, characterized in that Before acquiring the aggregated dynamic attribute data of the target user account within the target time period from the first database, the method further includes: Acquire a splicing time window corresponding to the dynamic attribute data from a configuration database; A target time period is determined according to the target acquisition time and the splicing time window.
7. The method according to claim 1, characterized in that After obtaining the target associated data, it also includes: The target associated data is stored in a second database, and the second database is used for a third server to query the associated data.
8. A data processing device, characterized in that: Applied to the first server, comprising: The receiving module is configured to receive a target user account and a target collection time sent by the second server; the target user account and the target collection time are extracted by the second server from the first database; the first database stores user data collected by various business parties, the user data includes the user account, the collection time and dynamic attribute data, the dynamic attribute data is data whose attribute value changes according to time, and the dynamic attribute data includes multiple data categories; An acquisition module is configured to acquire aggregated dynamic attribute data of the target user account within a target time period from the first database; the target time period is determined based on the target collection time, and the aggregated dynamic attribute data is all dynamic attribute data collected by each business party within the target time period; The association processing module is configured to perform association processing on the dynamic attribute data of multiple data categories in the aggregated dynamic attribute data according to the target collection time to obtain target association data.
9. The device according to claim 8, characterized in that The association processing module is specifically configured to execute: Determining a correspondence between dynamic attribute data of multiple data categories in the aggregated dynamic attribute data according to the target collection time; The dynamic attribute data of the multiple data categories are associated according to the corresponding relationship to obtain target associated data.
10. The device according to claim 9, characterized in that The aggregated dynamic attribute data includes two data categories: location data and base station data. The location data includes the geographical location of the user account in the target time period. The base station data includes the base station identifier to which the user account is connected in the target time period. The association processing module is further configured to execute: Determining a correspondence between the base station identifier and the geographical location; The association processing module is further configured to execute: Determine a geographical location list corresponding to the base station identifier according to the corresponding relationship; The base station identifier and the geographical location list are associated to obtain target association data.
11. The device according to claim 8, characterized in that The device also includes: A number acquisition module is configured to execute acquisition of the number of attribute values of the aggregated dynamic attribute data corresponding to each data category; The calibration module is configured to calibrate the dynamic attribute data of the target data category if there are two or more target data categories with attribute values.
12. The device according to claim 11, characterized in that The calibration module is specifically configured to perform: Clustering the multiple attribute values to obtain a clustering result; Determine the number of cluster center points in the clustering result; If the number of the cluster center point is one, the attribute value corresponding to the cluster center point is used as the calibration result of the dynamic attribute data; If there are multiple cluster center points, multiple attribute values corresponding to the multiple cluster center points are used as calibration results of the dynamic attribute data.
13. The device according to claim 8, characterized in that The device also includes: A time window acquisition module is configured to acquire a splicing time window corresponding to the dynamic attribute data from a configuration database; The target time period determination module is configured to determine the target time period according to the target acquisition time and the splicing time window.
14. The device according to claim 8, characterized in that The device also includes: The first storage module is configured to store the target associated data in a second database, where the second database is used for a third server to query the associated data.
15. A server, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the data processing method according to any one of claims 1 to 7. 16 . A storage medium, when instructions in the storage medium are executed by a processor of a server, enables the server to execute the data processing method according to any one of claims 1 to 7.
