A data processing system based on multi-source heterogeneous data

Through a multi-source heterogeneous data processing system, the grid position is confirmed by comprehensively utilizing multiple data sources, which solves the problem of inaccurate data in the existing technology and achieves more accurate grid information acquisition.

CN116723231BActive Publication Date: 2025-08-26HANGZHOU YUNSHEN TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310949990.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-07-31
Publication Date
2025-08-26
Estimated Expiration
2043-07-31

AI Technical Summary

Technical Problem

The existing methods of obtaining the number of active populations and grid information of the target geographical area are based on the registration method, resulting in inaccurate data and inaccurate updates.

Method used

A data processing system based on multi-source heterogeneous data is adopted, and a computer program is executed through a processor, and the grid position is determined by comprehensively using user ID information, activity track information, designated grid position information, network environment information, associated grid position information, historical stay grid position information and confirmed grid position information.

Benefits of technology

Improve the accuracy of grid position information and ensure that the obtained data is more accurate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116723231B_ABST
    Figure CN116723231B_ABST
Patent Text Reader

Abstract

The present invention provides a data processing system based on multi-source heterogeneous data, comprising a processor and a database in communicative communication. The database contains user ID information (MT) and multi-source heterogeneous data regarding a target geographic area. The multi-source heterogeneous data includes activity trajectory information, designated grid location information, mobile terminal network environment location information, associated grid location information, historical stay grid location information, and confirmed grid location information. The processor is configured to obtain current grid location information based on historical multi-source heterogeneous data. The present invention can improve the accuracy of obtained grid location information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data processing system based on multi-source heterogeneous data. Background Art

[0002] In some application scenarios, it is necessary to know the actual number of active people in a target geographic area during a set time period, as well as accurate grid information for individuals (including grids where they spend a lot of time during the day and at night). Existing solutions for obtaining this information are generally based on registration, and the data obtained by this method is static. For example, if the original data is not updated due to leaving the target geographic area, the information obtained may be inaccurate. Summary of the Invention

[0003] In view of the above technical problems, the technical solution adopted by the present invention is:

[0004] An embodiment of the present invention provides a data processing system based on multi-source heterogeneous data, the system comprising a processor and a database in communication connection, wherein the database contains user ID information MT and multi-source heterogeneous data about a target geographical area, the multi-source heterogeneous data including activity trajectory information, designated grid location information, network environment information, associated grid location information, historical stay grid location information, and confirmed grid location information; wherein MT = {MT1, MT2, ..., MT i ,…,MT m}, where MT i is the ID of the i-th user in the MT, where i ranges from 1 to m, and m is the number of users included in the current database; the designated grid location information is the grid location information associated with the transportation service provider, and the associated grid location information is the grid location information associated with the corresponding user; the grid belongs to the target geographical area;

[0005] The processor is configured to execute a computer program to implement the following steps:

[0006] S100, set i=1;

[0007] S200, if i≤m, execute S300; otherwise, exit the current control program;

[0008] S300, for MT i , obtain the activity trajectory information, designated grid location information, and network environment information within the first set time period closest to the current time from the current multi-source heterogeneous data as MT itarget activity trajectory information, target designated grid location information, target network environment information; and obtaining historical stay grid location information within a second set time period closest to the current time from the current multi-source heterogeneous data as the target historical stay grid location information; the time length corresponding to the second set time period is greater than the time length corresponding to the first set time period;

[0009] S400, based on MT i The target activity trajectory information, target designated grid location information, target network environment information, and associated grid location information are obtained by MT i Corresponding first candidate grid position information, second candidate grid position information, third candidate grid position information and fourth candidate grid position information; each candidate grid position information includes a first stop grid position and a second stop grid position;

[0010] S500, if MT i If the corresponding first candidate grid position information, second candidate grid position information, third candidate grid position information, and fourth candidate grid position information are all Null, execute S600; otherwise, execute S700;

[0011] S600, based on MT i The corresponding target historical stay grid location information and the confirmed grid location information obtain MT i Corresponding target grid position information; the target grid position information includes a first target stop grid position and a second target stop grid position;

[0012] S700, based on MT i The corresponding first to fourth candidate grid location information and the confirmed grid location information obtain MT i The corresponding target grid location information.

[0013] The present invention has at least the following beneficial effects:

[0014] The data processing system based on multi-source heterogeneous data provided by the embodiment of the present invention can make the obtained grid location more accurate because it confirms the grid location based on multiple data such as activity trajectory information, specified grid location information, network environment information, associated grid location information, historical stop grid location information and confirmed grid location information. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 This is a structural block diagram of a data processing system based on multi-source heterogeneous data provided by an embodiment of the present invention.

[0017] Figure 2 A flowchart of a method implemented by executing a computer program on a data processing system based on multi-source heterogeneous data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0019] The embodiment of the present invention provides a data processing system based on multi-source heterogeneous data, such as Figure 1 As shown, the system may include a processor and a database in communication connection, wherein the database contains user ID information MT and multi-source heterogeneous data about the target geographical area, wherein the multi-source heterogeneous data includes activity trajectory information, designated grid location information, network environment information, associated grid location information, historical stay grid location information, and confirmed grid location information; wherein MT = {MT1, MT2, ..., MT i ,…,MT m}, where MT i is the i-th user ID in MT, i ranges from 1 to m, and m is the number of users included in the current database.

[0020] In an embodiment of the present invention, a user ID may be a unique identifier that represents the identity of a user. For example, it may be a user's ID number or the ID of a device to which the user belongs, such as a mobile terminal. The mobile terminal ID may be a unique code used to distinguish different mobile terminals. For example, the terminal ID may be any one or a combination of the IMEI, MAC address, and SIM card number of the mobile terminal, or any one or a combination of the IMEI, MAC address, and SIM card number of the mobile terminal that has been encrypted (e.g., MD5 encrypted), and the embodiment of the present invention does not specifically limit this.

[0021] In embodiments of the present invention, activity trajectory information may be user activity trajectory information, including but not limited to user location information obtained through a user's device, such as a mobile terminal, user location information identified by a sensing device, and user location information obtained through the user's internet behavior data. In embodiments of the present invention, data obtained through a mobile terminal may be reported by the mobile terminal.

[0022] In the embodiment of the present invention, the network environment information is the address information of a relatively fixed network environment where the user is located and which can support the user's device, such as a mobile terminal, to perform communication services.

[0023] In an embodiment of the present invention, the designated grid location information is grid location information for establishing contact with a transportation service provider to receive items transported by the transportation service provider.

[0024] The associated grid location information is grid location information with which corresponding users have an associated relationship, such as a legal ownership relationship.

[0025] In the embodiment of the present invention, the target geographical area may be determined according to actual needs, and may be an area divided according to administrative regions, for example, Hangzhou.

[0026] In the embodiment of the present invention, the grid belongs to the target geographical area. The grid can be a target space, such as a house.

[0027] In an embodiment of the present invention, multi-source heterogeneous data can be obtained through a third-party data providing platform that has data acquisition qualifications.

[0028] In an embodiment of the present invention, historical stay grid location information may include a first stay grid location and a second stay grid location historically obtained during each stay grid location information calculation period. In a specific embodiment, the first stay grid location may be a home address, and the second stay grid location may be an office address. Those skilled in the art will appreciate that if there is no office address, the second stay grid location will be Null.

[0029] In an embodiment of the present invention, the confirmed grid location information may be manually confirmed grid location information, including the confirmed first and second stop grid locations and the confirmation time. Those skilled in the art will appreciate that if there is no office address, the second stop grid location is Null.

[0030] In a specific application scenario, the number of currently active people in the target geographical area may be determined based on the number of mobile terminals included in the current database.

[0031] Furthermore, in an embodiment of the present invention, the processor is configured to execute a computer program to implement the following Figure 2 Steps shown:

[0032] S100, set i=1.

[0033] S200, if i≤m, execute S300; otherwise, exit the current control program.

[0034] S300, for MT i , obtain the activity trajectory information, designated grid location information, and network environment information within the first set time period closest to the current time from the current multi-source heterogeneous data as MT i target activity trajectory information, target designated grid location information, and target network environment information; and obtaining historical stay grid location information within a second set time period closest to the current time from the current multi-source heterogeneous data as the target historical stay grid location information; the time length corresponding to the second set time period is greater than the time length corresponding to the first set time period.

[0035] In an embodiment of the present invention, the processor calculates the grid location information within the target geographic area according to a set time period. In an exemplary embodiment, the set time period can be a time window. In an embodiment of the present invention, the current time is the time at which the grid location information of the target geographic area is currently calculated.

[0036] In an embodiment of the present invention, the first set time period may include n1 consecutively arranged time windows, and the second set time period may include n2 consecutively arranged time windows. The length of each time window can be set based on actual needs, for example, one day. n1 and n2 can be set based on actual needs. In a preferred embodiment, n1 = 30 and n2 = 90.

[0037] S400, based on MT i The target activity trajectory information, target designated grid location information, target network environment information, and associated grid location information are obtained by MT i The corresponding first candidate grid position information, second candidate grid position information, third candidate grid position information and fourth candidate grid position information. Each candidate grid position information includes a first stop grid position and a second stop grid position.

[0038] Furthermore, in an embodiment of the present invention, the activity track information may include n activity track information lists, wherein the kth row of the jth activity track information list includes (MA jk , MAD jk ), MA jk is the kth user ID in the jth activity trajectory information list, MAD jk is the MA obtained in the jth time windowjk The location information of MA in the jth time window jk The corresponding location data and the corresponding acquisition time, k ranges from 1 to f(j), f(j) is the number of user IDs obtained in the jth time window, j ranges from 1 to n, and n is the number of time windows; among them, the acquired location data is latitude and longitude data, that is, longitude and latitude coordinates.

[0039] In S400, the MT is obtained based on the target activity trajectory information. i The corresponding first candidate grid location information includes:

[0040] S401, for the location information within any time window in the target activity trajectory information, if any acquisition time in the location information within the time window belongs to the first preset time range, the location data corresponding to the acquisition time is stored in the first candidate stay grid location information table; if any acquisition time in the location information within the time window belongs to the second preset time range, the location data corresponding to the acquisition time is stored in the second candidate stay grid location information table. The sum of the time lengths of the first preset time range and the second preset time range is equal to the time length of one time window. For example, the first preset time range is from 6 pm to 7 am, and the second preset time range is from 7 am to 6 pm.

[0041] S402 : Convert each location data in the first candidate stay grid location information table and the second candidate stay grid location information table into a GeoHash grid.

[0042] Those skilled in the art know that converting each location data in the first candidate stay grid location information table and the second candidate stay grid location information table into a GeoHash grid is an existing technology.

[0043] S403, obtaining the location data corresponding to the h-th candidate stop grid location information table and the preset N-digit GeoHash grid sequence G1=(G11, G12, ..., G1) corresponding to the mapping of the target geographical area. r1 ,…,G1 x1 )N1=(N11,N12,…,N1 r1 ,…,N1 x1 ), N1 r1 The target activity trajectory information and grid G1 r The number of user IDs that have intersections, r1 ranges from 1 to x1, where x1 is the number of GeoHash grids in G1; h ranges from 1 to 2.

[0044] In a preferred embodiment of the present invention, N=5.

[0045] S404, set N=N+1, if N≤Q, execute S405, otherwise execute S408; Q is a preset bit threshold, preferably, Q=8.

[0046] S405 , obtaining the grid corresponding to the grid sequence corresponding to max(Ns) as the Ns-th target grid.

[0047] S406, obtain the activity trajectory information corresponding to the Ns-th target grid and the preset N-digit GeoHash grid sequence Gs corresponding to the mapping of the Ns-th target grid = (Gs1, Gs2, ..., Gs r2 ,…,Gs x2 )Ns=(Ns1,Ns2,…,Ns rs ,…,Ns x2 ), Ns rs The activity trajectory information corresponding to the Ns-th target grid and the grid Gs r2 The number of user IDs that have intersections. rs ranges from 1 to xs, where xs is the number of GeoHash grids in Gs.

[0048] S407, set s=s+1, and execute S403.

[0049] S408, the mobile terminal location data corresponding to the current target grid is used as MT i The corresponding h-th candidate target stay grid position data.

[0050] S409: Based on the first candidate target stop grid location data and the second candidate target stop grid location data, obtain MT i The corresponding first candidate grid position information.

[0051] Furthermore, in an embodiment of the present invention, the designated grid location information may include n designated grid location information lists, and the g-th row of the e-th designated grid location information list includes (MA eg , MAS eg ), MA jk For the gth user ID in the eth specified grid location information list, MAS eg MA eg The corresponding designated grid location information in the e-th time window, e is 1 to n, n is the number of time windows, g is 1 to y(e), y(e) is the number of user IDs in the e-th designated grid location information list; MAS eg Including the designated grid location data and the end time when the transportation service provider completes the transportation service at the corresponding designated grid location data, such as the signing time;

[0052] In S400, the MT is obtained based on the target specified grid position information. i The corresponding second candidate grid location information specifically includes:

[0053] S410, based on the set grid position identification tool, the type of the designated grid position data in the target designated grid position information is identified. If any designated grid position data is identified as a first type of grid position, the designated grid position data is stored in a third candidate stay grid position information table. If any designated grid position data is identified as a second type of grid position, the designated grid position data is stored in a fourth candidate stay grid position information table.

[0054] In an embodiment of the present invention, the grid location identification tool may be an existing address identification tool. The first type of grid location may be a type characterized as a residential address, and the second type of grid location may be a type characterized as an office address.

[0055] S412 , respectively obtain the word frequency, that is, the number of occurrences, of each grid location data in the third candidate stay grid location information table and the fourth candidate stay grid location information table.

[0056] S414 , using the grid position data corresponding to the maximum word frequency in the third candidate stay grid position information table and the fourth candidate stay grid position information table as the third candidate target stay grid position data and the fourth candidate target stay grid position data, respectively.

[0057] S416: Obtain MT based on the third candidate target stop grid location data and the fourth candidate target stop grid location data. i The corresponding second candidate grid position information.

[0058] Furthermore, in an embodiment of the present invention, the network environment information may include n network environment information lists, and the dth row of the bth network environment information list includes (MA bd , MWF bd ), MA bd is the dth user ID in the bth network environment information list, MWF bd MA bd The network environment location information of the corresponding mobile terminal in the dth time window, b is 1 to n, n is the number of time windows, d is 1 to h(b), h(b) is the number of user IDs in the bth network environment information list; MWF bd Including MA bd The network environment location data and the corresponding residence time in the d-th time window, that is, the residence time in the network environment, for example, the Wi-Fi connection time of the mobile terminal.

[0059] Among them, S400, based on MT i Target network environment information acquisition MT i The corresponding third candidate grid location information specifically includes:

[0060] S420, for any network environment location data in the target network environment information, if the data belongs to the first type of network environment connection location, the data is stored in the first candidate network environment location information table; if the data belongs to the second type of network environment connection location, the data is stored in the second candidate network environment location information table.

[0061] In an embodiment of the present invention, a first-type network environment connection location may be a location characterized as a residential location, and a second-type network environment connection location may be a location characterized as an office location. For any network environment location data, if the connection time corresponding to the data falls within a first preset range, the location is determined to be a first-type network environment connection location; if the connection time falls within a second preset range, the location is determined to be a second-type network environment connection location.

[0062] S422: If both the first candidate network environment location information table and the second candidate network environment location information table are not Null, the network environment location data corresponding to the maximum stay time in the first candidate network environment location information table and the second candidate network environment location information table are respectively used as the first candidate network environment location data and the second candidate network environment location data.

[0063] S424, obtaining the associated user ID associated with the first candidate network environment location data from the database, and obtaining the corresponding associated user ID set MA1 = {MA11, MA12, ..., MA1 a1 ,…,MA1 g1}, and obtain the associated grid data set DR1 = {DR11, DR12, ..., DR1 a1 ,…,DR1 g1}, MA1 a1 DR1 is the a1th user ID associated with the first candidate network environment location data, where a1 ranges from 1 to g1, and g1 is the number of user IDs associated with the first candidate network environment location data; a1 MA1 a1 The corresponding grid location data corresponding to the first candidate network environment location data; and, if a1=1, and DR1 a1 If it is not Null, DR1 a1As the candidate stay grid location data corresponding to the first candidate network environment location data; if a1>1, and the number of data that is not Null in DR1 is greater than 1, then the data with the maximum stay time in DR1 is used as the candidate stay grid location data corresponding to the first network environment location data; if DR1 is Null, then the network address corresponding to the first candidate network environment location data, such as the wifimac address, is used as the candidate stay grid location data corresponding to the first candidate network environment location data.

[0064] S426, obtaining the associated user ID associated with the second candidate network environment location data from the database, and obtaining the corresponding associated user ID set MA2 = {MA21, MA22, ..., MA2 a2 ,…,MA2 g2}, and obtain the associated grid data set DR2 from the database = {DR21, DR22, ..., DR2 a2 ,…,DR2 g2}, MA2 a2 DR2 is the a2th user ID associated with the second candidate network environment location data, where a2 ranges from 1 to g2, and g2 is the number of user IDs associated with the second candidate network environment location data; a2 MA2 a2 The corresponding grid location data corresponding to the second candidate network environment location data; and, if a2=1, and DR2 a2 If it is not Null, DR2 a2 as the candidate stay grid location data corresponding to the second candidate network environment location data; if a2>1, and the number of data that is not Null in DR2 is greater than 1, then the data with the maximum stay time in DR2 is used as the candidate stay grid location data corresponding to the second network environment location data; if DR2 is Null, then the network address corresponding to the first candidate network environment location data, such as the wifimac address, is used as the candidate stay grid location data corresponding to the second candidate network environment location data.

[0065] S428, obtain MT based on the obtained candidate stay grid location data i The corresponding third candidate grid position information.

[0066] In the embodiment of the present invention, if at least one of the first candidate network environment location information table and the second candidate network environment location information table is Null, the grid position corresponding to the Null candidate network environment location information table is Null. For example, if the second candidate network environment location information table is Null, then MT i The second stop grid position in the corresponding third candidate grid position information is Null.

[0067] Furthermore, in the embodiment of the present invention, at S400, based on MT i The associated grid location information of MT is obtained i The corresponding fourth candidate grid location information specifically includes:

[0068] S430, if MT i If the associated grid position information of is not Null, execute S431; otherwise, set the fourth candidate grid position information to Null and exit the current control program;

[0069] For any network environment location data in the target network environment information, if the data belongs to the first type of network environment connection location, the data is stored in the first candidate network environment location information table; if the data belongs to the second type of network environment connection location, the data is stored in the second candidate network environment location information table;

[0070] S431, for MT i If the grid location data w belongs to the first type of grid location, the grid location data q is stored in the first candidate associated grid location information table; if the grid location data w belongs to the second type of grid location, the grid location data q is stored in the second candidate associated grid location information table; the value of q ranges from 1 to z, where z is MT i the number of grid location data included in the associated grid location information;

[0071] S432: If the value of the oth candidate associated grid location information table is not Null, for the grid location data A in the oth candidate associated grid location information table, o w, respectively obtain A o The distance between w and any position data in the target activity trajectory information is obtained as the first distance set D1 o w ; The value of o is 1 or 2;

[0072] S433, get D1 o w The distances that are less than or equal to the preset distance are obtained to obtain the second distance set D2 o w ;

[0073] S434, if D2 o w If it is Null, then A o The candidate grid position information WI corresponding to w o w Set to Null; otherwise, A o The candidate grid position information WI corresponding to w ow Set to include A o w and the corresponding number of time windows TP o wTP o wBased on D2 o w The acquisition time corresponding to each distance in is determined; the candidate associated grid position information set WI corresponding to the oth candidate associated grid position information table is obtained. o ={WI o 1. WI o 2,…,WI o w ,…,WI o w};

[0074] S435, if WI o If there is at least one candidate grid position data that is not Null, then WI o The candidate associated grid position data corresponding to the maximum number of time windows in the MA is used as the corresponding target associated grid position data MA o w;

[0075] S436, based on MA o wGetMT i The corresponding fourth candidate grid position information is the first stop grid position and the second stop grid position in the fourth candidate grid position information.

[0076] S500, if MT i If the corresponding first candidate grid position information, second candidate grid position information, third candidate grid position information, and fourth candidate grid position information are all Null, execute S600; otherwise, execute S700.

[0077] S600, based on MT i The corresponding target historical stay grid location information and the confirmed grid location information obtain MT i Corresponding target grid position information; the target grid position information includes the first target stop grid position and the second target stop grid position.

[0078] Furthermore, S600 specifically includes:

[0079] S610, if MT i The corresponding confirmed grid location information and the target historical stop grid location information are not Null, and MT i The confirmation time in the corresponding confirmed grid location information is earlier than the time corresponding to the target historical stay grid location information, then the MT i The corresponding confirmed grid location information is used as MT iThe corresponding target grid location information, otherwise, the MT i The corresponding target historical stay grid location information is used as MT i The corresponding target grid location information.

[0080] S620, if MT i The corresponding confirmed grid location information and the target grid historical stop grid location information are both Null, indicating that MT i The corresponding user has left the target geographical area, then the MT i Remove from the current MT.

[0081] S700, based on MT i The corresponding first to fourth candidate grid location information and the confirmed grid location information obtain MT i The corresponding target grid location information.

[0082] In an exemplary embodiment, S700 may specifically include:

[0083] S701, if MT i The corresponding confirmed grid location information is not Null, and the corresponding confirmation time is within the set time period, indicating that MT i The corresponding confirmed grid location information is more accurate, then the MT i The corresponding confirmed grid location information is used as MT i Corresponding target grid position information; otherwise, execute S702.

[0084] S702: If the first candidate grid position information is not Null, use the first candidate grid position information as the first target candidate grid position information; otherwise, use the second candidate grid position information as the first target candidate grid position information.

[0085] S703: If the fourth candidate grid position information is not Null, use the fourth candidate grid position information as the second target candidate grid position information; otherwise, use the third candidate grid position information as the second target candidate grid position information.

[0086] S704: If the first target candidate grid location information is the first candidate grid location information, the second target candidate grid location information is the fourth candidate grid location information, and if the number of time windows corresponding to the fourth candidate grid location information is greater than the set number of time windows, then the fourth candidate grid location information is used as the MT i The corresponding target grid location information, otherwise, the first candidate grid location information is used as MT iIn an embodiment of the invention, the number of time windows may be set to an empirical value, for example, two-thirds of the length of the first set time period.

[0087] S705: If the first target candidate grid position information is the first candidate grid position information or the second candidate grid position information, and the second target candidate grid position information is the third candidate grid position information, the first target candidate grid position information is used as the MT i The corresponding target grid location information.

[0088] Furthermore, in another embodiment, S700 specifically includes:

[0089] S710, respectively from MT i The first category grid location data and the second category grid location data are obtained from the corresponding first to fourth candidate grid location information and the confirmed grid location information, and a first category grid location information table and a second category grid location information table are obtained respectively.

[0090] S720 , clustering the grid location data in the first type grid location information table and the second type grid location information table, respectively, to obtain a first target class and a second target class, wherein the distance between any two grid location data in each target class is less than a set distance.

[0091] Those skilled in the art will appreciate that existing distance clustering methods may be used to cluster the grid location data in the first type grid location information table and the second type grid location information table. The set distance may be set based on actual needs, for example, 1 km.

[0092] S730: If the u-th target class includes the fourth candidate grid position information, that is, includes the associated grid position, the included fourth candidate grid position information is used as the MT i The corresponding u-th target stay grid position; otherwise, the center position of the u-th target class is used as the MT i The corresponding grid position where the uth target stays; the value of u is 1 or 2.

[0093] Those skilled in the art know that the center position of the u-th target class can be obtained based on existing technologies.

[0094] S740, obtaining MT based on the u-th target stop grid position i The corresponding target grid location information.

[0095] The data processing system based on multi-source heterogeneous data provided by the embodiment of the present invention can make the obtained grid location more accurate because it confirms the grid location based on multiple data such as activity trajectory information, specified grid location information, network environment information, associated grid location information, historical stop grid location information and confirmed grid location information.

[0096] Although some specific embodiments of the present invention have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It should also be understood by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A data processing system based on multi-source heterogeneous data, characterized in that: The system includes a processor and a database in communication connection, wherein the database contains user ID information MT and multi-source heterogeneous data about a target geographical area, wherein the multi-source heterogeneous data includes activity trajectory information, designated grid location information, network environment information, associated grid location information, historical stay grid location information, and confirmed grid location information; wherein MT = {MT1, MT2, ..., MT i ,…,MT m }, where MT i is the ID of the i-th user in the MT, where i ranges from 1 to m, and m is the number of users included in the current database; the designated grid location information is the grid location information associated with the transportation service provider, and the associated grid location information is the grid location information associated with the corresponding user; the grid belongs to the target geographical area; The processor is configured to execute a computer program to implement the following steps: S100, set i=1; S200, if i≤m, execute S300; otherwise, exit the current control program; S300, for MT i , obtain the activity trajectory information, designated grid location information, and network environment information within the first set time period closest to the current time from the current multi-source heterogeneous data as MT i target activity trajectory information, target designated grid location information, target network environment information; and obtaining historical stay grid location information within a second set time period closest to the current time from the current multi-source heterogeneous data as the target historical stay grid location information; the time length corresponding to the second set time period is greater than the time length corresponding to the first set time period; S400, based on MT i The target activity trajectory information, target designated grid location information, target network environment information, and associated grid location information are obtained by MT i Corresponding first candidate grid position information, second candidate grid position information, third candidate grid position information and fourth candidate grid position information; each candidate grid position information includes a first stop grid position and a second stop grid position; S500, if MT i If the corresponding first candidate grid position information, second candidate grid position information, third candidate grid position information, and fourth candidate grid position information are all Null, execute S600; otherwise, execute S700; S600, based on MT i The corresponding target historical stay grid location information and the confirmed grid location information obtain MT i Corresponding target grid position information; the target grid position information includes a first target stop grid position and a second target stop grid position; S700, based on MT i The corresponding first to fourth candidate grid location information and the confirmed grid location information obtain MT i The corresponding target grid location information.

2. The system according to claim 1, wherein: The activity track information includes n activity track information lists, wherein the kth row of the jth activity track information list includes (MA jk , MAD jk ), MA jk is the kth user ID in the jth activity trajectory information list, MAD jk is the MA obtained in the jth time window jk The location information includes the acquired location data and the corresponding acquisition time, where k ranges from 1 to f(j), f(j) is the number of user IDs acquired in the jth time window, j ranges from 1 to n, and n is the number of time windows; the acquired location data is the latitude and longitude data; In S400, the MT is obtained based on the target activity trajectory information. i The corresponding first candidate grid location information includes: S401: For location information within any time window in the target activity trajectory information, if any acquisition time in the location information within the time window falls within a first preset time range, the location data corresponding to the acquisition time is stored in a first candidate stay grid location information table; if any acquisition time in the location information within the time window falls within a second preset time range, the location data corresponding to the acquisition time is stored in a second candidate stay grid location information table. The sum of the time lengths of the first preset time range and the second preset time range is equal to the time length of one time window. S402, converting each location data in the first candidate stay grid location information table and the second candidate stay grid location information table into a GeoHash grid; S403, obtaining the location data corresponding to the h-th candidate stop grid location information table and the preset N-digit GeoHash grid sequence G1=(G11, G12, ..., G1) corresponding to the mapping of the target geographical area. r1 ,…,G1 x1 ) The number of users’ IDs that have intersection is N1 = (N11, N12, ..., N1 r1 ,…,N1 x1 ), N1 r1 The target activity trajectory information and grid G1 r The number of user IDs that have intersections, r1 ranges from 1 to x1, where x1 is the number of GeoHash grids in G1; h ranges from 1 to 2. S404, set N=N+1, if N≤Q, execute S405, otherwise execute S408; Q is the preset bit threshold; S405, obtaining the grid corresponding to the grid sequence corresponding to max(Ns) as the Ns-th target grid; S406, obtain the activity trajectory information corresponding to the Ns-th target grid and the preset N-digit GeoHash grid sequence Gs corresponding to the mapping of the Ns-th target grid = (Gs1, Gs2, ..., Gs r2 ,…,Gs x2 )Ns=(Ns1,Ns2,…,Ns rs ,…,Ns x2 ), Ns rs The activity trajectory information corresponding to the Ns-th target grid and the grid Gs r2 The number of user IDs that have intersections. The value of rs ranges from 1 to xs, where xs is the number of GeoHash grids in Gs. S407, set s=s+1, and execute S403; S408, the position data corresponding to the current target grid is used as MT i The corresponding h-th candidate target stop grid location data; S409: Based on the first candidate target stop grid location data and the second candidate target stop grid location data, obtain MT i The corresponding first candidate grid position information.

3. The system according to claim 1, wherein: The specified grid location information includes n specified grid location information lists, and the g-th row of the e-th specified grid location information list includes (MA eg , MAS eg ), MA jk For the gth user ID in the eth specified grid location information list, MAS eg MA eg The corresponding designated grid location information in the e-th time window, e is 1 to n, n is the number of time windows, g is 1 to y(e), y(e) is the number of user IDs in the e-th designated grid location information list; MAS eg including the designated grid location data and the end time at which the transportation service provider completes the transportation service at the corresponding designated grid location data; In S400, the MT is obtained based on the target specified grid position information. i The corresponding second candidate grid location information specifically includes: S410: Identifying the type of designated grid location data in the target designated grid location information based on a set grid location identification tool; if any designated grid location data is identified as a first type of grid location, storing the designated grid location data in a third candidate stay grid location information table; and if any designated grid location data is identified as a second type of grid location, storing the designated grid location data in a fourth candidate stay grid location information table. S412, respectively obtaining the word frequency of each grid location data in the third candidate stay grid location information table and the fourth candidate stay grid location information table; S414, using the grid position data corresponding to the maximum word frequency in the third candidate stop grid position information table and the fourth candidate stop grid position information table as the third candidate target stop grid position data and the fourth candidate target stop grid position data respectively; S416: Obtain MT based on the third candidate target stop grid location data and the fourth candidate target stop grid location data. i The corresponding second candidate grid position information.

4. The system according to claim 1, wherein: The network environment information includes n network environment information lists, and the dth row of the bth network environment information list includes (MA bd , MWF bd ), MA bd is the dth user ID in the bth network environment information list, MWF bd MA bd In the network environment location information corresponding to the dth time window, the value of b ranges from 1 to n, n is the number of time windows, and the value of d ranges from 1 to h(b), h(b) is the number of user IDs in the bth network environment information list; MWF bd Including MA bd Network environment location data and corresponding stay duration in the dth time window; Among them, S400, based on MT i Target network environment information acquisition MT i The corresponding third candidate grid location information specifically includes: S420: For any network environment location data in the target network environment information, if the data belongs to the first type of network environment connection location, store the data in the first candidate network environment location information table; if the data belongs to the second type of network environment connection location, store the data in the second candidate network environment location information table; S422: If both the first candidate network environment location information table and the second candidate network environment location information table are null, the network environment location data corresponding to the maximum stay time in the first candidate network environment location information table and the second candidate network environment location information table are used as the first candidate network environment location data and the second candidate network environment location data respectively; S424, obtaining the associated user ID associated with the first candidate network environment location data from the database, and obtaining the corresponding associated user ID set MA1 = {MA11, MA12, ..., MA1 a1 ,…,MA1 g1 }, and obtain the associated grid data set DR1 = {DR11, DR12, ..., DR1 a1 ,…,DR1 g1 }, MA1 a1 DR1 is the a1th user ID associated with the first candidate network environment location data, where a1 ranges from 1 to g1, and g1 is the number of user IDs associated with the first candidate network environment location data; a1 MA1 a1 The corresponding grid location data corresponding to the first candidate network environment location data; and, if a1=1, and DR1 a1 If it is not Null, DR1 a1 as the candidate stay grid location data corresponding to the first candidate network environment location data; if a1>1, and the number of data in DR1 that is not Null is greater than 1, then the data with the maximum stay duration in DR1 is used as the candidate stay grid location data corresponding to the first network environment location data; if DR1 is Null, then the network address corresponding to the first candidate network environment location data is used as the candidate stay grid location data corresponding to the first candidate network environment location data; S426, obtaining the associated user ID associated with the second candidate network environment location data from the database, and obtaining the corresponding associated user ID set MA2 = {MA21, MA22, ..., MA2 a2 ,…,MA2 g2 }, and obtain the associated grid data set DR2 from the database = {DR21, DR22, ..., DR2 a2 ,…,DR2 g2 }, MA2 a2 DR2 is the a2th user ID associated with the second candidate network environment location data, where a2 ranges from 1 to g2, and g2 is the number of user IDs associated with the second candidate network environment location data; a2 MA2 a2 The corresponding grid location data corresponding to the second candidate network environment location data; and, if a2=1, and DR2 a2 If it is not Null, DR2 a2 as the candidate stay grid location data corresponding to the second candidate network environment location data; if a2>1, and the number of data in DR2 that is not Null is greater than 1, then the data with the maximum stay duration in DR2 is used as the candidate stay grid location data corresponding to the second network environment location data; if DR2 is Null, then the wifimac address corresponding to the first candidate network environment location data is used as the candidate stay grid location data corresponding to the second candidate network environment location data; S428, obtain MT based on the obtained candidate stay grid location data i The corresponding third candidate grid position information.

5. The system according to claim 2, wherein: In S400, based on MT i The associated grid location information of MT is obtained i The corresponding fourth candidate grid location information specifically includes: S430, if MT i If the associated grid position information of is not Null, execute S431; otherwise, set the fourth candidate grid position information to Null and exit the current control program; For any network environment location data in the target network environment information, if the data belongs to the first type of network environment connection location, the data is stored in the first candidate network environment location information table; if the data belongs to the second type of network environment connection location, the data is stored in the second candidate network environment location information table; S431, for MT i If the grid location data w belongs to the first type of grid location, the grid location data q is stored in the first candidate associated grid location information table; if the grid location data w belongs to the second type of grid location, the grid location data q is stored in the second candidate associated grid location information table; the value of q ranges from 1 to z, where z is MT i the number of grid location data included in the associated grid location information; S432: If the value of the oth candidate associated grid location information table is not Null, for the grid location data A in the oth candidate associated grid location information table, o w, respectively obtain A o The distance between w and any position data in the target activity trajectory information is obtained as the first distance set D1 o w ; The value of o is 1 or 2; S433, get D1 o w The distances that are less than or equal to the preset distance are obtained to obtain the second distance set D2 o w ; S434, if D2 o w If A is Null, o The candidate grid position information WI corresponding to w o w Set to Null; otherwise, A o The candidate grid position information WI corresponding to w o w Set to include A o w and the corresponding number of time windows TP o wTP o wBased on D2 o w The acquisition time corresponding to each distance in is determined; the candidate associated grid position information set WI corresponding to the oth candidate associated grid position information table is obtained. o ={WI o 1. WI o 2,…,WI o w ,…,WI o w }; S435, if WI o If there is at least one candidate grid position data that is not Null, then WI o The candidate associated grid position data corresponding to the maximum number of time windows in the MA is used as the corresponding target associated grid position data MA o w; S436, based on MA o wGetMT i The corresponding fourth candidate grid position information.

6. The system according to claim 1, wherein: The confirmed grid location information includes the confirmed first and second stop grid locations and a confirmation time; S600 specifically includes: S610, if MT i If the corresponding confirmed grid location information and the target historical stay grid location information are not Null, and the confirmed time is earlier than the time corresponding to the target historical stay grid location information, then MT i The corresponding confirmed grid location information is used as MT i The corresponding target grid location information, otherwise, the MT i The corresponding target historical stay grid location information is used as MT i Corresponding target grid location information; S620, if MT i If the corresponding confirmed grid location information and the target grid historical stay grid location information are both Null, then the MT i Remove from the current MT.

7. The system according to claim 5, characterized in that The confirmed grid location information includes the confirmed first and second stop grid locations and a confirmation time; S700 specifically includes: S701, if MT i If the corresponding confirmed grid location information is not Null and the corresponding confirmation time is within the set time period, the MT i The corresponding confirmed grid location information is used as MT i Corresponding target grid position information; otherwise, execute S702; S702: If the first candidate grid position information is not Null, use the first candidate grid position information as the first target candidate grid position information; otherwise, use the second candidate grid position information as the first target candidate grid position information; S703: If the fourth candidate grid position information is not Null, use the fourth candidate grid position information as the second target candidate grid position information; otherwise, use the third candidate grid position information as the second target candidate grid position information; S704: If the first target candidate grid location information is the first candidate grid location information, the second target candidate grid location information is the fourth candidate grid location information, and if the number of time windows corresponding to the fourth candidate grid location information is greater than the set number of time windows, then the fourth candidate grid location information is used as the MT i The corresponding target grid location information, otherwise, the first candidate grid location information is used as MT i Corresponding target grid location information; S705: If the first target candidate grid position information is the first candidate grid position information or the second candidate grid position information, and the second target candidate grid position information is the third candidate grid position information, the first target candidate grid position information is used as the MT i The corresponding target grid location information.

8. The system according to claim 1, wherein: S700 specifically includes: S710, respectively from MT i Obtaining first-category grid location data and second-category grid location data from the corresponding first to fourth candidate grid location information and the confirmed grid location information, and obtaining a first-category grid location information table and a second-category grid location information table respectively; S720, clustering the grid location data in the first type grid location information table and the second type grid location information table, respectively, to obtain a first target class and a second target class, wherein the distance between any two grid location data in each target class is less than a set distance; S730: If the u-th target class includes the fourth candidate grid position information, the included fourth candidate grid position information is used as the MT i The corresponding u-th target stay grid position; otherwise, the center position of the u-th target class is used as the MT i The corresponding grid position where the uth target stops; the value of u is 1 or 2; S740, obtaining MT based on the u-th target stop grid position i The corresponding target grid location information.

Citation Information

Patent Citations

  • Terminal data processing system

    CN112507046A

  • Multi-source heterogeneous online service platform user data association fusion method and system

    CN116304981A