Loading and unloading point data real-time updating method and device and storage medium
Through cascading fusion of real-time order data and grid coding update, the problem of insufficient timeliness of recommended system data is solved, and the point library and feature library are updated in real time, reducing the risk of database crashes and maintenance costs, and improving user experience.
Patent Information
- Application Number
- CN202411125255.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-26
AI Technical Summary
In the prior art, in freight and passenger transport scenarios, the data timeliness of the recommendation system are insufficient, resulting in users that may obtain inaccurate or untimely information, and large-scale data updates have a negative impact on database performance, increasing maintenance costs.
By obtaining real-time order data, cascading and fusion generate grid encoding, updating historical point library and behavior feature data list, real-time update of point library and feature library, converting batch updates to streaming data updates, reducing the risk of database crashes, and improving data timeliness sex.
It significantly reduces the risk of database crashes, reduces the maintenance costs of managers, and improves the timeliness of feature data, ensuring that users can obtain timely and accurate information.
Smart Images

Figure CN119025535A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method, device and storage medium for real-time updating of loading and unloading point data. Background Art
[0002] In the scenarios of freight and passenger transport, the point library and feature library are the data basis of the recommendation system. The point library usually refers to the database that stores information about various locations, while the feature library contains data used to describe the various attributes and characteristics of these locations. These data are crucial to the recommendation system and determine what content and information the system can provide to users. In high-frequency scenarios such as passenger transport, the timeliness of data is particularly important because user needs and location information may change rapidly. If the data used by the recommendation system is not updated in a timely manner, it may not be able to accurately respond to user needs, thus affecting the user experience.
[0003] In order to alleviate the problem of data timeliness, the industry may adopt the method of delaying data push by two to three days. Although this can ensure that users can get relatively timely data most of the time, it still does not solve the problem of timeliness of data updates, which may cause users to get inaccurate or untimely information at certain times. In addition, large amounts of data insertion may affect database performance, resulting in slow response or service unavailability, which may increase the maintenance costs of database administrators. Summary of the invention
[0004] Based on this, it is necessary to provide a method, device and storage medium for real-time updating of loading and unloading point data in response to the above technical problems, so as to solve at least one of the above technical problems.
[0005] The present invention provides a method for real-time updating of loading and unloading point data, comprising:
[0006] Acquire real-time order data, wherein the real-time order data is generated by cascading and fusing a user request data stream, a user order data stream, and a driver loading and unloading data stream;
[0007] Encoding the loading and unloading completion points in the real-time order data to obtain a grid code;
[0008] Based on the grid code, query to obtain a list of historical warehouse points, and based on the tag information in the real-time order data, query to obtain a list of historical behavior feature data;
[0009] Based on the loading and unloading completion points, the historical warehouse point list is updated, and based on the user behavior data in the real-time order data, the historical behavior feature data list is updated.
[0010] Optionally, according to a method for real-time updating of loading and unloading point data provided by the present invention, the step of obtaining real-time order data includes:
[0011] When an address search request of a user is monitored, generating the user request data stream based on detailed information in the address search request;
[0012] Report the user request data stream to the Kafka topic and cache it in a preset cache area;
[0013] When a user's order request is detected, a user order data stream is generated based on the order information in the order request and reported to the Kafka topic;
[0014] Cascading and fusing the user order data stream and the user request data stream to obtain a fused data stream, and caching the fused data stream in a preset cache area and deleting the user request data stream in the preset cache area;
[0015] When a driver's order completion request is detected, based on the order completion information in the order completion request, a driver loading and unloading data stream is generated and reported to the Kafka topic;
[0016] The fused data stream and the driver loading and unloading data stream are cascaded and fused to obtain real-time order data, and the real-time order data is cached in a preset cache area and the fused data stream in the preset cache area is deleted.
[0017] Optionally, according to a method for real-time updating of loading and unloading point data provided by the present invention, the updating of the historical point warehouse point list based on the loading and unloading completion point includes:
[0018] Determine the distance between the loading and unloading completion point and each point in the historical point list;
[0019] Based on the distance, a point location to be updated is selected from each point location in the point library;
[0020] Determine the historical unloading frequency corresponding to the storage point to be updated;
[0021] Perform weighted calculation on the to-be-updated warehouse location and its corresponding historical unloading frequency, as well as the loading and unloading completion point, to obtain the target warehouse location;
[0022] The target point warehouse point replaces the point warehouse point to be updated, and a new point warehouse point list is obtained and stored.
[0023] Optionally, according to a method for real-time updating of loading and unloading point data provided by the present invention, after selecting the point warehouse location to be updated from each point warehouse location based on the distance, the method further includes:
[0024] The historical unloading frequency is updated to obtain a target unloading frequency, and the to-be-updated point warehouse location is associated with the target unloading frequency and stored.
[0025] Optionally, according to a method for real-time updating of loading and unloading point data provided by the present invention, after encoding the loading and unloading completion points in the real-time order data to obtain grid codes, the method further comprises:
[0026] If no historical point warehouse location list is found based on the grid code, the loading and unloading completion point is stored in the database as a newly added point warehouse location, and the unloading frequency of the newly added point warehouse location is set to 1.
[0027] Optionally, according to a method for real-time updating of loading and unloading point data provided by the present invention, the updating of the historical behavior feature data list based on the user behavior data in the real-time order data includes:
[0028] According to a preset data format, each data in the user behavior data is spliced to obtain a target feature data list;
[0029] Based on the target feature data list, the feature data in the historical behavior feature data list is updated.
[0030] Optionally, according to a method for real-time updating of loading and unloading point data provided by the present invention, before querying and obtaining a list of historical behavior feature data based on the tag information in the real-time order data, the method further includes:
[0031] If the historical behavior feature data list is not found, the target feature data list is added to the preset feature database.
[0032] Optionally, according to a method for real-time updating of loading and unloading point data provided by the present invention, after caching the real-time order data in a preset cache area, the method further comprises:
[0033] When it is monitored that the cache expiration time corresponding to the user request data flow or the user order data flow has arrived, the user request data flow or the user order data flow cached in the preset cache area is deleted.
[0034] The present invention also provides a device for real-time updating of loading and unloading point data, comprising:
[0035] An acquisition module, used to acquire real-time order data, wherein the real-time order data is generated by cascading and fusing a user request data stream, a user order data stream, and a driver loading and unloading data stream;
[0036] An encoding module, used for encoding the loading and unloading completion points in the real-time order data to obtain a grid code;
[0037] A query module, used to query and obtain a list of historical point warehouse locations based on the grid code, and to query and obtain a list of historical behavior feature data based on the tag information in the real-time order data;
[0038] An updating module is used to update the historical warehouse point list based on the loading and unloading completion point, and to update the historical behavior feature data list based on the user behavior data in the real-time order data.
[0039] The present invention also provides a computer device, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for real-time updating of loading and unloading point data when executing the computer-readable instructions.
[0040] The present invention also provides one or more readable storage media storing computer-readable instructions, and when the computer-readable instructions are executed by a processor, the above-mentioned method for real-time updating of loading and unloading point data is implemented.
[0041] The above-mentioned method, device and storage medium for real-time updating of loading and unloading point data include: obtaining real-time order data, wherein the real-time order data is generated by cascading fusion based on user request data stream, user order data stream and driver loading and unloading data stream; encoding the loading and unloading completion points in the real-time order data to obtain grid codes; querying and obtaining a list of historical point warehouse locations based on the grid codes, and querying and obtaining a list of historical behavior feature data based on the tag information in the real-time order data; updating the list of historical point warehouse locations based on the loading and unloading completion points, and updating the list of historical behavior feature data based on the user behavior data in the real-time order data. Based on the real-time order data generated by cascading fusion of user request data stream, user order data stream and driver loading and unloading data stream, the point warehouse is located and the feature data is updated based on the real-time order data, realizing the conversion of the update mode from batch data update to real-time update of stream data, which can reduce the sudden large-scale insertion or update of data in the database, significantly reduce the risk of database crash, and thus reduce the maintenance cost of management personnel. In addition, the real-time update process of online data streaming has a great improvement in feature timeliness. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0043] Figure 1 It is a flow chart of a method for real-time updating of loading and unloading point data in one embodiment of the present invention;
[0044] Figure 2 It is a schematic diagram of equipment deployment of a method for real-time updating of loading and unloading point data in one embodiment of the present invention;
[0045] Figure 3 is a complete flow chart of data updating provided by an embodiment of the present invention;
[0046] Figure 4 It is a structural schematic diagram of a device for real-time updating of loading and unloading point data in one embodiment of the present invention;
[0047] Figure 5 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION
[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0049] The terms used in one or more embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present invention. The singular forms of "a", "said" and "the" used in one or more embodiments of the present invention are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present invention refers to and includes any or all possible combinations of one or more associated listed items.
[0050] In one embodiment, specifically, Figure 1 As shown, Figure 1 1 is a flow chart of a method for real-time updating of loading and unloading point data in an embodiment of the present invention. The embodiment of the present invention provides a method for real-time updating of loading and unloading point data, comprising the following steps:
[0051] Step S11, obtaining real-time order data, wherein the real-time order data is generated by cascading and fusing the user request data stream, the user order data stream, and the driver loading and unloading data stream;
[0052] It should be noted that the user request data stream includes detailed information such as user ID, requested location (starting point and end point), request time, contact information, etc.; the user order data stream includes order information such as order ID, starting and ending locations used for placing the order, order time, user ID and contact information, etc.; the driver loading and unloading data stream includes information such as loading and unloading completion point, completion time, and order ID, etc.
[0053] It should be noted that, refer to Figure 2 , Figure 2 It is a schematic diagram of a module for obtaining real-time order data online provided by an embodiment of the present invention. The general order process is that the user selects the starting point and the end point - the user places an order - the driver loads and unloads the goods. Therefore, the user request data flow will first reach the background service, followed by the user order data flow, and finally the driver loading and unloading data flow.
[0054] Specifically, when a user's address search request is monitored, the user request data stream is generated based on the user id, the requested location (starting point and end point), the request time, the contact information and other information, and the user request data stream is reported to the Kafka topic in the form of a data stream and cached in a preset cache area; subsequently, when a user's order request is monitored, the user order data stream is generated based on the order id, the starting and ending locations used for the order, the order time and the order information such as the user id and contact information, and reported to the Kafka topic; and then the user request data stream and the user order data stream are directly combined. Cascade to obtain a fused data stream, and cache the fused data stream in the preset cache area and delete the user request data stream in the preset cache area, thereby reducing the cache pressure of the preset cache area; finally, when the driver's order completion request is monitored, based on the loading and unloading completion point, completion time, order ID and other information, a driver loading and unloading data stream is generated and reported to the Kafka topic; further, the fused data stream and the driver loading and unloading data stream belonging to the same order ID are cascaded and fused to obtain real-time order data, and the real-time order data is cached in the preset cache area and the fused data stream in the preset cache area is deleted.
[0055] Step S12, encoding the loading and unloading completion points in the real-time order data to obtain a grid code;
[0056] Specifically, the loading and unloading completion points in the real-time order data are H3-encoded, where H3 refers to the hexagonal spatial index designed by Uber, where each point in the geographic space clearly corresponds to a hexagon (i.e., an H3 grid). Specifically, the hexagon corresponding to the point can be determined by the latitude and longitude information of the point, and each hexagon uniquely corresponds to an H3index (i.e., an H3 grid code). When encoding, the Uberi open source H3 spatial index algorithm can be used to convert the code in the real-time order data into the corresponding H3 code (i.e., the grid code in this embodiment).
[0057] Step S13, querying and obtaining a list of historical point warehouse locations based on the grid code, and querying and obtaining a list of historical behavior feature data based on the tag information in the real-time order data;
[0058] Specifically, using the H3 grid code as the key value, the historical point library point list can be queried in the H3 point library, and the historical loading and unloading frequency corresponding to each point library point can also be queried. In addition, it should be noted that the tag information includes tags such as the user ID, point of interest ID, contact ID, H3 grid code corresponding to the requested location, and grid code ID corresponding to the loading and unloading completion point in the real-time order data; thus, based on the tag information, the corresponding historical behavior feature data list can be queried in the preset feature library.
[0059] Step S14, based on the loading and unloading completion points, updating the historical point warehouse point list, and based on the user behavior data in the real-time order data, updating the historical behavior feature data list.
[0060] Specifically, refer to Figure 3 , Figure 3 This is a complete flow chart of data update provided by an embodiment of the present invention. For the update of point warehouse locations: first calculate the distance between the loading and unloading completion point and each point warehouse location in the historical point warehouse location list, and then select the point warehouse location with the closest distance as the point warehouse location to be updated, and determine the historical unloading frequency corresponding to the point warehouse location to be updated, and then perform weighted average calculation on the point warehouse location to be updated and its corresponding historical unloading frequency and the loading and unloading completion point (the current unloading frequency is set to 1) to obtain the target point warehouse location, and add 1 to the historical unloading frequency to obtain the target unloading frequency. Then, using the H3 grid code as the key value, the target point warehouse location and its target unloading frequency replace the point warehouse location to be updated and its corresponding historical unloading frequency for caching.
[0061] In addition, if no historical point warehouse location list is found, it proves that there is no historical point warehouse location data near the loading and unloading completion point, that is, there is no point warehouse location data associated with the H3 grid code in the database, and then the loading and unloading completion point is directly added as a new point warehouse location and stored in the database, and the unloading frequency of the new point warehouse location is set to 1.
[0062] Update of feature data: Update the historical behavior feature data list according to the behavior data such as loading and unloading completion point, completion time, loading and unloading mark in the real-time order data. In one embodiment, the loading and unloading completion point, completion time, loading and unloading mark and other information in the user behavior data are spliced according to a specific data format to obtain a target feature data list; then based on the target feature data list, update the feature data in the historical behavior feature data list.
[0063] In addition, if no relevant historical behavior feature data list is found, the target feature data list is directly added to the preset feature database.
[0064] The embodiment of the present invention, through the above scheme, includes: obtaining real-time order data, wherein the real-time order data is generated by cascading fusion based on user request data stream, user order data stream and driver loading and unloading data stream; encoding the loading and unloading completion point in the real-time order data to obtain a grid code; querying to obtain a list of historical point warehouse locations based on the grid code, and querying to obtain a list of historical behavior feature data based on the tag information in the real-time order data; updating the list of historical point warehouse locations based on the loading and unloading completion point, and updating the list of historical behavior feature data based on the user behavior data in the real-time order data. The real-time order data is generated by cascading fusion based on the user request data stream, the user order data stream and the driver loading and unloading data stream, and then the point warehouse is located and the feature data is updated based on the real-time order data, realizing the conversion of the update mode from batch data update to real-time update of stream data, which can reduce the sudden large-scale insertion or update of data in the database, significantly reduce the risk of database crash, and thus reduce the maintenance cost of management personnel. In addition, the real-time update process of online data streaming has a great improvement in feature timeliness.
[0065] In one embodiment of the present invention, the obtaining of real-time order data includes:
[0066] When an address search request of a user is monitored, generating the user request data stream based on detailed information in the address search request;
[0067] Report the user request data stream to the Kafka topic and cache it in a preset cache area;
[0068] When a user's order request is detected, a user order data stream is generated based on the order information in the order request and reported to the Kafka topic;
[0069] Cascading and fusing the user order data stream and the user request data stream to obtain a fused data stream, and caching the fused data stream in a preset cache area and deleting the user request data stream in the preset cache area;
[0070] When a driver's order completion request is detected, based on the order completion information in the order completion request, a driver loading and unloading data stream is generated and reported to the Kafka topic;
[0071] The fused data stream and the driver loading and unloading data stream are cascaded and fused to obtain real-time order data, and the real-time order data is cached in a preset cache area and the fused data stream in the preset cache area is deleted.
[0072] It should be noted that the real-time update service will connect to the Kafka service through a specific interface and consume data from a specific topic, where a Topic can be regarded as a message channel in Kafka. The producer will publish messages to a specific Topic, and the consumer will subscribe to one or more Topics to receive messages. The embodiment of the present invention needs to consume three streams of data in total.
[0073] 1: Request point data stream topic (that is, user request data stream in this embodiment): When the user searches for an address on the front end, when the user's address search request is monitored, the detailed information in the address search request will be reported to the Kafka topic in the form of a data stream. The user may have multiple requests, where the detailed information includes the user ID, contact person for loading and unloading, and request time, location, name, surrounding points of interest, and other information.
[0074] 2: User order data stream topic (that is, user order data stream in this embodiment): After the user selects the starting and ending points, he will click to place an order. When the user's order request is monitored, the order information in the order request will be reported to the Kafka topic in the form of a data stream. Even if the user requests the location multiple times, the user only has one order operation. The order information includes user ID, contact person for loading and unloading, order ID, order time, and the starting and ending locations used for the order.
[0075] 3: Driver operation data stream (that is, the driver loading and unloading data stream in this embodiment): When the driver clicks on the completion of loading and unloading, when the driver's order completion request is monitored, the order completion information in the order completion request will be reported to the kafka topic in the form of a data stream. The driver only has one click operation. Among them, the order completion information includes order ID, order completion time, loading and unloading completion click-in information, etc.
[0076] Specifically, when a user's address search request is monitored, the user request data stream is generated based on the detailed information in the address search request; the user request data stream is then reported to the topic of Kafka, and the user request data stream is cached in the preset cache area redis with the user id as the key value. Optionally, in order to prevent excessive cache pressure in redis, a cache expiration time of the user request data stream can be set, for example, set to 4 hours. When the cache expiration time corresponding to the user request data stream is detected to be reached, the user request data stream cached in the preset cache area is deleted.
[0077] Furthermore, when a user's order request is monitored, based on the order information in the order request, a user order data stream is generated and reported to the Kafka topic, and then according to the user id and the order time, a user request data stream with the same user id and the closest to the order time is filtered in the preset cache area redis; and then the user order data stream and the user request data stream are cascaded and merged to obtain a fused data stream. Furthermore, the user request data stream cached in the preset cache area is deleted according to the user id, and the fused data stream is cached in the preset cache area with the order id as the key value. Optionally, in order to prevent excessive redis cache pressure, a cache expiration time of the fused data stream is set, for example, set to 24 hours. When it is detected that the cache expiration time corresponding to the user order data stream is reached, the user order data stream cached in the preset cache area is deleted.
[0078] Furthermore, after the driver clicks on the completion of loading and unloading, when the driver's order completion request is monitored, the driver's loading and unloading data stream is generated based on the order completion information in the order completion request and reported to the Kafka topic, and then based on the order id, the preset cache area is queried to obtain the fused data stream, and then the fused data stream and the driver's loading and unloading data stream are cascaded and fused to obtain real-time order data, and the real-time order data is cached in the preset cache area and the fused data stream in the preset cache area is deleted.
[0079] The embodiment of the present invention adopts the above scheme, reports each data to the platform in the form of data stream, and the platform generally collects these data streams into the cache area using the Kafka framework, thereby realizing a real-time online update process, and converting the subsequent data update method from batch data update to data stream update, which can eliminate the sudden large-scale insertion or update of data in the database.
[0080] In one embodiment of the present invention, the updating of the historical point warehouse point list based on the loading and unloading completion point includes:
[0081] Determine the distance between the loading and unloading completion point and each point in the historical point list;
[0082] Based on the distance, a point location to be updated is selected from each point location in the point library;
[0083] Determine the historical unloading frequency corresponding to the storage point to be updated;
[0084] Perform weighted calculation on the to-be-updated warehouse location and its corresponding historical unloading frequency, as well as the loading and unloading completion point, to obtain the target warehouse location;
[0085] The target point warehouse point replaces the point warehouse point to be updated, and a new point warehouse point list is obtained and stored.
[0086] Specifically, the distance between the loading and unloading completion point and each point warehouse point in the historical point warehouse point list is calculated respectively, and then the point warehouse point with the closest distance is selected as the point warehouse point to be updated. It is also necessary to determine the historical unloading frequency corresponding to the point warehouse point to be updated. Further, the point warehouse point to be updated and its corresponding historical unloading frequency and the loading and unloading completion point are weighted averaged. It should be noted that the unloading frequency associated with the loading and unloading completion point is set to 1 to obtain the target point warehouse point. Then the target point warehouse point replaces the point warehouse point to be updated. In addition, the historical unloading frequency is updated, for example, the historical unloading frequency + 1, to obtain the target unloading frequency, and then the new point warehouse point list and the target unloading frequency are associated and stored with the H3 grid code as the key value.
[0087] In one embodiment, after encoding the loading and unloading completion points in the real-time order data to obtain the grid code, the following further includes:
[0088] If no historical point warehouse location list is found based on the grid code, the loading and unloading completion point is stored in the database as a newly added point warehouse location, and the unloading frequency of the newly added point warehouse location is set to 1.
[0089] It should be noted that there is no historical warehouse point data near the loading and unloading completion point, that is, based on the grid code, it is impossible to query the historical warehouse point list. At this time, the update operation will be changed to a new operation. Then the loading and unloading completion point is directly stored in the database as a new warehouse point, and the unloading frequency of the new warehouse point is set to 1.
[0090] The embodiment of the present invention adopts the above scheme, obtains the point warehouse location data from the H3 point warehouse through query, and obtains the new point warehouse location through weighted average combined with the loading and unloading completion points in this data stream, thereby realizing online update of the point warehouse location and improving the timeliness of data update.
[0091] In one embodiment of the present invention, updating the historical behavior feature data list based on the user behavior data in the real-time order data includes:
[0092] According to a preset data format, each data in the user behavior data is spliced to obtain a target feature data list;
[0093] Based on the target feature data list, the feature data in the historical behavior feature data list is updated.
[0094] Specifically, the loading and unloading completion point, completion time, loading and unloading marks, and other information in the user behavior data are spliced according to a specific data format to obtain a target feature data list; and then based on the target feature data list, the feature data in the historical behavior feature data list is updated.
[0095] In addition, if the historical behavior feature data list is not found in the adjustment database, the target feature data list is added to the preset feature database, thereby realizing online update of feature data and improving the timeliness of data update.
[0096] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.
[0097] In one embodiment, a device for real-time updating of loading and unloading point data is provided, and the device for real-time updating of loading and unloading point data corresponds one-to-one to the method for real-time updating of loading and unloading point data in the above embodiment. Figure 4 As shown, Figure 4 1 is a schematic diagram of a device for updating data of a loading and unloading point in real time according to an embodiment of the present invention. The device for updating data of a loading and unloading point in real time comprises:
[0098] An acquisition module 21 is used to acquire real-time order data, wherein the real-time order data is generated by cascading and fusing a user request data stream, a user order data stream, and a driver loading and unloading data stream;
[0099] An encoding module 22, used for encoding the loading and unloading completion points in the real-time order data to obtain a grid code;
[0100] A query module 23, configured to query and obtain a list of historical point warehouse locations based on the grid code, and query and obtain a list of historical behavior feature data based on the tag information in the real-time order data;
[0101] The updating module 24 is used to update the historical warehouse point list based on the loading and unloading completion point, and to update the historical behavior feature data list based on the user behavior data in the real-time order data.
[0102] The acquisition module 21 is also used for:
[0103] When an address search request of a user is monitored, generating the user request data stream based on detailed information in the address search request;
[0104] Report the user request data stream to the Kafka topic and cache it in a preset cache area;
[0105] When a user's order request is detected, a user order data stream is generated based on the order information in the order request and reported to the Kafka topic;
[0106] Cascading and fusing the user order data stream and the user request data stream to obtain a fused data stream, and caching the fused data stream in a preset cache area and deleting the user request data stream in the preset cache area;
[0107] When a driver's order completion request is detected, based on the order completion information in the order completion request, a driver loading and unloading data stream is generated and reported to the Kafka topic;
[0108] The fused data stream and the driver loading and unloading data stream are cascaded and fused to obtain real-time order data, and the real-time order data is cached in a preset cache area and the fused data stream in the preset cache area is deleted.
[0109] The update module 24 is also used to:
[0110] Determine the distance between the loading and unloading completion point and each point in the historical point list;
[0111] Based on the distance, a point location to be updated is selected from each point location in the point library;
[0112] Determine the historical unloading frequency corresponding to the storage point to be updated;
[0113] Perform weighted calculation on the to-be-updated warehouse location and its corresponding historical unloading frequency, as well as the loading and unloading completion point, to obtain the target warehouse location;
[0114] The target point warehouse point replaces the point warehouse point to be updated, and a new point warehouse point list is obtained and stored.
[0115] The update module 24 is also used to:
[0116] The historical unloading frequency is updated to obtain a target unloading frequency, and the to-be-updated point warehouse location is associated with the target unloading frequency and stored.
[0117] The real-time updating device for loading and unloading point data also includes:
[0118] A new storage module is added, which is used to store the loading and unloading completion point as a new point warehouse point in the database if no historical point warehouse point list is found based on the grid code, and set the unloading frequency of the new point warehouse point to 1.
[0119] The update module 24 is also used to:
[0120] According to a preset data format, each data in the user behavior data is spliced to obtain a target feature data list;
[0121] Based on the target feature data list, the feature data in the historical behavior feature data list is updated.
[0122] The real-time updating device for loading and unloading point data also includes:
[0123] The adding module is used to add the target feature data list to the preset feature database if the historical behavior feature data list is not found.
[0124] The real-time updating device for loading and unloading point data also includes:
[0125] The deletion module is used to delete the user request data flow or the user order data flow cached in the preset cache area when it is monitored that the cache expiration time corresponding to the user request data flow or the user order data flow has arrived.
[0126] The specific definition of the real-time updating device for loading and unloading point data can be found in the definition of the real-time updating method for loading and unloading point data above, which will not be repeated here. Each module in the above-mentioned real-time updating device for loading and unloading point data can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0127] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, Figure 5 : is a schematic diagram of a computer device in an embodiment of the present invention. The computer device includes a processor, a memory, a network interface and a database connected by a device bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium and an internal memory. The readable storage medium stores an operating device, a computer-readable instruction and a database. The internal memory provides an environment for the operation of the operating device and the computer-readable instructions in the readable storage medium. The database of the computer device is used to store data involved in the real-time update method of loading and unloading point data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer-readable instruction is executed by the processor, a real-time update method of loading and unloading point data is implemented. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.
[0128] In one embodiment, a computer device is provided. The computer device may be a terminal device, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, and a network interface connected through a device bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium. The readable storage medium stores computer-readable instructions. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer-readable instructions are executed by the processor, a method for real-time updating of loading and unloading point data is implemented. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.
[0129] In one embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, the steps of the above-mentioned method for real-time updating of loading and unloading point data are implemented.
[0130] In one embodiment, a readable storage medium is provided, the readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the real-time update method for loading and unloading point data as described above are implemented. A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When the computer-readable instructions are executed, they may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0131] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0132] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. A method for real-time updating of loading and unloading point data, characterized in that: include: Acquire real-time order data, wherein the real-time order data is generated by cascading and fusing a user request data stream, a user order data stream, and a driver loading and unloading data stream; Encoding the loading and unloading completion points in the real-time order data to obtain a grid code; Based on the grid code, query to obtain a list of historical warehouse points, and based on the tag information in the real-time order data, query to obtain a list of historical behavior feature data; Based on the loading and unloading completion points, the historical warehouse point list is updated, and based on the user behavior data in the real-time order data, the historical behavior feature data list is updated.
2. The method for real-time updating of loading and unloading point data according to claim 1, characterized in that: The obtaining of real-time order data includes: When an address search request of a user is monitored, generating the user request data stream based on detailed information in the address search request; Report the user request data stream to the Kafka topic and cache it in a preset cache area; When a user's order request is detected, a user order data stream is generated based on the order information in the order request and reported to the Kafka topic; Cascading and fusing the user order data stream and the user request data stream to obtain a fused data stream, and caching the fused data stream in a preset cache area and deleting the user request data stream in the preset cache area; When a driver's order completion request is detected, based on the order completion information in the order completion request, a driver loading and unloading data stream is generated and reported to the Kafka topic; The fused data stream and the driver loading and unloading data stream are cascaded and fused to obtain real-time order data, and the real-time order data is cached in a preset cache area and the fused data stream in the preset cache area is deleted.
3. The method for real-time updating of loading and unloading point data according to claim 1, characterized in that: The updating of the historical point warehouse point list based on the loading and unloading completion point includes: Determine the distance between the loading and unloading completion point and each point in the historical point list; Based on the distance, a point location to be updated is selected from each point location in the point library; Determine the historical unloading frequency corresponding to the storage point to be updated; Perform weighted calculation on the to-be-updated warehouse location and its corresponding historical unloading frequency, as well as the loading and unloading completion point, to obtain the target warehouse location; The target point warehouse point replaces the point warehouse point to be updated, and a new point warehouse point list is obtained and stored.
4. The method for real-time updating of loading and unloading point data according to claim 3, characterized in that: After selecting the point locations to be updated from the various point locations based on the distance, the method further includes: The historical unloading frequency is updated to obtain a target unloading frequency, and the to-be-updated point warehouse location is associated with the target unloading frequency and stored.
5. The method for real-time updating of loading and unloading point data according to claim 3, characterized in that: After encoding the loading and unloading completion points in the real-time order data to obtain a grid code, the method further includes: If no historical point warehouse location list is found based on the grid code, the loading and unloading completion point is stored in the database as a newly added point warehouse location, and the unloading frequency of the newly added point warehouse location is set to 1.
6. The method for real-time updating of loading and unloading point data according to claim 1, characterized in that: The updating of the historical behavior feature data list based on the user behavior data in the real-time order data includes: According to a preset data format, each data in the user behavior data is spliced to obtain a target feature data list; Based on the target feature data list, the feature data in the historical behavior feature data list is updated.
7. The method for real-time updating of loading and unloading point data according to claim 6, characterized in that: Before querying and obtaining a list of historical behavior feature data based on the tag information in the real-time order data, the method further includes: If the historical behavior feature data list is not found, the target feature data list is added to the preset feature database.
8. The method for real-time updating of loading and unloading point data according to claim 2, characterized in that: After caching the real-time order data into a preset cache area, the method further includes: When it is monitored that the cache expiration time corresponding to the user request data flow or the user order data flow has arrived, the user request data flow or the user order data flow cached in the preset cache area is deleted.
9. A device for real-time updating of loading and unloading point data, characterized in that: include: An acquisition module, used to acquire real-time order data, wherein the real-time order data is generated by cascading and fusing a user request data stream, a user order data stream, and a driver loading and unloading data stream; An encoding module, used for encoding the loading and unloading completion points in the real-time order data to obtain a grid code; A query module, used to query and obtain a list of historical point warehouse locations based on the grid code, and to query and obtain a list of historical behavior feature data based on the tag information in the real-time order data; An updating module is used to update the historical warehouse point list based on the loading and unloading completion point, and to update the historical behavior feature data list based on the user behavior data in the real-time order data.
10. A readable storage medium having computer readable instructions stored thereon, characterized in that: When the computer-readable instructions are executed by a processor, the real-time updating method for loading and unloading point data according to any one of claims 1 to 8 is implemented.
Citation Information
Patent Citations
Streaming data processing method and device, computer equipment and storage medium
CN110851477A
Streaming processing method and device for information data
CN111367953A
Workflow data storage method and device
CN114490662A
Out-of-order data real-time synchronization method based on flink, electronic equipment and computer readable storage medium
CN115757382A
Vehicle and road cloud collaborative big data management system for park
CN116052420A
