Data processing method and device

Abstract

The invention discloses a data processing method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: in response to a received task execution request sent by a logistics gateway, analyzing task parameters of a to-be-executed task from the task execution request; the task parameters are sent to the logistics gateway through an intranet system; the task parameters comprise a task type and an address type; determining target data of the to-be-executed task according to the task type and the address type; and sending a result indicating successful execution of the task to the intranet system through the logistics gateway, so that the intranet system obtains the target data. According to the implementation mode, cross-network real-time interaction is realized through the logistics gateway, the AI engine is used for capturing the data and analyzing the data, and then the result is synchronized to the intranet system, so that the data is provided for operation, and convenience is provided for optimizing the logistics network.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to a method and device for data processing. Background Art

[0002] In the field of logistics technology, the layout of the logistics network has a great impact on the logistics timeliness. By obtaining and analyzing the logistics data of other logistics companies, it is helpful to optimize its own logistics network. However, there is currently a lack of effective technical means to meet the business requirements, and it is impossible to analyze the differences in the logistics network efficiency of each logistics company, resulting in low logistics efficiency. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a method and device for data processing, which realize cross-network real-time interaction through a logistics gateway, use the AI engine of the external network system to capture and analyze data, and then synchronize the results to the internal network system for operation to provide data, facilitating the optimization of the logistics network.

[0004] To achieve the above object, according to one aspect of the embodiments of the present invention, a method for data processing is provided, including:

[0005] In response to receiving a task execution request sent by the logistics gateway, parsing task parameters of a to-be-executed task from the task execution request; the task parameters are sent to the logistics gateway through the internal network system; the task parameters include a task type and an address type;

[0006] Determining target data of the to-be-executed task according to the task type and the address type;

[0007] Sending a result indicating successful task execution to the internal network system through the logistics gateway, so that the internal network system obtains the target data.

[0008] Optionally, determining the target data of the to-be-executed task according to the task type and the address type includes:

[0009] Determining network address data corresponding to the to-be-executed task according to the task type and the address type, and writing the network address data into a network address file;

[0010] Obtaining logistics data corresponding to the to-be-executed task according to the network address file;

[0011] Filtering out the target data from the logistics data according to a preset rule.

[0012] Optionally, the task type is the first type or the second type, and determining the network address data corresponding to the to-be-executed task according to the task type and the address type includes:

[0013] If the task type is the first type, obtain the network address corresponding to the first type, and convert the network address; determine the network address data according to the converted network address and the task type;

[0014] If the task type is the second type, determine the network address data according to the address type.

[0015] Optionally, the address type is a full - volume task or a flow task. Determining the network address data according to the address type includes:

[0016] If the address type is a full - volume task, obtain the local full - volume address data, and determine the network address data according to the current timestamp and the local full - volume address data;

[0017] If the address type is a flow task, download the flow file corresponding to the flow task, and determine the network address data according to the current timestamp and the flow file.

[0018] Optionally, after determining the target data of the task to be executed according to the task type and the address type, it further includes:

[0019] Upload the target data to cloud storage, so that the intranet system obtains the target data from the cloud storage and visualizes the target data according to a preset dimension and a preset display method.

[0020] Optionally, the task parameter includes a task status. Before determining the target data of the task to be executed according to the task type and the address type, it further includes:

[0021] Determine that the task status is a status indicating task start.

[0022] Optionally, after the intranet system receives the result indicating that the task execution is successful, update the task status and task progress of the task to be executed.

[0023] According to another aspect of the embodiments of the present invention, there is provided a data processing device, including:

[0024] A receiving module, in response to receiving a task execution request sent by a logistics gateway, parsing task parameters of a task to be executed from the task execution request; the task parameters are sent to the logistics gateway by the intranet system; the task parameters include a task type and an address type;

[0025] A determining module, determining the target data of the task to be executed according to the task type and the address type;

[0026] A sending module that sends the result indicating successful task execution to the intranet system through the logistics gateway, so that the intranet system can obtain the target data.

[0027] According to another aspect of the embodiments of the present invention, an electronic device is provided, including:

[0028] One or more processors;

[0029] A storage device for storing one or more programs,

[0030] When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method provided by the present invention.

[0031] According to still another aspect of the embodiments of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the data processing method provided by the present invention is implemented.

[0032] One of the embodiments of the above invention has the following advantages or beneficial effects: The data processing method of the embodiments of the present invention, after receiving a task execution request sent by the logistics gateway, parses the task parameters of the task to be executed from the task execution request, and determines the target data of the task to be executed according to the task type and address type in the task parameters, so that after the intranet system receives the result of successful task execution through the logistics network, it can obtain the target data. This method realizes real-time parsing of the task to be executed through the AI engine across networks through the logistics gateway, and synchronizes the obtained target data of the task to be executed to the intranet system, so that the operation personnel can obtain the target data through the intranet system, analyze the competitor data, optimize our logistics network, and improve the user experience.

[0033] The further effects of the above non-conventional optional methods will be described in conjunction with the specific embodiments below. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention. Among them:

[0035] Figure 1 is a schematic diagram of the main process of a data processing method according to an embodiment of the present invention;

[0036] Figure 2 is a flowchart of a data processing method according to an embodiment of the present invention;

[0037] Figure 3 is a schematic diagram of system interaction of a data processing method according to an embodiment of the present invention;

[0038] Figure 4It is a schematic diagram of the process flow of data processing in an intranet system according to an embodiment of the present invention;

[0039] Figure 5 It is a schematic diagram of the process of data processing in an extranet system according to an embodiment of the present invention;

[0040] Figure 6 It is a schematic diagram of the main modules of a data processing device according to an embodiment of the present invention;

[0041] Figure 7 It is an exemplary system architecture diagram to which an embodiment of the present invention can be applied;

[0042] Figure 8 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. Detailed implementation manners

[0043] The following describes exemplary embodiments of the present invention with reference to the accompanying drawings. Various details of the embodiments of the present invention are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0044] Figure 1 It is a schematic diagram of the main process of a data processing method according to an embodiment of the present invention. As Figure 1 shown, the data processing method includes the following steps:

[0045] Step S101: In response to receiving a task execution request sent by a logistics gateway, parse task parameters of a task to be executed from the task execution request; the task parameters are sent to the logistics gateway through an intranet system; the task parameters include a task type and an address type;

[0046] Step S102: Determine target data of the task to be executed according to the task type and the address type;

[0047] Step S103: Send a result indicating successful task execution to the intranet system through the logistics gateway so that the intranet system can obtain the target data.

[0048] In an embodiment of the present invention, the data processing method can be executed by an external network system. The external network system and the internal network system interact through a logistics gateway. The internal network system can be an application system deployed within an enterprise, and the external network system can be an AI engine deployed on the Internet side (the AI engine is a tool kernel for automated data scraping, downloading, conversion, and storage based on task instructions). The internal network system can cross-network call the AI engine of the external network system through the logistics gateway to send task instructions. The external network system can send a message notification to the internal network system through the logistics gateway after processing the task, enabling the internal network system and the external network system to achieve cross-network real-time interaction.

[0049] In an embodiment of the present invention, a task execution request sent by the logistics gateway is received, and the task execution request is parsed to parse out the task parameters of the task to be executed. The task parameters include a task type and an address type, and may also include one or more of a task identifier, a task status, a manufacturer name, an order time, an order weight, etc. The task parameters are sent by the internal network system to the logistics gateway, that is, the internal network system sends a task instruction to the logistics gateway, and the logistics gateway can parse out the task parameters from the task instruction.

[0050] In an embodiment of the present invention, both the task parameters sent by the internal network system to the logistics gateway and the results returned by the external network system to the logistics gateway have undergone encryption processing by an encryption algorithm. The encryption algorithm can be the AES (Advanced Encryption Standard) encryption algorithm. That is, the task execution request sent by the logistics gateway is also encrypted. When the external network system receives the task execution request sent by the logistics gateway, it decrypts the task execution request. After successful decryption, the task parameters are parsed and the task is executed. After receiving the task instruction sent by the internal network system, the logistics gateway obtains the IP address of the internal network system. If the IP address of the internal network system is in the access whitelist of the logistics gateway, the internal network system has access rights. The requests sent and received by the logistics gateway can be in the jsf or http manner. Through the security mechanism of the logistics gateway, the security of the internal network system can be effectively guaranteed, and concurrent requests can be controlled.

[0051] In the embodiment of the present invention, before the intranet system sends a task instruction to the logistics gateway, it creates a task, creates a task according to task parameters, that is, creates a task according to the task name, manufacturer name, task type, address type, start and end time, weight, etc. Before creating the task, it checks whether there is a task of the same type as this task. If not, it creates a new task and generates a task identifier according to the rule. For example, it can generate a task identifier according to the manufacturer identifier and date, and stores the task information in the database; if it exists, it updates the task status. It can also create a batch task according to the task parameters, that is, generates a batch task identifier according to the task parameters and the specified rule, calculates the number of task executions according to the execution period and start and end time of the batch task, adds the batch task identifier to the batch task table, and creates tasks in a loop according to the newly added batch task identifier in the batch task table.

[0052] In the embodiment of the present invention, different manufacturer names correspond to different tasks to be started. For example, the manufacturer names include SN, SF, and DP. The SN task only supports starting tasks on the current day, the SF task only supports tasks within 24 hours, and the DP task only supports starting tasks within 3 days; and the start time of the SN task cannot be less than the current time. When determining the target data of the task to be executed, the target data can be determined according to the task type, manufacturer name, and address type.

[0053] In the embodiment of the present invention, determining the target data of the task to be executed according to the task type and address type includes:

[0054] Determine the network address data corresponding to the task to be executed according to the task type and address type, and write the network address data into the network address file;

[0055] Obtain the logistics data corresponding to the task to be executed according to the network address file;

[0056] Filter out the target data from the logistics data according to the preset rule.

[0057] In an embodiment of the present invention, the task types may include a first type and a second type. The first type may be the type of self-operated logistics, and the second type may be the type of third-party logistics. The address types include full-volume tasks and flow tasks. A full-volume task is a task for local full-volume address data, that is, the Cartesian product of provincial, municipal, and county-level addresses at the third level nationwide. A flow task is a task from a specified origin third-level address to a destination third-level address. According to whether the task type is the first type or the second type and whether the address type is a full-volume task or a flow task, the network address data corresponding to the task to be executed can be determined, such as url (uniform resource locator) address data, and then the network address data is written into a network address file, such as a url address file. The network address file is read, and the corresponding logistics data is obtained according to each network address data in the network address file. Then, the logistics data corresponding to each network address data is traversed item by item, and the target data is filtered out according to a preset rule. The preset rule may include one or more preset fields, and the target data is filtered out from the logistics data according to the one or more fields. The one or more fields may include the origin third-level address, the destination third-level address, the order placement time, the order placement weight, the expected delivery time, etc.

[0058] In an embodiment of the present invention, after determining the target data of the task to be executed according to the task type and the address type, uploading the target data to cloud storage may include: writing the target data into a text file, such as a csv (a file format) file, and the text file can be compressed and cut before being uploaded to cloud storage, such as OSS (a massive, secure, low-cost, and highly reliable cloud storage service). Compressing and cutting the text file can avoid the problem of upload timeout caused by excessive target data volume.

[0059] In an embodiment of the present invention, after the target data is successfully uploaded to cloud storage, a result indicating successful task execution is sent to the logistics gateway, so that the logistics gateway notifies the intranet system of the result, and the intranet system synchronizes the result.

[0060] In an embodiment of the present invention, the task type is the first type or the second type. Determining the network address data corresponding to the task to be executed according to the task type and the address type includes:

[0061] If the task type is the first type, obtain the network address corresponding to the first type, and convert the network address; determine the network address data according to the converted network address and the task type;

[0062] If the task type is the second type, determine the network address data according to the address type.

[0063] In an embodiment of the present invention, if the task type is the first type, the first type indicates a network address, such as a URL address. Then, the network address is converted according to the current timestamp, that is, the current timestamp is added to the network address to obtain the converted network address, so that each network address requests the latest data. Then, the network address data can be determined according to the converted network address and the task type. If the task type is the first type, the network address data can be directly determined according to the task type.

[0064] In an embodiment of the present invention, the address type is a full-volume task or a flow task. Determining the network address data according to the address type includes:

[0065] If the address type is a full-volume task, obtain the local full-volume address data, and determine the network address data according to the current timestamp and the local full-volume address data;

[0066] If the address type is a flow task, download the flow file corresponding to the flow task, and determine the network address data according to the current timestamp and the flow file.

[0067] In an embodiment of the present invention, if the address type is a full-volume task, then parse and obtain the full-volume address data, that is, the Cartesian product list of addresses. Then, perform network address conversion on the full-volume address data according to the current timestamp, that is, add the current timestamp to each network address in the full-volume address data to obtain the network address data. If the address type is a flow task, download the flow file corresponding to the flow task from cloud storage. The flow file includes each network address corresponding to the flow task. Parse the flow file to obtain each network address, and perform conversion on each network address according to the current timestamp, that is, add the current timestamp to each network address to obtain the network address data. Then, write the network address data into the network address file.

[0068] In an embodiment of the present invention, determining the network address data according to the current timestamp and the local full-volume address data can be to perform network address conversion on the full-volume address data according to information such as the current timestamp, the order placement time, and the order placement weight, that is, splice the current timestamp, the order placement time, the order placement weight, etc. on each network address in the full-volume address data to obtain the network address data.

[0069] Determining the network address data according to the current timestamp and the flow file can be to perform conversion on each network address of the flow file according to information such as the current timestamp, the order placement time, and the order placement weight, that is, splice the current timestamp, the order placement time, the order placement weight, etc. on each network address to obtain the network address data.

[0070] In an embodiment of the present invention, after obtaining logistics data corresponding to a to-be-executed task according to a network address file, the logistics data is written into a cache, such as a Redis cache, and the task progress of the to-be-executed task is updated. The logistics data is obtained from Redis, and the logistics data is processed according to a preset rule to filter out target data.

[0071] In an embodiment of the present invention, the task parameter includes a task status. Before determining the target data of the to-be-executed task according to the task type and the address type, it further includes: determining that the task status is a status indicating task start.

[0072] In an embodiment of the present invention, after receiving a task execution request sent by a logistics gateway, task parameters are parsed from the task execution request, and it is determined whether the task type is a first type or a second type. If it is the first type, a network address corresponding to the first type is obtained; then the task status is determined. If the task status is a task start status, that is, the intranet system starts the to-be-executed task, then preprocessing is performed on the to-be-executed task, including: recording the start time of the to-be-executed task and storing the task identifier of the to-be-executed task in the cache. If the task status is a task stop status, that is, the intranet system stops the to-be-executed task, then the to-be-executed task is terminated, the to-be-executed task is terminated by calling kill, and a request is sent to the intranet system through an interface to notify the termination of the execution of the to-be-executed task. Then the manufacturer name is determined, and the to-be-executed task is classified to start a task corresponding to the manufacturer name of the to-be-executed task. The first manufacturer of the first type of task.

[0073] In an embodiment of the present invention, after the intranet system receives a result indicating successful task execution, the task status and task progress of the to-be-executed task are updated. That is, after the intranet system obtains the target data from the cloud storage, the target data is visually displayed according to a preset dimension and a preset display method. The logistics gateway receives a notification request sent by an external network system, and the notification request includes a result indicating successful task execution. The logistics gateway sends a request to the intranet system, and the request includes a result indicating successful task execution. After receiving the request, the intranet system obtains the task identifier of the to-be-executed task corresponding to the result indicating successful task execution, queries the task information according to the task identifier, and updates the task status and task progress.

[0074] In the embodiment of the present invention, after the intranet system obtains the target data from the cloud storage, it visualizes the target data according to a preset dimension and a preset display mode. The preset display mode may include a table, a pie chart, a bar chart, a trend chart, etc. The preset dimension may include at least one of a three-level address, an order placement time, an order placement type, an estimated delivery time, etc. The table can display the three-level address + order placement time + weight + estimated delivery time data and the timeliness that can be queried according to different dimensions; the pie chart can screen the origin province and the destination province according to conditions, and count the order placement ratio. At the same time, the map supports counting the order placement ratio of the next-level address, which can be used to analyze and optimize the network layout; the multi-segment bar chart: screen the origin province and the destination province according to conditions, count the timeliness data of each city for a period of time, and display the proportion of different timeliness of each city, which can be used to analyze the timeliness changes in different time dimensions; the line chart: screen the origin district and the destination district according to conditions, count the timeliness data for a period of time, draw a line chart, and view the timeliness change trend.

[0075] In the embodiment of the present invention, after visualizing the target data according to the preset dimension and the preset display mode, a notification message may be sent to the user so that the user can view it in time, such as pushing a message, etc.

[0076] Figure 2 It is a schematic flowchart of a data processing method according to an embodiment of the present invention. The intranet management platform, i.e., the intranet system, creates a task by customizing task parameters, generates a task instruction according to a task start or stop instruction, encrypts the task instruction by using an encryption and decryption tool, and then sends the task instruction to the logistics gateway through the http method; the logistics gateway decrypts the message of the task instruction, and after signature verification and permission verification, sends a task execution request to the external network AI system through the http method. The external network AI system decrypts the task execution request by using an encryption and decryption tool, then parses the task instruction parameters from the task execution request, and grabs data according to the task instruction parameters by using an AI engine; analyzes the data by using the AI engine, writes the result into a file and uploads it to OSS, encrypts the result of successful execution by using an encryption and decryption tool, and then sends a request to the logistics gateway; the logistics gateway signs and distributes the request, and then sends a request to the intranet system to send the result of successful task execution to the intranet system; the intranet system decrypts the request by using an encryption and decryption tool, and updates the task progress according to the result of successful task execution; the intranet system downloads the task data from OSS, visualizes the task data, and obtains a visualization result.

[0077] Figure 3It is a schematic diagram of system interaction for data processing according to an embodiment of the present invention. The intranet system creates a task, encrypts and signs the task parameters, and then sends a request to start or terminate the task to the logistics gateway. The request includes the task parameters. After receiving the request, the logistics gateway performs signature verification and sends a gateway request to the AI engine of the extranet system. After receiving the gateway request, the AI engine decrypts the gateway request, verifies the accuracy of the message in the gateway request, and returns the task status and processing progress to the intranet system through the logistics gateway. The AI engine queues up to process the task, parses the task parameters from the gateway request, grabs and analyzes data according to the task parameters to obtain the result. The AI engine writes the result into a file and uploads it to OSS, and regularly requests the intranet interface through the logistics gateway to synchronize the task processing progress and task status to the intranet system.

[0078] Figure 4 It is a schematic flowchart of the data processing process of an intranet system according to an embodiment of the present invention. The data processing process includes:

[0079] After receiving the gateway request from the logistics gateway, use the encryption and decryption tool to decrypt the gateway request; then perform message parsing and serialization saving on the gateway request;

[0080] Process according to the message status respectively;

[0081] If the message status indicates that the task is in execution, query the task according to the task identifier, update the number of tasks in progress, and update the task status to in execution; if the message status indicates that the task execution fails, query the task according to the task identifier, update the task status to failed, update the task execution time, and asynchronously send a notification message; if the message status indicates that the task execution is successful, query the task according to the task identifier, obtain the OSS file address; download the corresponding file, decompress it and then re-upload the file and return the OSS address, update the task status to successful, and asynchronously send a notification message; if the message status indicates that the task is being parsed, query the task according to the task identifier, update the number of tasks in progress, and update the task status to being parsed;

[0082] The tasks in execution receive external network requests at regular intervals to update the task progress;

[0083] Download the result file of the successfully executed task. The result includes the three-level address, order placement time, order placement weight, and estimated delivery time. After parsing the file, write it into the database; perform visual display on the result and send a notification message to notify the user. Among them, the visual display includes tables, pie charts, multi-segment bar charts, and line charts.

[0084] Figure 5 It is a schematic flowchart of the data processing process of the AI engine of the extranet system according to an embodiment of the present invention. The data processing process includes:

[0085] After receiving the gateway request sent by the logistics gateway, use the encryption and decryption tool to decrypt the gateway request, parse the message in the gateway request, and obtain the task parameters;

[0086] Judge the task type; if tasktype = 0, the task type is the first type, and obtain the specified url parameter; if tasktype = 1, the task type is the second type;

[0087] Judge the task status. If taskstate = 7, it means that the intranet stops the task, then the extranet terminates the task. The extranet system calls the system kill to terminate the executing task, and calls the intranet interface of the logistics gateway to send a request to the intranet to notify the intranet; if taskstate = 1, perform the task preprocessing process, that is, the intranet starts the task, the extranet starts to process the task, records the task start time, and sets the key in the redis cache as the task identifier and the value as the task progress;

[0088] Judge the manufacturer name, and start the corresponding task according to the manufacturer name. If the manufacturer name is SN, start the SN task; if the manufacturer name is SF, start the SF task; if the manufacturer name is DP, start the DP task;

[0089] The task analysis engine updates the task status of the corresponding task;

[0090] Judge the task address range for different task types;

[0091] For the SN task, if the address type is a full - volume task and the local full - volume address is used, verify and parse the url and write it to a file; if the address type is a flow task, download the flow address file, verify and parse the url and write it to a file;

[0092] For the SF / DP task, if the address type is a full - volume task, use the local full - volume address information, read the address list to calculate the Cartesian product, convert the url address and write it to a file. If the address type is a flow task, download the flow address file, verify and parse the url address and write it to a file;

[0093] The task execution engine reads the url address file, uses the AI engine to grab data, writes the result to redis, and updates the task progress according to the task identifier;

[0094] The data analysis engine processes the data in redis according to the preset rules, filters out the valid data of the third - level address, order time, order weight, and expected delivery time, and writes the data to a csv file;

[0095] Compress and cut the file and upload it to oss, and send a notification message to the intranet system.

[0096] The data processing method according to an embodiment of the present invention, after receiving a task execution request sent by a logistics gateway, parses task parameters of a to-be-executed task from the task execution request, determines target data of the to-be-executed task according to the task type and address type in the task parameters, and uploads the target data to cloud storage, so that after the intranet system receives the result of successful task execution through the logistics network, it can obtain the target data from the cloud storage. This method realizes real-time cross-network parsing of the to-be-executed task through an AI engine by the logistics gateway, and synchronizes the target data of the to-be-executed task obtained to the intranet system, so that the operation personnel can obtain the target data through the intranet system, analyze the competitor data, optimize our logistics network, and improve the user experience.

[0097] According to still another aspect of the embodiments of the present invention, as Figure 6 shown, a data processing apparatus 600 is provided, including:

[0098] A receiving module 601, in response to receiving a task execution request sent by a logistics gateway, parses task parameters of a to-be-executed task from the task execution request; the task parameters are sent to the logistics gateway by the intranet system; the task parameters include a task type and an address type;

[0099] A determining module 602, determines target data of the to-be-executed task according to the task type and address type;

[0100] A sending module 603, sends the result indicating successful task execution to the intranet system through the logistics gateway, so that the intranet system can obtain the target data.

[0101] In the embodiment of the present invention, the determining module 602 is further configured to: determine network address data corresponding to the to-be-executed task according to the task type and address type, write the network address data into a network address file; obtain logistics data corresponding to the to-be-executed task according to the network address file; filter out the target data from the logistics data according to a preset rule.

[0102] In the embodiment of the present invention, the task type is the first type or the second type, and the determining module 602 is further configured to: if the task type is the first type, obtain a network address corresponding to the first type, perform conversion on the network address; determine the network address data according to the converted network address and the task type; if the task type is the second type, determine the network address data according to the address type.

[0103] In an embodiment of the present invention, the address type is a full-volume task or a flow task. The determination module 602 is further configured to: if the address type is a full-volume task, obtain local full-volume address data, and determine network address data according to the current timestamp and the local full-volume address data; if the address type is a flow task, download a flow file corresponding to the flow task, and determine network address data according to the current timestamp and the flow file.

[0104] In an embodiment of the present invention, the determination module 602 is further configured to: after determining the target data of the to-be-executed task according to the task type and the address type, upload the target data to cloud storage, so that the intranet system obtains the target data from the cloud storage and visually displays the target data according to a preset dimension and a preset display mode.

[0105] In an embodiment of the present invention, the task parameter includes a task status. The determination module 602 is further configured to: before determining the target data of the to-be-executed task according to the task type and the address type, determine that the task status is a status indicating task start.

[0106] In an embodiment of the present invention, after receiving a result indicating that the task execution is successful, the intranet system updates the task status and task progress of the to-be-executed task.

[0107] According to another aspect of the embodiments of the present invention, an electronic device is provided, including: one or more processors; a storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the data processing method provided by the present invention.

[0108] According to still another aspect of the embodiments of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the data processing method provided by the present invention is implemented.

[0109] Figure 7 An exemplary system architecture 700 to which the data processing method or data processing apparatus according to the embodiments of the present invention can be applied is shown.

[0110] As Figure 7 shown, the system architecture 700 may include terminal devices 701, 702, 703, a network 704, and a server 705. The network 704 is used to provide a medium for a communication link between the terminal devices 701, 702, 703 and the server 705. The network 704 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0111] Users can use terminal devices 701, 702, and 703 to interact with server 705 via network 704 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 701, 702, and 703, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (for example only).

[0112] Terminal devices 701, 702, and 703 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop computers, desktop computers, and so on.

[0113] Server 705 can be a server that provides various services, such as a background management server that supports shopping websites browsed by users using terminal devices 701, 702, and 703 (for example only). The background management server can analyze and process data such as product information query requests received, and feedback the processing results (such as target push information, product information - for example only) to the terminal device.

[0114] It should be noted that the data processing method provided by the embodiments of the present invention is generally executed by server 705. Correspondingly, the data processing device is generally set in server 705.

[0115] It should be understood that Figure 7 the numbers of terminal devices, networks, and servers in

[0116] are merely illustrative. According to actual needs, there can be any number of terminal devices, networks, and servers. Figure 8 Shown below with reference to Figure 8 is a schematic structural diagram of a computer system 800 of a terminal device suitable for implementing the embodiments of the present invention.

[0117] As Figure 8 shown, computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to programs stored in read-only memory (ROM) 802 or programs loaded from storage section 808 into random access memory (RAM) 803. In RAM 803, various programs and data required for the operation of system 800 are also stored. CPU 801, ROM 802, and RAM 803 are connected to each other via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0118] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is installed on the drive 810 as needed so that a computer program read therefrom is installed into the storage section 808 as needed.

[0119] Specifically, according to the embodiments disclosed by the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed by the present invention include a computer program product which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 809, and / or installed from the removable medium 811. When the computer program is executed by a central processing unit (CPU) 801, the above-described functions defined in the system of the present invention are executed.

[0120] It should be noted that the computer-readable medium shown in the present invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0121] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram can represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in a block can occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and the combination of blocks in a block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0122] The modules involved in the embodiments of the present invention can be implemented in software or in hardware. The described modules can also be provided in a processor. For example, it can be described as: a processor includes a receiving module, a determining module, and a sending module. Among them, the names of these modules do not constitute a limitation to the module itself in some cases. For example, the receiving module can also be described as "a module that parses the task parameters of the task to be executed from the task execution request in response to receiving the task execution request sent by the logistics gateway."

[0123] As another aspect, the present invention also provides a computer-readable medium, which can be included in the device described in the above embodiments; or it can exist alone without being assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device includes: parsing the task parameters of the task to be executed from the task execution request in response to receiving the task execution request sent by the logistics gateway; the task parameters are sent to the logistics gateway through the intranet system; the task parameters include a task type and an address type; determining the target data of the task to be executed according to the task type and the address type; and sending the result indicating the successful execution of the task to the intranet system through the logistics gateway so that the intranet system can obtain the target data.

[0124] According to the technical solution of the embodiments of the present invention, for the data processing method of the embodiments of the present invention, after receiving the task execution request sent by the logistics gateway, parse the task parameters of the task to be executed from the task execution request, determine the target data of the task to be executed according to the task type and the address type in the task parameters, and upload the target data to the cloud storage so that after the intranet system receives the result of the successful execution of the task through the logistics network, it can obtain the target data from the cloud storage. This method realizes cross-network real-time parsing of the task to be executed through the AI engine via the logistics gateway, and synchronizes the obtained target data of the task to be executed to the intranet system, so that the operation personnel can obtain the target data through the intranet system, analyze the competitor data, optimize our logistics network, and improve the user experience.

[0125] The above specific embodiments do not constitute a limitation to the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for data processing, characterized in that, including: In response to receiving a task execution request sent by the logistics gateway, parsing task parameters of the task to be executed from the task execution request; The task parameters are sent to the logistics gateway through the intranet system; the task parameters include a task type and an address type; Determining target data of the task to be executed according to the task type and the address type; Sending a result indicating successful task execution to the intranet system through the logistics gateway, so that the intranet system can obtain the target data.

2. The method according to claim 1, characterized in that Determining target data of the task to be executed according to the task type and the address type includes: Determining network address data corresponding to the task to be executed according to the task type and the address type, and writing the network address data into a network address file; Obtaining logistics data corresponding to the task to be executed according to the network address file; Filtering out the target data from the logistics data according to a preset rule.

3. The method according to claim 2, wherein The task type is the first type or the second type. Determining network address data corresponding to the task to be executed according to the task type and the address type includes: If the task type is the first type, obtaining a network address corresponding to the first type and converting the network address; determining the network address data according to the converted network address and the task type; If the task type is the second type, determining the network address data according to the address type.

4. The method according to claim 3, characterized in that The address type is a full-volume task or a flow task. Determining the network address data according to the address type includes: If the address type is a full-volume task, obtaining local full-volume address data, and determining the network address data according to the current timestamp and the local full-volume address data; If the address type is a flow task, downloading a flow file corresponding to the flow task, and determining the network address data according to the current timestamp and the flow file.

5. The method according to claim 1, wherein After determining the target data of the task to be executed according to the task type and the address type, further including: Uploading the target data to cloud storage, so that the intranet system can obtain the target data from the cloud storage and perform visual display on the target data according to a preset dimension and a preset display method.

6. The method according to claim 1, wherein The task parameters include a task status. Before determining the target data of the task to be executed according to the task type and the address type, further including: Determining that the task status is a status indicating task start.

7. The method according to claim 1, characterized in that, After receiving the result indicating successful task execution, the intranet system updates the task status and task progress of the task to be executed.

8. A data processing device, characterized in that, including: A receiving module, which, in response to receiving a task execution request sent by the logistics gateway, parses task parameters of the task to be executed from the task execution request; The task parameters are sent to the logistics gateway through the intranet system; the task parameters include a task type and an address type; A determining module, which determines target data of the task to be executed according to the task type and the address type; A sending module, which sends a result indicating successful task execution to the intranet system through the logistics gateway, so that the intranet system can obtain the target data.

9. An electronic device, characterized in that, including: One or more processors; A storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, the method according to any one of claims 1-7 is implemented.

Images