Cross-border trade big data platform operation task scheduling method, system and device

Abstract

The application discloses a cross-border trade big data platform operation task scheduling method, system and equipment, relates to the technical field of task scheduling, and establishes a shared scheduling graph between a plurality of operation tasks and a big data platform through receiving a plurality of operation tasks for cross-border trade data processing, updates a private scheduling path of each operation task on the shared scheduling graph based on an analysis result of each operation task, constructs a big data platform resource pool, determines demand scheduling resources of each operation task in the big data platform resource pool, determines a corresponding scheduling enclosure set of the operation task in the big data platform based on the demand scheduling resources, sets food sources and prey points in the scheduling enclosure set, determines foraging relationships between the food sources and the prey points, and executes final scheduling of each operation task based on the foraging relationships. The application realizes common scheduling decision of a plurality of operation tasks, matches the best scheduling resources for each operation task, and improves the overall scheduling efficiency of the operation task.

Description

Technical Field

[0001] This invention relates to the field of task scheduling technology, specifically to a method, system, and equipment for scheduling tasks on a cross-border trade big data platform. Background Technology

[0002] Cross-border trade involves numerous business processes and a long data chain. Its big data platform needs to process a large number of heterogeneous tasks in parallel, such as customs document review, logistics tracking, and cross-border payment reconciliation. These tasks differ significantly in terms of computational complexity, data dependence, and real-time requirements, which places dynamic and demanding requirements on underlying computing, storage, and network resources.

[0003] Existing task scheduling methods in cross-border trade big data platforms typically treat each task as an independent entity, lacking analysis of the inherent relationships between tasks. This leads to isolated scheduling decisions, making it difficult to achieve resource collaboration and sharing. At the same time, the resource allocation process is relatively crude, failing to accurately define the optimal resource search range for each task, resulting in blind matching and low efficiency, which affects overall operational efficiency. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, and equipment for scheduling tasks on a cross-border trade big data platform, in order to address the shortcomings in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for scheduling tasks on a cross-border trade big data platform, comprising the following steps:

[0006] Step S1: Receive several job tasks for cross-border trade data processing, establish a shared scheduling graph between several job tasks and the big data platform, and update the private scheduling path of each job task on the shared scheduling graph based on the parsing results of each job task.

[0007] Step S2: Construct a big data platform resource pool, determine the required scheduling resources for each job task in the big data platform resource pool, and determine the scheduling encirclement set corresponding to the job task on the big data platform based on the required scheduling resources;

[0008] Step S3: Set food sources and prey points within the scheduling encirclement set, determine the foraging relationship between food sources and prey points, and execute the final scheduling of each task based on the foraging relationship.

[0009] Furthermore, the process of receiving several task assignments for cross-border trade data processing and establishing a shared scheduling graph between these task assignments and the big data platform includes:

[0010] Several tasks are labeled, and the label is combined with the timestamp when the task is received as their respective identifiers. An integrated data area is set up on the big data platform to store the task scheduling model obtained after analyzing the data corresponding to several historical cross-border trade orders.

[0011] Based on the number of tasks, the integrated data area is divided into several sub-data areas of the corresponding number. In each sub-data area, several corresponding graph layers are created based on the number of task scheduling models. Each graph layer is used to store one task scheduling model.

[0012] The graph layer containing the task scheduling model is labeled as the scheduling layer;

[0013] Inter-layer communication interaction points are set between every two scheduling layers to establish data communication between different scheduling layers under the same sub-data area, and cross-domain communication interaction points are set to enable data communication between corresponding scheduling layers in different sub-data areas.

[0014] When all scheduling layers under the same sub-data region establish data communication, a shared scheduling graph of the corresponding job tasks in the big data platform is established. The shared scheduling graph of the current sub-data region is copied to other unoperated sub-data regions, and a shared scheduling graph of the job tasks in each sub-data region and the big data platform is established.

[0015] Furthermore, the process of updating the private scheduling path of each job task on the shared scheduling graph based on the parsing results of each job task includes:

[0016] A job time sequence window is set for each job task. The job time sequence window includes several time sequence slice sequences. The time sequence slice sequences are used to divide the job task into sub-stage tasks corresponding to a time period.

[0017] Data analysis is performed on sub-stage tasks to obtain corresponding stage data features. The stage data features of the same task corresponding to the sub-stage tasks in all time periods are integrated to obtain the global data features of the task as the analysis result.

[0018] A real-time scheduling model for job tasks is established based on global data characteristics. A local scheduling model for each time period is generated based on the real-time scheduling model. The real-time scheduling model of job tasks is then mapped to their respective common scheduling graphs.

[0019] For each scheduling layer in their respective shared scheduling graph, the time period of their respective task real-time scheduling model is divided into several scheduling slice windows. All scheduling slice windows in the same time period are merged into a set of scheduling path points. Based on the local scheduling model of each time period, it is determined whether there are suitable scheduling path points in the set of scheduling path points of the corresponding time period.

[0020] If so, then filter the scheduling path points to construct the scheduling path for the job task in the corresponding time period;

[0021] If not, create a new private path point, connect to the existing scheduling path in other time periods, and update it to the private scheduling path corresponding to the current job task.

[0022] Furthermore, the process of determining whether a suitable scheduling path exists in the set of scheduling path points for the corresponding time period includes:

[0023] When the local scheduling model of a task in a certain time period is consistent with the task scheduling model of any scheduling layer on the common scheduling graph in the corresponding time period, it is determined that there is a suitable scheduling path point in the corresponding time period.

[0024] If none of the scheduling layers can correspond to the local scheduling model for the current time period, it is determined that there is no suitable scheduling path point.

[0025] Furthermore, the process of constructing a big data platform resource pool and determining the resource scheduling requirements of each job task within the big data platform resource pool includes:

[0026] A number of resource allocation points are set up on the big data platform. These resource allocation points are used for the distribution of cross-border trade-related resources. A corresponding scheduling area is set up at each resource allocation point, thereby constructing several resource area sub-pools.

[0027] The dispatch area is the region where resource allocation points can be used for resource distribution.

[0028] Connect several resource area sub-pools to build the big data platform into a corresponding big data platform resource pool and obtain the task quota information for each job task;

[0029] Based on task quota information, resources are filtered in several resource sub-pools corresponding to the resource pool of the big data platform, the resource sub-pools related to each job task are locked, and all resources corresponding to the resource sub-pools are determined as the required scheduling resources for the corresponding job task.

[0030] Furthermore, the process of determining the set of scheduling encirclements for job tasks on the big data platform based on demand scheduling resources includes:

[0031] Set the resource acquisition distance for each job task, construct the resource capture circle corresponding to each job task based on the resource acquisition distance, and capture the resource area sub-pool within the resource capture circle as the scheduling object;

[0032] Using the midpoint of the communication distance between the task and the resource area sub-pool as the center, and half the corresponding communication distance as the radius, an encircling circle is established between each scheduling object and the task.

[0033] There are several enclosing circles of different sizes between different scheduling objects and tasks. By integrating the enclosing circles formed by the same task and different scheduling objects, a set of scheduling enclosing circles corresponding to the task on the big data platform can be constructed.

[0034] Furthermore, within the scheduling encirclement set, food sources and prey points are defined, the foraging relationship between food sources and prey points is determined, and the final scheduling process for each task based on the foraging relationship includes:

[0035] Mark each encirclement within the set of encirclements as a food source and the task itself as a prey point, and obtain the resource supply capacity value from each food source to the prey point;

[0036] Food sources are ranked based on their resource supply capacity values, and a foraging priority relationship between food sources and prey locations is established based on resource supply capacity evaluation values.

[0037] Based on foraging priority, a resource scheduling request is initiated from the prey point to the food source with the highest priority, and the corresponding required scheduling resources are locked;

[0038] When the highest priority food source cannot meet all the resource requirements of the task, the prey point initiates a scheduling request to the next highest priority food source according to the foraging priority order, until all the scheduling resources required for the task are successfully locked. Based on the finally locked scheduling resources, the task is driven to perform the final scheduling on the big data platform.

[0039] Furthermore, the present invention also provides a cross-border trade big data platform operation task scheduling system, the system comprising:

[0040] The scheduling data processing module receives several tasks for cross-border trade data processing, establishes a shared scheduling graph between the tasks and the big data platform, and updates the private scheduling path of each task on the shared scheduling graph based on the parsing results of each task.

[0041] The scheduling resource determination module constructs a big data platform resource pool, determines the required scheduling resources for each job task in the big data platform resource pool, and determines the set of scheduling encirclements for the job task on the big data platform based on the required scheduling resources.

[0042] The scheduling and execution module sets up food sources and prey points within the scheduling encirclement set, determines the foraging relationship between food sources and prey points, and executes the final scheduling of each task based on the foraging relationship.

[0043] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-mentioned cross-border trade big data platform task scheduling method.

[0044] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0045] 1. This invention establishes a shared scheduling graph between job tasks and a big data platform, incorporating previously isolated job tasks into a shared graph model set. This enables the system to analyze the relationships between different job tasks, construct a big data platform resource pool, determine the scheduling resources required for each job task, generate a corresponding scheduling encirclement set, and define the optimal resource search range for each job task on the big data plane. This reduces the blindness and computational overhead in the resource matching process and improves the scheduling efficiency of big data platform resources.

[0046] 2. Abstract the supply and demand sides of resources as food sources and prey points respectively, establish the foraging relationship between food sources and prey points, so that each task seeks the most suitable resources to initiate scheduling. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0048] Figure 1 This is a flowchart of the method of the present invention.

[0049] Figure 2 This is a schematic diagram illustrating the construction process of the shared scheduling graph in this invention.

[0050] Figure 3 This is a system block diagram of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Please see Figure 1 As shown, the task scheduling method for the cross-border trade big data platform includes the following steps:

[0053] Step S1: Receive several job tasks for cross-border trade data processing, establish a shared scheduling graph between several job tasks and the big data platform, and update the private scheduling path of each job task on the shared scheduling graph based on the parsing results of each job task.

[0054] Step S2: Construct a big data platform resource pool, determine the required scheduling resources for each job task in the big data platform resource pool, and determine the scheduling encirclement set corresponding to the job task on the big data platform based on the required scheduling resources;

[0055] Step S3: Set food sources and prey points within the scheduling encirclement set, determine the foraging relationship between food sources and prey points, and execute the final scheduling of each task based on the foraging relationship.

[0056] It should be further explained that, in the specific implementation process, the process of receiving several operational tasks for cross-border trade data processing and establishing a shared scheduling graph between these operational tasks and the big data platform includes:

[0057] Set a receiving period. During the receiving period, receive several tasks for cross-border trade data processing and label the tasks. Let the label be i, then i = 1, 2, 3, ..., n, where n is a natural number greater than 0.

[0058] The task number and the corresponding timestamp at the time of receipt are combined to form their respective identity identifiers, and a task is uniquely identified by the identity identifier.

[0059] An integrated data area is set up on the big data platform. The integrated data area is used to store several task scheduling models obtained after analyzing the data corresponding to several historical cross-border trade orders. Each task scheduling model corresponds to a detailed scheduling process of cross-border trade.

[0060] Based on the number of tasks, the integrated data area is divided into several sub-data areas of the corresponding number. In each sub-data area, several corresponding graph layers are created based on the number of task scheduling models. Each graph layer is used to store one task scheduling model.

[0061] It should be noted that the task scheduling models for cross-border trade in different actual scenarios include, but are not limited to, the following: the customs declaration document processing scheduling model, which is used to handle intensive document review tasks in the customs declaration process. When a new customs declaration document processing task enters the system, this model will be activated. Its detailed scheduling process is as follows: the task is first assigned computing nodes with high-performance OCR (Optical Character Recognition) and natural language processing capabilities to quickly extract key fields (such as commodity codes, amounts, and quantities) from documents such as invoices, packing lists, and certificates of origin. Then, the model will guide the task to access the storage area that integrates the customs commodity classification database and the trade control policy database to perform automatic data comparison and compliance verification. Throughout the process, the model defines strict data flow and calculation order, and presets an exception handling path that automatically diverts the task to the manual review queue when the recognition confidence is lower than the threshold.

[0062] Logistics trajectory tracking and scheduling model: This model is designed for complex tasks in cross-border logistics, such as real-time tracking of massive parcel trajectories and prediction of arrival times. The scheduling task establishes network connections with the API interfaces of multiple logistics providers to continuously acquire the latest logistics status information. The computing cluster performs real-time cleaning, correlation, and aggregation of the latest logistics status information. The scheduling task accesses the historical logistics timeliness database and calls machine learning prediction algorithms deployed on high-performance GPU nodes. It dynamically calculates the estimated delivery time by considering external variables such as the current logistics node, transportation mode, historical performance, and weather.

[0063] Cross-border payment detection and scheduling model: Applied to the payment process in cross-border trade, it is used to identify and intercept suspicious transactions in real time. When a cross-border trade payment transaction occurs, the model schedules a task to route it to in-memory computing resources for response, calls a rule engine pre-deployed in the network environment, and executes a series of predefined risk control rules (such as single transaction amount limit, frequency limit). At the same time, the model schedules tasks in parallel to quickly obtain the user's historical transaction network relationship and analyze abnormal patterns of transaction behavior in real time.

[0064] The graph layer containing the task scheduling model is labeled as the scheduling layer;

[0065] Inter-layer communication interaction points are set between every two scheduling layers. Inter-layer communication interaction points are used to establish data communication between different scheduling layers under the same sub-data area. Cross-domain communication interaction points are set. Cross-domain communication interaction points are used to enable data communication between corresponding scheduling layers of different sub-data areas.

[0066] When all scheduling layers under the same sub-data region establish data communication, a shared scheduling graph of the corresponding job tasks in the big data platform is established. The shared scheduling graph of the current sub-data region is copied to other unoperated sub-data regions, thereby establishing a shared scheduling graph of the job tasks in each sub-data region and the big data platform.

[0067] For a schematic diagram of the construction process of the shared scheduling graph, please refer to [link / reference needed]. Figure 2 As shown.

[0068] It should be further explained that, in the specific implementation process, the process of updating the private scheduling path on the shared scheduling graph based on the parsing results of each job task includes:

[0069] A job time sequence window is set for each job task. The job time sequence window includes several time sequence slice sequences. Each time sequence slice sequence is used to divide the job task into a sub-stage task corresponding to a time period.

[0070] Data analysis is performed on the sub-stage tasks of each time period to obtain the stage data features of the corresponding sub-stage tasks. The stage data features of the same task in all time periods are integrated to obtain the global data features of the task, and the global data features are used as the corresponding analysis results.

[0071] Based on the global data characteristics, a real-time scheduling model for the corresponding job tasks is established, and a local scheduling model is established on the real-time scheduling model for each sub-stage task. The local scheduling model is used to characterize the relevant process of scheduling job tasks in a certain period of time.

[0072] The real-time scheduling model of each task is mapped to its respective common scheduling graph. For each scheduling layer in the common scheduling graph, the time period of each task's real-time scheduling model is divided into several scheduling slice windows.

[0073] All scheduling slice windows in the same time period are merged into a scheduling path point set, thereby obtaining several scheduling path point sets corresponding to different time periods. Based on the local scheduling model of the job task in each time period, it is determined whether there is a suitable scheduling path point in the scheduling path point set of the corresponding time period.

[0074] If so, then filter the scheduling path points to construct the scheduling path for the job task in the corresponding time period;

[0075] If not, create a new private path point, connect to the existing scheduling path in other time periods, and update it to the private scheduling path corresponding to the current job task;

[0076] Specifically, if the local scheduling model of a task in a certain time period is consistent with the task scheduling model of any scheduling layer on the common scheduling graph in the corresponding time period, it is determined that there is a suitable scheduling path point in the corresponding time period; if none of the scheduling layers can correspond to the local scheduling model of the current time period, it is determined that there is no suitable scheduling path point.

[0077] It should be further explained that, in the specific implementation process, the process of constructing a big data platform resource pool and determining the resource scheduling requirements of each job task within the big data platform resource pool includes:

[0078] A number of resource allocation points are set up on the big data platform. These resource allocation points are used for the distribution of cross-border trade-related resources. A corresponding scheduling area is set up at each resource allocation point, thereby constructing several resource area sub-pools.

[0079] The scheduling area is the region where resource allocation points can be used for resource distribution.

[0080] By connecting several resource area sub-pools, the big data platform is constructed into a corresponding big data platform resource pool, and the task quota information of each job task is obtained. The task quota information is a set representing the upper limit of resources that the job task can use during execution. It serves as the data basis for subsequent resource allocation and management, ensuring fair resource sharing among various job tasks and preventing operation crashes caused by excessive resource consumption of a single job task.

[0081] Task quota information includes computing resource quotas, storage resource quotas, and network resource quotas;

[0082] Computing resource quotas include CPU quotas and memory quotas. CPU quotas define the maximum number of virtual CPU cores that a job can use and the CPU usage time. Memory quotas consist of two parts: on-heap memory and off-heap memory.

[0083] The total memory available to a job is the sum of on-heap memory and off-heap memory.

[0084] Storage resource quotas include disk space quotas and I / O bandwidth quotas;

[0085] The disk space quota is further divided into temporary space and output space. Job tasks are executed in the temporary space, and the execution results of the corresponding job tasks are output in the output space. This method of executing job tasks in the temporary space and outputting results in the output space prevents the disk from being filled up.

[0086] The I / O bandwidth quota is used to limit the disk read / write throughput of job tasks to a preset safe throughput range, thereby preventing high I / O job tasks from affecting the operation of other job tasks with normal throughput.

[0087] The network resource quota includes the network bandwidth available for each job task, and the upper limit of the number of access interfaces corresponding to each job task when making a connection. The job task obtains the network resources it lacks from the outside through the access interface.

[0088] Based on task quota information, resources are filtered in several resource sub-pools corresponding to the resource pool of the big data platform, thereby locking in the resource sub-pool related to each job task, and determining all resources corresponding to the resource sub-pool as the required scheduling resources for the corresponding job task.

[0089] The computing, storage, and network resources in each resource region sub-pool satisfy the resource usage of several job tasks within the corresponding resource region sub-pool. By matching the task quota information of the job tasks with the resources in the resource region sub-pool, corresponding demand scheduling resources are provided for each job task.

[0090] It should be further explained that, in the specific implementation process, the process of determining the set of scheduling encirclements for job tasks on the big data platform based on demand scheduling resources includes:

[0091] Set the resource acquisition distance for each task, and construct a resource grabbing circle for each task based on the resource acquisition distance. The resource grabbing circle is constructed with the storage location of the task on the big data platform as the center and the resource acquisition distance as the radius.

[0092] The resource acquisition distance is set by scheduling resources according to the requirements of the job task;

[0093] It should be noted that resource acquisition distance is a comprehensive logical quantitative indicator, rather than a physical spatial distance. Its specific settings include: scheduling the type and attributes of resources based on the locked requirements of the job task, and extracting key influencing factors. Key influencing factors include, but are not limited to: for computing resources, considering the network latency and bandwidth cost between them and the task execution location; for storage resources, considering the time overhead of data migration and I / O access latency; and for network resources, considering the time it takes to establish a connection and transmission stability.

[0094] Each key influencing factor is assigned a standardized cost weight. Based on the specific attribute values ​​of the resources to be scheduled, a quantified resource acquisition distance is output through a preset distance calculation function. The larger the value, the higher the overall cost of acquiring and allocating resources from that resource location to the task execution location; conversely, the lower the overall cost. The resource acquisition distance obtained through multi-dimensional data reflects the efficiency of resource scheduling for job tasks at different resource locations.

[0095] The resource area sub-pool within the resource capture circle is captured as the scheduling object. The midpoint of the communication distance between the job task and the resource area sub-pool is taken as the center, and half the value of the corresponding communication distance is taken as the radius. An encircling circle is established between each scheduling object and the job task.

[0096] Furthermore, there are several enclosing circles of different sizes between different scheduling objects and tasks. By integrating the enclosing circles formed by the same task and different scheduling objects, a set of scheduling enclosing circles corresponding to the task on the big data platform can be constructed.

[0097] It should be further explained that, in the specific implementation process, the final scheduling process for each task, which involves setting up food sources and prey points within the scheduling encirclement set, determining the foraging relationship between food sources and prey points, and executing each task based on the foraging relationship, includes:

[0098] Mark the encirclement corresponding to each resource region sub-pool within the scheduling encirclement set as a food source, and mark the task itself as the prey point, to obtain the resource supply capacity value from each food source to the prey point;

[0099] Food sources are sorted based on their resource supply capacity values. A foraging priority relationship between food sources and prey points is established based on the resource supply capacity evaluation value. Based on the foraging priority relationship, the prey point initiates a resource scheduling request to the food source with the highest priority and locks the corresponding demand scheduling resources.

[0100] When the highest priority food source cannot meet all the resource requirements of the task, the prey point initiates a scheduling request to the next highest priority food source according to the foraging priority order, until all the scheduling resources required for the task are successfully locked. Based on the finally locked scheduling resources, the task is driven to perform the final scheduling on the big data platform.

[0101] It should be noted that the resource supply capacity value is used to quantitatively evaluate the comprehensive score of a food source (i.e., a resource area sub-pool)'s ability to meet the resource requirements of a specific prey point (i.e., a task). The process of obtaining the resource supply capacity value is as follows: collect the current status data of the food source, including resource sufficiency (such as the number of available CPU cores, the amount of remaining memory, and whether the free disk space meets the respective task quotas), resource performance status (such as the current average I / O throughput and network bandwidth utilization), and its health and stability indicators (such as historical failure rate and current load level).

[0102] These status data are matched and analyzed with the task quota information and real-time demand characteristics of the prey points. The matching and analysis process is completed by a comprehensive evaluation model, which assigns different importance coefficients to status data of different dimensions and performs weighted calculations.

[0103] For example, for a computationally intensive task, the sufficiency and performance of computing resources will be given higher weight. The comprehensive evaluation model outputs a quantitative score as a resource supply capacity value. The higher the value, the better and more efficiently the food source can meet the resource needs of the prey point, and the higher its position in the foraging priority ranking. The foraging priority relationship is a dynamic ranking relationship established based on the descending order of resource supply capacity values. The prey point will initiate resource scheduling requests to the food source in turn according to this ranking.

[0104] Please see Figure 3 As shown, the present invention also provides a cross-border trade big data platform task scheduling system, which includes:

[0105] The scheduling data processing module receives several tasks for cross-border trade data processing, establishes a shared scheduling graph between the tasks and the big data platform, and updates the private scheduling path of each task on the shared scheduling graph based on the parsing results of each task.

[0106] The scheduling resource determination module constructs a big data platform resource pool, determines the required scheduling resources for each job task in the big data platform resource pool, and determines the set of scheduling encirclements for the job task on the big data platform based on the required scheduling resources.

[0107] The scheduling and execution module sets up food sources and prey points within the scheduling encirclement set, determines the foraging relationship between food sources and prey points, and executes the final scheduling of each task based on the foraging relationship.

[0108] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-mentioned cross-border trade big data platform task scheduling method.

[0109] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for scheduling tasks on a cross-border trade big data platform, characterized in that, Includes the following steps: Step S1: Receive several job tasks for cross-border trade data processing, establish a shared scheduling graph between several job tasks and the big data platform, and update the private scheduling path of each job task on the shared scheduling graph based on the parsing results of each job task. The process of establishing a shared scheduling graph includes: labeling several tasks, merging the label with the timestamp when the task was received as their respective identifiers, setting up an integrated data area on the big data platform, and storing the task scheduling model obtained after analyzing the data corresponding to several historical cross-border trade orders. Based on the number of tasks, the integrated data area is divided into several sub-data areas of the corresponding number. In each sub-data area, several corresponding graph layers are created based on the number of task scheduling models. Each graph layer is used to store one task scheduling model. The graph layer containing the task scheduling model is labeled as the scheduling layer; Inter-layer communication interaction points are set between every two scheduling layers to establish data communication between different scheduling layers under the same sub-data area, and cross-domain communication interaction points are set to enable data communication between corresponding scheduling layers in different sub-data areas. When all scheduling layers under the same sub-data area establish data communication, a shared scheduling graph of the corresponding job tasks in the big data platform is established. The shared scheduling graph of the current sub-data area is copied to other unoperated sub-data areas, and a shared scheduling graph of the job tasks in each sub-data area and the big data platform is established. Step S2: Construct a big data platform resource pool, determine the required scheduling resources for each job task in the big data platform resource pool, and determine the scheduling encirclement set corresponding to the job task on the big data platform based on the required scheduling resources; A number of resource allocation points are set up on the big data platform for the distribution of cross-border trade-related resources. A scheduling area is set up at each resource allocation point, thereby constructing several resource area sub-pools. The process of determining the set of scheduling encirclements includes: Set the resource acquisition distance for each job task, construct the resource capture circle corresponding to each job task based on the resource acquisition distance, and capture the resource area sub-pool within the resource capture circle as the scheduling object; Using the midpoint of the communication distance between the task and the resource area sub-pool as the center, and half the corresponding communication distance as the radius, an encircling circle is established between each scheduling object and the task. There are several enclosing circles of different sizes between different scheduling objects and tasks. By integrating the enclosing circles formed by the same task and different scheduling objects, a set of scheduling enclosing circles corresponding to the task on the big data platform can be constructed. Step S3: Set food sources and prey points within the scheduling encirclement set, determine the foraging relationship between food sources and prey points, and execute the final scheduling of each task based on the foraging relationship.

2. The cross-border trade big data platform task scheduling method according to claim 1, characterized in that, The process of updating each job's private scheduling path on the shared scheduling graph based on the parsing results of each job task includes: A job time sequence window is set for each job task. The job time sequence window includes several time sequence slice sequences. The time sequence slice sequences are used to divide the job task into sub-stage tasks corresponding to a time period. Data analysis is performed on sub-stage tasks to obtain corresponding stage data features. The stage data features of the same task corresponding to the sub-stage tasks in all time periods are integrated to obtain the global data features of the task as the analysis result. A real-time scheduling model for job tasks is established based on global data characteristics. A local scheduling model for each time period is generated based on the real-time scheduling model. The real-time scheduling model of job tasks is then mapped to their respective common scheduling graphs. For each scheduling layer in their respective shared scheduling graph, the time period of their respective task real-time scheduling model is divided into several scheduling slice windows. All scheduling slice windows in the same time period are merged into a set of scheduling path points. Based on the local scheduling model of each time period, it is determined whether there are suitable scheduling path points in the set of scheduling path points of the corresponding time period. If so, then filter the scheduling path points to construct the scheduling path for the job task in the corresponding time period; If not, create a new private path point, connect to the existing scheduling path in other time periods, and update it to the private scheduling path corresponding to the current job task.

3. The cross-border trade big data platform task scheduling method according to claim 2, characterized in that, The process of determining whether a suitable scheduling path exists in the set of scheduling path points for the corresponding time period includes: When the local scheduling model of a task in a certain time period is consistent with the task scheduling model of any scheduling layer on the common scheduling graph in the corresponding time period, it is determined that there is a suitable scheduling path point in the corresponding time period. If none of the scheduling layers can correspond to the local scheduling model for the current time period, it is determined that there is no suitable scheduling path point.

4. The cross-border trade big data platform task scheduling method according to claim 3, characterized in that, The process of constructing a big data platform resource pool and determining the resource scheduling requirements of each job task within the big data platform resource pool includes: The dispatch area is the region where resource allocation points can be used for resource distribution. Connect several resource area sub-pools to build the big data platform into a corresponding big data platform resource pool and obtain the task quota information for each job task; Based on task quota information, resources are filtered in several resource sub-pools corresponding to the resource pool of the big data platform, the resource sub-pools related to each job task are locked, and all resources corresponding to the resource sub-pools are determined as the required scheduling resources for the corresponding job task.

5. The cross-border trade big data platform task scheduling method according to claim 4, characterized in that, The final scheduling process, which involves defining food sources and prey locations within the scheduling encirclement set, determining the foraging relationships between food sources and prey locations, and executing each task based on these foraging relationships, includes: Mark each encirclement within the set of encirclements as a food source and the task itself as a prey point, and obtain the resource supply capacity value from each food source to the prey point; Food sources are ranked based on their resource supply capacity values, and a foraging priority relationship between food sources and prey locations is established based on resource supply capacity evaluation values. Based on foraging priority, a resource scheduling request is initiated from the prey point to the food source with the highest priority, and the corresponding required scheduling resources are locked; When the highest priority food source cannot meet all the resource requirements of the task, the prey point initiates a scheduling request to the next highest priority food source according to the foraging priority order, until all the scheduling resources required for the task are successfully locked. Based on the finally locked scheduling resources, the task is driven to perform the final scheduling on the big data platform.

6. A cross-border trade big data platform task scheduling system, used to implement the big data platform task scheduling method according to any one of claims 1 to 5, characterized in that, The system includes: The scheduling data processing module receives several tasks for cross-border trade data processing, establishes a shared scheduling graph between the tasks and the big data platform, and updates the private scheduling path of each task on the shared scheduling graph based on the parsing results of each task. The scheduling resource determination module constructs a big data platform resource pool, determines the required scheduling resources for each job task in the big data platform resource pool, and determines the set of scheduling encirclements for the job task on the big data platform based on the required scheduling resources. The scheduling and execution module sets up food sources and prey points within the scheduling encirclement set, determines the foraging relationship between food sources and prey points, and executes the final scheduling of each task based on the foraging relationship.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

Images