Cross-border trade big data platform job task scheduling method, system and equipment

Abstract

The invention discloses a cross-border trade big data platform job task scheduling method, system and device, and relates to the technical field of task scheduling. A plurality of job tasks for cross-border trade data processing are received, and a common scheduling graph between the plurality of job tasks and a big data platform is established; and updating a private scheduling path of each job task on the common scheduling graph based on an analysis result of each job task, constructing a big data platform resource pool, determining a demand scheduling resource of each job task in the big data platform resource pool, determining a scheduling encirclement set corresponding to the job task on the big data platform based on the demand scheduling resource, and scheduling the job task on the big data platform according to the scheduling encirclement set. And setting a food source and a prey point in the scheduling encirclement set, determining a foraging relationship between the food source and the prey point, and executing final scheduling of each job task based on the foraging relationship. The joint scheduling decision of a plurality of job tasks is realized, the optimal scheduling resource is matched for each job task, and the overall scheduling efficiency of the job tasks is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of task scheduling, in particular to a cross-border trade big data platform job task scheduling method, system and device. BACKGROUND

[0002] The cross-border trade business links are numerous, and the data chain is long. Its big data platform needs to process a large number of heterogeneous job tasks in parallel, such as customs declaration document review, logistics track tracking, cross-border payment reconciliation, etc. These tasks have significant differences in computational complexity, data dependency and real-time requirements, and put forward dynamic and stringent demands on underlying computing, storage and network resources.

[0003] The existing cross-border trade big data platform job task scheduling method usually treats each task as an independent individual for processing, lacks analysis of the internal correlation between tasks, resulting in isolated scheduling decisions, making it difficult to achieve resource collaborative sharing. At the same time, the resource allocation process is relatively extensive, and the best resource search range of each task cannot be accurately determined, resulting in matching blindness and low efficiency, affecting the overall job efficiency. SUMMARY

[0004] The purpose of the present application is to provide a cross-border trade big data platform job task scheduling method, system and device to solve the problems in the background art.

[0005] In order to achieve the above purpose, the present application provides the following technical scheme: a cross-border trade big data platform job task scheduling method, comprising the following steps: Step S1: receiving a plurality of job tasks for cross-border trade data processing, establishing a common scheduling graph between the plurality of job tasks and the big data platform, and updating the private scheduling path of each job task on the common scheduling graph based on the analysis result of each job task; Step S2: constructing a big data platform resource pool, determining the demand scheduling resource of each job task in the big data platform resource pool, and determining the scheduling enclosure set of the job task corresponding to the big data platform based on the demand scheduling resource; Step S3: setting food sources and prey points in the scheduling enclosure set, determining the foraging relationship between the food sources and the prey points, and executing the final scheduling of each job task based on the foraging relationship.

[0006] Further, the process of receiving a plurality of job tasks for cross-border trade data processing and establishing a common scheduling graph between the plurality of job tasks and the big data platform comprises: Labeling the plurality of job tasks, combining the label with the time stamp when receiving the job task as the identity of each job task, setting an integrated data area on the big data platform, and the integrated data area is used to store the task scheduling model obtained after data analysis of a plurality of historical cross-border trade orders. based on the number of task scheduling models, a corresponding number of atlas layers are created in each sub-data area, and each atlas layer is used to store one task scheduling model; The atlas layer storing the task scheduling model is marked as a scheduling layer. An inter-layer communication interaction point is set between every two scheduling layers to establish data intercommunication of different scheduling layers under the same sub-data area, and a cross-domain communication interaction point is set to establish data intercommunication of corresponding scheduling layers of different sub-data areas. When all scheduling layers under the same sub-data area establish data intercommunication, a common scheduling atlas of the corresponding job task in the big data platform is established, and the common scheduling atlas of the current sub-data area is copied to other non-operated sub-data areas, and a common scheduling atlas of the corresponding job task and the big data platform is established for each sub-data area.

[0007] Further, the process of updating the private scheduling path of each job task on the common scheduling atlas based on the analysis result of each job task includes: A job timing window is set for each job task, which includes a plurality of timing slice sequences, and the timing slice sequences are used to divide the job task into sub-phase tasks in a corresponding time period. The sub-phase tasks are analyzed to obtain corresponding phase data features, and the phase data features of the sub-phase tasks of the same job task in all time periods are integrated to obtain the global data features of the job task as the analysis result. A task real-time scheduling model of the job task is established according to the global data features, a local scheduling model corresponding to each time period is generated based on the task real-time scheduling model, and the task real-time scheduling model of the job task is mapped into the respective common scheduling atlas. For each scheduling layer in the respective common scheduling atlas, the time period is divided into a plurality of scheduling slice windows based on the respective task real-time scheduling model, all scheduling slice windows of the same time period are combined into a scheduling path point set, and based on the local scheduling model of each time period, it is judged whether there is an adaptive scheduling path point in the scheduling path point set of the corresponding time period. If yes, the scheduling path point is filtered to construct the scheduling path of the job task in the corresponding time period. If not, a new private path point is created, connected to the existing scheduling path of other time periods, and updated as the private scheduling path corresponding to the current job task.

[0008] Further, the process of judging whether there is an adaptive scheduling path point in the scheduling path point set of the corresponding time period includes: When the local scheduling model of a job task in a time period is consistent with the task scheduling model of any scheduling layer on the common scheduling graph in the corresponding time period, it is determined that there is an adaptive scheduling path point in the corresponding time period; If all scheduling layers cannot correspond to the local scheduling model of the current time period, it is determined that there is no adaptive scheduling path point.

[0009] Further, the process of determining the demand scheduling resource of each job task in the big data platform resource pool includes: A number of resource allocation points are set on the big data platform, and the resource allocation points are used for cross-border trade related resource distribution. A corresponding scheduling area is set at each resource allocation point, and then a plurality of resource area sub-pools are constructed; The scheduling area is the area range available for resource distribution of the resource allocation point; Connect a plurality of resource area sub-pools to construct the big data platform into a corresponding big data platform resource pool, and obtain the task quota information of each job task; Based on the task quota information, the resource is screened in the corresponding plurality of resource area sub-pools of the big data platform resource pool, the resource area sub-pool related to each job task is locked, and all resources corresponding to the resource area sub-pool are determined as the demand scheduling resource of the corresponding job task.

[0010] Further, the process of determining the demand scheduling resource of each job task in the big data platform resource pool includes: Set the resource acquisition distance of each job task, and construct the resource acquisition circle corresponding to each job task based on the resource acquisition distance. The resource area sub-pool within the resource acquisition circle range is grabbed as a scheduling object; The middle position between the communication distance of the job task and the resource area sub-pool is taken as the center, and the value of half of the corresponding communication distance is taken as the radius to establish a surrounding circle between each scheduling object and the job task; There are a plurality of surrounding circles with different areas between different scheduling objects and the job task. The plurality of surrounding circles formed by the same job task and different scheduling objects are integrated to obtain the scheduling surrounding circle set corresponding to the job task on the big data platform.

[0011] Further, the process of setting food sources and prey points in the scheduling surrounding circle set, determining the foraging relationship between the food sources and the prey points, and executing the final scheduling of each job task based on the foraging relationship includes: Each surrounding circle in the scheduling surrounding circle set is marked as a food source, and the job task itself is marked as a prey point to obtain the resource supply capability value from each food source to the prey point; Sort the food sources based on the size of the resource supply capability value, and establish a foraging priority relationship between the food sources and the prey points based on the resource supply capability evaluation value; Based on the foraging priority relationship, the prey point initiates a resource scheduling request to the food source with the highest priority, and locks the corresponding demand scheduling resource; When the highest priority food source cannot meet the entire resource demand of the operation task, the prey point initiates a scheduling request to the next priority food source according to the foraging priority order until the entire demand scheduling resource of the operation task is successfully locked, and based on the finally locked entire demand scheduling resource, the operation task is driven to execute the final scheduling on the big data platform.

[0012] Further, the application also provides a cross-border trade big data platform operation task scheduling system, which comprises: The scheduling data processing module receives a plurality of operation tasks for cross-border trade data processing, establishes a common scheduling graph between the plurality of operation tasks and the big data platform, and updates the private scheduling path of each operation task on the common scheduling graph based on the analysis result of each operation task; The scheduling resource determination module constructs a big data platform resource pool, determines the demand scheduling resource of each operation task in the big data platform resource pool, and determines the scheduling enclosure set corresponding to the operation task in the big data platform based on the demand scheduling resource; The scheduling execution module sets food sources and prey points in the scheduling enclosure set, determines the foraging relationship between the food sources and the prey points, and executes the final scheduling of each operation task based on the foraging relationship.

[0013] A computer device comprising a memory and a processor, the memory storing a computer program, and the processor executing the above-mentioned computer program to realize the steps in the above-mentioned cross-border trade big data platform operation task scheduling method.

[0014] In the above technical solution, the application provides technical effects and advantages: 1、The application establishes a common scheduling graph between the operation tasks and the big data platform, and integrates the originally isolated operation tasks into a shared graph model set, so that the system can analyze the association between different operation tasks, construct a big data platform resource pool and determine the demand scheduling resource of each operation task, generate a corresponding scheduling enclosure set, and define the optimal resource search range of each operation task on the big data platform, thereby reducing the blindness and computational overhead in the resource matching process and improving the scheduling efficiency of the big data platform resources; 2、The resource supply and demand are respectively abstracted as food sources and prey points, and the foraging relationship between the food sources and the prey points is established, so that each operation task seeks the most suitable resource to initiate scheduling. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed in the embodiments will be briefly introduced as follows. Obviously, the accompanying drawings in the following description only represent some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained based on these drawings.

[0016] Figure 1 The method flowchart of the present application.

[0017] Figure 2 The construction flowchart of the common scheduling atlas in the present application.

[0018] Figure 3 The system block diagram of the present application. DETAILED DESCRIPTION

[0019] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0020] Please refer to Figure 1 As shown in the figure, the cross-border trade big data platform job task scheduling method comprises the following steps: Step S1: receiving a plurality of job tasks for cross-border trade data processing, establishing a common scheduling atlas between the plurality of job tasks and a big data platform, and updating a private scheduling path of each job task on the common scheduling atlas based on an analysis result of each job task; Step S2: constructing a big data platform resource pool, determining the required scheduling resources of each job task in the big data platform resource pool, and determining a scheduling enclosure set corresponding to the job task in the big data platform based on the required scheduling resources; Step S3: setting a food source and a prey point in the scheduling enclosure set, determining a foraging relationship between the food source and the prey point, and executing the final scheduling of each job task based on the foraging relationship.

[0021] It needs to be further explained that in the specific implementation process, the process of receiving a plurality of job tasks for cross-border trade data processing and establishing a common scheduling atlas between the plurality of job tasks and a big data platform comprises: A receiving period is set, in which a plurality of job tasks for cross-border trade data processing are received, and the plurality of job tasks are labeled, with the label being denoted as i, i.e., i = 1, 2, 3, …, n, where n is a natural number greater than 0; The label of the job task is combined with the time stamp corresponding to the receiving to serve as the respective identity, and an identity is used to uniquely determine a job task; An integrated data area is set on the big data platform, and the integrated data area is used to store a plurality of task scheduling models obtained after a plurality of historical cross-border trade order corresponding data analyses, and each task scheduling model corresponds to a detailed scheduling process of cross-border trade; Based on the number of job tasks, the integrated data area is divided into a plurality of sub-data areas corresponding to the number, and a plurality of graph layers corresponding to the number of task scheduling models are created in each sub-data area, and each graph layer is used to store a task scheduling model; It should be noted that the task scheduling model corresponding to the detailed scheduling process of cross-border trade in different actual scenarios includes but is not limited to the following: a customs document processing scheduling model, which is used to process intensive document auditing tasks in the customs clearance link. When a new customs document processing job task enters the system, the model will be activated, and the detailed scheduling process is as follows: the job task is preferentially assigned to a computing node with high-performance OCR (optical character recognition) and natural language processing capability, which is used to quickly extract key fields (such as commodity code, amount, quantity) in invoices, packing lists, and certificates of origin. Then, the model directs the task to access a storage area integrated with a customs commodity classification database and a trade control policy database to perform automatic data comparison and compliance verification. In the entire process, the model defines a strict data flow and calculation order, and presets an abnormal processing path for automatically diverting the task to a manual review queue when the recognition confidence is lower than a threshold.

[0022] A logistics track tracking scheduling model: used for complex job tasks for real-time tracking of massive package tracks in cross-border logistics and predicting arrival time, the scheduling job task establishes a network connection with the API interface of a plurality of logistics suppliers to continuously obtain the latest logistics state information; the latest logistics state information is cleaned, associated, and aggregated in real time by a computing cluster, the scheduling job task accesses a historical logistics time efficiency database, and a machine learning prediction algorithm deployed on a high-performance GPU node is called to dynamically calculate the predicted delivery time by comprehensively considering the current logistics node, transportation mode, historical performance, and weather and other external variables.

[0023] The cross-border payment detection scheduling model is applied to the payment link in the cross-border trade process, is used for identifying and intercepting suspicious transactions in real time, when a cross-border trade payment transaction occurs, the model scheduling job task routes it to the memory computing resource for response, calls the rule engine pre-deployed in the network environment, executes a series of pre-defined risk control rules (such as single-amount upper limit, frequency limit) of the rule engine, meanwhile, the model will schedule tasks in parallel, quickly acquire the historical transaction network relationship of the user, and analyze the abnormal mode of the transaction behavior in real time.

[0024] The graph layer storing the task scheduling model is marked as a scheduling layer; An inter-layer communication interaction point is arranged between every two scheduling layers, the inter-layer communication interaction point is used for establishing data intercommunication between different scheduling layers in a same sub-data area, a cross-domain communication interaction point is arranged, and the cross-domain communication interaction point is used for establishing data intercommunication between scheduling layers corresponding to different sub-data areas; When all the scheduling layers in a same sub-data area establish data intercommunication, a common scheduling graph of the corresponding job task in the big data platform is established, the common scheduling graph of the current sub-data area is copied to other sub-data areas which are not operated, and then the common scheduling graph of the corresponding job task and the big data platform is established in each sub-data area.

[0025] The construction process of the common scheduling graph is shown in Figure 2 .

[0026] It should be further explained that, in the specific implementation process, the process of updating the private scheduling path of each job task in the common scheduling graph based on the analysis result of each job task includes: A job time sequence window is set for each job task, the job time sequence window includes a plurality of time sequence slice sequences, each time sequence slice sequence is used for dividing the job task into a sub-stage task in a corresponding time period; Data analysis is performed on each time period sub-stage task to obtain stage data characteristics of the corresponding sub-stage task, the stage data characteristics of all time period sub-stage tasks of the same job task are integrated, and then global data characteristics of the job task are obtained, and the global data characteristics are taken as the corresponding analysis result; The task real-time scheduling model of the corresponding job task is established according to the global data characteristics, and the local scheduling model of each sub-stage task corresponding to the time period in the task real-time scheduling model is divided, and the local scheduling model is used for representing the related process of scheduling the job task in a time period; The task real-time scheduling model of the job task is mapped into the respective common scheduling graph, and for each scheduling layer in the respective common scheduling graph, a plurality of scheduling slice windows are divided based on the time period division of the respective task real-time scheduling model; merge all scheduling slice windows of the same period into one scheduling path point set, and then obtain several scheduling path point sets corresponding to different periods, and determine whether there is an adaptive scheduling path point in the scheduling path point set of the corresponding period based on the local scheduling model of the job task in each period; If yes, then screen the scheduling path point to construct the scheduling path of the job task in the corresponding period; If no, then create a new private path point, connect the existing scheduling path of other periods, and update it to the private scheduling path corresponding to the current job task; Wherein, when the local scheduling model of the job task in a certain period is consistent with the task scheduling model of any scheduling layer on the common scheduling atlas in the corresponding period, it is determined that there is an adaptive scheduling path point in the corresponding period; if all scheduling layers cannot correspond to the local scheduling model of the current period, it is determined that there is no adaptive scheduling path point.

[0027] It needs to be further explained that in the specific implementation process, the process of constructing a big data platform resource pool and determining the demand scheduling resources of each job task in the big data platform resource pool includes: A number of resource distribution points are set on the big data platform, which are used for cross-border trade related resource distribution, and a corresponding scheduling area is set at each resource distribution point, and then a number of resource area sub-pools are constructed; The scheduling area is the area range available for resource distribution by the resource distribution point; Connect several resource area sub-pools to build the big data platform into a corresponding big data platform resource pool, obtain the task quota information of each job task, and the task quota information is a set of upper limits of resources that can be used by the job task during execution, which is the data basis for subsequent resource allocation and management, ensures fair sharing of resources among various job tasks, and prevents running collapse caused by excessive consumption of resources by a single job task; The task quota information includes computing resource quota, storage resource quota and network resource quota; The computing resource quota includes CPU quota and memory quota, the CPU quota defines the upper limit of the number of virtual CPU cores that can be used by the job task and the CPU usage time, and the memory quota consists of heap memory and off-heap memory; The heap memory and off-heap memory are cumulatively composed of the total memory available to the job task; The storage resource quota includes disk space quota and I / O bandwidth quota; The disk space quota is further divided into temporary space and output space, the execution of the job task is performed in the temporary space, and the output of the execution result corresponding to the job task is performed in the output space; performing the job task in the temporary space and outputting the result in the output space prevents the disk from being full; The I / O bandwidth quota is used to limit the disk read-write throughput of the operation task in a preset safe throughput range, thereby avoiding the influence of high I / O operation tasks on the running of other normal throughput operation tasks. The network resource quota includes the network bandwidth available for each operation task, and includes the upper limit of the number of access interfaces corresponding to each operation task when connecting, and the operation task obtains the network resource lacking by itself from the outside through the access interface; Based on the task quota information, resource screening is performed in a plurality of resource area sub-pools corresponding to the resource pool of the big data platform, thereby locking the resource area sub-pool related to each operation task, and determining all resources corresponding to the resource area sub-pool as the demand scheduling resources of the corresponding operation task.

[0028] Among them, the corresponding computing resources, storage resources and network resources in each resource area sub-pool meet the resource use of a plurality of operation tasks in the corresponding resource area sub-pool. By matching the task quota information of the operation task with the resources in the resource area sub-pool, the corresponding demand scheduling resources are provided for each operation task.

[0029] It needs to be further explained that in the specific implementation process, the process of determining the scheduling enclosure set corresponding to the operation task on the big data platform based on the demand scheduling resources includes: Set the resource acquisition distance of each operation task, and construct the resource grabbing circle corresponding to each operation task based on the resource acquisition distance. The resource grabbing circle is constructed by taking the storage position of the operation task on the big data platform as the coordinate center and taking the resource acquisition distance as the radius. The resource acquisition distance is set by the demand scheduling resources corresponding to the operation task; It needs to be explained that the resource acquisition distance is a comprehensive logical quantitative index, rather than a physical spatial distance. Its specific setting includes: extracting key influence factors according to the type and attribute of the demand scheduling resources locked by the operation task, the key influence factors including but not limited to: for computing resources, considering the network delay and bandwidth cost between the computing resources and the task execution position; for storage resources, considering the time overhead of data migration and I / O access delay; for network resources, considering the time consumption of connection establishment and transmission stability.

[0030] A standardized cost weight is given to each key influence factor, a quantified resource acquisition distance value is outputted based on the specific attribute value of the demand scheduling resource through a preset distance calculation function, the greater the value, the higher the comprehensive cost paid for acquiring and deploying resources from the resource location to the task execution location; on the contrary, the lower the comprehensive cost, the resource acquisition distance obtained through multi-dimensional data reflects the efficiency of resource scheduling for the job task at different demand resource locations.

[0031] The resource area sub-pool within the resource grabbing circle range is grabbed as the scheduling object, the middle position of the communication distance between the job task and the resource area sub-pool is taken as the center of the circle, and a surrounding circle is established between each scheduling object and the job task with the value of half of the corresponding communication distance as the radius. Further, there are several surrounding circles of different sizes between different scheduling objects and the job task, and the several surrounding circles formed by the same job task and different scheduling objects are integrated to obtain the scheduling surrounding circle set corresponding to the job task on the big data platform.

[0032] It needs to be further explained that in the specific implementation process, food sources and prey points are set in the scheduling surrounding circle set, the foraging relationship between the food sources and the prey points is determined, and the process of executing the final scheduling of each job task based on the foraging relationship includes: Each resource area sub-pool in the scheduling surrounding circle set is marked as a food source, and the job task itself is marked as a prey point, and the resource supply capacity value from each food source to the prey point is obtained; The food sources are sorted based on the size of the resource supply capacity value, the foraging priority relationship between the food sources and the prey points based on the resource supply capacity evaluation value is established, and based on the foraging priority relationship, the prey point initiates a resource scheduling request to the highest priority food source and locks the corresponding demand scheduling resource; When the highest priority food source cannot meet the full resource demand of the job task, the prey point initiates a scheduling request to the next priority food source according to the foraging priority order until the full demand scheduling resource of the job task is successfully locked, and based on the finally locked full demand scheduling resource, the job task is driven to execute the final scheduling on the big data platform.

[0033] It should be noted that the resource supply capability value is used to quantitatively evaluate the comprehensive score value of the ability of a food source (i.e., a resource area sub-pool) to meet the resource demand of a specific prey point (i.e., a job task). The process of obtaining the resource supply capability value is as follows: collecting the current state data of the food source, including resource sufficiency (such as the number of available CPU cores, the size of the remaining memory, and whether the idle disk space meets the respective task quota), resource performance state (such as the current average I / O throughput rate and network bandwidth utilization rate), and health and stability indicators (such as historical failure rate and current load level).

[0034] The state data is matched and analyzed with the task quota information and real-time demand characteristics of the prey point. The matching and analysis process is completed by a comprehensive evaluation model, which assigns different importance coefficients to different dimensions of state data and performs weighted calculation. For example, for a computing-intensive job task, the sufficiency and performance state of the computing resource will be given a higher weight. The comprehensive evaluation model outputs a quantitative score as the resource supply capability value. The higher the value, the better and more efficiently the food source can meet the resource demand of the prey point, and the higher the position in the foraging priority ranking. The foraging priority relationship is a dynamic ranking relationship established based on the descending order of the resource supply capability value. The prey point will initiate resource scheduling requests to the food source in turn according to this ranking.

[0035] Please refer to Figure 3 The present application also provides a cross-border trade big data platform job task scheduling system. The system includes: A scheduling data processing module receives a plurality of job tasks for cross-border trade data processing, establishes a shared scheduling graph between the plurality of job tasks and the big data platform, and updates the private scheduling path of each job task on the shared scheduling graph based on the analysis result of each job task. A scheduling resource determination module constructs a big data platform resource pool, determines the demand scheduling resource of each job task in the big data platform resource pool, and determines the corresponding scheduling enclosure set of the job task in the big data platform based on the demand scheduling resource. A scheduling execution module sets food sources and prey points within the scheduling enclosure set, determines the foraging relationship between the food sources and the prey points, and executes the final scheduling of each job task based on the foraging relationship.

[0036] A computer device includes a memory and a processor. The memory stores a computer program. The processor executes the above-mentioned computer program to implement the steps of the above-mentioned cross-border trade big data platform job task scheduling method.

[0037] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for scheduling tasks of a cross-border trade big data platform, characterized in that, The method comprises the following steps: Step S1: receiving a plurality of job tasks for cross-border trade data processing, establishing a common scheduling graph between the plurality of job tasks and a big data platform, and updating a private scheduling path of each job task on the common scheduling graph based on an analysis result of each job task; Step S2: constructing a big data platform resource pool, determining a demand scheduling resource of each job task in the big data platform resource pool, and determining a scheduling enclosure set corresponding to the job task in the big data platform based on the demand scheduling resource; Step S3: setting a food source and a prey point in the scheduling enclosure set, determining a foraging relationship between the food source and the prey point, and executing final scheduling of each job task based on the foraging relationship. 2.The method of claim 1, wherein, The process of receiving a plurality of job tasks for cross-border trade data processing and establishing a common scheduling graph between the plurality of job tasks and a big data platform comprises: labeling the plurality of job tasks, combining the label with a time stamp when the job task is received as an identity of each job task, setting an integrated data area on the big data platform, and storing a task scheduling model obtained after data analysis of a plurality of historical cross-border trade orders in the integrated data area; based on the number of job tasks, dividing the integrated data area into a corresponding number of sub-data areas, creating a corresponding number of graph layers based on the number of task scheduling models in each sub-data area, and storing one task scheduling model in each graph layer; marking the graph layer storing the task scheduling model as a scheduling layer; setting an inter-layer communication interaction point between each two scheduling layers for establishing data intercommunication of different scheduling layers under the same sub-data area, and setting a cross-domain communication interaction point for data intercommunication of scheduling layers corresponding to different sub-data areas; when all scheduling layers under the same sub-data area establish data intercommunication, a common scheduling graph of the job task corresponding to the corresponding sub-data area in the big data platform is established, the common scheduling graph of the current sub-data area is copied to other sub-data areas that are not operated, and the common scheduling graph of the job task corresponding to each sub-data area and the big data platform is established. 3.The method of claim 2, wherein, The process of updating a private scheduling path of each job task on the common scheduling graph based on an analysis result of each job task comprises: setting a job time sequence window for each job task, the job time sequence window comprising a plurality of time sequence slice sequences, the time sequence slice sequences being used to divide the job task into a sub-phase task in a corresponding time period; obtaining corresponding phase data features by data analysis on the sub-phase task, integrating the phase data features of the sub-phase task corresponding to each job task in all time periods to obtain global data features of the job task as the analysis result; establishing a task real-time scheduling model of the job task according to the global data features, generating a local scheduling model corresponding to each time period based on the task real-time scheduling model, and mapping the task real-time scheduling model of the job task to the respective common scheduling graph. For each scheduling layer in the respective common scheduling graph, based on the time period division of the respective task real-time scheduling model, a plurality of scheduling slice windows are obtained, all scheduling slice windows in the same time period are combined into a scheduling path point set, and based on the local scheduling model of each time period, it is determined whether there is an adaptive scheduling path point in the scheduling path point set of the corresponding time period; If yes, the scheduling path of the job task in the corresponding time period is constructed by screening the scheduling path point; If no, a new private path point is created, connected to the existing scheduling path of other time periods, and updated as the private scheduling path corresponding to the current job task. 4.The method of claim 3, wherein, The process of determining whether there is an adaptive scheduling path point in the scheduling path point set of the corresponding time period includes: When the local scheduling model of the job task in a certain time period is consistent with the task scheduling model of any scheduling layer on the common scheduling graph in the corresponding time period, it is determined that there is an adaptive scheduling path point in the corresponding time period; If all scheduling layers cannot correspond to the local scheduling model of the current time period, it is determined that there is no adaptive scheduling path point. 5.The method of scheduling tasks of cross-border trade big data platform according to claim 4, characterized in that, The process of constructing a big data platform resource pool and determining the demand scheduling resource of each job task in the big data platform resource pool includes: A plurality of resource allocation points are set on the big data platform, which are used for cross-border trade related resource distribution, a corresponding scheduling area is set at each resource allocation point, and a plurality of resource area sub-pools are constructed; The scheduling area is the area range available for resource distribution of the resource allocation point; Connect a plurality of resource area sub-pools to construct the corresponding big data platform resource pool, and obtain the task quota information of each job task; Based on the task quota information, the resource area sub-pool related to each job task is locked in the corresponding plurality of resource area sub-pools of the big data platform resource pool, and all resources corresponding to the resource area sub-pool are determined as the demand scheduling resource of the corresponding job task. 6.The method of scheduling tasks of cross-border trade big data platform according to claim 5, characterized in that, The process of determining the scheduling enclosure set of each job task on the big data platform based on the demand scheduling resource includes: Set the resource acquisition distance of each job task, construct the resource capture circle corresponding to each job task based on the resource acquisition distance, and capture the resource area sub-pool within the resource capture circle range as the scheduling object; Take the middle position between the job task and the resource area sub-pool as the center, and take the value of half of the corresponding communication distance as the radius to establish an enclosure between each scheduling object and the job task; There are a plurality of enclosures with different areas between different scheduling objects and the job task, and the plurality of enclosures formed by the same job task and different scheduling objects are integrated to obtain the scheduling enclosure set corresponding to the job task on the big data platform.

7. The method of claim 6, wherein, In the scheduling enclosure set, set the food source and the prey point, determine the foraging relationship between the food source and the prey point, and execute the final scheduling of each job task based on the foraging relationship, which includes: Mark each enclosure in the scheduling enclosure set as a food source, and mark the job task itself as a prey point to obtain the resource supply capacity value from each food source to the prey point; The food sources are ranked based on the size of the resource supply capability value, and a foraging priority relationship between the food sources and the prey points based on the resource supply capability evaluation value is established; Based on the foraging priority relationship, the prey point initiates a resource scheduling request to the food source with the highest priority, and locks the corresponding demand scheduling resource; When the highest priority food source cannot meet the entire resource demand of the operation task, the prey point initiates a scheduling request to the next priority food source according to the foraging priority order until the entire demand scheduling resource of the operation task is successfully locked, and based on the finally locked entire demand scheduling resource, the operation task is driven to execute the final scheduling on the big data platform.

8. A cross-border trade big data platform job task scheduling system for implementing the big data platform job task scheduling method of any one of claims 1 to 7, characterized in that, The system comprises: A scheduling data processing module receives a plurality of operation tasks for cross-border trade data processing, establishes a common scheduling graph between the plurality of operation tasks and the big data platform, and updates the private scheduling path of each operation task on the common scheduling graph based on the analysis result of each operation task; A scheduling resource determination module constructs a big data platform resource pool, determines the demand scheduling resource of each operation task in the big data platform resource pool, and determines the scheduling enclosure set corresponding to the operation task in the big data platform based on the demand scheduling resource; A scheduling execution module sets food sources and prey points in the scheduling enclosure set, determines the foraging relationship between the food sources and the prey points, and executes the final scheduling of each operation task based on the foraging relationship. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to realize the steps of the method of any one of claims 1 to 7.

Images