Foreign trade business data collaborative analysis method and system based on big data

Abstract

The invention discloses a big data-based foreign trade business data collaborative analysis method and system, and belongs to the technical field of data collaborative analysis management. The method comprises the following steps: monitoring transmission performance data of each department in real time, and dynamically optimizing a data transmission process according to the transmission performance data of each department; monitoring the currently completed task load of each department in real time, and dynamically transmitting the task data according to the currently completed task load of each department and the transmission performance of each department; updating information of each department is monitored in real time, and the task data updating frequency is dynamically adjusted according to the updating information of each department and the transmission performance of the system. According to the method, dynamic optimization of data transmission, intelligent transmission of task data and accurate adjustment of updating frequency are realized by monitoring transmission performance, task completion amount and updating information of each department in real time, so that foreign trade business data processing efficiency is remarkably improved, and reliable data support is provided for foreign trade business.

Description

Technical Field

[0001] The present invention relates to the technical field of data collaborative analysis and management, and in particular to a method and system for collaborative analysis of foreign trade business data based on big data. Background Art

[0002] With the acceleration of globalization and the rapid development of digital technology, the foreign trade industry is facing an increasingly complex market environment and fierce competition. The rise of big data technology has provided new opportunities for foreign trade companies. The collaborative analysis method for foreign trade business data based on big data aims to integrate multi-source data from both internal and external sources, breaking down information silos and enabling cross-departmental and cross-platform data sharing and collaborative analysis.

[0003] Existing collaborative analysis methods for foreign trade business data integrate and analyze large amounts of real-time data from different departments. With the help of big data technology and intelligent algorithms, they can achieve rapid data transmission and updating, optimize collaboration processes, and improve decision-making efficiency and accuracy. This is mainly to address problems such as data delays and information asymmetry that exist in cross-departmental collaboration among foreign trade companies.

[0004] For example, the foreign trade logistics processing system and processing method announced in the invention patent with announcement number CN113793106B include: a waybill creation module, a warehouse management module, a route planning module and a collaborative management module; the waybill creation module is used for the shipper to enter logistics requirements, and the shipper creates a logistics order and generates a cargo ID after reaching an agreement with the carrier; the route planning module is used to obtain logistics order data, data of various port storage stations and historical transportation route plan data from the logistics monitoring center to plan the optimal transportation route plan for the logistics order; the collaborative management module conducts collaborative analysis on the estimated arrival time of the corresponding logistics to achieve real-time update of the estimated arrival time of the corresponding logistics.

[0005] For example, the invention patent announcement with announcement number: CN118485415B discloses a business collaboration management system and method based on data analysis, including: compiling business execution ports and business execution status interfaces to characterize a business collaboration relationship tree model; capturing interactive behavior paths, and analyzing the impact of the triggering behavior of the protection mechanism on the interactive behavior path based on the business's execution feedback instructions; analyzing the correlation between businesses and fitting a related function; analyzing the hedging degree between businesses under the influence of different interactive behavior paths; forming an association channel between businesses, outputting a business collaboration warning set, and realizing hedging warnings; being able to characterize business collaboration relationships and sort out the feedback path of instructions during business execution, quantify the correlation in the business collaboration process through the triggering behavior of the protection mechanism, and analyze the hedging between businesses under the joint action of the feedback path and the protection mechanism.

[0006] However, in the process of implementing the technical solutions of the invention in the embodiments of the present application, the present application found that the above technology has at least the following technical problems:

[0007] In the existing technology, during the cross-departmental collaboration to execute a single task in foreign trade business, updates may not be made in a timely manner, resulting in data from certain departments being delayed in being transmitted to other departments, causing data from subsequent departments not to be updated in a timely manner. Information asymmetry in cross-departmental collaboration causes workflow delays, and there is a problem of low quality of collaborative data analysis. Summary of the Invention

[0008] The present invention provides a method and system for collaborative analysis of foreign trade business data based on big data, thereby solving the problem in the prior art that, during the cross-departmental collaborative execution of a single task in foreign trade business, updates are only performed according to preset task conditions, which may lead to untimely updates of data from certain departments, resulting in delayed transmission of data to other departments, and failure to update data of subsequent departments in a timely manner. The information asymmetry in cross-departmental collaboration causes workflow delays, and the quality of data collaborative analysis is low, thereby achieving an improvement in the quality of data collaborative analysis.

[0009] The present invention provides a method for collaborative analysis of foreign trade business data based on big data, comprising the following steps: real-time monitoring of the transmission performance data of each department, and dynamic optimization of the data transmission process based on the transmission performance data of each department; real-time monitoring of the current task volume completed by each department, and dynamic transmission of task data based on the current task volume completed by each department and the transmission performance of each department; real-time monitoring of the update information of each department, and dynamic adjustment of the task data update frequency based on the update information of each department and the system transmission performance.

[0010] An embodiment of the present application provides a foreign trade business data collaborative analysis system based on big data, including a transmission performance optimization module, a task data scheduling module, an update frequency adjustment module and a foreign trade collaborative database; wherein, the transmission performance optimization module is used to monitor the transmission performance data of each department in real time, and dynamically optimize the data transmission process according to the transmission performance data of each department; the task data scheduling module is used to monitor the current task volume completed by each department in real time, and dynamically transmit the task data according to the current task volume completed by each department and the transmission performance of each department; the update frequency adjustment module is used to monitor the update information of each department in real time, and dynamically adjust the task data update frequency according to the update information of each department and the system transmission performance.

[0011] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0012] 1. The present invention provides a method and system for collaborative analysis of foreign trade business data based on big data, thereby monitoring the transmission performance, task completion status and update information of each department in real time, and dynamically optimizing the transmission process, adjusting the transmission strategy and update frequency of task data based on these data, thereby achieving efficient scheduling of system resources, ensuring the timely transmission and update of task data, and improving overall work efficiency and data processing accuracy.

[0013] 2. The present invention obtains the system transmission performance evaluation value by real-time monitoring of the transmission performance data of each department, and dynamically optimizes the data transmission process according to the system transmission performance evaluation value. When the system transmission performance is insufficient, the transmission process is optimized through the load balancing mode and task data caching, thereby ensuring the stability and efficiency of data transmission, and improving the transmission capacity of the overall system and the foreign trade collaboration efficiency.

[0014] 3. The present invention performs dynamic transmission based on the task completion status of each department and the department's transmission performance, thereby giving priority to transmitting completed task data when the task progress lags behind or the transmission performance is insufficient, and flexibly adjusting the block size and parallel transmission mode to ensure efficient and fast transmission of task data, thereby improving the task completion efficiency and the timeliness of data transmission, and optimizing the overall system resource utilization and task scheduling effect.

[0015] 4. The present invention dynamically adjusts the task data update frequency according to the update information of each department and the system transmission performance, thereby optimizing the task data update strategy based on the comparison between the system update intensity evaluation value and the transmission performance evaluation threshold, giving priority to high-priority data, and regularly updating low-priority data, thereby achieving efficient utilization of system resources and timeliness of data updates. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of a collaborative analysis method for foreign trade business data based on big data is provided in an embodiment of the present application.

[0017] Figure 2 A graph showing changes in task data priority evaluation values ​​based on a collaborative analysis method for foreign trade business data provided in an embodiment of the present application.

[0018] Figure 3 A structural diagram of a foreign trade business data collaborative analysis system based on big data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The embodiment of the present application provides a method and system for collaborative analysis of foreign trade business data based on big data, which solves the problem in the prior art that in the process of cross-departmental collaboration in foreign trade business to execute a single task, the data of some departments may be delayed in being transmitted to other departments due to the update only according to the preset task conditions, resulting in the data of subsequent departments not being updated in time, and the information asymmetry in cross-departmental collaboration causing workflow delays, and the problem of low quality of collaborative data analysis. By real-time monitoring of the transmission performance data of each department, the data transmission process is dynamically optimized according to the transmission performance data of each department; real-time monitoring of the current task volume completed by each department, and dynamic transmission of task data according to the current task volume completed by each department and the transmission performance of each department; real-time monitoring of the update information of each department, and dynamic adjustment of the task data update frequency according to the update information of each department and the transmission performance of each department, not only the efficiency and responsiveness of the method are improved, but also the rational use of resources is ensured, thereby improving the efficiency of the entire workflow.

[0020] The technical solution in the embodiments of the present application is to solve the above-mentioned problem in the cross-departmental collaboration for executing a single task in foreign trade business. Since updates are only performed according to preset task conditions, updates may not be timely, resulting in delayed transmission of data from certain departments to other departments, causing data from subsequent departments not to be updated in a timely manner. Information asymmetry in cross-departmental collaboration causes workflow delays, and there is a problem of low quality of collaborative data analysis. The overall concept is as follows:

[0021] By dynamically adjusting the transmission strategy based on the transmission performance data of each department, intelligently distributing task data based on the current amount of completed tasks, and dynamically adjusting the update frequency of task data based on the updated information of each department and the system transmission performance, we can maximize task processing efficiency, optimize resource utilization, and agile system response, ensuring that tasks are completed on time and reducing transmission delays and resource waste.

[0022] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0023] like Figure 1 As shown, a flow chart of a collaborative analysis method for foreign trade business data based on big data provided by an embodiment of the present application includes the following steps: real-time monitoring of the transmission performance data of each department, and dynamic optimization of the data transmission process based on the transmission performance data of each department; real-time monitoring of the current amount of tasks completed by each department, and dynamic transmission of task data based on the current amount of tasks completed by each department and the transmission performance of each department; real-time monitoring of the update information of each department, and dynamic adjustment of the task data update frequency based on the update information of each department and the system transmission performance.

[0024] In this embodiment, foreign trade business often involves the collaborative work of multiple departments (such as procurement, sales, finance, logistics, etc.), and each department completes specific tasks at different time nodes. In the process of cross-departmental collaboration, data transmission, task completion status and information update frequency between different departments will affect the smoothness and efficiency of the overall workflow. Without real-time data tracking and optimization, the data of some departments may lag behind, resulting in work delays or information asymmetry in subsequent departments, affecting the overall business process. By monitoring the transmission performance of each department in real time, the system can be optimized according to the network status of each department, avoiding transmission bottlenecks caused by insufficient network performance or transmission delays; by monitoring the task completion volume of each department and combining transmission performance data, data transmission can be intelligently allocated between different departments to ensure priority transmission of key data and avoid delays caused by unreasonable resource allocation; by adjusting the update frequency of task data in real time, frequent update operations can be avoided from occupying bandwidth, ensuring that network resources are not frequently occupied by unnecessary updates, thereby improving the efficiency of data transmission.

[0025] In addition, the foreign trade collaborative database is used to store relevant data based on the collaborative analysis method of foreign trade business data, including: transmission performance evaluation threshold, transmission performance evaluation influencing factor corresponding to the task data transmission rate, critical task data transmission rate, critical task data transmission delay time, critical task data reception delay time and critical task data processing delay time, etc. The data in the foreign trade collaborative database can be directly queried through public databases such as the United Nations Commodity Trade Statistics Database and the official website of the General Administration of Customs of China, or obtained through cooperation with industry associations such as the Foreign Trade Industry Association.

[0026] Furthermore, the transmission performance data of each department is monitored in real time, and the data transmission process is dynamically optimized according to the transmission performance data of each department. The steps are as follows: based on the transmission performance data of each department, a quantitative evaluation of the transmission performance is performed to obtain the system transmission performance evaluation value; the transmission performance evaluation threshold is obtained from the preset foreign trade collaborative database, and the system transmission performance evaluation value is compared with the transmission performance evaluation threshold; if the system transmission performance evaluation value is greater than or equal to the transmission performance evaluation threshold, no additional processing is performed; if the system transmission performance evaluation value is less than the transmission performance evaluation threshold, the load balancing mode is turned on; if the system transmission performance evaluation value is still less than the transmission performance evaluation threshold after the load balancing mode is turned on, the current system transmission performance evaluation value is matched with the cache capacity corresponding to the transmission performance evaluation values ​​of each system in the preset foreign trade collaborative database to obtain the corresponding cache capacity, and the task data is queued and cached according to the data criticality parameters and the obtained cache capacity.

[0027] In this embodiment, the load balancing mode distributes traffic to multiple routers or paths, which can reduce congestion on a single path and reduce queuing delays. By monitoring the transmission performance of each department in real time and dynamically optimizing it based on the evaluation value, the system can always operate in the optimal transmission state. By introducing the load balancing mode and making adjustments when the transmission performance is insufficient, the transmission load can be effectively dispersed, preventing transmission bottlenecks in local departments or systems from affecting the entire workflow. By dynamically adjusting the cache capacity and data transmission method, resources can be reasonably allocated according to the current network status, avoiding excessive bandwidth waste, while ensuring that critical mission data is transmitted first, improving the resource utilization efficiency of the entire system. By queuing and caching task data according to the data's criticality parameters, the system can ensure that high-priority tasks and critical data are transmitted first, thereby reducing transmission delays and ensuring that key links in the business are not affected.

[0028] Cache capacity can be directly obtained from the foreign trade collaborative database. For example, a mapping table is created in the database, which maps transmission performance evaluation values ​​to cache capacities. This table records each transmission performance evaluation value and its corresponding cache capacity. These relationships can be one-to-one or many-to-one. To obtain cache capacity, simply enter the system transmission performance evaluation value into the mapping table. The database will then quickly locate and return the cache capacity corresponding to that system transmission performance evaluation value.

[0029] Furthermore, the transmission performance data includes the data transmission rate of each task, the data transmission delay time of each task, the data reception delay time of each task and the data processing delay time of each task; the quantitative evaluation of the transmission performance based on the transmission performance data of each department and the step of obtaining the system transmission performance evaluation value include: obtaining the critical task data transmission rate, critical task data transmission delay time, critical task data reception delay time and critical task data processing delay time from the foreign trade collaborative database; performing an average calculation of the proportion of the data transmission rate of each task and the critical task data transmission rate, and then performing a weighted calculation on the calculation results to obtain the task data transmission rate Influencing parameters; perform a proportion approximation operation on the data transmission delay time of each task, the data reception delay time of each task and the data processing delay time of each task respectively with the critical task data transmission delay time, the critical task data reception delay time and the critical task data processing delay time, perform weighted operation on the operation results and then perform coupling processing, average the coupling processing results and then perform inverse proportional operation to obtain the data delay time influencing parameters; couple the task data transmission rate influencing parameters and the data delay time influencing parameters to obtain the transmission performance evaluation value of each department, and record the result of the average processing of the transmission performance evaluation value of each department as the system transmission performance evaluation value.

[0030] The transmission performance evaluation value of each department is obtained as follows:

[0031]

[0032] Where TP i represents the transmission performance evaluation value of the i-th department, α1 represents the transmission performance evaluation factor corresponding to the task data transmission rate, α2 represents the transmission performance evaluation factor corresponding to the task data transmission delay time, α3 represents the transmission performance evaluation factor corresponding to the task data reception delay time, α4 represents the transmission performance evaluation factor corresponding to the task data processing delay time, TR 1ij represents the task data transmission rate of the jth task data of the i-th department, TR0 represents the critical task data transmission rate, DD 1ij represents the task data transmission delay time of the jth task data of the i-th department, DD0 represents the critical task data transmission delay time, RD 1ij represents the task data receiving delay time of the jth task data of the i-th department, RD0 represents the critical task data receiving delay time, PD 1ij It represents the task data processing delay time of the jth task data of the i-th department, PD0 represents the critical task data processing delay time, where i is the department number, i = 1, 2, 3, ..., N, N is the total number of departments, j is the task data number, j = 1, 2, 3, ..., M, M is the total number of task data.

[0033] α1, α2, α3, and α4 are the transmission performance evaluation impact factors corresponding to the task data transmission rate, task data transmission delay time, task data reception delay time, and task data processing delay time preset in the foreign trade collaborative database. These impact factors are numerical indicators that measure the impact of the above-mentioned transmission performance data on the transmission performance evaluation value. Specifically, there is a mapping relationship table for each task data transmission rate, task data transmission delay time, task data reception delay time, and task data processing delay time. The table records each possible transmission performance data value and its corresponding transmission performance evaluation impact factor. These mapping relationships can be one-to-one or many-to-one. For example, in actual applications, when it is necessary to evaluate the transmission performance evaluation value of a department, the measured task data transmission rate, task data transmission delay time, task data reception delay time, and task data processing delay time can be entered into their respective corresponding mapping relationship tables, and the transmission performance evaluation impact factors corresponding to these values ​​can be quickly found. The impact factors range in value from 0 to 1.

[0034] The system transmission performance evaluation value is obtained as follows:

[0035]

[0036] Where TP1 represents the system transmission performance evaluation value, TP i represents the transmission performance evaluation value of the i-th department.

[0037] In this embodiment, the task data transmission rate represents the speed at which data is transmitted in the network, which can be directly monitored using network traffic analysis tools such as traffic counters and bandwidth measurement tools. The task data transmission delay refers to the time required for data to travel from the source node to the destination node, which can be measured using network measurement tools (such as the ping command) to measure the packet transmission time from the sender to the receiver. The task data reception delay refers to the waiting time after the destination node receives the data and begins processing the data, which can be calculated by recording the time the data arrives and the time the processing begins at the receiving end. The task data processing delay refers to the time required for the system to perform necessary processing after receiving the data, which can be measured from the time the data arrives at the receiving end to the time the processing result is output. The four are interrelated. For example, a longer reception delay means that the receiving end may be waiting for other data or tasks, which in turn increases the processing delay. A higher transmission rate generally means that data can be transmitted faster, but if there are problems with network equipment or lines, the transmission delay may also be higher. A higher processing delay may lead to a smaller congestion window, which in turn limits the transmission rate. The transmission performance evaluation value obtained through comprehensive analysis reflects whether the system is working efficiently. It can accurately determine which links need to be optimized, and then rationally allocate resources and adjust priorities to ensure that the most critical tasks are completed first and maximize transmission efficiency.

[0038] Furthermore, the data criticality parameters include the fluctuation amplitude, change frequency and number of related departments in each detection time period; the step of queuing and caching the task data through the data criticality parameters and the obtained cache capacity includes: obtaining the critical fluctuation amplitude, critical change frequency and critical number of related departments from the preset foreign trade collaborative database; performing a proportion proximity operation on the fluctuation amplitude and change frequency of each detection time period and the critical fluctuation amplitude and critical change frequency respectively, performing a weighted operation on the proportion proximity operation results and then performing coupling processing, and recording the homogenized result of the coupling processing results as the data change influencing parameter; performing a proportion proximity operation on the number of related departments and the critical number of related departments, and then performing a weighted operation on the proportion proximity operation results to obtain the related department number influencing parameter; coupling the data change influencing parameter and the related department number influencing parameter to obtain the priority evaluation value of each task data; sorting each task data from large to small according to the priority evaluation value of each task data, and queuing and caching the task data within the cache capacity according to the sorting result.

[0039] The priority evaluation value of each task data is obtained as follows:

[0040]

[0041] Where, DP j represents the jth task data priority evaluation value, α5 represents the task data priority evaluation impact factor corresponding to the fluctuation amplitude, α6 represents the task data priority evaluation impact factor corresponding to the change frequency, α7 represents the task data priority evaluation impact factor corresponding to the number of associated departments, FA 1jk It represents the fluctuation range of the jth data in the kth monitoring period, FA0 represents the critical fluctuation range, CF 1jk Indicates the change frequency of the jth data in the kth monitoring period, CF0 indicates the critical change frequency, RD 1j represents the number of associated departments of the jth data, RD0 represents the number of critical associated departments, where k is the number of each monitoring time period, k = 1, 2, 3, ..., Z, and Z is the total number of monitoring time periods.

[0042] α5, α6, and α7 are the task data priority assessment impact factors corresponding to the fluctuation amplitude, change frequency, and number of related departments preset in the foreign trade collaborative database. These impact factors are numerical indicators that measure the impact of the above-mentioned data criticality parameters on the task data priority assessment value. Specifically, there is a mapping relationship table for each of the fluctuation amplitude, change frequency, and number of related departments. The table records each possible data criticality parameter value and its corresponding task data priority assessment impact factor. These mapping relationships can be one-to-one or many-to-one. For example, in actual applications, when it is necessary to evaluate a task data priority assessment value, the measured fluctuation amplitude, change frequency, and number of related departments can be entered into their respective corresponding mapping relationship tables, and the task data priority assessment impact factors corresponding to these values ​​can be quickly found. The impact factor value range is between 0 and 1.

[0043] In this embodiment, the fluctuation amplitude represents the maximum value of the fluctuation range of a data indicator within a certain detection time period, which can be obtained by calculating the difference between the maximum and minimum values ​​of the data within the detection time period. The change frequency refers to the frequency of data changes within a specific detection time period, which can be obtained by recording the number of data changes within the time period. The number of associated departments refers to the number of different departments involved in a given task, which can be directly obtained through a task management system or project management tool. The three are interrelated. For example, a larger fluctuation amplitude may also mean a higher change frequency, because frequent changes in task status and requirements will lead to greater fluctuations. Similarly, frequent changes may cause instability in the system's task processing, thereby increasing the fluctuation amplitude. When a task involves more departments, coordination and communication between departments may lead to instability in task data, thereby increasing the fluctuation amplitude. The task data priority evaluation value obtained through comprehensive analysis can better assess the importance and urgency of tasks, thereby effectively allocating and scheduling resources. This can ensure that, given limited resources, tasks that may have a greater impact on overall task completion are prioritized, thereby improving the overall efficiency and responsiveness of the system.

[0044] The task data priority assessment impact factor corresponding to the fluctuation amplitude is set to 0.4, the task data priority assessment impact factor corresponding to the change frequency is set to 0.4, the task data priority assessment impact factor corresponding to the number of associated departments is set to 0.2, the total number of monitoring time periods is set to 1, the fluctuation amplitude of the jth data is set to 4, the critical fluctuation amplitude is set to 2, the change frequency of the jth data is set to 10 times, the critical change frequency is set to 5 times, and the critical number of associated departments is set to 5. The priority assessment value of the task data is calculated under the assumption that the number of associated departments of the jth data continues to increase. This is shown in Table 1, a data table of task data priority assessment values ​​based on the collaborative analysis method for foreign trade business data.

[0045] Table 1 Task data priority evaluation value table based on the collaborative analysis method of foreign trade business data

[0046] serial number <![CDATA[RD 1j (items)]]> DPj 1 3 1.72 2 4 1.76 3 5 1.8 4 6 1.84 5 7 1.88

[0047] like Figure 2 As shown in Table 1 and Figure 2It can be seen that when the task data priority evaluation influencing factor corresponding to the fluctuation amplitude, the task data priority evaluation influencing factor corresponding to the change frequency, the task data priority evaluation influencing factor corresponding to the number of associated departments, the total number of monitoring time periods, the fluctuation amplitude of the j-th data, the critical fluctuation amplitude, the change frequency of the j-th data, the critical change frequency and the critical number of associated departments remain unchanged, and the number of associated departments of the j-th data continues to increase, the task data priority evaluation value also continues to increase.

[0048] Furthermore, the current amount of tasks completed by each department is monitored in real time, and the steps of dynamically transmitting task data according to the current amount of tasks completed by each department and the transmission performance of each department include: obtaining the amount of tasks of each department, the warning time of each department and the amount of tasks completed by each department before the warning; when a department reaches the department warning time, if the current amount of tasks completed by the department does not reach the amount of tasks completed by the warning of the department, and the transmission performance evaluation value of the department is less than the transmission performance evaluation threshold, the completed task data will be transmitted first, and the block size will be dynamically adjusted according to the transmission performance evaluation value of the department and the current amount of tasks completed, and the unfinished task data will be transmitted according to the priority of each block The evaluation value is transported in blocks; if the current task volume of the department has not reached the department's early warning completed task volume, and the department's transmission performance evaluation value is greater than or equal to the transmission performance evaluation threshold, the completed task data will be transported synchronously, and the unfinished task data will be quickly distributed through parallel transmission; if the current task volume of the department has reached the department's early warning completed task volume, or the current task volume of the department has not reached the department's early warning completed task volume but the task data can be processed in segments, the completed task data will be transmitted in full, and the unfinished task data will only be transmitted in increments; when the department's early warning time has not been reached, task processing will continue.

[0049] In this embodiment, by dynamically adjusting the transmission strategy, the data transmission process is optimized according to the actual situation of each department, unnecessary waiting and delays are reduced, thereby improving the overall transmission efficiency; for departments that have reached the warning time but have insufficient task completion, the timely transmission of important data is ensured and data lag is reduced by giving priority to transmitting completed task data and dynamically adjusting the block size; transmission tasks are allocated according to the transmission performance evaluation value of each department, avoiding certain departments from becoming bottlenecks due to insufficient transmission performance, and achieving balanced distribution of transmission load; this mechanism can flexibly adjust the transmission strategy according to different department situations and task characteristics, thereby enhancing the adaptability and flexibility of the system.

[0050] Furthermore, the block size is dynamically adjusted according to the transmission performance evaluation value of the department and the current completed task volume, and the steps of transporting the unfinished task data in blocks according to the priority evaluation value of each block include: marking the difference between the task volume of the department and the current completed task volume of the department as the unfinished task volume; obtaining the reference block size, reference unfinished task volume, unit transmission performance evaluation value and unit unfinished task volume from the preset foreign trade collaborative database; marking the transmission performance evaluation threshold and the unfinished task volume with the difference between the transmission performance evaluation value of the department and the reference unfinished task volume as the deviation transmission performance evaluation value and the deviation unfinished task volume respectively; obtaining the transmission performance adjustment unit based on the deviation transmission performance evaluation value and the unit transmission performance evaluation value, and the transmission performance adjustment unit represents the deviation An integer multiple relationship between the differential transmission performance evaluation value and the unit transmission performance evaluation value; a task quantity adjustment unit is obtained based on the deviation of the uncompleted task quantity and the unit uncompleted task quantity, and the task quantity adjustment unit represents the integer multiple relationship between the deviation of the uncompleted task quantity and the unit uncompleted task quantity; based on the reference block size, the transmission performance adjustment unit is reduced in real time, and the task quantity adjustment unit is increased at the same time to obtain the block size of each time monitoring point, and the unfinished task data is divided according to the block size of each time monitoring point to obtain each block data; the task data priority evaluation value of each block data is averaged to obtain each block priority evaluation value, the block priority evaluation value is sorted from large to small, and the unfinished task data is transported in blocks according to the sorting.

[0051] The priority evaluation value of each block is obtained as follows:

[0052]

[0053] Where, BP a Indicates the priority evaluation value of the a-th block, DP aj represents the priority evaluation value of the j-th task data in the a-th block, where a is the block number, a=1, 2, 3, ..., A, and A is the total number of blocks.

[0054] In this embodiment, the unit transmission performance evaluation value and the unit uncompleted task volume represent the basic units within which the transmission performance evaluation value and the uncompleted task volume are allowed to change, respectively, and are used to dynamically adjust the size of task blocks. By dynamically adjusting the transmission priority of the block size and task volume, the system can flexibly adapt to different task processing requirements and network transmission conditions, ensuring efficient data transmission. The priority evaluation value processing of the block data ensures that task data is properly sorted and prioritized according to its importance and urgency, thereby avoiding delays in the transmission of important or urgent task data and improving the overall efficiency of task completion. By adjusting the transmission performance evaluation value and task volume evaluation value, the block size is dynamically adjusted to avoid over-reliance on large-scale data transmission when network performance is unstable.

[0055] Furthermore, the steps of real-time monitoring of the updated information of each department and dynamically adjusting the task data update frequency according to the updated information of each department and the system transmission performance include: quantitatively evaluating the system update intensity based on the updated information of each department to obtain the system update intensity evaluation value; matching the system transmission performance evaluation value with the system update intensity evaluation threshold value corresponding to each system transmission performance evaluation value preset in the foreign trade collaborative database to obtain the system update intensity evaluation threshold value corresponding to the system transmission performance evaluation value; comparing the system update intensity evaluation value with the system update intensity evaluation threshold value: if the system update intensity evaluation value is less than or equal to the system update intensity evaluation threshold value, no additional processing is performed; if the system update intensity evaluation value is greater than the system update intensity evaluation threshold value, the task data update frequency is dynamically adjusted according to the priority evaluation value of each task data.

[0056] In this embodiment, the system update intensity assessment threshold can be directly obtained from the foreign trade collaborative database. For example, the foreign trade collaborative database forms a mapping relationship table that records each system transmission performance assessment value and its corresponding system update intensity assessment threshold. These relationships can be one-to-one or many-to-one. When obtaining the system update intensity assessment threshold, simply input the system transmission performance assessment value into the mapping relationship table, and the foreign trade collaborative database can quickly locate and return the system update intensity assessment threshold corresponding to the system transmission performance assessment value. By monitoring the update information of each department in real time and dynamically adjusting the task data update frequency accordingly, the system can ensure a rapid response to important and urgent update information, improving the overall response efficiency of the system. Dynamically adjusting the update frequency based on system transmission performance and update intensity can avoid unnecessary resource waste, ensure that transmission resources are used in the most critical areas, and improve resource utilization efficiency. By properly adjusting the update frequency, the system can avoid overload or performance degradation caused by frequent updates, thereby enhancing the stability and reliability of the system.

[0057] Furthermore, the department update information includes task processing frequency, data update frequency and data change frequency; the quantitative evaluation of system update strength based on the updated information of each department, and the step of obtaining the system update strength evaluation value includes: obtaining the critical task processing frequency, critical data update frequency and critical data change frequency from the foreign trade collaborative database; performing a proportion approach calculation on the task processing frequency and the critical task processing frequency, and then performing an update strength impact proportion calculation on the proportion approach calculation result to obtain the task processing frequency impact parameter; performing a proportion approach calculation on each data update frequency and each data change frequency and the critical data update frequency and the critical data change frequency, and then performing an update strength impact proportion calculation on the proportion approach calculation results, and then coupling processing, and recording the homogenized result of the coupling processing as the data update impact parameter; coupling the task processing frequency impact parameter and the data update impact parameter to obtain the system update strength evaluation value.

[0058] The system update intensity evaluation value is obtained as follows:

[0059]

[0060] Where UI represents the system update strength evaluation value, β1 represents the system update strength evaluation factor corresponding to the task processing frequency, β2 represents the system update strength evaluation factor corresponding to the data update frequency, β3 represents the system update strength evaluation factor corresponding to the data change frequency, TF1 represents the task processing frequency, TF0 represents the critical task processing frequency, UF 1j Indicates the data update frequency of the jth task data, UF0 indicates the critical data update frequency, CF 1j represents the data change frequency of the j-th task data, and CF0 represents the critical data change frequency.

[0061] β1, β2, and β3 are the system update strength assessment impact factors corresponding to the task processing frequency, data update frequency, and data change frequency preset in the foreign trade collaborative database. These impact factors are numerical indicators that measure the impact of the above-mentioned department update information on the system update strength assessment value. Specifically, there is a mapping relationship table for each task processing frequency, data update frequency, and data change frequency. The table records each possible department update information value and its corresponding system update strength assessment impact factor. These mapping relationships can be one-to-one or many-to-one. For example, in actual applications, when it is necessary to evaluate a certain system update strength assessment value, the measured task processing frequency, data update frequency, and data change frequency can be input into their respective corresponding mapping relationship tables, and the system update strength assessment impact factors corresponding to these values ​​can be quickly found, where the impact factor value ranges from 0 to 1.

[0062] In this embodiment, the task processing frequency represents the number of times a task is processed or executed within a monitoring time period, the data update frequency represents the number of times task data is updated within a monitoring time period, and the data change frequency refers to the number of times the state of task data changes within a monitoring time period, all of which can be directly obtained through a task management system, system logs, etc. Among them, the three are interrelated. For example, when there are more tasks to be processed in the system, the number of data updates in the system will usually increase; high-frequency task processing may cause frequent changes in data status because the progress and results of the task will affect the status of the data; when the data update frequency is high, the data change frequency is also high because frequent data updates are often accompanied by changes in data content or status. The system update intensity evaluation value obtained through comprehensive analysis can evaluate the update work intensity of the system, and the system can adjust resource allocation according to the update intensity evaluation value.

[0063] Furthermore, the step of dynamically adjusting the task data update frequency according to the priority evaluation value of each task data includes: obtaining the data priority evaluation threshold from the foreign trade collaborative database; comparing the data priority evaluation value of each task with the data priority evaluation threshold, if a task data priority evaluation value is greater than or equal to the data priority evaluation threshold, then marking the task data as high priority data; if a task data priority evaluation value is less than the data priority evaluation threshold, then marking the task data as low priority data; counting each high priority data and each low priority data, performing real-time update processing on each high priority data, and performing periodic update processing on each low priority data.

[0064] In this embodiment, real-time update processing refers to the process of immediately synchronizing data to related systems when it changes, and periodic update processing refers to the process of updating data according to a predetermined time interval or plan (such as daily, weekly, or monthly updates). By classifying data according to priority and performing real-time update processing on high-priority data, it is possible to ensure that critical and urgent data is transmitted and processed in a timely manner, improving the system's response speed and accuracy; performing periodic update processing on low-priority data can avoid unnecessary resource waste, focusing limited transmission and processing resources on high-priority data, and improving resource utilization efficiency; dynamically adjusting the update frequency according to data priority allows the system to more flexibly respond to the update needs of different data types, improving the system's adaptability and scalability.

[0065] like Figure 3As shown, it is a structural diagram of a foreign trade business data collaborative analysis system based on big data provided by an embodiment of the present application. The foreign trade business data collaborative analysis system based on big data provided by an embodiment of the present application includes: a transmission performance optimization module, a task data scheduling module, an update frequency adjustment module and a foreign trade collaborative database; wherein, the transmission performance optimization module is used to monitor the transmission performance data of each department in real time, and dynamically optimize the data transmission process according to the transmission performance data of each department; the task data scheduling module is used to monitor the current task volume completed by each department in real time, and dynamically transmit the task data according to the current task volume completed by each department and the transmission performance of each department; the update frequency adjustment module is used to monitor the update information of each department in real time, and dynamically adjust the task data update frequency according to the update information of each department and the system transmission performance.

[0066] In summary, the embodiments of the present application monitor the transmission performance data of each department in real time, and dynamically optimize the data transmission process according to the transmission performance data of each department; monitor the current task volume completed by each department in real time, and dynamically transmit the task data according to the current task volume completed by each department and the transmission performance of each department; monitor the update information of each department in real time, and dynamically adjust the task data update frequency according to the update information of each department and the system transmission performance, which not only improves the efficiency and responsiveness of the method, but also ensures the rational use of resources, thereby improving the efficiency of the entire workflow.

[0067] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0069] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0070] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0071] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0072] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A collaborative analysis method for foreign trade business data based on big data, characterized in that: The following steps are involved: Real-time monitoring of the transmission performance data of each department, and dynamic optimization of the data transmission process based on the transmission performance data of each department; Monitor the current task volume of each department in real time, and dynamically transmit task data based on the current task volume and transmission performance of each department; Monitor the updated information of each department in real time, and dynamically adjust the task data update frequency based on the updated information of each department and the system transmission performance.

2. The method for collaborative analysis of foreign trade business data based on big data according to claim 1, characterized in that: The steps of real-time monitoring of the transmission performance data of each department and dynamic optimization of the data transmission process according to the transmission performance data of each department are as follows: Perform quantitative evaluation of transmission performance based on the transmission performance data of each department to obtain the system transmission performance evaluation value; Obtaining a transmission performance evaluation threshold from a preset foreign trade collaboration database, and comparing the system transmission performance evaluation value with the transmission performance evaluation threshold; If the system transmission performance evaluation value is greater than or equal to the transmission performance evaluation threshold, no additional processing is performed; If the system transmission performance evaluation value is less than the transmission performance evaluation threshold, the load balancing mode is turned on; If the system transmission performance evaluation value is still less than the transmission performance evaluation threshold after the load balancing mode is turned on, the current system transmission performance evaluation value is matched with the cache capacity corresponding to the transmission performance evaluation values ​​of each system in the preset foreign trade collaborative database to obtain the corresponding cache capacity, and the task data is queued and cached according to the data criticality parameters and the obtained cache capacity.

3. The method for collaborative analysis of foreign trade business data based on big data according to claim 2, characterized in that: The transmission performance data includes the data transmission rate of each task, the data transmission delay time of each task, the data reception delay time of each task, and the data processing delay time of each task; The step of performing a quantitative evaluation of transmission performance based on the transmission performance data of each department and obtaining a system transmission performance evaluation value includes: Obtaining the critical task data transmission rate, critical task data transmission delay time, critical task data reception delay time and critical task data processing delay time from the foreign trade collaborative database; Perform an average calculation of the proportion of the data transmission rate of each task and the data transmission rate of the critical task, and then perform a weighted calculation on the calculation results to obtain the influencing parameters of the task data transmission rate; Perform a proportion approximation operation on the data transmission delay time of each task, the data reception delay time of each task, and the data processing delay time of each task, respectively, with the critical task data transmission delay time, the critical task data reception delay time, and the critical task data processing delay time, perform weighted operation on the operation results, and then perform coupling processing. After averaging the coupling processing results, perform inverse proportional operation to obtain the data delay time influencing parameters; The task data transmission rate influencing parameters and data delay time influencing parameters are coupled and processed to obtain the transmission performance evaluation value of each department. The result of the mean processing of the transmission performance evaluation value of each department is recorded as the system transmission performance evaluation value.

4. The method for collaborative analysis of foreign trade business data based on big data according to claim 2, characterized in that: The key parameters of the data include the fluctuation range, change frequency and number of related departments in each detection time period; The step of queuing and caching the task data according to the data criticality parameter and the obtained cache capacity includes: Obtain critical fluctuation amplitude, critical change frequency and critical number of related departments from the preset foreign trade collaboration database; The fluctuation amplitude and change frequency of each detection time period are respectively subjected to a proportion approaching degree calculation with the critical fluctuation amplitude and critical change frequency, and the proportion approaching degree calculation results are weighted and then coupled, and the homogenized result of the coupling processing result is recorded as the data change influencing parameter; Perform a ratio approximation operation on the number of related departments and the number of critical related departments, and then perform a weighting operation on the result of the ratio approximation operation to obtain the influencing parameter of the number of related departments; The data change influencing parameters and the related department quantity influencing parameters are coupled to obtain the priority evaluation value of each task data; The task data are sorted from large to small according to the priority evaluation value of each task data, and the task data are queued and cached within the cache capacity according to the sorting result.

5. The method for collaborative analysis of foreign trade business data based on big data according to claim 1, characterized in that: The steps of real-time monitoring of the current amount of tasks completed by each department and dynamically transmitting task data according to the current amount of tasks completed by each department and the transmission performance of each department include: Obtain the task volume, warning time and completed task volume of each department; When a department reaches the department warning time, if the department's current completed task volume has not reached the department's warning completed task volume, and the department's transmission performance evaluation value is less than the transmission performance evaluation threshold, the completed task data will be transmitted first, and the block size will be dynamically adjusted according to the department's transmission performance evaluation value and the current completed task volume. The unfinished task data will be transported in blocks according to the priority evaluation value of each block; If the department's current completed task volume does not reach the department's early warning completed task volume, and the department's transmission performance evaluation value is greater than or equal to the transmission performance evaluation threshold, the completed task data will be transported synchronously, and the unfinished task data will be quickly distributed through parallel transmission; If the department's current completed task volume reaches the department's pre-warning completed task volume, or the department's current completed task volume does not reach the department's pre-warning completed task volume but the task data can be processed in segments, the completed task data will be transmitted in full, and only the incremental part of the unfinished task data will be transmitted.

6. The method for collaborative analysis of foreign trade business data based on big data according to claim 5, characterized in that: The step of dynamically adjusting the block size according to the department's transmission performance evaluation value and the current amount of completed tasks, and transporting unfinished task data in blocks according to the priority evaluation value of each block includes: The difference between the department's task volume and the department's currently completed task volume is marked as unfinished task volume; Obtain reference block size, reference unfinished task volume, unit transmission performance evaluation value and unit unfinished task volume from a preset foreign trade collaboration database; The differences between the transmission performance evaluation threshold and the unfinished task volume and the transmission performance evaluation value of the department and the reference unfinished task volume are marked as the deviation transmission performance evaluation value and the deviation unfinished task volume respectively; obtaining a transmission performance adjustment unit based on the deviation transmission performance evaluation value and the unit transmission performance evaluation value, wherein the transmission performance adjustment unit represents an integer multiple relationship between the deviation transmission performance evaluation value and the unit transmission performance evaluation value; Obtaining a task quantity adjustment unit based on the deviation uncompleted task quantity and the unit uncompleted task quantity, wherein the task quantity adjustment unit represents an integer multiple relationship between the deviation uncompleted task quantity and the unit uncompleted task quantity; Based on the reference block size, the transmission performance adjustment unit is reduced in real time, while the task volume adjustment unit is increased, to obtain the block size of each time monitoring point. The unfinished task data is divided according to the block size of each time monitoring point to obtain the data of each block; The task data priority evaluation values ​​of each block data are averaged to obtain the priority evaluation value of each block, and the priority evaluation values ​​of each block are sorted from large to small, and the unfinished task data are transported in blocks according to the sorting.

7. The method for collaborative analysis of foreign trade business data based on big data according to claim 1, characterized in that: The steps of real-time monitoring of the updated information of each department and dynamically adjusting the task data update frequency according to the updated information of each department and the system transmission performance include: Conduct a quantitative assessment of system update intensity based on the update information of each department and obtain the system update intensity assessment value; Matching the system transmission performance evaluation value with the system update strength evaluation threshold corresponding to each system transmission performance evaluation value preset in the foreign trade collaborative database to obtain the system update strength evaluation threshold corresponding to the system transmission performance evaluation value; Compare the system update strength assessment value with the system update strength assessment threshold: If the system update strength assessment value is less than or equal to the system update strength assessment threshold, no additional processing is performed; If the system update intensity evaluation value is greater than the system update intensity evaluation threshold, the task data update frequency is dynamically adjusted according to the priority evaluation value of each task data.

8. The method for collaborative analysis of foreign trade business data based on big data according to claim 7, characterized in that: The department update information includes task processing frequency, data update frequency and data change frequency; The step of performing a quantitative evaluation of the system update intensity based on the update information of each department and obtaining the system update intensity evaluation value includes: Obtain critical task processing frequency, critical data update frequency and critical data change frequency from the foreign trade collaborative database; After performing a proportion approximation operation on the task processing frequency and the critical task processing frequency, the proportion approximation operation result is subjected to an update intensity influence proportion operation to obtain the task processing frequency influence parameter; Perform a proportion approximation operation on each data update frequency and each data change frequency with the critical data update frequency and the critical data change frequency, perform a proportion approximation operation on the results of the proportion approximation operation, and then perform a coupling process on the results of the update intensity influence proportion, and record the homogenized result of the coupling process as the data update influence parameter; The task processing frequency influencing parameters and data update influencing parameters are coupled to obtain the system update intensity evaluation value.

9. The method for collaborative analysis of foreign trade business data based on big data according to claim 7, characterized in that: The step of dynamically adjusting the task data update frequency according to the priority evaluation value of each task data includes: Obtain data priority assessment thresholds from the foreign trade collaborative database; Compare the priority evaluation value of each task data with the data priority evaluation threshold. If the priority evaluation value of a task data is greater than or equal to the data priority evaluation threshold, the task data is marked as high priority data. If the priority evaluation value of a task data is less than the data priority evaluation threshold, the task data is marked as low priority data. Collect statistics on each high-priority data and each low-priority data, perform real-time update processing on each high-priority data, and perform regular update processing on each low-priority data.

10. A foreign trade business data collaborative analysis system based on big data, characterized in that: It includes transmission performance optimization module, task data scheduling module, update frequency adjustment module and foreign trade collaboration database; The transmission performance optimization module is used to monitor the transmission performance data of each department in real time and dynamically optimize the data transmission process according to the transmission performance data of each department; The task data scheduling module is used to monitor the current task volume of each department in real time and dynamically transmit task data based on the current task volume of each department and the transmission performance of each department; The update frequency adjustment module is used to monitor the update information of each department in real time and dynamically adjust the task data update frequency according to the update information of each department and the system transmission performance.

Images