Technical trade barrier risk assessment system based on big data

Abstract

The invention relates to the cross technical field of big data processing and international trade risk assessment, in particular to a big data-based technical trade barrier risk assessment system, which comprises a data acquisition module, a data preprocessing module, a risk assessment module and a result output module, the data acquisition module collects multi-source trade related data through a distributed crawler node cluster and transmits the multi-source trade related data to the data preprocessing module; the data preprocessing module performs abnormal value elimination and feature dimension reduction processing on the received original data and then transmits a standardized data set to the risk assessment module; the risk assessment module calculates a risk value based on a multi-dimensional index system and a mixed weighting algorithm and transmits an assessment result to the result output module; and the result output module displays the risk level and the index contribution degree distribution through an interactive visual interface. According to the technical trade barrier risk assessment method, accurate assessment, credible evidence storage and coping of technical trade barrier risks are realized by acquiring multi-source trade data, optimizing data quality, calculating risk values, storing evidence and performing early warning.

Description

Technical Field

[0001] This invention relates to the field of big data processing and international trade risk assessment, and in particular to a big data-based technical trade barrier risk assessment system. Background Technology

[0002] With the acceleration of global trade integration, technical barriers to trade, involving multiple dimensions such as technical standards, quarantine and testing, and environmental restrictions, have gradually become key factors affecting international trade flows, and related risk assessment technologies have also developed accordingly. Early risk assessments of technical barriers to trade relied heavily on manual processing of limited trade data, combined with industry experience for qualitative judgments, resulting in low efficiency and high subjectivity. With the rise of big data technology, the industry began to introduce distributed web crawling tools to collect multi-source trade data, using single algorithms such as the analytic hierarchy process (AHP) and entropy weighting to calculate risk values, and attempting to display assessment results through basic charts, which improved the objectivity and efficiency of the assessment to some extent. However, the overall technical system remains in a fragmented development stage, and the coordination of data processing, model calculation, result presentation, and storage has not yet formed a complete architecture.

[0003] Although existing risk assessment technologies have initially integrated big data and algorithm tools, many shortcomings remain to be addressed in practical applications. In the data processing stage, current technologies often only perform simple cleaning on multi-source trade data, resulting in incomplete outlier removal and poor feature dimensionality reduction. This leads to inconsistent dataset quality input into the assessment model, directly impacting the accuracy of risk calculations. Regarding the assessment models, weight determination often relies on a single subjective or objective weighting algorithm, making it difficult to balance human experience with objective data laws. Furthermore, the model parameters lack a dynamic iterative optimization mechanism, failing to adapt to the differentiated needs of various industry trade scenarios. In addition, the storage of assessment data lacks reliable tamper-proof protection, easily leading to disputes over data credibility. The presentation of risk results is simplistic, failing to intuitively show the risk distribution and indicator contribution relationship. Moreover, risk warning thresholds are often fixed and cannot be dynamically adjusted based on historical data and industry benchmarks, resulting in delayed or misjudged warning responses and hindering effective support for early avoidance of trade risks. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned problems and provide a technical trade barrier risk assessment system based on big data. To achieve the above objective, this invention adopts the following technical solution:

[0005] A big data-based technical trade barrier risk assessment system includes a data acquisition module, a data preprocessing module, a risk assessment module, and a result output module. The data acquisition module collects multi-source trade-related data through a distributed crawler node cluster and transmits it to the data preprocessing module. The data preprocessing module performs outlier removal and feature dimensionality reduction on the received raw data and then transmits the standardized dataset to the risk assessment module. The risk assessment module calculates the risk value based on a multi-dimensional indicator system and a hybrid weighted algorithm and transmits the assessment result to the result output module. The result output module displays the risk level and indicator contribution distribution through an interactive visualization interface.

[0006] Furthermore, the data preprocessing module includes an outlier handling module, which uses the Z-score algorithm to identify outlier data. The calculation formula is as follows:

[0007]

[0008] Among them, Z i Let X be the standardized deviation value of the i-th sample. i Let |Z| be the index value of the i-th sample, μ be the mean of all samples of this index, and σ be the standard deviation of all samples of this index. i |>Z α / 2 At that time, this sample was marked as an outlier and corrected using the nearest neighbor mean replacement method, where α is the significance level and Z is the mean. α / 2 α represents the standard normal distribution quantile corresponding to the significance level α.

[0009] Furthermore, the risk assessment module includes a dynamic weighting module, which integrates subjective and objective weights through a combination algorithm of the analytic hierarchy process (AHP) and the entropy weighting method. Specifically, the dynamic weighting module includes the following steps:

[0010] Step S1: Use the Analytic Hierarchy Process (AHP) to determine subjective weights; construct the judgment matrix: A = (a ij ) n×n ;

[0011] Where n is the total number of evaluation indicators, a ij Let a be a scale representing the relative importance of the i-th indicator and the j-th indicator. ij >0 and Find the largest eigenvalue λ of the matrix max The reliability of the results is verified by a consistency test, where CR is the consistency ratio. When CR < 0.1, the test is considered passed.

[0012] Step S2: Calculate the objective weights using the entropy weight method;

[0013] Step S21: Calculate the information entropy of the indicator:

[0014]

[0015] Among them, H j Let X be the information entropy of the j-th indicator, n be the number of samples, j be the indicator number, j = 1, 2, ..., m, m be the total number of indicators, and X be the information entropy of the j-th indicator. ij Let j be the value of the i-th indicator in the i-th sample. Let X be the normalized value of the i-th sample of the j-th indicator, and X ij ≥0;

[0016] Step S22: Calculate entropy weights based on information entropy:

[0017]

[0018] Among them, e j Let H be the objective weight of the j-th indicator, m be the total number of indicators, and H be the weight of the j-th indicator. j Let the information entropy of the j-th indicator be denoted as .

[0019] Step S3: Integrate subjective weights and objective weights:

[0020] The weighted formula is: W j =α·w j +(1-α)·e j ;

[0021] Among them, W j Let w be the combined weight of the j-th indicator, α be the adjustment coefficient, 0 < α < 1, and w j Let e ​​be the subjective weight of the j-th indicator obtained by the analytic hierarchy process. j Let be the objective weight of the j-th index obtained by the entropy weight method.

[0022] Furthermore, the risk assessment module calculates the comprehensive risk value using a weighted summation algorithm, as shown in the following formula:

[0023]

[0024] Where R is the overall risk value, m is the total number of indicators, and W j S represents the combined weight of the j-th indicator. j S is the standardized score of the j-th indicator; j Calculated using the exponential membership function:

[0025] S j =1-exp(-k·d j )

[0026] Among them, S j Let d be the standardized score of the j-th indicator. j Let be the deviation rate between the actual value and the benchmark value of the j-th indicator. X j X is the actual measured value of the j-th indicator, X0 is the industry benchmark value of the j-th indicator and X0>0, and k is the industry adjustment coefficient and k>0, which is determined according to industry characteristics.

[0027] Furthermore, the risk assessment module also includes a model optimization module, which uses stochastic gradient descent to iteratively update the weight parameters. The iterative formula is as follows:

[0028]

[0029] Among them, W t+1 Let W be the weight parameter vector updated in the (t+1)th iteration. t Let be the weight parameter vector for the t-th iteration, where t is the iteration number and η is the learning rate (η>0), with values ​​chosen based on the model training requirements. For the cross-entropy loss function in W t The gradient at point L is the cross-entropy loss function;

[0030] The cross-entropy loss function is defined as follows:

[0031]

[0032] Where L is the cross-entropy loss function value, n is the number of samples, and y i Let y be the true risk level label for the i-th sample. i =0 indicates low risk, y i =1 indicates high risk. Let the model predict the probability that the i-th sample belongs to high risk.

[0033] Furthermore, the data acquisition module includes a multi-protocol interface module, which includes a distributed message queue architecture and connects to a Kafka cluster. The Kafka cluster constructs an asynchronous data channel. In the asynchronous data channel, each data source corresponds to an independent topic and has data isolation attributes. The number of message partitions matches the concurrency of the data sources and has load balancing attributes. The multi-protocol interface module includes a consumer group and a message acknowledgment mechanism. The consumer group is equipped with a polling mechanism to allocate message consumption tasks, and the message acknowledgment mechanism is associated with data transmission and has reliability attributes.

[0034] Furthermore, the blockchain storage module adopts a consortium blockchain architecture to achieve tamper-proof storage of evaluation data. Each block includes a data header, a data body, and consensus mechanism information: the data header includes a version number, the previous block hash value, Merkle root hash, and a timestamp; the data body includes the original data hash digest, the evaluation result signature, and node consensus information; the consensus mechanism adopts a practical Byzantine fault-tolerant algorithm, and achieves data consistency verification between nodes through a three-stage submission process of pre-preparation, preparation, and submission.

[0035] Furthermore, the multi-dimensional indicator system includes primary indicators and secondary indicators. The primary indicators include technical standard compliance, certification process complexity, quarantine and testing requirements, environmental restrictions, intellectual property barriers, and trade remedy measures. Each primary indicator is further divided into secondary indicators. Technical standard compliance includes international standard matching degree, standard update frequency, and standard differences.

[0036] Furthermore, the results output module supports three interactive visualization formats, including a risk heatmap, a parameter sensitivity curve, and a contribution dome chart. The risk heatmap is based on a geographic information system and uses red, yellow, and green gradients to indicate the spatial distribution of global trade barrier risks. The parameter sensitivity curve displays the impact of key indicators on the overall risk value within a fluctuation range in the form of a line graph. The contribution dome chart visually presents the contribution weight of each primary indicator to the total risk value through a two-layered dome sector ratio, with the inner ring showing the subjective weight ratio and the outer ring showing the objective weight ratio.

[0037] Furthermore, it also includes an early warning response module, which responds when the comprehensive risk value R ≥ R 阈值 The system automatically triggers a three-level early warning mechanism and calls upon the response strategy library to generate differentiated solutions; the R... 阈值 The calculation formula is dynamically adjusted based on historical data and industry benchmarks, as follows:

[0038]

[0039] Among them, R 阈值 As a risk warning threshold, R is the historical average risk value. 行业 The risk value is the industry benchmark, β is the adjustment coefficient and 0 < β < 1, and the value is determined based on the weight requirements of historical data and industry benchmark; the response strategy library includes standard alternatives, simplified compliance process paths and negotiation scripts for trading partners.

[0040] The advantages of this invention are:

[0041] 1. This invention utilizes a distributed crawler node cluster to collect multi-source trade-related data through a data acquisition module. A data preprocessing module performs outlier removal and feature dimensionality reduction to optimize data quality. A risk assessment module then calculates risk values ​​based on a multi-dimensional indicator system and a hybrid weighted algorithm. This achieves a comprehensive and accurate assessment of technical trade barriers risks, effectively solving the problem of risk assessment bias caused by the disorganization and inconsistent quality of multi-source trade data, and providing reliable data support for trade risk prediction.

[0042] 2. This invention uses a consortium blockchain architecture to store assessment data through a blockchain storage module, ensuring that the data is tamper-proof. At the same time, the result output module displays risk information in interactive and visual forms such as risk heatmaps, parameter sensitivity curves, and contribution ring charts. This achieves the assurance of the credibility of assessment data and the intuitive presentation of risk information, allowing users to clearly understand the spatial distribution of risks, the magnitude of the influence of indicators, and their weight contributions, thereby improving the readability and efficiency of risk information.

[0043] 3. This invention improves the accuracy of the evaluation model by using a stochastic gradient descent method to iteratively update weight parameters through a model optimization module. Combined with an early warning response module, it dynamically adjusts the risk threshold based on historical data and industry benchmarks, triggers a three-level early warning mechanism, and calls upon the response strategy library to generate solutions. This enables accurate early warning and differentiated response to technical trade barrier risks, reduces trade losses caused by delayed risk prediction or a single response plan, and helps users efficiently avoid the impact of trade barriers. Attached Figure Description

[0044] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application.

[0045] In the attached diagram:

[0046] Figure 1 This is a system framework diagram of the big data-based technical trade barrier risk assessment system in Example 1.

[0047] Figure 2 This is a flowchart of the early warning response mechanism in the big data-based technical trade barrier risk assessment system in Example 1. Detailed Implementation

[0048] The present invention will now be described in detail and specifically through specific embodiments to enable a better understanding of the invention. However, the following embodiments do not limit the scope of protection of the present invention.

[0049] Example 1

[0050] like Figure 1-2As shown, the big data-based technical trade barrier risk assessment system includes a data acquisition module, a data preprocessing module, a risk assessment module, and a result output module. The data acquisition module collects multi-source trade-related data through a distributed crawler node cluster and transmits it to the data preprocessing module. The data preprocessing module performs outlier removal and feature dimensionality reduction on the received raw data and then transmits the standardized dataset to the risk assessment module. The risk assessment module calculates the risk value based on a multi-dimensional indicator system and a hybrid weighted algorithm and transmits the assessment result to the result output module. The result output module displays the risk level and indicator contribution distribution through an interactive visualization interface.

[0051] In a specific embodiment, when building a big data-based technical trade barrier risk assessment system, the data acquisition module uses an Alibaba Cloud ECS t6.xlarge server to construct a distributed crawler node cluster, collecting multi-source trade-related data from the WTO technical trade barrier notification database, customs import and export statistics, and industry standard databases, and transmitting it to the data preprocessing module in real time. The data preprocessing module uses the Python Pandas library to perform outlier removal and PCA feature dimensionality reduction, outputting a standardized dataset. The risk assessment module calculates risk values ​​based on a multi-dimensional indicator system covering six primary indicators, combined with a hybrid weighted algorithm of analytic hierarchy process (AHP) and entropy weighting. The results output module uses ECharts 5.4.3 to develop an interactive visualization interface, displaying the risk level and indicator contribution distribution, effectively improving the comprehensiveness of trade data collection and the intuitiveness of risk assessment, providing efficient support for enterprise trade decisions.

[0052] Furthermore, the data preprocessing module includes an outlier handling module, which uses the Z-score algorithm to identify outlier data. The calculation formula is as follows:

[0053]

[0054] Among them, Z i Let X be the standardized deviation value of the i-th sample. i Let |Z| be the index value of the i-th sample, μ be the mean of all samples of this index, and σ be the standard deviation of all samples of this index. i |>Z α / 2 At that time, this sample was marked as an outlier and corrected using the nearest neighbor mean replacement method, where α is the significance level and Z is the mean. α / 2 α represents the standard normal distribution quantile corresponding to the significance level α.

[0055] In a specific embodiment, when the outlier handling module of the data preprocessing module adopts the Z-score algorithm, it selects RoHS test index data of 100 products in a certain electronics industry as samples. The mean μ of all samples of this index is set to 0.5 mg / kg, the standard deviation σ is set to 0.1 mg / kg, the significance level α is set to 0.05, and the corresponding standard normal distribution quantile Z α / 2 The value was 1.96. When the index value X of the 32nd sample was detected... i When the concentration is 0.8 mg / kg, the calculated value is... The sample with an absolute value greater than 1.96 was marked as an outlier and was subsequently replaced with the mean of the 31st and 33rd nearest samples, which was 0.52 mg / kg. This effectively eliminated the interference of outlier data and ensured the accuracy of the dataset used in the subsequent risk assessment.

[0056] Furthermore, the risk assessment module includes a dynamic weighting module, which integrates subjective and objective weights through a combination algorithm of the analytic hierarchy process (AHP) and the entropy weighting method. Specifically, the dynamic weighting module includes the following steps:

[0057] Step S1: Use the Analytic Hierarchy Process (AHP) to determine subjective weights; construct the judgment matrix: A = (a ij ) n×n ;

[0058] Where n is the total number of evaluation indicators, a ij Let a be a scale representing the relative importance of the i-th indicator and the j-th indicator. ij >0 and Find the largest eigenvalue λ of the matrix max The reliability of the results is verified by a consistency test, where CR is the consistency ratio. When CR < 0.1, the test is considered passed.

[0059] Step S2: Calculate the objective weights using the entropy weight method;

[0060] Step S21: Calculate the information entropy of the indicator:

[0061]

[0062] Among them, H j Let X be the information entropy of the j-th indicator, n be the number of samples, j be the indicator number, j = 1, 2, ..., m, m be the total number of indicators, and X be the information entropy of the j-th indicator. ij Let j be the value of the i-th indicator in the i-th sample. Let X be the normalized value of the i-th sample of the j-th indicator, and X ij ≥0;

[0063] Step S22: Calculate entropy weights based on information entropy:

[0064]

[0065] Among them, e j Let H be the objective weight of the j-th indicator, m be the total number of indicators, and H be the weight of the j-th indicator. j Let the information entropy of the j-th indicator be denoted as .

[0066] Step S3: Integrate subjective weights and objective weights:

[0067] The weighted formula is: W j =α·w j +(1-α)·e j ;

[0068] Among them, W j Let w be the combined weight of the j-th indicator, α be the adjustment coefficient, 0 < α < 1, and w j Let e ​​be the subjective weight of the j-th indicator obtained by the analytic hierarchy process. j Let be the objective weight of the j-th index obtained by the entropy weight method.

[0069] When implementing the dynamic weighting module of the risk assessment module, the subjective weights are first determined using the analytic hierarchy process: for the three secondary indicators under technical standard compliance, with n=3, a judgment matrix A is constructed, where a... 12 The value is 2, meaning that the international standard matching degree is twice as important as the standard update frequency. 13 The value is 3, meaning that the international standard matching degree is 3 times more important than the standard difference. 23 The value is 2, meaning the standard update frequency is twice as important as the standard variability. Solving for the maximum eigenvalue λ yields this value. max The coefficient of performance is 3.005, the eigenvector w is [0.54, 0.29, 0.17], and the consistency ratio CR is 0.004, which is less than 0.1, thus passing the test. Next, the objective weights are calculated using the entropy weight method: 50 samples are selected, and the information entropy of each indicator is calculated as follows: H1 = 0.82, H2 = 0.88, H3 = 0.91, corresponding to entropy weights e of [0.42, 0.33, 0.25]. Finally, the adjustment coefficient α is set to 0.6, and the weights are calculated according to the formula W... j =0.6×w j +0.4×e j The combined weights W are [0.49, 0.31, 0.20], which balances subjective and objective factors and improves the rationality of weight allocation.

[0070] Furthermore, the risk assessment module calculates the comprehensive risk value using a weighted summation algorithm, as shown in the following formula:

[0071]

[0072] Where R is the overall risk value, m is the total number of indicators, and W j S represents the combined weight of the j-th indicator. jis the standardized score of the j-th indicator;

[0073] The S j Calculated using the exponential membership function:

[0074] S j =1-exp(-k·d j )

[0075] Among them, S j Let d be the standardized score of the j-th indicator. j Let be the deviation rate between the actual value and the benchmark value of the j-th indicator. X j X is the actual measured value of the j-th indicator, X0 is the industry benchmark value of the j-th indicator and X0>0, and k is the industry adjustment coefficient and k>0, which is determined according to industry characteristics.

[0076] In a specific embodiment, when the risk assessment module calculates the comprehensive risk value, the total number of indicators m is taken as 3, and the combined weights W1, W2, and W3 are 0.49, 0.31, and 0.20, respectively. When calculating the standardized score S1 of the first indicator, the actual measured value X1 of this indicator is 85, which is the international standard matching score, the industry benchmark value X0 is 90, and the deviation rate is... The industry adjustment coefficient k is 0.8. Substituting this into the exponential membership function, we get S1 = 1 - exp(-0.8 × 0.056) ≈ 0.044. Similarly, we calculate S2 as 0.032 and S3 as 0.021, and then apply the formula... The calculated value of R is approximately 0.49 × 0.044 + 0.31 × 0.032 + 0.20 × 0.021 ≈ 0.035, which accurately quantifies the overall risk level and truly reflects the degree of technical trade barrier risk faced by the company's products.

[0077] Furthermore, the risk assessment module also includes a model optimization module, which uses stochastic gradient descent to iteratively update the weight parameters. The iterative formula is as follows:

[0078]

[0079] Among them, W t+1 Let W be the weight parameter vector updated in the (t+1)th iteration. t Let be the weight parameter vector for the t-th iteration, where t is the iteration number and η is the learning rate (η>0), with values ​​chosen based on the model training requirements. For the cross-entropy loss function in W t The gradient at point L is the cross-entropy loss function;

[0080] The cross-entropy loss function is defined as follows:

[0081]

[0082] Where L is the cross-entropy loss function value, n is the number of samples, and y i Let y be the true risk level label for the i-th sample. i =0 indicates low risk, y i =1 indicates high risk. Let the model predict the probability that the i-th sample belongs to high risk.

[0083] In a specific embodiment, when the model optimization module of the risk assessment module adopts the stochastic gradient descent method, the initial iteration t is set to 1, and the initial weight parameter vector W... t The learning rate η is set to [0.49, 0.31, 0.20], and the learning rate η is 0.01. When calculating the cross-entropy loss function, 10 samples are selected, with the 5th sample having a true risk level label y5 of 1. The model predicts that this sample belongs to the high-risk category. The value is 0.82. The loss function value is calculated for the remaining samples. Then the gradient is obtained. Given [0.02, 0.015, 0.01], according to the iterative formula... Updated W t+1 With values ​​of [0.4898, 0.30985, 0.1999], the weight parameters were continuously optimized through multiple rounds of iteration, which significantly improved the prediction accuracy of the risk assessment model.

[0084] Furthermore, the data acquisition module includes a multi-protocol interface module, which includes a distributed message queue architecture and connects to a Kafka cluster. The Kafka cluster constructs an asynchronous data channel. In the asynchronous data channel, each data source corresponds to an independent topic and has data isolation attributes. The number of message partitions matches the concurrency of the data sources and has load balancing attributes. The multi-protocol interface module includes a consumer group and a message acknowledgment mechanism. The consumer group is equipped with a polling mechanism to allocate message consumption tasks, and the message acknowledgment mechanism is associated with data transmission and has reliability attributes.

[0085] In a specific embodiment, the multi-protocol interface module of the data acquisition module uses the Huawei CloudLink EC600S model. This module integrates the RabbitMQ distributed message queue architecture and constructs an asynchronous data channel through a Confluent Kafka 7.0 cluster. In the asynchronous data channel, the WTO notification data source corresponds to the WTO-TBT independent topic, and the customs data corresponds to the Customs-Data independent topic, achieving data isolation. The number of message partitions is set to 4, matching the concurrency of the 4 data sources to ensure load balancing. The consumer group of the multi-protocol interface module uses the Apache Kafka Consumer component, which allocates message consumption tasks through a polling mechanism and enables Kafka's ACK message confirmation mechanism to associate data transmission, ensuring the stability and reliability of multi-source trade data transmission and avoiding data loss or transmission delay issues.

[0086] Furthermore, the blockchain storage module adopts a consortium blockchain architecture to achieve tamper-proof storage of evaluation data. Each block includes a data header, a data body, and consensus mechanism information: the data header includes a version number, the previous block hash value, Merkle root hash, and a timestamp; the data body includes the original data hash digest, the evaluation result signature, and node consensus information; the consensus mechanism adopts a practical Byzantine fault-tolerant algorithm, and achieves data consistency verification between nodes through a three-stage submission process of pre-preparation, preparation, and submission.

[0087] In a specific embodiment, the blockchain storage module adopts the AntChain consortium blockchain architecture. Each block's header is set to version number V1.0, the previous block hash value is 7a3f92e5d1c4b8a06789012345abcdef, and the Merklegen hash is 5d8e7c6b5a4f3e2d1c0b9a8f7e6d5c4b. The timestamp is accurate to 2025-05-2014:30:00. The data body contains the SHA-256 hash digest of the original trade data (9b2c8d7e6f5a4b3c2d1e0f9a8b7c6d5e), the RSA algorithm signature of the risk assessment result, and consensus information from the five consortium nodes. The consensus mechanism employs a practical Byzantine fault-tolerant algorithm, using a three-stage submission process (pre-preparation, preparation, and submission) to verify data consistency between nodes, ensuring the immutability of the assessment data and improving data credibility.

[0088] Furthermore, the multi-dimensional indicator system includes primary indicators and secondary indicators. The primary indicators include technical standard compliance, certification process complexity, quarantine and testing requirements, environmental restrictions, intellectual property barriers, and trade remedy measures. Each primary indicator is further divided into secondary indicators. Technical standard compliance includes international standard matching degree, standard update frequency, and standard differences.

[0089] In a specific embodiment, the primary indicators of the multi-dimensional indicator system include technical standard compliance, certification process complexity, quarantine and testing requirements, environmental restrictions, intellectual property barriers, and trade remedy measures. Each primary indicator has secondary indicators. The secondary indicators for technical standard compliance specifically include international standard matching degree, such as the matching degree between the company's products and the ISO 9001 standard, standard update frequency, such as the technical standards in the target market being updated twice a year, and standard difference, such as the difference rate between the company's implemented standard and the EU CE standard being 15%. Other primary indicators, such as certification process complexity, have two secondary indicators: the number of certification steps and the certification cycle. This comprehensively covers the core impact dimensions of technical trade barriers and provides a systematic and complete indicator support for risk assessment.

[0090] Furthermore, the results output module supports three interactive visualization formats, including a risk heatmap, a parameter sensitivity curve, and a contribution dome chart. The risk heatmap is based on a geographic information system and uses red, yellow, and green gradients to indicate the spatial distribution of global trade barrier risks. The parameter sensitivity curve displays the impact of key indicators on the overall risk value within a fluctuation range in the form of a line graph. The contribution dome chart visually presents the contribution weight of each primary indicator to the total risk value through a two-layered dome sector ratio, with the inner ring showing the subjective weight ratio and the outer ring showing the objective weight ratio.

[0091] In specific embodiments, the three interactive visualization formats supported by the results output module are all developed based on ECharts 5.4.3. The risk heatmap, based on the Gaode GIS map, uses red to mark regions with a comprehensive risk value greater than 0.8 (e.g., EU member states), yellow to mark regions with a comprehensive risk value between 0.5 and 0.8 (e.g., Southeast Asian countries), and green to mark regions with a comprehensive risk value less than 0.5 (e.g., some South American countries), visually presenting the spatial distribution of global risk. The parameter sensitivity curve, in the form of a line graph, shows the change in the comprehensive risk value corresponding to ±0.08 when the international standard matching degree fluctuates by ±10%. The contribution pie chart visually presents the contribution weight of each primary indicator to the total risk value through a two-layer pie chart sector ratio. The inner ring shows the subjective weight ratio (30% for technical standard compliance), and the outer ring shows the objective weight ratio (28% for technical standard compliance), allowing users to quickly grasp the key influencing factors of risk and improving the efficiency of risk information interpretation.

[0092] Furthermore, it also includes an early warning response module, which responds when the comprehensive risk value R ≥ R 阈值 The system automatically triggers a three-level early warning mechanism and calls upon the response strategy library to generate differentiated solutions; the R... 阈值 The calculation formula is dynamically adjusted based on historical data and industry benchmarks, as follows:

[0093]

[0094] Among them, R 阈值 As a risk warning threshold, R is the historical average risk value. 行业 The risk value is the industry benchmark, β is the adjustment coefficient and 0 < β < 1, and the value is determined based on the weight requirements of historical data and industry benchmark; the response strategy library includes standard alternatives, simplified compliance process paths and negotiation scripts for trading partners.

[0095] In a specific embodiment, when the early warning response module is running, the risk early warning threshold R 阈值 By dynamically adjusting historical data and industry benchmarks, with an adjustment coefficient β of 0.7, the historical average risk value is... The risk level is 0.4, compared to the industry benchmark risk value R. 行业 The value is 0.35, which is then substituted into the formula. Calculated R 阈值 =0.7×0.4+0.3×0.35=0.385. When the system calculates the comprehensive risk value R of a batch of products to be 0.4 and greater than 0.385, it automatically triggers a three-level early warning mechanism and simultaneously calls upon the response strategy library: for technical standard compliance risks, it provides a standard alternative to ISO 14001 using ISO 9001; for certification process risks, it provides a simplified compliance process path for the two audit stages; for communication with trading partners, it provides negotiation script templates adapted to the EU market, effectively helping companies to respond to trade barrier risks in a timely manner and reduce potential losses.

[0096] The specific embodiments of the present invention have been described in detail above, but they are merely examples, and the present invention is not equivalent to the specific embodiments described above. For those skilled in the art, any equivalent modifications and substitutions to the present invention are also within the scope of the present invention. Therefore, all equivalent transformations and modifications made without departing from the spirit and scope of the present invention should be covered within the scope of the present invention.

Claims

1. A big data-based technical trade barrier risk assessment system, characterized in that: The system includes a data acquisition module, a data preprocessing module, a risk assessment module, and a result output module. The data acquisition module collects multi-source trade-related data through a distributed crawler node cluster and transmits it to the data preprocessing module. The data preprocessing module performs outlier removal and feature dimensionality reduction on the received raw data and then transmits the standardized dataset to the risk assessment module. The risk assessment module calculates the risk value based on a multi-dimensional indicator system and a hybrid weighted algorithm and transmits the assessment result to the result output module. The results output module displays the risk level and indicator contribution distribution through an interactive visual interface.

2. The technical trade barrier risk assessment system based on big data according to claim 1, characterized in that, The data preprocessing module includes an outlier handling module, which uses the Z-score algorithm to identify outlier data. The calculation formula is as follows: Among them, Z i Let X be the standardized deviation value of the i-th sample. i Let |Z| be the index value of the i-th sample, μ be the mean of all samples of this index, and σ be the standard deviation of all samples of this index. i |>Z α / 2 At that time, this sample was marked as an outlier and corrected using the nearest neighbor mean replacement method, where α is the significance level and Z is the mean. α / 2 α represents the standard normal distribution quantile corresponding to the significance level α.

3. The technical trade barrier risk assessment system based on big data according to claim 2, characterized in that, The risk assessment module includes a dynamic weighting module, which integrates subjective and objective weights through a combination algorithm of the analytic hierarchy process and the entropy weighting method. Specifically, it includes the following steps: Step S1: Use the Analytic Hierarchy Process (AHP) to determine subjective weights; construct the judgment matrix: A = (a ij ) n×n ; Where n is the total number of evaluation indicators, a ij Let a be a scale representing the relative importance of the i-th indicator and the j-th indicator. ij >0 and Find the largest eigenvalue λ of the matrix max The reliability of the results is verified by a consistency test, where CR is the consistency ratio. When CR < 0.1, the test is considered passed. Step S2: Calculate the objective weights using the entropy weight method; Step S21: Calculate the information entropy of the indicator: Among them, H j Let X be the information entropy of the j-th indicator, n be the number of samples, j be the indicator number, j = 1, 2, ..., m, m be the total number of indicators, and X be the information entropy of the j-th indicator. ij Let j be the value of the i-th indicator in the i-th sample. Let X be the normalized value of the i-th sample of the j-th indicator, and X ij ≥0; Step S22: Calculate entropy weights based on information entropy: Among them, e j Let H be the objective weight of the j-th indicator, m be the total number of indicators, and H be the weight of the j-th indicator. j Let the information entropy of the j-th indicator be denoted as . Step S3: Integrate subjective weights and objective weights: The weighted formula is: W j =α·w j +(1-α)·e j ; Among them, W j Let w be the combined weight of the j-th indicator, α be the adjustment coefficient, 0 < α < 1, and w j Let e ​​be the subjective weight of the j-th indicator obtained by the analytic hierarchy process. j Let be the objective weight of the j-th index obtained by the entropy weight method.

4. The technical trade barrier risk assessment system based on big data according to claim 3, characterized in that, The risk assessment module calculates the comprehensive risk value using a weighted summation algorithm, as shown in the following formula: Where R is the overall risk value, m is the total number of indicators, and W j S represents the combined weight of the j-th indicator. j is the standardized score of the j-th indicator; The S j Calculated using the exponential membership function: S j =1-exp(-k·d j ) Among them, S j Let d be the standardized score of the j-th indicator. j Let be the deviation rate between the actual value and the benchmark value of the j-th indicator. X j X is the actual measured value of the j-th indicator, X0 is the industry benchmark value of the j-th indicator and X0>0, and k is the industry adjustment coefficient and k>0, which is determined according to industry characteristics.

5. The big data-based technical trade barrier risk assessment system according to claim 4, characterized in that, The risk assessment module also includes a model optimization module, which uses stochastic gradient descent to iteratively update the weight parameters. The iterative formula is as follows: Among them, W t+1 Let W be the weight parameter vector updated in the (t+1)th iteration. t Let be the weight parameter vector for the t-th iteration, where t is the iteration number and η is the learning rate (η>0), with values ​​chosen based on the model training requirements. For the cross-entropy loss function in W t The gradient at point L is the cross-entropy loss function; The cross-entropy loss function is defined as follows: Where L is the cross-entropy loss function value, n is the number of samples, and y i Let y be the true risk level label for the i-th sample. i =0 indicates low risk, y i =1 indicates high risk. Let the model predict the probability that the i-th sample belongs to high risk.

6. The technical trade barrier risk assessment system based on big data according to claim 5, characterized in that, The data acquisition module includes a multi-protocol interface module, which comprises a distributed message queue architecture connected to a Kafka cluster. The Kafka cluster establishes an asynchronous data channel. In the asynchronous data channel, each data source corresponds to an independent topic and has data isolation attributes. The number of message partitions matches the concurrency of the data sources and has load balancing attributes. The multi-protocol interface module includes a consumer group and a message acknowledgment mechanism. The consumer group is equipped with a polling mechanism to allocate message consumption tasks, and the message acknowledgment mechanism is associated with data transmission and has reliability attributes.

7. The technical trade barrier risk assessment system based on big data according to claim 6, characterized in that, The blockchain storage module adopts a consortium blockchain architecture to achieve tamper-proof storage of evaluation data. Each block includes a data header, a data body, and consensus mechanism information: the data header includes a version number, the previous block hash value, Merkle root hash, and a timestamp; the data body includes the original data hash digest, the evaluation result signature, and node consensus information; the consensus mechanism adopts a practical Byzantine fault-tolerant algorithm, and achieves data consistency verification between nodes through a three-stage submission process of pre-preparation, preparation, and submission.

8. The big data-based technical trade barrier risk assessment system according to claim 7, characterized in that, The multi-dimensional indicator system includes primary indicators and secondary indicators. The primary indicators include technical standard compliance, certification process complexity, quarantine and testing requirements, environmental restrictions, intellectual property barriers, and trade remedy measures. Each primary indicator has a secondary indicator. The technical standard compliance includes international standard matching degree, standard update frequency, and standard differences.

9. The big data-based technical trade barrier risk assessment system according to claim 8, characterized in that, The output module supports three interactive visualization formats, including a risk heatmap, a parameter sensitivity curve, and a contribution pie chart. The risk heatmap, based on a geographic information system, uses red, yellow, and green gradients to indicate the spatial distribution of global trade barrier risks. The parameter sensitivity curve displays the impact of key indicators on the overall risk value within a fluctuation range in the form of a line graph. The contribution pie chart visually presents the contribution weight of each primary indicator to the total risk value through a two-layered pie chart, with the inner ring showing the subjective weight percentage and the outer ring showing the objective weight percentage.

10. The big data-based technical trade barrier risk assessment system according to claim 9, characterized in that, It also includes an early warning response module, when the comprehensive risk value R ≥ R 阈值 The system automatically triggers a three-level early warning mechanism and calls upon the response strategy library to generate differentiated solutions; the R... 阈值 The calculation formula is dynamically adjusted based on historical data and industry benchmarks, as follows: Among them, R 阈值 As a risk warning threshold, R is the historical average risk value. 行业 The risk value is the industry benchmark, β is the adjustment coefficient and 0 < β < 1, and the value is determined based on the weight requirements of historical data and industry benchmark; the response strategy library includes standard alternatives, simplified compliance process paths and negotiation scripts for trading partners.

Images