CNN-Kriging model-based emergence performance evaluation optimization method

Abstract

The invention relates to an emergence performance evaluation optimization method based on a CNN-Kriging model, and the method comprises the steps: extracting local spectrum features through a convolutional neural network, reconstructing a global continuous performance field from discrete node measurement data through combining with a Kriging spatial interpolation technology, and evaluating the emergence characteristics of a system. According to the method, the construction efficiency and the simulation optimization efficiency of an unknown model can be improved, the calculation cost of an algorithm is remarkably reduced, and the calculation amount of solution is reduced while the precision of a calculation result is ensured; by guaranteeing the wireless communication quality and reliability of key production links, the risk of production pause caused by network problems is directly reduced, and a solid network layer support is provided for stable and efficient operation of an intelligent factory.

Description

Technical Field

[0001] This invention belongs to the field of performance evaluation technology and relates to an emergent performance evaluation optimization method based on the CNN-Kriging model. Background Technology

[0002] Evaluating system performance requires simulation experiments to assess the impact of different performance index values ​​on system performance. However, simulations of complex processes in specific application scenarios can take minutes, hours, or even days to complete, failing to meet the timeliness requirements of actual work. Surrogate models, built on a data-driven approach, assume that the analytical representation of actual processes is too complex and dynamically changing to be accurately processed. They rely on fitting the model's input-output data. Therefore, surrogate models essentially use fitting or interpolation methods to construct a function with sufficient accuracy from known sample points to predict the response value at unknown points.

[0003] Regression prediction, as one of the core tasks of data analysis, has significant application value in engineering fields such as finance, energy, and industry. With the development of deep learning technology, models based on convolutional neural networks (CNN), long short-term memory networks (LSTM), and Transformers have made significant progress in regression prediction. However, traditional deep learning methods often face problems such as vanishing gradients, high computational complexity, and insufficient sensitivity to local features. For example, while LSTM can capture long-term dependencies, its ability to extract high-frequency fluctuating features is limited; and while Transformers improve global modeling capabilities through self-attention mechanisms, they are prone to parameter redundancy in small sample scenarios.

[0004] To address these challenges, researchers have begun exploring the construction of hybrid models. Existing research has successfully combined the advantages of Graph Convolutional Networks (GCNs) and Kriging. Compared to standard GCNs, KCNs directly utilize neighboring observations when generating predictions, significantly improving model performance. Integrating Convolutional Neural Networks (CNNs) with attention-based Bidirectional Long Short-Term Memory (ABiLSTM) has also yielded excellent results in long-term wind power forecasting. Furthermore, while traditional methods such as Support Vector Machines (SVMs) and Kriging models demonstrate stability in nonlinear predictions, their reliance on manual feature engineering makes them ill-suited for complex temporal patterns. Summary of the Invention

[0005] To address the problems existing in the above-mentioned traditional methods, this invention proposes an emergent performance evaluation optimization method based on the CNN-Kriging model.

[0006] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions: On the one hand, an emergent performance evaluation and optimization method based on the CNN-Kriging model is provided, including the following steps: Step 1: Construct an evaluation index system for the emergent effectiveness of the scenario to be evaluated.

[0007] Step 2: Based on the emergence performance evaluation index system, guide the design of corresponding simulation experiments or actual experiments, and construct experimental samples, which are divided into training samples and test samples.

[0008] Step 3: Train the CNN-Kriging-based performance evaluation optimization surrogate model using training samples, and then test it using test samples to obtain the trained CNN-Kriging-based performance evaluation optimization surrogate model; the CNN-Kriging-based performance evaluation optimization surrogate model includes: a CNN network and a Kriging module; the CNN network is used to extract features from the training samples, and the Kriging module is used to construct the Kriging surrogate model using the extracted features as input.

[0009] Step 4: Based on the CNN-Kriging-based performance evaluation optimization surrogate model, multiple data points meeting accuracy requirements are generated through active sampling. Using all data points meeting accuracy requirements, the evaluation index system is optimized through an index system screening and optimization method. New index inputs and end-point index calculation output sample sets are set. Finally, the selected key impact indicators are used as decision variables, and the surrogate model of system performance is used as the objective function. An optimization algorithm is used to solve the multi-objective problem and obtain the optimal solution for the system performance.

[0010] One of the above technical solutions has the following advantages and beneficial effects: The aforementioned emergent performance evaluation and optimization method based on the CNN-Kriging model extracts local spectral features through a convolutional neural network and combines this with Kriging space interpolation to reconstruct a global continuous performance field from discrete node measurement data, thereby evaluating the system's emergent characteristics. This method improves the efficiency of constructing unknown models and simulation optimization, significantly reduces algorithm computational costs, and decreases the computational load while ensuring the accuracy of the calculation results. By ensuring the quality and reliability of wireless communication in key production processes, it directly reduces the risk of production stoppages caused by network problems, providing solid network layer support for the stable and efficient operation of smart factories. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating an emergent performance evaluation and optimization method based on a CNN-Kriging model in one embodiment. Figure 2 This is a schematic diagram of the structure of a performance evaluation optimization agent model based on CNN-Kriging in one embodiment. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0014] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0015] It should be noted that, in this document, the reference to "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The presentation of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will understand that the embodiments described herein can be combined with other embodiments. The term "and / or" as used herein refers to any combination of one or more of the associated listed items, and all possible combinations, including such combinations.

[0016] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0017] In one embodiment, such as Figure 1 As shown, an emergent performance evaluation and optimization method based on a CNN-Kriging model is provided, characterized by the following steps: Step 1: Construct an evaluation index system for the emergent effectiveness of the scenario to be evaluated.

[0018] Specifically, the scenario to be evaluated can be, but is not limited to, a distributed intelligent spectrum access system that includes a central node and multiple distributed edge nodes.

[0019] The distributed intelligent spectrum access system consists of a central node (such as a cloud server or regional network controller) and multiple edge access gateways (i.e., distributed edge nodes) deployed in various areas of the smart factory. IoT devices access the network through their associated gateways.

[0020] The central node periodically collects data from each edge gateway, including: signal strength indication (RSSI), signal-to-noise ratio (SNR), and packet error rate on different channels sensed by each gateway; location information of each access device (such as obtained through triangulation or beacons), device type (such as sensors, AGVs, robotic arms), service mode (periodic reporting, event triggering, streaming transmission); and performance data such as actual throughput and latency of each channel.

[0021] The emergence performance evaluation metrics for distributed intelligent spectrum access systems include: global network emergence performance metrics (calculated by the central node based on the CNN-Kriging model) and system-level emergence performance metrics (derived by the central node based on long-term data and higher-level analysis).

[0022] By fusing observation data from all nodes and performing spatial interpolation and prediction using the CNN-Kriging model, global emergent performance metrics for the network are obtained. These metrics include global spectral efficiency, global coverage probability, global latency distribution, network capacity, spectrum resource utilization, network resilience, network energy efficiency, and service consistency. Global Spectral Efficiency is the data throughput (bps / Hz / m²) of the entire network per unit area and per unit bandwidth.

[0023] Global Coverage Probability is the probability of meeting the minimum quality of service (such as SINR being higher than a threshold) at any random location.

[0024] Global latency distribution is the spatial distribution of latency across the entire network, particularly in areas where latency exceeds the standard.

[0025] Network capacity is the maximum number of connections or total throughput that a network can support under a given quality of service requirement.

[0026] Spectrum resource utilization is the ratio of spectrum resources occupied by the entire network to available spectrum resources, as well as the spatiotemporal uniformity of spectrum use.

[0027] Network resilience is the ability of a network to maintain service when some nodes or channels fail. It is usually measured by the magnitude of performance degradation and recovery time.

[0028] Network energy efficiency is the ratio of total network throughput to total energy consumption.

[0029] Service consistency refers to the quality difference in services provided by a network in different areas, which can be measured by the variance or coefficient of variation of service quality indicators (such as SINR and latency).

[0030] System-level emergent metrics reflect a system's ability to adapt, learn, and optimize over longer time scales, specifically including: policy convergence speed, learning efficiency, adaptive capability, and scalability, among which: Policy convergence speed is the time required for a system to adjust its policy and reach a stable state when the network environment changes.

[0031] Learning efficiency is the relationship between the improvement in system performance after updating the CNN-Kriging model and the amount of training data required.

[0032] Adaptability is the ability of a system to maintain stable performance when it responds to dynamic changes (such as the addition of new equipment or sudden interference).

[0033] Scalability is the trend of system performance as the network size (number of nodes, number of devices) increases.

[0034] Step 2: Based on the emergence performance evaluation index system, guide the design of corresponding simulation experiments or actual experiments, and construct experimental samples, which are divided into training samples and test samples.

[0035] Specifically, based on the emergent performance evaluation index system, corresponding simulation experiments are conducted to construct a data input sample set, and experimental samples for performance evaluation results are obtained through methods such as analytic hierarchy process. Construct a data output sample set, through This experiment constructs a complete data sample set. The experimental samples were divided into training samples. and test samples .

[0036] Step 3: Train the CNN-Kriging-based performance evaluation optimization surrogate model using training samples, and then test it using test samples to obtain the trained CNN-Kriging-based performance evaluation optimization surrogate model; the CNN-Kriging-based performance evaluation optimization surrogate model includes: a CNN network and a Kriging module; the CNN network is used to extract features from the training samples, and the Kriging module is used to construct the Kriging surrogate model using the extracted features as input.

[0037] Specifically, the training samples As input and output data for the surrogate model, the system performance of the surrogate model is optimized based on the training samples, and the test samples are used. The prediction accuracy of the surrogate model is then evaluated. If the prediction accuracy meets the requirements, the surrogate model is considered complete; otherwise, a new surrogate model is selected or training sample points are added, and the surrogate model is reconstructed until the accuracy requirements are met, thus obtaining the optimized surrogate model for system performance. .

[0038] The structure of the survivability performance evaluation optimization surrogate model based on the CNN-Kriging model is as follows: Figure 2 As shown.

[0039] Model Training and Updates. A CNN-Kriging model is trained based on collected historical and real-time data. The model's input consists of all feature variables collected periodically from data collected by the central node from each edge gateway. The model output is the predicted interference level at that location on that channel (e.g., normalized interference power) and the Kriging variance (representing uncertainty). The CNN part extracts deep spatial-spectral features from the combination of location and channel, while the Kriging part performs interpolation predictions based on these features and spatial correlations.

[0040] This study aims to construct a Kriging model that accurately reflects the complex mapping relationship between key input parameters and output performance response of a system. While ensuring sufficient predictive accuracy for the input-output behavior of the original model, the time required for a single evaluation is reduced from hours or days to seconds or even milliseconds, enabling large-scale, high-efficiency computational experiments. For example, Support Vector Regression (SVR) is often chosen due to its excellent generalization ability in handling small samples and nonlinear problems; the GMDH method in self-organizing data mining algorithms can effectively handle high-dimensional complex systems with transparent model structures through automatic network structure growth and selection; Gaussian process regression not only provides point estimates of predicted values ​​but also gives a measure of prediction uncertainty; and for problems with extremely complex input-output relationships and abundant data, deep neural networks, with their powerful nonlinear fitting capabilities and hierarchical feature extraction capabilities, can construct highly expressive surrogate models. Regardless of the specific technology used, the construction process usually follows a standard supervised learning workflow: first, run the full-order model to obtain a certain number of input-output sample data to form a training set; then, train and cross-validate the selected surrogate model based on the training set; finally, ensure that the model reaches the preset prediction accuracy threshold on an independent test set, so that it can reliably replace the original complex model to perform fast performance inference and prediction.

[0041] The proposed CNN-Kriging model employs an end-to-end fusion of feature extraction and regression modeling. It automatically extracts multi-scale local features from the data using CNN and utilizes Kriging's spatial interpolation method to achieve efficient regression, thus avoiding the limitations of manual feature design.

[0042] Step 4: Based on the CNN-Kriging-based performance evaluation optimization surrogate model, multiple data points meeting accuracy requirements are generated through active sampling. Using all data points meeting accuracy requirements, the evaluation index system is optimized through an index system screening and optimization method. New index inputs and end-point index calculation output sample sets are set. Finally, the selected key impact indicators are used as decision variables, and the surrogate model of system performance is used as the objective function. An optimization algorithm is used to solve the multi-objective problem and obtain the optimal solution for the system performance.

[0043] Specifically, using a trained CNN-Kriging model, an analysis dataset containing tens of thousands of {input feature combinations -> model prediction output} samples is generated within its input domain (the entire factory region, all channels) using the Latin hypercube sampling method. To improve efficiency, the sampling density can be increased preferentially in regions with high model prediction variance (i.e., regions where the model's cognition is uncertain).

[0044] By using a surrogate model, a large amount of data that meets accuracy requirements can be obtained without relying on the original model, driven by only a limited sample data. Using this data, the evaluation indicator system can be optimized through indicator system screening and optimization methods, and new indicator inputs and end indicator calculation output sample sets can be set. Finally, the selected key influencing indicators are used as decision variables, and the surrogate model of system effectiveness is used as the objective function. The multi-objective optimization problem is thus expressed as: , This method employs multi-objective optimization based on optimization algorithms to obtain the optimal solution for maximizing system performance. While maintaining accuracy, it significantly reduces equipment and time costs, thereby improving computational efficiency.

[0045] In the aforementioned emergent performance evaluation and optimization method based on the CNN-Kriging model, the method extracts local spectral features through a convolutional neural network and combines it with Kriging space interpolation technology to reconstruct the global continuous performance field from discrete node measurement data, thereby evaluating the emergent characteristics of the system. This method can improve the efficiency of constructing unknown models and simulation optimization, significantly reduce the computational cost of the algorithm, and reduce the computational load while ensuring the accuracy of the calculation results. By ensuring the quality and reliability of wireless communication in key production links, it directly reduces the risk of production stoppages caused by network problems, providing solid network layer support for the stable and efficient operation of smart factories.

[0046] In one embodiment, step 1 includes: based on the task of the scenario to be evaluated, according to the definition of system effectiveness and its evaluation framework and the construction principles of effectiveness evaluation indicators, based on the definition of effectiveness of the dataset to be evaluated and the analysis of effectiveness influencing factors, initially constructing an evaluation indicator system, optimizing the evaluation indicator system based on the indicator system screening and optimization method, screening key influencing factors to optimize the design of system effectiveness evaluation indicators, obtaining the optimized indicator system construction scheme, and determining the construction mode of the emergent effectiveness evaluation indicator system for the scenario to be evaluated.

[0047] In one embodiment, step 3 includes: normalizing the training samples and test samples; inputting the normalized training samples into the CNN network for feature extraction to obtain a high-dimensional feature vector; using the high-dimensional feature vector as the input feature of the Kriging module to establish a Kriging surrogate model; the Kriging module, based on Gaussian process regression theory, establishes a mapping relationship between input features and output response by optimizing the covariance function and hyperparameters; training, cross-validating, and regularizing the CNN-Kriging-based performance evaluation optimization surrogate model based on the normalized training samples to obtain the trained CNN-Kriging-based performance evaluation optimization surrogate model; and testing the trained CNN-Kriging-based performance evaluation optimization surrogate model using the normalized test set to obtain the trained CNN-Kriging-based performance evaluation optimization surrogate model.

[0048] Specifically, the CNN-Kriging-based performance evaluation optimization surrogate model improves the model's prediction accuracy and generalization ability by combining the interpolation capability of the Kriging model with the feature extraction and nonlinear fitting capabilities of CNN.

[0049] Data preprocessing is the primary step in the CNN-Kriging-based performance evaluation and optimization surrogate model framework. Its main task is to standardize the raw data and divide it into training and test sets to ensure efficient training and evaluation of the subsequent model. The fundamental purpose of data preprocessing is to eliminate dimensional differences between different features, avoiding the unbalanced impact of excessively large numerical ranges on model training, thereby ensuring the stability and convergence of model training. Through normalization, data is mapped to a uniform interval (such as [0,1] or [-1,1]), which not only improves the training efficiency of the model but also effectively prevents problems such as gradient vanishing or gradient exploding. Especially in deep learning models, normalization has a significant impact on the convergence speed and final performance of the model. Commonly used normalization methods include min-max normalization and Z-score normalization.

[0050] Feature extraction is performed using a CNN network. The CNN network is a core component of the CNN-Kriging model framework. Its main function is to extract features from the input data at multiple levels, generating high-dimensional feature vectors, thus providing high-quality input for subsequent Kriging model construction. Through its unique convolutional and pooling layer structure, the CNN network can effectively capture local features and nonlinear relationships in the data, making it particularly suitable for processing high-dimensional data. Convolutional layers perform local convolution operations on the input data using kernels (filters), extracting spatial or temporal local features. This local connectivity and weight sharing not only significantly reduces the number of model parameters but also improves computational efficiency. Pooling layers further reduce the spatial dimensionality of the data through downsampling operations (such as max pooling or average pooling), reducing computational complexity and enhancing the model's robustness to small changes in the data. Through the alternating combination of convolutional and pooling layers, CNNs can progressively extract multi-level features from the data, from low-level simple features such as edges and textures to high-level features, forming a high-dimensional feature representation with rich expressive power. In this application, the output of the CNN network is a high-dimensional feature vector. This vector not only retains the key information of the original data but also enhances the expressive power of the features through nonlinear transformations, providing a more discriminative input for the subsequent Kriging model. This multi-level feature extraction mechanism enables CNNs to perform exceptionally well when processing complex data. CNNs can effectively capture the inherent structure and patterns of the data, significantly improving the model's predictive performance. By using the high-dimensional features extracted by the CNN as input to the Kriging model, the hybrid surrogate model can fully utilize the local and nonlinear characteristics of the data while ensuring prediction accuracy, further enhancing the model's generalization ability and robustness.

[0051] The Kriging model, as another core component of the CNN-Kriging-based performance evaluation and optimization surrogate model framework, plays a crucial role in achieving accurate predictions based on feature selection. The Kriging model is an interpolation method based on Gaussian process theory. Its core idea is to predict unknown points by modeling the spatial correlation between known sample points. This model not only provides point prediction results but also estimates the uncertainty of the predicted values. In the proposed hybrid surrogate model framework, the input of the Kriging model is a high-dimensional feature representation generated by the CNN feature extraction module. These features, through multi-layer nonlinear transformations of the convolutional neural network, effectively capture the complex patterns and underlying structures of the input data. By modeling these high-dimensional features, the Kriging model establishes a mapping relationship between the input features and the objective function, thereby achieving high-precision predictions.

[0052] The construction of the Kriging model involves several key steps, among which the choice of the correlation function has a significant impact on model performance. The correlation function quantifies the spatial correlation between sample points, and its selection needs to be tailored to the specific characteristics of the problem. Commonly used correlation functions include the Gaussian exponential function, cubic spline function, and Matrn function, which can describe spatial dependencies at different scales and intensities. The Gaussian exponential function is widely used due to its smoothness and differentiability, while the Matrn function is more flexible and can adapt to different spatial correlation patterns. After determining the correlation function, the model parameters are typically trained and optimized using the maximum likelihood estimation optimization method. By maximizing the likelihood function, the parameters of the correlation function are determined, allowing the model to better fit the training data. The prediction results of the Kriging model can be combined with other machine learning methods to form a hybrid prediction framework to further improve model performance. In the proposed CNN-Kriging model framework, the construction of the Kriging model is closely integrated with the CNN feature extraction module, forming a complete hybrid surrogate model. By using the high-dimensional features extracted by the CNN as input to the Kriging model, not only can the advantages of CNN in feature extraction be fully utilized, but also the characteristics of the Kriging model in high-precision interpolation and uncertainty quantification can be leveraged. This combination approach enables the hybrid proxy model to exhibit excellent performance when dealing with high-dimensional, nonlinear problems.

[0053] The model training and prediction process specifically includes: by integrating the functions of a CNN network and the Kriging model building module, a complete hybrid surrogate model system is constructed, undertaking the key tasks of model optimization and performance evaluation. In the model training phase, the training set data is first processed by a CNN network, which automatically extracts high-dimensional feature representations of the input data through a multi-layer convolutional neural network structure. These features effectively capture the nonlinear relationships and spatial correlations of the data. Subsequently, the extracted high-dimensional features are input into the Kriging model. This module, based on Gaussian process regression theory, establishes the mapping relationship between input features and output response by optimizing the covariance function and hyperparameters. The training process of the Kriging model involves methods such as maximum likelihood estimation to ensure that the model can accurately fit the statistical characteristics of the training data. During training, cross-validation and regularization are used to prevent overfitting. In the model prediction phase, the test set data is also processed by the CNN feature extraction module to obtain feature representations consistent with the training set. These features are then input into the trained CNN-Kriging model to predict the output response.

[0054] To comprehensively evaluate the model's predictive performance, various statistical indicators were employed for quantitative analysis. Mean squared error (MSE), a commonly used error metric, reflects the overall deviation between predicted and true values. The coefficient of determination (R²) assesses the model's ability to explain data variability; a value closer to 1 indicates a better fit. In addition, mean absolute error (MAE) and root mean square error (RMSE) were used to evaluate the model's predictive accuracy from different perspectives. Furthermore, to ensure good generalization ability, strict random sampling principles were followed when dividing the training and test sets, maintaining the uniformity of data distribution. In the performance evaluation phase, in addition to quantitative indicator analysis, visualization methods such as residual plots and prediction-true value scatter plots were used to visually demonstrate the model's predictive performance.

[0055] In one embodiment, the CNN network includes: a first convolutional layer, a first downsampling layer, a second convolutional layer, a second downsampling layer, a fully connected layer, and an output layer; the output layer includes a fully connected layer and a Softmax activation function.

[0056] Specifically, CNN (Convolutional Neural Network) is a core component of the hybrid agent model framework. Its main function is to extract multi-level features from input data using CNNs, generating high-dimensional feature representations to provide high-quality input for subsequent Kriging model construction. Through its unique convolutional and pooling layer structure, CNNs effectively capture local features and non-linear relationships in data, making them particularly suitable for processing high-dimensional data. Convolutional layers perform local convolution operations on the input data using kernels (filters), extracting spatial or temporal local features. This local connectivity and weight sharing not only significantly reduces the number of model parameters but also improves computational efficiency. Pooling layers further reduce the spatial dimensionality of the data through downsampling operations (such as max pooling or average pooling), reducing computational complexity and enhancing the model's robustness to small changes in the data. Through the alternating combination of convolutional and pooling layers, CNNs can progressively extract multi-level features from the data, from low-level simple features such as edges and textures to high-level features, forming high-dimensional feature representations with rich expressive power. In this paper's framework, the output of the CNN feature extraction module is a high-dimensional feature vector. This vector not only retains the key information of the original data but also enhances the expressive power of the features through nonlinear transformations, providing a more discriminative input for the subsequent Kriging model. This multi-level feature extraction mechanism enables CNNs to perform exceptionally well when processing complex data. CNNs can effectively capture the inherent structure and patterns of the data, significantly improving the model's predictive performance. By using the high-dimensional features extracted by the CNN as input to the Kriging model, the hybrid surrogate model can fully utilize the local and nonlinear characteristics of the data while ensuring prediction accuracy, further enhancing the model's generalization ability and robustness.

[0057] In one embodiment, the construction process of the Kriging agent model follows a standard supervised learning process; the construction process of the Kriging agent model specifically includes: obtaining [data / performance] based on the simulation platform and engine through a comprehensive performance evaluation method. N The input-output sample data constitute the training set; based on the Gaussian process regression theory, the mapping relationship between input features and output response is established by optimizing the covariance function and hyperparameters to obtain the Kriging model; the Kriging model is trained and optimized using the maximum likelihood estimation optimization method.

[0058] Specifically, the construction process of the surrogate model follows a standard supervised learning workflow. Let the original simulation model be... ,in The dimension of the input parameters. Obtained through a comprehensive performance evaluation method based on the simulation platform and engine. The training set consists of input-output sample data. ,in For the input vector, This is the corresponding output response calculated using a comprehensive evaluation method. (Proxy model) The goal is to use the training set D Learn a mapping function such that For all Established, among which The input space is defined as follows. The training process is accomplished by minimizing the loss function. Techniques such as cross-validation are used to ensure the effectiveness of the system and to evaluate the surrogate model. It achieves a preset prediction accuracy threshold on an independent test set, thus enabling it to reliably replace the original complex model for fast performance extrapolation and prediction.

[0059] After successfully constructing a high-precision surrogate model, it is necessary to design and define a multi-dimensional input space that can comprehensively cover various external practical conditions and internal uncertainty parameters that may be encountered during system operation. Each of them Indicates the first iThe range or probability distribution of the input parameters is defined. By reasonably discretizing or setting the probability distribution of these parameters, and considering their possible interaction range, a high-dimensional input parameter space is defined to characterize the system's operating state in a real and complex environment, ensuring that the sensitivity analysis results have practical physical meaning and broad applicability. The dimension of this input space corresponds to all factors considered to potentially affect the system's performance, which constitute the initial pool of indicators to be screened. The input parameters need to systematically cover the following aspects: first, the key performance parameters of the system itself, such as the operating distance and accuracy of various sensors, the bandwidth and latency of communication links, etc.; second, the organizational structure parameters of the system, such as network topology connections, the number of command levels, information flow paths, etc.; third, external environmental conditions; and fourth, the statistical characteristics of various uncertainty factors, such as the tolerance range of equipment performance, the accuracy of operator skill fluctuations, etc. These uncertainties are usually described in the form of probability distributions. By reasonably discretizing or setting the probability distribution of these parameters, and considering their possible interaction range, a high-dimensional input parameter space is defined to characterize the system's operating state in a real and complex environment, thus defining clear boundaries for subsequent computational experiments.

[0060] In one embodiment, the relevant function used in the construction of the Kriging proxy model is the Gaussian exponential function.

[0061] In one embodiment, the normalization process employs either the min-max normalization method or the Z-score normalization method.

[0062] Specifically, min-max normalization maps data to a specified interval through a linear transformation. The normalization formula is:

[0063] in, Represents the normalized value. Represents data to be normalized. and These represent the minimum and maximum values ​​of the data, respectively.

[0064] Z-score standardization transforms data into a distribution with a mean of 0 and a standard deviation of 1 by subtracting the mean and dividing by the standard deviation. The normalization formula is:

[0065] in, Represents the normalized value. Represents data to be normalized. and These are the mean and standard deviation of the data, respectively.

[0066] Z-score normalization is particularly suitable for situations with imbalanced data distribution or outliers, effectively reducing the interference of outliers on model training. After normalization, the data is divided into training and test sets. The training set is used for model parameter optimization and feature learning, while the test set is used to evaluate the model's generalization ability and prediction performance. This partitioning method not only avoids model overfitting but also ensures the model's robustness in practical applications. Through scientific data preprocessing, the hybrid surrogate model can exhibit higher accuracy and stability in subsequent feature extraction and prediction tasks.

[0067] In a specific implementation, this method is applied to the distributed intelligent spectrum access system of a smart factory for emergent performance evaluation. The distributed intelligent spectrum access system of the smart factory adopts a center-edge collaborative architecture. Local spectrum data is collected by edge access gateways deployed in various regions. The central node performs deep feature extraction and spatial interpolation based on the CNN-Kriging model, thereby emerging continuous spectrum situation awareness and performance evaluation of the entire factory area, and dynamically optimizing accordingly.

[0068] The central node of the distributed intelligent spectrum access system in the smart factory is a high-performance industrial cloud server deployed in the factory's information center computer room; equipped with a dedicated GPU acceleration card to run deep convolutional neural network (CNN) models and geospatial statistics (Kriging) algorithms; and running the "intelligent spectrum brain" platform software, which is responsible for data aggregation, model calculation, global evaluation, and strategy generation.

[0069] The distributed intelligent spectrum access system for the smart factory utilizes distributed edge access gateways: 12 industrial-grade intelligent edge access gateways are deployed at key locations throughout the factory, serving as distributed edge nodes; specifically, 3 gateways are deployed in the welding area, 4 in the assembly area, 2 in the quality inspection area, 2 in the logistics and warehousing area, and 1 in the equipment maintenance area. The physical coordinates (x, y, z) of each industrial-grade intelligent edge access gateway have been precisely calibrated within the system. The industrial-grade intelligent edge access system integrates a dual-band Wi-Fi 6 / 5G industrial wireless access module, a spectrum sensing module (supporting 2.4GHz, 5.8GHz, and proprietary frequency band scanning), an edge computing unit, and a BeiDou / GPS positioning module. Each industrial-grade intelligent edge access gateway provides wireless access services to fixed and mobile IoT devices (such as robotic arms, AGVs, and handheld terminals) within its coverage area.

[0070] The specific process for evaluating and optimizing the emergent performance of a distributed intelligent spectrum access system in a smart factory includes: Step S201: Distributed sensing and data acquisition.

[0071] Each edge gateway autonomously performs spectrum environment sensing within its coverage area every 100 milliseconds, and collects communication performance data from associated devices to form a local sensing data packet. This data packet mainly includes: Spatiotemporal tags: gateway ID, precise 3D coordinates, timestamp.

[0072] Raw spectrum situation data: Scan 20 predefined channels and record the real-time received signal strength (RSSI), signal-to-interference-plus-noise ratio (SINR), and occupancy status (busy / idle) of each channel.

[0073] Access performance data: average uplink / downlink throughput, data transmission latency, packet loss rate, and access request success rate of all associated devices under this gateway.

[0074] Preliminary Local Performance Assessment: The gateway calculates a preliminary comprehensive local performance score based on throughput and latency using a preset formula.

[0075] Step S202: Data aggregation and feature extraction.

[0076] All edge gateways upload locally sensed data packets to the central node in real time through the factory's industrial ring network.

[0077] Data formatting: The central node organizes the spectrum scan data from each gateway into a multi-channel spectrum image. For example, the RSSI values ​​of 20 channels form one channel, and the SINR values ​​form another channel, forming an input matrix for CNN processing.

[0078] CNN Deep Feature Extraction: The central node uses a pre-trained CNN model to process the "spectral image" of each gateway. This CNN model, trained on historical data, is able to identify complex patterns such as "periodic strong interference," "broadband noise," and "severe multipath fading." The model output is a 128-dimensional deep feature vector, which highly abstractly represents the essential characteristics of the electromagnetic environment at the gateway's location.

[0079] Step S203: Kriging spatial interpolation and global situation "emergence". Constructing a spatial dataset: The central node obtains a dataset of 12 spatially discrete points: {location coordinates (x_i, y_i), depth feature vector F_i, local performance score E_i}.

[0080] Spatial correlation modeling: Spatial variability analysis was performed on the feature vector F_i and the performance value E_i, and their variability function was fitted. The analysis revealed that the spectral characteristics exhibit strong spatial correlation within a range of approximately 25 meters in this factory environment.

[0081] Continuous field reconstruction: The entire factory area is divided into a 1m × 1m grid. For each grid point P, the Kriging algorithm calculates the expected depth feature vector F(P) and the comprehensive performance value E(P) of that point using weighted interpolation based on the known data from multiple surrounding gateways.

[0082] Generate an emergent cognitive map: Through the above interpolation, two key maps are generated, including: a continuous heat map of the spectrum health of the entire plant area: which intuitively displays the expected communication quality of each location (represented by color gradient from excellent to poor); and a spatial distribution inference map of interference sources: which, combined with depth features, infers potential areas of concentrated interference (such as multipath areas caused by large motors or metal corridors).

[0083] Thus, the system "emerged" a continuous and detailed understanding of the spectral situation of the entire 20,000-square-meter factory area from data from 12 discrete points.

[0084] Step S204: Calculation and evaluation of the emergent performance index system.

[0085] Based on the plant-wide continuous situation generated in step S203, the central node calculates system-level emergent performance indicators that cannot be directly obtained from local data: Global spectrum spatiotemporal utilization: Calculate the overall utilization rate of available spectrum resources across the entire plant in terms of time and space, and assess the adequacy of resource utilization.

[0086] Service Consistency Index: Assess the differences in communication service quality obtained by equipment in different areas (such as assembly line vs. warehouse) to ensure service availability in critical production areas.

[0087] Network resilience score: By simulating any gateway failure, observe the degree of overall system performance degradation and recovery speed to assess network resilience.

[0088] Collaborative decision gain: The ratio (>1) of the overall throughput under the current distributed collaborative access to the theoretical throughput of each gateway making independent decisions is the collaborative gain, which quantifies the effect of "the whole is greater than the sum of its parts".

[0089] Step S205: Intelligent Optimization Decision Generation and Distribution. Based on the emergent evaluation results, the central node formulates and distributes optimization strategies: Case 1 (Addressing Spatial Interference): A spectrum heatmap showed an "inefficient band" in the middle of the logistics channel. Analysis of the central node suggested that this was caused by multiple AGVs passing through simultaneously, resulting in superimposed interference on the same frequency. Decision: Instruct the two gateways in this area to temporarily switch their operating channels to predicted idle backup channels while the AGV fleet is passing through.

[0090] Case 2 (Improving Service Consistency): The assessment revealed that the latency performance of the assembly area was better than that of the quality inspection area. Decision: Implement a more aggressive channel reservation strategy for the gateway in the quality inspection area and slightly increase its transmit power to narrow the service quality gap between areas.

[0091] Case 3 (Preventative Maintenance): Feature analysis shows that the deep feature vectors near a certain gateway consistently exhibit a "gradually increasing noise floor" pattern. Decision: The system automatically generates an alert, prompting maintenance personnel to check for any abnormal electrical equipment in the area.

[0092] Step S206: Closed-loop verification and model evolution.

[0093] After the edge gateway implements the new strategy, it collects and uploads a new round of data. The central node calculates and compares the performance indicators before and after optimization to verify the effectiveness of the strategy (e.g., whether the "low-performance band" has disappeared). Simultaneously, these new data pairs will be added to the training set for periodic fine-tuning of the CNN-Kriging model, enabling the system to adapt to the long-term evolution of the factory environment and equipment.

[0094] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0095] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and all such modifications and improvements fall within the scope of protection of this application.

Claims

1. A method for evaluating and optimizing emergent performance based on a CNN-Kriging model, characterized in that, Including the following steps: Step 1: Construct an emergent performance evaluation index system for the scenario to be evaluated; Step 2: Based on the emergence performance evaluation index system, guide the design of corresponding simulation experiments or actual experiments, and construct experimental samples, which are divided into training samples and test samples; Step 3: Train the CNN-Kriging-based performance evaluation optimization surrogate model using training samples, and then test it using test samples to obtain the trained CNN-Kriging-based performance evaluation optimization surrogate model; the CNN-Kriging-based performance evaluation optimization surrogate model includes: a CNN network and a Kriging module; the CNN network is used to extract features from the training samples, and the Kriging module is used to construct the Kriging surrogate model using the extracted features as input; Step 4: Based on the CNN-Kriging-based performance evaluation optimization surrogate model, multiple data points meeting accuracy requirements are generated through active sampling. Using all data points meeting accuracy requirements, the evaluation index system is optimized through an index system screening and optimization method. New index inputs and end-point index calculation output sample sets are set. Finally, the selected key impact indicators are used as decision variables, and the surrogate model of system performance is used as the objective function. An optimization algorithm is used to solve the multi-objective problem and obtain the optimal solution for the system performance.

2. The emergence performance evaluation and optimization method based on the CNN-Kriging model according to claim 1, characterized in that, Step 1 includes: Based on the task of the scenario to be evaluated, according to the definition of system effectiveness and its evaluation framework and the construction principles of effectiveness evaluation indicators, based on the definition of effectiveness of the dataset to be evaluated and the analysis of effectiveness influencing factors, an initial evaluation indicator system is constructed. The evaluation indicator system is optimized based on the indicator system screening and optimization method. Key influencing factors are screened to optimize the design of system effectiveness evaluation indicators, resulting in an optimized indicator system construction scheme. The construction mode of the emergent effectiveness evaluation indicator system for the scenario to be evaluated is determined.

3. The emergence performance evaluation and optimization method based on the CNN-Kriging model according to claim 1, characterized in that, Step 3 includes: Normalize the training and test samples; The normalized training samples are input into the CNN network for feature extraction to obtain high-dimensional feature vectors; The high-dimensional feature vector is used as the input feature of the Kriging module to establish a Kriging surrogate model; the Kriging module is based on Gaussian process regression theory and establishes the mapping relationship between input features and output response by optimizing the covariance function and hyperparameters; The CNN-Kriging-based performance evaluation optimization surrogate model is trained, cross-validated, and regularized based on the normalized training samples to obtain the trained CNN-Kriging-based performance evaluation optimization surrogate model. The trained CNN-Kriging-based performance evaluation optimization proxy model was tested using a normalized test set to obtain the trained CNN-Kriging-based performance evaluation optimization proxy model.

4. The emergence performance evaluation and optimization method based on the CNN-Kriging model according to claim 3, characterized in that, The CNN network includes: a first convolutional layer, a first downsampling layer, a second convolutional layer, a second downsampling layer, a fully connected layer, and an output layer; the output layer includes a fully connected layer and a Softmax activation function.

5. The emergence performance evaluation and optimization method based on the CNN-Kriging model according to claim 3, characterized in that, The construction process of the Kriging agent model follows the standard supervised learning process; the specific steps in the construction process of the Kriging agent model include: Based on the simulation platform and engine, a comprehensive performance evaluation method is used to obtain... The input-output sample data constitutes the training set; Based on the Gaussian process regression theory, the Kriging model is obtained by optimizing the covariance function and hyperparameters to establish the mapping relationship between input features and output response. The Kriging model is trained and optimized using the maximum likelihood estimation optimization method.

6. The method for dynamically constructing an indicator system based on the CNN-Kriging model according to claim 4, characterized in that, The relevant function used in the construction of the Kriging proxy model is the Gaussian exponential function.

7. The method for dynamically constructing an indicator system based on the CNN-Kriging model according to claim 3, characterized in that, Normalization is performed using either the min-max normalization method or the Z-score standardization method.

Images