Soft measurement modeling data consumption analysis method and system

Abstract

The application provides a soft measurement modeling data consumption analysis method and system; the method first gives a data consumption cube model, including three data dimensions: time dimension, space dimension and feature dimension. Among them, the time dimension represents the number of data samples required to build a soft measurement model, the space dimension represents the number of sampling points required to be set in space to build a soft measurement model, and the feature dimension represents the feature quantity required to be collected at each sampling point. Through data consumption analysis in three single dimensions and combined dimensions, the soft measurement model with the most saved data amount and the modeling data set with the optimal performance-price ratio can be obtained. The method described in the application also provides a new soft measurement model evaluation method different from model performance, which has practical significance for high-cost data soft measurement modeling.

Description

Technical Field

[0001] This invention relates to the fields of data science and machine learning applications, and in particular to a method and system for analyzing data consumption through soft measurement modeling. Background Technology

[0002] The statements in this section merely refer to the background art related to this invention and do not necessarily constitute prior art.

[0003] Soft measurement technology is a data-driven estimation measurement technique that organically combines data with domain knowledge. For the important features of the target being measured, it selects other known or easily accessible features and constructs a data relationship model to infer or estimate the target features. In recent years, the widespread application of machine learning methods has provided important technical support for the modeling and maturation of soft measurement technology. Therefore, soft measurement has gradually been widely applied in many fields such as petrochemicals, wastewater treatment, and environmental monitoring.

[0004] High-cost data refers to data that is difficult to obtain or has a high acquisition cost, such as water quality monitoring data. Obtaining such data requires steps such as on-site sampling, laboratory testing, or sensor deployment, which requires significant expenditures on manpower, equipment, reagents, and sensor purchase and maintenance, resulting in high data collection costs.

[0005] Data consumption in soft sensor models refers to the amount of data required to build a model, i.e., how much data is needed for training, validating, and testing the model. For high-cost data, when building a soft sensor model, it's crucial to consider not only model performance (such as accuracy) but also the amount of data required for modeling, i.e., the cost of data acquisition. Typically, the performance of machine learning models is highly dependent on the amount of modeling data, but different models require significantly different amounts of data. Even for the same model, the relationship between modeling data volume and model performance is not linear; once model performance improves to a certain level, further increases in data volume do not significantly enhance performance. Therefore, data consumption analysis can identify soft sensor models that are more economical in terms of modeling data requirements and the most cost-effective modeling datasets.

[0006] There is currently limited research on data consumption, and existing soft measurement research is mainly based on the premise of sufficient data, without considering the modeling costs of high-cost data. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a method and system for analyzing data consumption in soft measurement modeling. It proposes a data consumption cube model to observe and compare the amount of modeling data from three single dimensions (time, space, and features) and combinations thereof, thereby identifying soft measurement models that require less modeling data and the most cost-effective modeling datasets, thus reducing the cost of soft measurement modeling.

[0008] In a first aspect, the present invention provides a method for analyzing data consumption in soft measurement modeling;

[0009] A method for analyzing data consumption in soft measurement modeling includes:

[0010] Given the target features of soft measurement, the dataset, and the set of candidate basic models for soft measurement, a data consumption cube model is generated. The data consumption cube model includes three dimensions: time, space, and features. The data consumption cube model forms a data consumption coordinate system, in which any point corresponds to a dataset instance, and the coordinate values ​​of each point correspond to the number of samplings, the number of sampling points, and the number of features collected at each sampling point in this dataset instance.

[0011] Set the data cost function of the data consumption cube model to calculate the cost of the dataset instance; set the cost-effectiveness evaluation function to calculate the cost-effectiveness index of the soft measurement model generated by the machine learning model using the dataset instance;

[0012] Based on the data cost function and cost-effectiveness evaluation function, the dataset is analyzed according to the time dimension, spatial dimension, and feature dimension respectively. The machine learning model with the best soft measurement performance in a single dimension, the optimal number of sampling times, the number of sampling points, and the optimal feature set collected for each sampling point are obtained.

[0013] By performing a combined analysis of the dataset across three dimensions—time, space, and features—we obtain machine learning models with optimal soft measurement performance across multiple dimensions, as well as cost-effective sampling times, number of sampling points, and the optimal feature set collected for each sampling point.

[0014] Furthermore, the method also includes: training, validating, and testing the machine learning model based on the machine learning model with optimal soft measurement performance, the most cost-effective number of samplings, the number of sampling points, and the optimal feature set collected at each sampling point; and implementing soft measurement based on the trained soft measurement model.

[0015] Furthermore, the given soft measurement target features, dataset, and set of candidate soft measurement base models specifically include:

[0016] Given the soft measurement target features y, the dataset D, and the set of candidate soft measurement base models M;

[0017] The dataset D includes the data acquisition time, the spatial location of the sampling points, and various features related to the target feature y collected at each sampling point. The soft measurement candidate machine learning base model M = {m1, m2, ..., m...} n}, where n is the number of models.

[0018] Furthermore, the data consumption cube model specifically includes:

[0019] The data consumption cube model, Cube_data, includes three data dimensions: time dimension T, spatial dimension S, and feature dimension F. Wherein:

[0020] The time dimension represents the number of data samples required to build the soft measurement model;

[0021] Spatial dimension represents the number of sampling points that need to be set in space to construct a soft measurement model;

[0022] Feature dimension refers to the set of features to be collected at each sampling point.

[0023] A data consumption coordinate system is formed by the data consumption cube model Cube_data, where any point coordinate (t,s,f) corresponds to a dataset instance D. (t,s,f) This dataset contains t sampling times, s sampling points, and f feature data for each sampling point.

[0024] Furthermore, the data cost function and cost-effectiveness evaluation function specifically include:

[0025] set up This is the data cost function for the Cube_data model, used to calculate the dataset D. (t,s,f) The cost. For any base model m∈M, using dataset D (t,s,f) The target soft measurement model generated after training, validation, and testing is denoted as Model The soft measurement performance is denoted as Set up a cost-performance evaluation function Dataset D is used to compute the basic model m. (t,s,f) The generated soft measurement model The cost-performance ratio.

[0026] Among them, the cost function Soft measurement performance indicators and cost-effectiveness evaluation function All settings need to be configured according to the actual application scenario.

[0027] Furthermore, after generating the data consumption cube model and before setting the data cost function of the data consumption cube model, the method further includes:

[0028] Based on dataset D, obtain the maximum values ​​of its corresponding time dimension T, spatial dimension S, and feature dimension F, denoted as t. max s max and f max ,Right now:

[0029] Furthermore, the dataset is analyzed according to the time dimension; specifically including:

[0030] Time-dimensional analysis means changing the data size only in the time dimension, while keeping the spatial and feature dimensions unchanged. A set of time points t1, t2, ..., t are selected sequentially from smallest to largest in the time dimension. k , where t k ≤t max Using datasets For model m1, m2, ..., m n The target soft measurement model was obtained by training, validating, and testing the model separately. Based on soft measurement performance This yields the base model with optimal soft measurement performance and the optimal number of samplings, based on... This will allow you to obtain the base model and number of samplings with the best cost-performance ratio.

[0031] Furthermore, the dataset is analyzed according to spatial dimensions; specifically including:

[0032] Spatial dimension analysis refers to changing the data size only in the spatial dimension, while keeping the time and feature dimensions unchanged. A sampling point σ is selected as the spatial center, and different numbers of sampling points are chosen around it, denoted as s1, s2, ..., s in ascending order. g , where s g ≤s max Using datasets For model m1, m2, ..., m n The target soft measurement model was obtained by training, validating, and testing the model separately. Based on soft measurement performance This yields the base model with optimal soft measurement performance and the optimal number of sampling points, based on... This will allow you to obtain the base model and number of sampling points with the best cost-performance ratio.

[0033] Furthermore, the dataset is analyzed according to feature dimensions; specifically including:

[0034] Feature dimension analysis involves changing the data size only along the feature dimension, while keeping the time and spatial dimensions unchanged. Features are sorted in descending order of relevance to the target feature y, and different numbers of features are selected, denoted as f1, f2, ..., f... h , where f h ≤f max Using datasets For model m1, m2, ..., mn The target soft measurement model was obtained by training, validating, and testing the model separately. Based on soft measurement performance This yields the base model with optimal soft measurement performance and the optimal number of features; based on This will yield the base model and number of features with the best cost-performance ratio.

[0035] Furthermore, a combined analysis of the dataset across three dimensions—time, space, and features—is conducted; specifically, this includes:

[0036] Simultaneously change the data in two or three of the time, space, and feature dimensions, and correlate multiple dimensions based on a single-dimensional analysis method.

[0037] In terms of time, select a set of time points t1, t2, ..., t in ascending order. k , where t k ≤t max In the spatial dimension, sampling point σ is selected as the spatial center, and different numbers of sampling points are selected around it, denoted as s1, s2, ..., s in ascending order. g , where s g ≤s max ;

[0038] In terms of feature dimension, features are sorted from largest to smallest according to their relevance to the target feature y, and different numbers of features are selected, denoted as f1, f2, ..., f in ascending order of feature quantity. h , where f h ≤f max ;

[0039] Using dataset For model m1, m2, ..., m n The target soft measurement model was obtained by training, validating, and testing the model separately. Based on soft measurement performance This yields the basic model with optimal soft measurement performance, as well as the optimal number of sampling times, sampling points, and sampling features; based on This will yield the basic model with the best cost-performance ratio, as well as the number of sampling times, the number of sampling points, and the number of sampling features.

[0040] Secondly, the present invention provides a soft measurement modeling data consumption analysis system;

[0041] A soft measurement modeling data consumption analysis system, comprising:

[0042] The model generation module is configured to: generate a data consumption cube model given soft measurement target features, dataset, and a set of candidate soft measurement base models; the data consumption cube model includes three dimensions: time, space, and features; a data consumption coordinate system is formed by the data consumption cube model, in which any point corresponds to a dataset instance, and the coordinate values ​​of each point correspond to the number of samplings, the number of sampling points, and the number of features collected at each sampling point in this dataset;

[0043] The configuration module is configured to: set the data cost function of the data consumption cube model to calculate the cost of the dataset instance; and set the cost-effectiveness evaluation function to calculate the cost-effectiveness index of the soft measurement model generated by the machine learning model using the dataset instance.

[0044] The analysis module is configured to: analyze the dataset according to the time dimension, spatial dimension, and feature dimension based on the data cost function and the cost-effectiveness evaluation function, respectively, to obtain the machine learning model with the best soft measurement performance in a single dimension, and the cost-effective sampling number, number of sampling points, and the best feature set collected at each sampling point; and perform combined analysis of the dataset in the time, space, and feature dimensions to obtain the machine learning model with the best soft measurement performance in multiple dimensions, and the cost-effective sampling number, number of sampling points, and the best feature set collected at each sampling point.

[0045] Furthermore, the system also includes a soft measurement module, which is configured to: train the machine learning model based on a machine learning model with optimal soft measurement performance, and the most cost-effective number of samplings, the number of sampling points, and the optimal feature set collected at each sampling point; and perform soft measurement of data based on the trained machine learning model.

[0046] Compared with the prior art, the beneficial effects of the present invention are:

[0047] To address the application needs of high-cost data-driven soft measurement modeling, this invention provides a method for discovering the soft measurement model with the least data requirement or finding the most cost-effective modeling dataset. The aim is to reduce modeling costs, that is, to achieve soft measurement modeling using as little high-cost data as possible while ensuring model engineering performance.

[0048] The method described in this application can also be used as a model evaluation method to evaluate the amount of data consumed by a machine learning model, which is different from existing model evaluation methods that only consider performance. Attached Figure Description

[0049] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0050] Figure 1 This is a flowchart of the main steps of the method described in Embodiment 1 of this application;

[0051] Figure 2 This is a schematic diagram of the data consumption cube model in Embodiment 1 of this application;

[0052] Figures 3(a) and 3(b) are schematic diagrams of the time dimension analysis results of the soft measurement model of dissolved inorganic nitrogen concentration in Example 1 of this application;

[0053] Figures 4(a) and 4(b) are schematic diagrams of the spatial dimension analysis results of the soft measurement model for dissolved inorganic nitrogen concentration in Example 1 of this application;

[0054] Figures 5(a) and 5(b) are schematic diagrams of the feature dimension analysis results of the soft measurement model for dissolved inorganic nitrogen concentration in Example 1 of this application. Detailed Implementation

[0055] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0056] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. Furthermore, it should be understood that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0057] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0058] All data acquisition in this embodiment is carried out in accordance with laws and regulations and with user consent, and the data is used legally.

[0059] Example 1

[0060] Taking the soft measurement model of dissolved inorganic nitrogen concentration (DIN) in water quality monitoring data of a certain water area as an example, the present invention is described in detail with reference to the accompanying drawings.

[0061] The target feature was set as dissolved inorganic nitrogen concentration (DIN). A dataset D was obtained for the studied water area, consisting of quarterly sampling data from 171 sampling points from the first quarter of 2015 to the fourth quarter of 2019. Various DIN-related features collected at each sampling point included: chemical features (total nitrogen concentration, dissolved inorganic silicon concentration, salinity, chemical oxygen demand, dissolved oxygen concentration, petroleum concentration, and suspended solids concentration); spatial features (longitude, latitude, and sampling depth); biological features (chlorophyll-a); physical features (water temperature); and structural features (dissolved inorganic phosphorus index and silicon-to-phosphorus ratio). Three machine learning models—Extreme Gradient Boosting (XGBoost), Support Vector Regression (SVR), and K-Nearest Neighbors (KNN)—were selected as the basic models for soft sensing, generating a candidate machine learning model set M = {XGBoost, SVR, KNN}.

[0062] like Figure 1 As shown, a soft measurement modeling data consumption analysis method includes the following steps.

[0063] Step S1, generate the data consumption cube model Cube_data, such as Figure 2 As shown, the Cube_data model comprises three data dimensions: time dimension T, spatial dimension S, and feature dimension F. The time dimension represents the number of data samples required to construct the DIN soft measurement model, the spatial dimension represents the number of sampling points that need to be set in space to construct the DIN soft measurement model, and the feature dimension represents the number of features to be collected at each sampling point. The Cube_data model forms a data consumption coordinate system, where any point coordinate (t, s, f) corresponds to a dataset instance D. (t,s,f) This dataset contains t sampling times, s sampling points, and f feature data for each sampling point.

[0064] Define the data cost function for the Cube_data model:

[0065]

[0066] Used to calculate dataset D (t,s,f) The cost. For any base model m∈M, using dataset D (t,s,f) The target soft measurement model generated after training, validation, and testing is denoted as Model The soft measurement performance is denoted as Using goodness of fit (R) 2 Mean absolute error (MAE) is used as a performance metric for soft measurement models.

[0067] Set up a cost-performance evaluation function:

[0068]

[0069] The dataset D is used to compute the underlying model m in M. (t,s,f) The generated soft measurement model The cost-performance ratio.

[0070] Step S2: Calculate the maximum data consumption cube. For dataset D, the time span is from Q1 2015 to Q4 2019, with 20 samplings per quarter. The maximum value t of the time dimension T of the Cube_data model is calculated. max =20; the spatial range covers 171 sampling points, and the maximum value of the spatial dimension S of the Cube_data model is s. max =171; Each sampling point contains 14 feature factors related to DIN, listed in descending order of correlation: total nitrogen concentration, dissolved inorganic phosphorus index, longitude, dissolved inorganic silicon, salinity, chlorophyll-a, silicon-to-phosphorus ratio, water temperature, dissolved oxygen concentration, sample collection depth, latitude, chemical oxygen demand, petroleum concentration, and suspended solids concentration. The maximum value f of the feature dimension F of the Cube_data model is... max =14, that is, D=D (20,171,14) .

[0071] Step S3, single-dimensional analysis of model data consumption, includes the following steps:

[0072] Step S3-1, Time Dimension Analysis. Only the data size is changed in the time dimension; the spatial and feature dimensions remain unchanged. A set of sampling times is selected in ascending order along the time dimension, starting from 1 and increasing by one sampling increment, and sequentially denoted as t1, t2, ..., t... k , where t k ≤20. Dataset used. The XGBoost, SVR, and KNN models were trained, validated, and tested respectively to obtain the target soft measurement model. Based on soft measurement performance This yields the base model with optimal soft measurement performance and the optimal number of samplings, based on... This allows us to obtain the basic model and sampling number with the best cost-effectiveness. The performance changes of the DIN concentration soft measurement model over time are shown in Figures 3(a) and 3(b).

[0073] Time dimension analysis results:

[0074] (1) Regardless of the goodness of fit (R) 2 Or should mean absolute error (MAE) be used as a performance metric for soft sensor models? XGBoost is a base model with optimal soft measurement performance.

[0075] (2) The performance of the model is not linearly related to the amount of data in the time dimension. For XGBoost, when the time dimension variable t < 10, the model performance continues to increase with the increase of t; when t ≥ 10, the performance of the soft measurement model tends to level off, and thereafter, the model performance does not improve significantly with the increase of the time span. According to the cost-effectiveness calculation formula, the optimal number of samplings is 10.

[0076] (3) The performance of SVR and KNN soft measurement models is not as good as XGBoost, and the optimal number of sampling times is 14. According to the cost-effectiveness calculation formula, the cost-effectiveness of SVR and KNN modeling is lower than that of XGBoost.

[0077] Step S3-2, Spatial Dimension Analysis. Only the data size is changed in the spatial dimension; the time and feature dimensions remain unchanged. A sampling point σ is selected as the spatial center, and different numbers of sampling points are selected around it, starting from 1 and increasing in increments of 1, sequentially denoted as s1, s2, ..., s g , where s g ≤171. Using dataset The XGBoost, SVR, and KNN models were trained, validated, and tested respectively to obtain the target soft measurement model. Based on soft measurement performance This yields the base model with optimal soft measurement performance and the optimal number of sampling points, based on... This allows for the acquisition of the optimal cost-effectiveness base model and number of sampling points. The spatial variation of the DIN concentration soft-sensor model performance is shown in Figures 4(a) and 4(b). Based on the soft-sensor performance... XGBoost is the base model with optimal soft measurement performance, and the optimal number of sampling points is 171 using all sampling points.

[0078] Spatial dimension analysis results:

[0079] The spatial dimension variation patterns of the three models XGBoost, SVR, and KNN are basically the same. When the performance of the three models tends to stabilize, the spatial variable s = 20. According to the cost-effectiveness calculation formula, the basic model with the best cost-effectiveness is XGBoost, and the number of sampling points with σ as the spatial location center with the highest cost-effectiveness is 20.

[0080] Step S3-3, Feature Dimension Analysis. Only the data size is changed along the feature dimensions; the time and spatial dimensions remain unchanged. Features are sorted from largest to smallest based on their relevance to DIN, and different numbers of features are selected, starting from 1 and increasing in increments of 1, denoted as f1, f2, ..., f... h, where f h ≤14. Using datasets The XGBoost, SVR, and KNN models were trained, validated, and tested respectively to obtain the target soft measurement model. Based on soft measurement performance This yields the base model with optimal soft measurement performance and the optimal number of features; based on This yields the base model and number of features with optimal cost-effectiveness. The performance variations of the DIN concentration soft measurement model across the feature dimensions are shown in Figures 5(a) and 5(b).

[0081] Feature dimension analysis results:

[0082] (1) Regardless of the goodness of fit (R) 2 Or should mean absolute error (MAE) be used as a performance metric for soft sensor models? XGBoost is a base model with optimal soft measurement performance.

[0083] (2) The feature variable f values ​​corresponding to the rapid changes in performance of XGBoost, SVR, and KNN models as they level off are different, with f values ​​of 8, 4, and 8 for the three models, respectively. Based on the cost-effectiveness calculation formula, XGBoost is the base model with the best cost-effectiveness, and the optimal number of features is 8. The above is a data consumption analysis for application scenarios focusing only on single-dimensional data consumption. If the application scenario focuses on multiple dimensions, the following combined-dimensional analysis can be used.

[0084] Step S4, combined dimensional analysis. This involves simultaneously changing the data in two or three of the time, space, and feature dimensions, and then linking multiple dimensions based on single-dimensional analysis methods. In the time dimension, a set of sampling times is selected from smallest to largest, starting with 1 and increasing by one sampling increment, sequentially denoted as t1, t2, ..., t k , where t k ≤20; In the spatial dimension, one sampling point σ is randomly selected from 171 sampling points as the spatial center. Different numbers of sampling points are selected around it, with the number of sampling points increasing from 1 with a step size of 1, and are sequentially denoted as s1, s2, ..., s g , where s g ≤171; Features are sorted from largest to smallest based on their relevance to DIN, and different numbers of features are selected, starting from 1 and increasing in increments of 1, and are denoted as f1, f2, ..., f in ascending order. h , where f h ≤14. Using datasets The XGBoost, SVR, and KNN models were trained, validated, and tested respectively to obtain the target soft measurement model. Based on soft measurement performance This yields the basic model with optimal soft measurement performance, as well as the optimal number of sampling times, sampling points, and sampling features; based on This will yield the basic model with the best cost-performance ratio, as well as the number of sampling times, the number of sampling points, and the number of sampling features.

[0085] Conclusions of combined dimension analysis:

[0086] If only soft measurement performance is considered, the optimal base model is XGBoost, and the corresponding modeling dataset is: D (10,171,14) According to the cost-effectiveness calculation formula, the base model with the best cost-effectiveness is still XGBoost, but the modeling dataset with the best cost-effectiveness is D. (10,20,8) That is, sampling 10 times, with 20 sampling points, and collecting 8 feature factors at each sampling point.

[0087] As can be seen from the above examples of single-dimensional and combined-dimensional data consumption analysis, the data consumption analysis method described in this invention can select the most cost-effective basic model and modeling dataset, significantly reducing the cost of modeling data. Furthermore, the method described in this application can also serve as a model evaluation method to assess the model's data consumption, unlike existing model evaluation methods that only consider performance.

[0088] Example 2

[0089] This embodiment provides a soft measurement modeling data consumption analysis system;

[0090] A soft measurement modeling data consumption analysis system, comprising:

[0091] The model generation module is configured to: generate a data consumption cube model given soft measurement target features, dataset, and a set of candidate soft measurement base models; the data consumption cube model includes three dimensions: time, space, and features; a data consumption coordinate system is formed by the data consumption cube model, in which any point corresponds to a dataset instance, and the coordinate values ​​of each point correspond to the number of samplings, the number of sampling points, and the number of features collected at each sampling point in this dataset;

[0092] The configuration module is configured to: set the data cost function of the data consumption cube model to calculate the cost of the dataset instance; and set the cost-effectiveness evaluation function to calculate the cost-effectiveness index of the soft measurement model generated by the machine learning model using the dataset instance.

[0093] The analysis module is configured to: analyze the dataset according to the time dimension, spatial dimension, and feature dimension based on the data cost function and the cost-effectiveness evaluation function, respectively, to obtain the machine learning model with the best soft measurement performance in a single dimension, and the cost-effective sampling number, number of sampling points, and the best feature set collected at each sampling point; and perform combined analysis of the dataset in the time, space, and feature dimensions to obtain the machine learning model with the best soft measurement performance in multiple dimensions, and the cost-effective sampling number, number of sampling points, and the best feature set collected at each sampling point.

[0094] Furthermore, the system also includes a soft measurement module, which is configured to: train the machine learning model based on a machine learning model with optimal soft measurement performance, and the most cost-effective number of samplings, the number of sampling points, and the optimal feature set collected at each sampling point; and perform soft measurement of data based on the trained machine learning model.

[0095] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for analyzing data consumption in soft measurement modeling, characterized in that, include: Given the target features of soft measurement, the dataset, and the set of candidate basic models for soft measurement, a data consumption cube model is generated. The data consumption cube model includes three dimensions: time, space, and features. The data consumption cube model forms a data consumption coordinate system, in which any point corresponds to a dataset instance, and the coordinate values ​​of each point correspond to the number of samplings, the number of sampling points, and the number of features collected at each sampling point in this dataset instance. Set the data cost function of the data consumption cube model to calculate the cost of the dataset instance; Set up a cost-performance evaluation function to calculate the cost-performance ratio of the soft measurement model generated by the machine learning model using dataset instances; Based on the data cost function and cost-effectiveness evaluation function, the dataset is analyzed according to the time dimension, spatial dimension, or feature dimension to obtain the machine learning model with the best soft measurement performance in a single dimension, and the cost-effective sampling number, sampling point number, and the best feature set collected for each sampling point. By performing a combined analysis of the dataset across three dimensions—time, space, and features—we obtained a machine learning model with optimal soft measurement performance across multiple dimensions, as well as the most cost-effective sampling times, number of sampling points, and the optimal feature set collected for each sampling point. The target feature is set as dissolved inorganic nitrogen concentration (DIN), and a dataset is obtained for the studied water area. D Each sampling point collected various DIN-related features, including chemical, spatial, biological, physical, and structural features. Chemical features included total nitrogen concentration, dissolved inorganic silicon concentration, salinity, chemical oxygen demand (COD), dissolved oxygen concentration, petroleum concentration, and suspended solids concentration. Spatial features included longitude, latitude, and sample sampling depth. Biological features included chlorophyll-a. Physical features included water temperature. Structural features included dissolved inorganic phosphorus index and silicon-to-phosphorus ratio. XGBoost, SVR, and KNN were selected as the basic soft sensing models to generate a set of candidate soft sensing machine learning models. M ={XGBoost, SVR, KNN}.

2. The soft sensor modeling data consumption analysis method as described in claim 1, characterized in that, The data consumption cube model specifically includes: Data Consumption Cube Model Cube_data It includes three data dimensions: time dimension T Spatial dimension S and feature dimensions F ;in: The time dimension represents the number of data samples required to build the soft measurement model; Spatial dimension represents the number of sampling points that need to be set in space to construct a soft measurement model; Feature dimension refers to the set of features to be collected at each sampling point.

3. The soft sensor modeling data consumption analysis method as described in claim 1, characterized in that, The dataset was analyzed according to the time dimension; specifically, this included: Time-dimensional analysis involves changing the data size only in the time dimension, while keeping the spatial and feature dimensions unchanged; a set of time points is selected sequentially from smallest to largest in the time dimension. ,..., ,in, Using datasets , For the model m 1, m 2, …, m n The target soft measurement model was obtained by training, validating, and testing the model separately. , Based on soft measurement performance ), , This yields the basic model with optimal soft measurement performance and the optimal number of samplings, based on... This will allow you to obtain the base model and number of samplings with the best cost-performance ratio.

4. The soft sensor modeling data consumption analysis method as described in claim 1, characterized in that, The dataset is analyzed according to its spatial dimension; specifically, this includes: Spatial dimension analysis refers to changing the data size only in the spatial dimension, while keeping the data unchanged in the time and feature dimensions; selecting sampling points. Using the spatial location center as an example, select different numbers of sampling points around it, denoted as follows: (The numbers are listed in ascending order). ,..., ,in, Using datasets , For the model m 1, m 2, …, m n The target soft measurement model was obtained by training, validating, and testing the model separately. , Based on soft measurement performance ), , This yields the basic model with optimal soft measurement performance and the optimal number of sampling points, based on... This will allow you to obtain the base model and number of sampling points with the best cost-performance ratio.

5. The soft sensor modeling data consumption analysis method as described in claim 1, characterized in that, The dataset is analyzed according to its feature dimensions; specifically, this includes: Feature dimension analysis, i.e., only changing the data size along the feature dimensions, while keeping the time and spatial dimensions unchanged; and adjusting the data along the feature dimensions according to the target features. y The correlations are sorted from largest to smallest, and different numbers of features are selected, denoted as follows: (The list of features is missing from the original text). ,..., ,in, Using datasets , For the model m 1, m 2, …, m n The target soft measurement model was obtained by training, validating, and testing the model separately. , Based on soft measurement performance ), , This yields the basic model with optimal soft measurement performance and the optimal number of features; based on This will yield the base model and number of features with the best cost-performance ratio.

6. The soft sensor modeling data consumption analysis method as described in claim 1, characterized in that, The dataset undergoes combined analysis across three dimensions: time, space, and features; specifically including: Simultaneously change the data in two or three of the time, space and feature dimensions, and correlate multiple dimensions based on the single-dimensional analysis method; In terms of time, select a set of time points in ascending order. ,..., ,in, ; In the spatial dimension, sampling points are selected. Using the spatial location center as an example, select different numbers of sampling points around it, denoted as follows: (The numbers are listed in ascending order). ,..., ,in, ; In terms of feature dimension, according to the target features y The correlations are sorted from largest to smallest, and different numbers of features are selected, denoted as follows: (The list of features is missing from the original text). ,..., ,in, ; Using dataset , ; For the model m 1, m 2,…, m n The target soft measurement model was obtained by training, validating, and testing the model separately. , Based on soft measurement performance ), , This yields the basic model with optimal soft measurement performance, as well as the optimal number of sampling times, sampling points, and sampling features; based on This will yield the basic model with the best cost-performance ratio, as well as the number of sampling times, the number of sampling points, and the number of sampling features.

7. The soft measurement modeling data consumption analysis method as described in claim 1, characterized in that, The method further includes: training, validating and testing the machine learning model based on the machine learning model with the best soft measurement performance, the most cost-effective number of samplings, the number of sampling points, and the best feature set collected at each sampling point; and implementing soft measurement based on the soft measurement model obtained after training.

8. The soft measurement modeling data consumption analysis method as described in claim 1, characterized in that, The given soft measurement target features, dataset, and set of candidate basic soft measurement models specifically include: Given soft measurement target features y、 Dataset D、 Soft Measurement Candidate Base Model Set M ; Among them, dataset D This includes data acquisition time, spatial location of sampling points, and the characteristics collected at each sampling point related to the target. y Related features; soft measurement candidate machine learning base models M ={ m 1, m 2, …, m n }, n This represents the number of models.

9. The soft measurement modeling data consumption analysis method as described in claim 1, characterized in that, Data consumption cube model Cube_data Form a data consumption coordinate system, where the coordinates of any point ( () corresponds to a dataset instance This dataset contains Number of samplings There are 10 sampling points, and each sampling point collects 10 samples. One feature data; The data cost function and cost-effectiveness evaluation function specifically include: set up )for Cube_data The model's data cost function is used to calculate the dataset. Cost; for any basic model Using dataset The target soft measurement model generated after training, validation, and testing is denoted as ,Model The soft measurement performance is denoted as ); Set up a cost-performance evaluation function Used to compute the basic model Using dataset The generated soft measurement model The cost-performance ratio.

10. A soft measurement modeling data consumption analysis system, characterized in that, include: The model generation module is configured to: generate a data consumption cube model given soft measurement target features, dataset, and a set of candidate soft measurement base models; the data consumption cube model includes three dimensions: time, space, and features; a data consumption coordinate system is formed by the data consumption cube model, in which any point corresponds to a dataset instance, and the coordinate values ​​of each point correspond to the number of samplings, the number of sampling points, and the number of features collected at each sampling point in this dataset; The configuration module is configured to: set the data cost function of the data consumption cube model for calculating the cost of dataset instances; Set up a cost-performance evaluation function to calculate the cost-performance ratio of the soft measurement model generated by the machine learning model using dataset instances; The analysis module is configured to: analyze the dataset according to the time dimension, spatial dimension, and feature dimension based on the data cost function and the cost-effectiveness evaluation function, respectively, to obtain the machine learning model with the best soft measurement performance in a single dimension, and the cost-effectiveness of the sampling number, the number of sampling points, and the optimal feature set collected at each sampling point; and perform combined analysis of the dataset in the time, space, and feature dimensions to obtain the machine learning model with the best soft measurement performance in multiple dimensions, and the cost-effectiveness of the sampling number, the number of sampling points, and the optimal feature set collected at each sampling point. The target feature is set as dissolved inorganic nitrogen concentration (DIN), and a dataset is obtained for the studied water area. D Each sampling point collected various DIN-related features, including chemical, spatial, biological, physical, and structural features. Chemical features included total nitrogen concentration, dissolved inorganic silicon concentration, salinity, chemical oxygen demand (COD), dissolved oxygen concentration, petroleum concentration, and suspended solids concentration. Spatial features included longitude, latitude, and sample sampling depth. Biological features included chlorophyll-a. Physical features included water temperature. Structural features included dissolved inorganic phosphorus index and silicon-to-phosphorus ratio. XGBoost, SVR, and KNN were selected as the basic soft sensing models to generate a set of candidate soft sensing machine learning models. M ={XGBoost, SVR, KNN}.

Images