Sea sand traceability big data modeling method fusing multi-dimensional test data

Abstract

The invention discloses a sea sand traceability big data modeling method fusing multi-dimensional test data, and relates to the technical field of marine science and geology, and the method comprises the steps: collecting multi-source data of marine geological survey and sea sand resource survey over the years, integrating 218 types of original test data, and carrying out the unified storage; and for data differences of different sources, cleaning and screening common test elements of each region, and generating a standardized data set. According to the method, the physical and chemical characteristics of the sea sand are comprehensively reflected by fusing the multi-dimensional test data including 218 kinds of original test data such as granularity, geochemistry and minerals, the limitation of single index analysis is avoided, the source of the sea sand can be more accurately identified, the complex mode of the source of the sea sand can be captured by the model due to integration of the multi-dimensional data, and the accuracy of the model is improved. Therefore, high-precision traceability can be realized for sea sand with similar indexes from different sea areas. And the accuracy and reliability of the traceability model are obviously improved.

Description

Technical Field

[0001] The present invention relates to the field of marine science and geological technology, and in particular to a sea sand traceability big data modeling method that integrates multi-dimensional test data. Background Art

[0002] With the rapid development of economic construction in coastal areas, the demand for sea sand, as an important building material and land reclamation material, has increased dramatically. However, illegal mining of sea sand has occurred frequently, seriously damaging the marine ecological environment and coastal geological safety. Therefore, it has become an urgent need to establish an effective sea sand traceability system and scientifically manage and protect sea sand resources. However, the data sources involved in sea sand traceability research are extensive, including marine geological surveys, sea sand resource surveys and other projects. These data have significant differences in time, space, testing elements and other aspects. With the rapid development of technologies such as big data and artificial intelligence, new ideas and methods have been provided for sea sand traceability research.

[0003] Existing sea sand traceability technology often relies on a single or a few test indicators. This single-dimensional analysis method is difficult to fully reflect the complex source characteristics of sea sand, especially when facing sea sand from different sea areas but with similar indicators. The traceability accuracy will drop significantly, and the existing traceability model may be too simple to capture the complex pattern of sea sand sources, or lacks generalization ability when processing new data. Therefore, how to integrate the multi-dimensional test data of sea sand traceability to fully reflect the physical and chemical properties of sea sand, and build a more accurate model structure, effectively extract the deep-level features in the data, and ensure that the constructed traceability model has high accuracy and generalization, is the problem to be solved by the present invention. To this end, a sea sand traceability big data modeling method that integrates multi-dimensional test data is proposed. Summary of the Invention

[0004] The purpose of the present invention is to provide a sea sand traceability big data modeling method that integrates multi-dimensional test data to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: A sea sand traceability big data modeling method integrating multi-dimensional test data includes the following steps: Step 1: Collect multi-source data from marine geological surveys and sea sand resource surveys over the years, integrate 218 types of original test data such as particle size, geochemistry, and mineralogy, and store them in a unified database; Step 2: Based on the differences in data from different sources, clean and filter the test elements common to each region, handle missing values ​​and outliers, generate a standardized data set, and ensure data quality; Step 3: Based on the standardized dataset, match the sample coordinates according to the 11 sea area labels to generate a multidimensional dataset with geo-tags; Step 4: Z-score standardization method is used to deal with numerical differences, filter out missing indicator samples, and use SMOTE technology to balance the samples of 11 geographical regions, synthesize data from a small number of sea areas, balance the category distribution, and form a comprehensive dataset with balanced category distribution; Step 5: After Z-score normalization, stratified sampling was performed with an 8:2 ratio to establish training and test sets. Random forest, support vector machine, and deep neural network models were constructed in parallel to extract complementary features. In step 6, the optimal model is selected based on weighted F1, confusion matrix, and feature importance, and generalization is continuously improved through iteration to form the final traceability model.

[0006] A further improvement of the technical solution of the present invention is that: the step 1 comprises: Collect multi-source data from marine geological surveys and sea sand resource surveys over the years, covering different survey projects, regions and periods, and conduct preliminary screening of the collected data. At the same time, record the data source and collection time to trace the original background of the data; The processed multi-source data are analyzed, and 218 types of original test data such as particle size, geochemistry, and minerals are classified and integrated according to data type and characteristics. A standardized database framework is established, and the integrated data are stored in a unified manner according to the preset database structure and rules. At the same time, a unique identifier is set for each data item, and a data backup mechanism is established to ensure data security.

[0007] A further improvement of the technical solution of the present invention is that: step 2 includes: To address the differences in test elements, parameter definitions, and measurement methods among multi-source data, we conducted data structure analysis and standardization. By comparing metadata documents, variable naming conventions, and analysis method descriptions across different datasets, we identified core test elements that recurred in data from each region. We then screened out key elements shared by all regions, established a unified data subset framework, and eliminated redundant or non-common variables. Missing values ​​and outliers are detected for the filtered common feature data. A hierarchical strategy is used for missing value processing: if the missing rate of a variable in the global data exceeds a preset threshold, it is marked as a high-missing variable and evaluated for removal. If a variable is locally missing, the mean / median of spatially or temporally adjacent samples is preferentially used for interpolation. Outlier detection combines statistical methods with domain knowledge thresholds to review values ​​that exceed the preset reasonable range. Once an error is confirmed, it is replaced with a nearby valid value or treated as missing to ensure that the data distribution conforms to geological laws. The processing results are verified through multi-dimensional quality checks, including completeness, accuracy and consistency. For the verified data, the units, coordinate systems and naming conventions are unified, standardized data sets are generated, and processing logs are recorded, ultimately forming high-quality, interoperable regional marine geological data resources.

[0008] A further improvement of the technical solution of the present invention is that: step 3 includes: Load the standardized dataset, extract the geographic coordinate information of each sample, standardize the coordinates, remove outliers, and fill in the missing coordinate information to generate a coordinate table containing sample ID, latitude and longitude; Based on the international maritime boundary agreement, the geographical scope of 11 sea areas is clarified. The boundaries of each sea area are described using polygonal vector files. The boundary coordinate range of each sea area is defined. The sample coordinates are matched with the sea area boundaries. The samples are assigned to the corresponding sea area labels according to the coordinate position. For samples located near the boundaries, the nearest neighbor algorithm is used to determine the sea area to which they belong, ensuring that each sample can be accurately matched with a unique sea area label. Then, a mapping relationship table between sea area labels and boundaries is established. The samples with matched sea area labels are integrated with the test data in the standardized dataset to form a multidimensional dataset with geographical labels. The integrity and consistency of the dataset are checked to ensure that the test data of each sample accurately corresponds to the sea area label.

[0009] A further improvement of the technical solution of the present invention is that: for samples located near the boundary, the specific process of using the nearest neighbor algorithm to determine the sea area to which they belong is: Load a polygon vector file containing 11 sea area boundaries, ensure that the coordinates of the boundary polygons of each sea area are clear and unambiguous, extract the coordinates of the polygon vertices of each sea area boundary to form a boundary coordinate set, and load a coordinate table containing sample IDs, latitude and longitude, and filter out samples located near the sea area boundary, that is, samples whose distance from the boundary is less than the preset proximity threshold; Define the Haversine formula to calculate the distance from the sample point to the sea area boundary, traverse the boundary polygon of each sea area, traverse all its vertices for each boundary polygon, calculate the distance from the sample point to each vertex, and retain the minimum distance and its corresponding sea area; For each sample point, compare its minimum distance to all sea area boundaries, determine the nearest sea area, assign the sample point to the nearest sea area, and record its sea area label; If the distances between multiple sea areas are the same, that is, the distances from the sample point to multiple sea areas are the same, the area weight strategy is adopted for allocation. The area weight selects the sea area with a larger area, and then adds the determined sea area label to the sample data to generate complete sample data with geographic labels.

[0010] A further improvement of the technical solution of the present invention is that the calculation formula of the distance from the sample point to the sea area boundary is as follows: ; ; ; Where d is the distance between two points, indicating the distance from the sample point to the sea boundary, and R is the radius of the earth, which is approximately 6371 km. and are the latitudes of the two points, and is the longitude of two points, is the latitude difference, is the longitude difference; The calculation formula of the distance from the sample point to each vertex is as follows: ; Where, is the distance from the sample point to each vertex, indicating that each sample point The distance between the vertex of the boundary polygon of each sea area i, and are the longitude and latitude of the sample point, and are the longitude and latitude of the jth vertex in the i-th sea area.

[0011] A further improvement of the technical solution of the present invention is that: step 4 includes: Perform Z-score normalization on all numerical indicators in the multidimensional dataset, calculate the mean and standard deviation of each indicator, and check the indicator integrity of each sample. If any key indicator is missing, the sample will be screened out; Based on the pre-assigned sea area labels, the number of samples in 11 geographical regions was counted, the proportion of each category was calculated, and the problem of uneven sample distribution was identified. The category differences were intuitively displayed through visualization (stacked bar charts), and the minority sea areas to be synthesized were clearly identified. At the same time, the original data of the majority class samples were retained, and only the minority class samples were oversampled to avoid information redundancy. For minority sea samples, the SMOTE algorithm is applied to generate synthetic data. For each minority sample, its k nearest neighbors (k=5) are calculated, a neighboring sample is randomly selected, and new samples are generated by linear interpolation until the number of samples in each category reaches the preset balance threshold. Finally, the original majority class samples and the synthesized minority class samples are integrated to form a comprehensive data set with balanced category distribution.

[0012] A further improvement of the technical solution of the present invention is that: step 5 includes: Perform stratified sampling on the Z-score normalized multidimensional dataset, dividing the training set and test set into a ratio of 8:2. After the division is completed, the training set and test set are saved separately to ensure data independence in the subsequent modeling process and prevent information leakage; Based on the divided training set, three classification models, random forest, support vector machine and deep neural network, are constructed in parallel. Among them, random forest captures nonlinear feature interactions by integrating multiple decision trees to achieve high accuracy and generalization ability; support vector machine uses kernel function to deal with nonlinear problems and optimizes classification boundaries; deep neural network extracts complex features through multi-layer neurons and learns the deep structure of data; each model is trained independently to extract complementary features. Among them, random forest outputs feature importance ranking, support vector machine provides key support vectors, and deep neural network extracts hidden layer features. The extracted complementary features are integrated to form a comprehensive feature set for subsequent model optimization and performance improvement, enhancing the model's understanding of sea sand traceability data and classification accuracy.

[0013] A further improvement of the technical solution of the present invention is that: step 6 includes: Independently evaluate the performance of random forests, support vector machines, and deep neural networks on the test set, calculating the weighted F1 score of each model (balancing precision and recall for multi-class classification). Analyze the misclassification patterns of the models for different sea area labels using a confusion matrix to identify the strengths and weaknesses of the models. Prioritize eliminating baseline solutions with significantly lower weighted F1 scores than other models, retaining the 2-3 candidate models with the best overall performance. Extract complementary features from candidate models, including feature importance ranking from random forests, key support vectors from support vector machines, and hidden layer features from deep neural networks. Construct a weighted integrated feature set based on feature importance scores. Use a feature selection algorithm to reduce dimensionality and remove redundant information to obtain an optimized weighted integrated feature set. Then, retrain the model using this optimized weighted integrated feature set. Compare the weighted F1 scores and confusion matrices of the models before and after fusion to verify the effectiveness of feature integration in improving classification boundaries and generalization capabilities. The performance of the optimized model is evaluated again on the test set. Combined with the weighted F1, the category-level performance of the confusion matrix and the stability of feature importance, the model with the strongest generalization ability is selected as the final traceability model. If the model still has classification bias in specific sea areas, it can be further optimized by adjusting the category weights or adding corresponding samples.

[0014] Due to the adoption of the above technical solution, the present invention has the following technical advancements compared to the prior art: The present invention provides a sea sand traceability big data modeling method that integrates multi-dimensional test data. By integrating multi-dimensional test data, including 218 types of original test data such as particle size, geochemistry, and mineralogy, the method comprehensively reflects the physical and chemical characteristics of sea sand, avoids the limitations of single-index analysis, and can more accurately identify the source of sea sand. The integration of multi-dimensional data enables the model to capture the complex patterns of sea sand sources, thereby achieving high-precision traceability when faced with sea sand from different sea areas but with similar indicators. This significantly improves the accuracy and reliability of the traceability model.

[0015] The present invention provides a sea sand traceability big data modeling method that integrates multi-dimensional test data. It balances the sample distribution through Z-score standardization processing and SMOTE technology, effectively solving the dimensional difference and category imbalance problems in the data. Among them, the standardization processing eliminates the dimensional difference between different indicators, making the model training more stable. The SMOTE technology balances the number of samples from different sea areas by synthesizing minority class samples, avoiding overfitting of the model to the majority class, and significantly enhancing the generalization ability of the model, so that it can maintain high accuracy and stability when processing new data, thereby better adapting to the sea sand traceability needs in different scenarios.

[0016] The present invention provides a sea sand traceability big data modeling method that integrates multi-dimensional test data. Through the parallel construction of three models: random forest, support vector machine and deep neural network, complementary features are extracted and integrated to form a comprehensive feature set. The feature selection algorithm is used to further reduce the dimension and remove redundant information, and optimize the feature set. This not only improves the training efficiency of the model, but also enhances the model's ability to understand sea sand traceability data, further improving the classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a schematic diagram of the workflow of the present invention; Figure 2 Schematic diagram of the method of the present invention. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0020] Example 1, as Figure 1 、 Figure 2 As shown, the present invention provides a sea sand traceability big data modeling method integrating multi-dimensional test data, comprising the following steps: Step 1: Collect multi-source data from marine geological surveys and sea sand resource surveys over the years, integrate 218 types of original test data such as particle size, geochemistry, and minerals, and store them in a unified database. Collect multi-source data from marine geological surveys and sea sand resource surveys over the years, covering different survey projects, regions, and periods, and conduct preliminary screening of the collected data to eliminate obviously erroneous or invalid data. At the same time, record the data source and collection time to trace the original background of the data, analyze the processed multi-source data, and classify and integrate the 218 types of original test data such as particle size, geochemistry, and minerals according to data type and characteristics, and establish a standardized database framework. The integrated data is stored in a unified database according to the preset database structure and rules. At the same time, set a unique identifier for each data item to facilitate subsequent query and management. At the same time, establish a data backup mechanism to ensure data security; The detailed process of step 1 is as follows: Data from the Qingdao Institute of Marine Geology and Oceanography over the years, including 218 types of original test data on particle size, geochemistry, light and heavy minerals, were collected. The data sources (specific survey projects, regions, and periods) and collection times were recorded in detail to ensure that the key fields (coordinates, test indicators) of each record were complete, and records with excessive missing values ​​were removed. According to the data types and characteristics, the 218 types of test data were divided into categories such as particle size data, geochemical data, and mineral data. After data sorting, the number of valid samples was as follows: 62,491 samples of particle size test data, 28,412 samples of geochemical test data, 12,167 samples of light mineral test data, and 10,884 samples of heavy mineral test data. A standardized database structure was designed, including data tables, fields, and data types, to ensure standardized data storage. A unique identifier was set for each data item, and the integrated data was imported into the database according to the preset database structure and rules. Indexes were established for key fields to improve query efficiency. The database was backed up regularly, and the backup data was stored in cloud storage to prevent data loss. Step 2: In view of the differences in data from different sources, clean and screen the test elements common to each region, handle missing values ​​and outliers, generate standardized data sets, ensure data quality, and conduct data structure analysis and standardization comparison for the differences in test elements, parameter definitions and measurement methods of multi-source data. By comparing the metadata documents, variable naming rules and analysis method descriptions of different data sets, identify the core test elements (such as particle size classification, geochemical indicators, mineral types, etc.) that appear repeatedly in the data of each region, and then screen out the key elements common to all regions, establish a unified data subset framework, and eliminate redundant or non-common variables. For the common element data after screening, detect missing values ​​and outliers. Among them, the missing value processing adopts a hierarchical strategy: if the missing rate of a variable in the global data exceeds the preset threshold, it is marked as a high missing variable. The data are collected and evaluated for elimination. If there are local missing elements, the mean / median of spatially or temporally adjacent samples is used for interpolation. Outlier detection is combined with statistical methods and domain knowledge thresholds to review values ​​that exceed the preset reasonable range. After confirming the error, it is replaced with a valid adjacent value or treated as missing to ensure that the data distribution conforms to geological laws. The processing effect is verified through multi-dimensional quality checks, including completeness, accuracy and consistency. In terms of completeness, it is confirmed that there are no systematic missing elements. In terms of accuracy, it is cross-validated whether key parameters meet theoretical constraints. In terms of consistency, the compatibility of statistical characteristics of the same element in different data sources is checked. For the data that pass the verification, the unit, coordinate system and naming convention are unified, a standardized data set is generated, and processing logs are recorded, ultimately forming a high-quality, interoperable regional marine geological data resource. The detailed process of step 2 is as follows: Obtain metadata documents of different data sets, including variable naming rules, analysis method descriptions, and test element definitions. By comparing metadata, identify the core test elements that appear repeatedly in the data of each region, including grain size classification, geochemical indicators, and mineral types. Among them, grain size classification such as GRAVEL, SAND, SILT, CLAY, etc., geochemical indicators such as As, Co, Cr, Cu, Nb, Ni, Pb, Zn, Zr, etc., mineral types such as light minerals, heavy minerals, etc., and then screen out the key elements shared by all regions, establish a unified data subset framework, eliminate redundant or non-common variables, and standardize and compare the data. The unit unification, coordinate system unification and naming convention unification are as follows: the unit unification is to convert the units of all data into a unified standard, the coordinate system unification is to convert the coordinates of all data into a unified coordinate system, and the naming convention unification is to standardize the variable naming to ensure that the same elements of different data sources have the same naming rules; the missing rate of each variable in the global data is calculated. If the missing rate exceeds the preset threshold, it is marked as a high missing variable and evaluated whether to remove it. For locally missing variables, the mean or median of spatially or temporally adjacent samples is preferably used for interpolation. For example, for geographically related data, the mean or median of spatially adjacent samples is used for interpolation, and for temporally adjacent samples, the mean or median of spatially adjacent samples is used for interpolation. For time series data, use the mean or median of samples adjacent in time to interpolate; record the methods and results of missing value processing in detail for easy follow-up, use statistical methods (such as Z-score, IQR) to detect outliers, combine domain knowledge thresholds to review values ​​that exceed the reasonable range, combine geological knowledge to verify outliers, and replace them with adjacent valid values ​​after confirming errors or treat them as missing. Record the methods and results of outlier processing in detail to ensure that the data distribution conforms to geological laws; conduct multi-dimensional quality checks on the data, and confirm through integrity checks that there are no systematic missing elements for common elements, and ensure that each key element has data in all areas. For areas with a large number of missing data, supplement data or re-collect data. Through accuracy checks, cross-validate key parameters to ensure that they meet theoretical constraints. Review data with questions to ensure data accuracy. Through consistency checks, check the compatibility of statistical characteristics (mean, standard deviation) of the same elements in different data sources. For data that pass the verification, unify the units, coordinate systems and naming conventions to ensure data consistency. Integrate data that pass the quality check into a standardized data set to ensure data high quality and interoperability. Record the entire data processing process in detail, including data sources, processing methods, processing results, etc., to facilitate subsequent traceability and verification; Step 3: Based on the standardized data set, match the sample coordinates according to the 11 sea area labels to generate a multidimensional data set with geographic tags. Load the standardized data set and extract the geographic coordinate information of each sample from it, ensuring that the coordinate format is unified as longitude and latitude (WGS84 coordinate system). Standardize the coordinates, remove outliers, and supplement the missing coordinate information to generate a coordinate table containing sample ID, longitude and latitude. Based on the international maritime boundary agreement, clarify the geographical scope of the 11 sea areas, describe the boundaries of each sea area through polygon vector files, define the boundary coordinate range of each sea area, and ensure that the vector file format of the sea area boundary is correct and contains the sea area name. The coordinate information of the sample and boundary polygon is used to match the sample coordinates with the sea area boundary. The samples are assigned to the corresponding sea area labels according to the coordinate position. For samples located near the boundary, the nearest neighbor algorithm is used to determine the sea area to which they belong, ensuring that each sample can be accurately matched with a unique sea area label. Then, a mapping relationship table between sea area labels and boundaries is established. The samples with matched sea area labels are integrated with the test data in the standardized data set (such as particle size, geochemistry, minerals and other indicators) to form a multidimensional data set with geographical tags. The integrity and consistency of the data set are checked to ensure that the test data of each sample accurately corresponds to the sea area label. In addition, for samples located near the boundary, the specific process of using the nearest neighbor algorithm to determine the sea area to which it belongs is as follows: Load the polygon vector file containing 11 sea area boundaries, ensure that the coordinates of the boundary polygon of each sea area are clear and unambiguous, extract the polygon vertex coordinates of each sea area boundary, form a boundary coordinate set, and load the coordinate table containing the sample ID, longitude and latitude, filter out the samples located near the sea area boundary, that is, the samples whose distance from the boundary is less than the preset proximity threshold, define the Haversine formula to calculate the distance from the sample point to the sea area boundary, traverse the boundary polygon of each sea area, for each boundary polygon, traverse all its vertices, calculate the distance from the sample point to each vertex, retain the minimum distance and its corresponding sea area, for each sample point, compare Compare the minimum distance to all sea area boundaries to determine the nearest sea area, assign the sample point to the nearest sea area, and record its sea area label. If the distances to multiple sea areas are the same, that is, the distances from the sample point to multiple sea areas are the same, an area weight strategy is used for allocation, where the area weight means selecting the sea area with the larger area. If the sample point is located on the boundary of two sea areas, the sample is marked as a special case in the data and will be taken into account in subsequent analysis. The determined sea area label is then added to the sample data to generate complete sample data with geographic labels, and the processing process of each sample is recorded in detail, including the calculated distance and the selected sea area. The calculation formula for the distance from the sample point to the sea area boundary is as follows: ; ; ; Where d is the distance between two points, indicating the distance from the sample point to the sea boundary, and R is the radius of the earth, which is approximately 6371 km. and is the latitude of the two points (in radians), and is the longitude of the two points (in radians), is the latitude difference, is the longitude difference; The calculation formula of the distance from the sample point to each vertex is as follows: ; Where, is the distance from the sample point to each vertex, indicating that each sample point The distance between the vertex of the boundary polygon of each sea area i, and are the longitude and latitude of the sample point (in radians), and is the longitude and latitude (in radians) of the jth vertex in the i-th sea area; For each sample point , traverse all vertices in all sea areas and calculate the minimum distance And its corresponding sea area i, its expression is as follows: ; For each sample point , compare its minimum distance to all sea area boundaries , determine the nearest sea area i, its expression is as follows: ; Where, Represents sample points Nearest sea area tag, Indicates that the Parameters that reach their minimum value; If the distances between multiple sea areas are the same, there are multiple i such that Equal, the area weight strategy is used for allocation. If the area of ​​the i-th sea area is , then select the sea area with the largest area, and its expression is as follows: ; Where, Indicates the sea area label selected according to the area weight strategy, Represents the area of ​​the i-th sea area, k represents the index of all sea areas, which is used to find the minimum distance from the sample point to all sea areas. The role of k is to traverse all sea areas and find the sample point Minimum distance to each sea area k ; Step 4: Use the Z-score standardization method to process numerical differences, screen out missing indicator samples, and use the SMOTE technology to balance the samples of 11 geographical regions, synthesize minority sea area data, balance the category distribution, and form a comprehensive data set with balanced category distribution. Perform Z-score standardization on all numerical indicators in the multidimensional data set, calculate the mean and standard deviation of each indicator, and convert the original value into a standard normal distribution with a mean of 0 and a standard deviation of 1 to eliminate the influence of dimensional differences and extreme values. The indicator integrity of each sample is checked. If any key indicator is missing (grain size classification, geochemical indicators, and mineral types, etc.), the sample is screened out. According to the pre-assigned sea area labels, the number of samples in 11 geographical regions is counted, the proportion of each category is calculated, the problem of sample distribution imbalance is identified, and the category differences are intuitively displayed through visualization (stacked bar charts) to clarify the minority sea areas to be synthesized. At the same time, the original data of the majority class samples are retained, and only the minority class samples are oversampled to avoid information redundancy. For the minority sea area samples, SMOTE (Synthetic Minority Equation) is applied. The over-sampling technique (OTT) algorithm generates synthetic data. For each minority class sample, its k nearest neighbors (k=5) are calculated. A random neighboring sample is selected and a new sample is generated by linear interpolation until the number of samples in each class reaches a preset balance threshold. Finally, the original majority class samples and the synthesized minority class samples are integrated to form a comprehensive dataset with a balanced class distribution. This ensures that the weights of samples in each sea area are consistent during model training, thereby improving classification performance. The calculation formula of the new sample is as follows: ; Where, is a new sample, x is a minority sample, is a random number ranging from [0,1]. is a neighboring sample; Step 5: After Z-score normalization, stratified sampling was performed with an 8:2 ratio to establish training and test sets. Random forest, support vector machine, and deep neural network models were constructed in parallel to extract complementary features. In step 6, the optimal model is selected based on weighted F1, confusion matrix, and feature importance, and generalization is continuously improved through iteration to form the final traceability model.

[0021] Example 2, as Figure 1 、 Figure 2 As shown, based on Example 1, the present invention provides a technical solution: preferably, the step 5 includes: The multidimensional data set after Z-score normalization was stratified and sampled, and the training set and test set were divided into training set and test set in a ratio of 8:2. Among them, stratified sampling ensured that the labels of each sea area were distributed in the original ratio in the training set and test set to ensure the representativeness of the data. After the division was completed, the training set and test set were saved separately to ensure the data independence of the subsequent modeling process and prevent information leakage. Based on the divided training set, three classification models, random forest, support vector machine and deep neural network, were constructed in parallel. Among them, random forest captured nonlinear feature interactions by integrating multiple decision trees to achieve high accuracy and generalization ability; support vector machine used kernel function to deal with nonlinear problems and optimize classification boundaries; deep neural network extracted complex features through multiple layers of neurons and learned the deep structure of data; each model was trained independently to extract complementary features. Among them, random forest output feature importance ranking, support vector machine provided key support vectors, and deep neural network extracted hidden layer features. The extracted complementary features were integrated to form a comprehensive feature set for subsequent model optimization and performance improvement, enhancing the model's understanding of sea sand traceability data and classification accuracy. The specific process of step 5 is as follows: The proportion of each sea area label in the complete data set was counted, and the data was stratified according to the sea area labels. Samples were randomly selected from each layer at a ratio of 8:2 as training sets and test sets respectively. The ratio of each sea area label in the training set and the test set was checked to ensure that it was consistent with the ratio of the original data set. The training set and the test set were divided at a ratio of 8:2 to ensure that the samples of the training set and the test set did not overlap with each other to prevent information leakage. The training set and the test set were saved separately. Based on the divided training set, three classification models, random forest, support vector machine and deep neural network, were constructed in parallel. Among them, for random forest, nonlinear feature interactions were captured by integrating multiple decision trees. Samples were randomly selected from the original training set with replacement. When constructing each node of each decision tree, some features were randomly selected. The decision tree was constructed using the CART algorithm without pruning. The splitting criterion used Gini impurity, and the majority voting method was used to determine the category of the final sample. For support vector machine, kernel function was used to deal with nonlinear problems, optimize the classification boundary, and find the maximized interval of the optimal hyperplane. , use the radial basis function (RBF) kernel to deal with nonlinear problems, transform them into dual problems through the Lagrange multiplier method, and solve the optimal solution; for deep neural networks, extract complex features through multi-layer neurons, learn the deep structure of data, adopt a multi-layer fully connected neural network, including input layer, hidden layer and output layer, use ReLU activation function and Softmax output layer, adopt Dropout strategy to prevent overfitting, and use Adam optimizer for training; extract complementary features from the three models, for random forest, use Gini importance method to evaluate feature importance, output feature importance ranking, for support vector machine, provide key support vector, define the key points of the optimal hyperplane, for deep neural network, extract hidden layer features, reflect the deep structure of the data, integrate the feature importance ranking of random forest, key support vector of support vector machine, and hidden layer features of deep neural network to form a comprehensive feature set, use the comprehensive feature set to retrain the model, and enhance the model's understanding of sea sand traceability data and classification accuracy; The step 6 comprises: The performance of random forests, support vector machines, and deep neural networks was independently evaluated on the test set. The weighted F1 score of each model (balancing the precision and recall of multi-category classification) was calculated. The misclassification patterns of the models for different sea area labels were analyzed through the confusion matrix to identify the strengths and weaknesses of the models. The baseline solutions with weighted F1 scores significantly lower than other models were eliminated first, and the 2-3 candidate models with the best overall performance were retained. The complementary features of the candidate models were extracted, including the feature importance ranking of random forests, the key support vectors of support vector machines, and the hidden layer features of deep neural networks. A weighted comprehensive feature set was constructed based on the feature importance scores. The feature selection algorithm was used to reduce the dimensionality and remove redundant information to obtain the optimized weighted comprehensive feature set. , and then use the optimized weighted comprehensive feature set to retrain the model, compare the weighted F1 scores and confusion matrix changes of the models before and after fusion, verify the effect of feature integration on the improvement of classification boundaries and generalization capabilities, and evaluate the performance of the optimized model on the test set again. Combined with the weighted F1, the category-level performance of the confusion matrix and the stability of feature importance, the model with the strongest generalization ability is selected as the final traceability model. If the model still has classification deviations in specific sea areas, it can be further optimized by adjusting the category weights or adding corresponding samples. The final traceability model must meet the following requirements: the weighted F1 reaches above the industry benchmark, the confusion matrix diagonal accounts for more than 85%, and the interpretability of key features is consistent with domain cognition, to ensure the reliability and reproducibility of the traceability results; The calculation formula of the weighted F1 score is as follows: ; ; ; Where WF1 is the weighted F1 score, C is the total number of categories (11 sea areas), is the weight of the g-th category, which indicates the proportion of samples in this category to the total number of samples. is the accuracy of the g-th category, is the recall rate of the g-th category, is the true number of examples of the g-th category, is the number of false positives for the g-th category, is the number of false negative examples of the g-th category; The confusion matrix is ​​used to analyze the misclassification patterns of different sea area labels. The confusion matrix is ​​a C×C matrix. The weighted comprehensive feature set is constructed based on the feature importance score, and its calculation formula is as follows: ; Where WFI is the sum of all feature importance scores, which is used to normalize the importance score of a single feature. M is the total number of features, that is, the number of features in the dataset. is the weight of the rth feature, which refers to the average importance score of the feature in all models, and is used to reflect the importance of the feature in the model. is the importance score of the rth feature, indicating the contribution of the feature to the model's predictive ability. t is used as the index variable for summation, which is used to traverse all features, from 1 to the total number of features M. It means summing up the importance scores of all features to get the sum of the importance of all features.

[0022] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A sea sand traceability big data modeling method integrating multi-dimensional test data, characterized in that: The following steps are involved: Step 1: Collect multi-source data from marine geological surveys and sea sand resource surveys over the years, integrate 218 types of original test data, and store them in a unified database; Step 2: Based on the differences in data from different sources, clean and filter the test elements common to each region to generate a standardized data set; Step 3: Based on the standardized dataset, match the sample coordinates according to the 11 sea area labels to generate a multidimensional dataset with geo-tags; Step 4: Z-score standardization method is used to deal with numerical differences, filter out missing indicator samples, and use SMOTE technology to balance the samples of 11 geographical regions and synthesize data from a few sea areas to form a comprehensive dataset with balanced category distribution; Step 5: After Z-score normalization, stratified sampling was performed with an 8:2 ratio to establish training and test sets. Random forest, support vector machine, and deep neural network models were constructed in parallel to extract complementary features. In step 6, the optimal model is selected based on weighted F1, confusion matrix, and feature importance, and the final traceability model is formed through continuous iteration.

2. The sea sand traceability big data modeling method integrating multi-dimensional test data according to claim 1 is characterized by: The step 1 comprises: Collect multi-source data from marine geological surveys and sea sand resource surveys over the years, covering different survey projects, regions and periods, and conduct preliminary screening of the collected data. At the same time, record the data source and collection time; The processed multi-source data was analyzed, and 218 types of original test data were classified and integrated according to data types and characteristics. A standardized database framework was established, and the integrated data was stored in the database according to the preset database structure and rules. At the same time, a unique identifier was set for each data item, and a data backup mechanism was established to ensure data security.

3. The sea sand traceability big data modeling method integrating multi-dimensional test data according to claim 1 is characterized by: The step 2 includes: To address the differences in test elements, parameter definitions, and measurement methods among multi-source data, we conducted data structure analysis and standardization. By comparing metadata documents, variable naming conventions, and analysis method descriptions of different data sets, we identified the core test elements that recur in the data from each region. We then screened out the key elements shared by all regions and established a unified data subset framework. For the common element data after screening, missing values ​​and outliers are detected. Among them, the missing value processing adopts a hierarchical strategy: if the missing rate of a variable in the global data exceeds the preset threshold, it is marked as a high-missing variable and evaluated for removal; if it is locally missing, the mean / median of spatially or temporally adjacent samples is preferentially used for interpolation. Outlier detection combines statistical methods with domain knowledge thresholds to review values ​​outside the preset range. After the error is confirmed, it is replaced with a nearby valid value or treated as missing; The processing effect is verified through multi-dimensional quality checks, including completeness, accuracy and consistency. For the verified data, the units, coordinate systems and naming conventions are unified, standardized data sets are generated, processing logs are recorded, and ultimately regional marine geological data resources are formed.

4. The sea sand traceability big data modeling method integrating multi-dimensional test data according to claim 1 is characterized by: The step 3 includes: Load the standardized dataset, extract the geographic coordinate information of each sample, standardize the coordinates, and supplement the missing coordinate information to generate a coordinate table containing the sample ID, latitude and longitude; Based on the international maritime boundary agreement, the geographical scope of 11 sea areas is clarified. The boundaries of each sea area are described using polygonal vector files. The boundary coordinate range of each sea area is defined. The sample coordinates are matched with the sea area boundaries. The samples are assigned to the corresponding sea area labels according to the coordinate position. For samples located near the boundaries, the nearest neighbor algorithm is used to determine the sea area to which they belong, and then a mapping relationship table between sea area labels and boundaries is established. The samples with matched sea area labels are integrated with the test data in the standardized dataset to form a multidimensional dataset with geographical labels. The integrity and consistency of the dataset are checked to ensure that the test data of each sample accurately corresponds to the sea area label.

5. The sea sand source tracing big data modeling method integrating multi-dimensional test data according to claim 4 is characterized by: For samples located near the boundary, the specific process of using the nearest neighbor algorithm to determine the sea area to which it belongs is as follows: Load a polygon vector file containing 11 sea area boundaries, extract the coordinates of the polygon vertices of each sea area boundary to form a boundary coordinate set, and load a coordinate table containing sample IDs, latitude and longitude to filter out samples located near the sea area boundary, that is, samples whose distance from the boundary is less than a preset proximity threshold; Define the Haversine formula to calculate the distance from the sample point to the sea area boundary, traverse the boundary polygon of each sea area, traverse all its vertices for each boundary polygon, calculate the distance from the sample point to each vertex, and retain the minimum distance and its corresponding sea area; For each sample point, compare its minimum distance to all sea area boundaries, determine the nearest sea area, assign the sample point to the nearest sea area, and record its sea area label; If the distances between multiple sea areas are the same, that is, the distances from the sample point to multiple sea areas are the same, the area weight strategy is adopted for allocation. The area weight selects the sea area with a larger area, and then adds the determined sea area label to the sample data to generate complete sample data with geographic labels.

6. The sea sand source tracing big data modeling method integrating multi-dimensional test data according to claim 5 is characterized by: The calculation formula for the distance from the sample point to the sea area boundary is as follows: ； ； ； Where d is the distance between two points, representing the distance from the sample point to the sea boundary, R is the radius of the earth, and are the latitudes of the two points, and is the longitude of two points, is the latitude difference, is the longitude difference; The calculation formula of the distance from the sample point to each vertex is as follows: ； Where, is the distance from the sample point to each vertex, indicating that each sample point The distance between the vertex of the boundary polygon of each sea area i, and are the longitude and latitude of the sample point, and are the longitude and latitude of the jth vertex in the i-th sea area.

7. The sea sand source tracing big data modeling method integrating multi-dimensional test data according to claim 1 is characterized by: The step 4 comprises: Perform Z-score normalization on all numerical indicators in the multidimensional dataset, calculate the mean and standard deviation of each indicator, and check the indicator integrity of each sample. If any key indicator is missing, the sample will be screened out; Based on the pre-assigned sea area labels, the number of samples in 11 geographical areas was counted, the proportion of each category was calculated, the imbalanced distribution of samples was identified, and the category differences were visually displayed to clearly identify the minority sea areas that needed to be synthesized. At the same time, the original data of the majority class samples was retained, and only the minority class samples were oversampled. For minority sea samples, the SMOTE algorithm is applied to generate synthetic data. For each minority sample, its k nearest neighbors are calculated, a neighboring sample is randomly selected, and new samples are generated by linear interpolation until the number of samples in each category reaches the preset balance threshold. Finally, the original majority class samples and the synthesized minority class samples are integrated to form a comprehensive data set with balanced category distribution.

8. The sea sand source tracing big data modeling method integrating multi-dimensional test data according to claim 1 is characterized by: The step 5 comprises: Perform stratified sampling on the Z-score normalized multidimensional data set, dividing the training set and the test set into a ratio of 8:

2. After the division is completed, save the training set and the test set separately; Based on the divided training set, three classification models, random forest, support vector machine and deep neural network, are constructed in parallel. Among them, random forest captures nonlinear feature interactions by integrating multiple decision trees; support vector machine uses kernel function to deal with nonlinear problems and optimizes classification boundaries; deep neural network extracts complex features through multi-layer neurons and learns the deep structure of data; each model is trained independently to extract complementary features. Among them, random forest outputs feature importance ranking, support vector machine provides key support vectors, and deep neural network extracts hidden layer features. The extracted complementary features are integrated to form a comprehensive feature set.

9. The method for sea sand traceability big data modeling based on multi-dimensional test data according to claim 8, characterized in that: The step 6 comprises: Independently evaluate the performance of random forests, support vector machines, and deep neural networks on the test set, calculate the weighted F1 score of each model, analyze the misclassification patterns of the models for different sea area labels using the confusion matrix, identify the strengths and weaknesses of the models, and prioritize eliminating baseline solutions with weighted F1 scores significantly lower than other models, retaining the 2-3 candidate models with the best overall performance; Extract complementary features from candidate models, including feature importance ranking from random forests, key support vectors from support vector machines, and hidden layer features from deep neural networks. Construct a weighted integrated feature set based on feature importance scores. Use a feature selection algorithm to reduce dimensionality and remove redundant information to obtain an optimized weighted integrated feature set. Then, retrain the model using this optimized weighted integrated feature set. Compare the weighted F1 scores and confusion matrices of the models before and after fusion to verify the effectiveness of feature integration in improving classification boundaries and generalization capabilities. The performance of the optimized model is evaluated again on the test set. Combined with the weighted F1, the category-level performance of the confusion matrix and the stability of feature importance, the model with the strongest generalization ability is selected as the final traceability model. If the model still has classification bias in specific sea areas, it can be further optimized by adjusting the category weights or adding corresponding samples.

Images