Method and device for evaluating signal strength prediction model of maritime wireless communication

Abstract

The present application provides a method and device for evaluating a signal strength prediction model for maritime wireless communication. The method provided by the present application includes: determining a set of machine learning models to be evaluated and a set of linear models to be evaluated, obtaining a plurality of models to be evaluated, and the output of the plurality of models to be evaluated is a predicted evaluation value of signal strength; determining, from a first initial feature set, the top n sensitive features with the highest correlation with the output of the model to be evaluated according to the parameters of each model to be evaluated, and the sensitive features include features of different objects associated with ship communication; measuring a parameter set of wireless communication in a target sea area, obtaining measured values corresponding to the sensitive features according to the parameter set, and constructing an evaluation data set; and using the evaluation data set to evaluate each model to be evaluated to obtain an evaluation result. The method and device for evaluating a signal strength prediction model for maritime wireless communication provided by the present application can solve the problems of inconsistent evaluation indicators and inability to intuitively compare the performance of prediction models.

Description

Technical Field

[0001] This application relates to the technical field of marine channel measurement, and particularly to a method and device for evaluating the signal strength prediction model of marine wireless communication. Background Art

[0002] Modeling of the marine wireless channel is very important in promoting the marine communication system and is an important supporting technology for the process of ocean digitization. With the development of technology, the marine wireless channel modeling technology has evolved from traditional modeling methods to artificial intelligence-based modeling methods.

[0003] Due to the different modeling methods, different evaluation metrics are used when evaluating different marine signal prediction models. The inconsistency of evaluation metrics makes it difficult to intuitively compare and analyze the prediction capabilities of different marine signal prediction models and difficult to determine which model's prediction results are the most accurate and reliable in a specific scenario. Summary of the Invention

[0004] In view of this, this application provides a method and device for evaluating the signal strength prediction model of marine wireless communication to solve the problem of inconsistent evaluation metrics and the inability to intuitively compare the performance of prediction models.

[0005] Specifically, this application is implemented through the following technical solutions:

[0006] In the first aspect of this application, a method for evaluating the signal strength prediction model of marine wireless communication is provided. The method includes:

[0007] Determine a set of machine learning models to be evaluated and a set of linear models to be evaluated, and obtain a plurality of models to be evaluated. The outputs of the plurality of models to be evaluated are prediction evaluation values of signal strength;

[0008] Determine the top n sensitive features with the highest correlation with the output of the model to be evaluated from the first initial feature set according to the parameters of each model to be evaluated. The sensitive features include features of different objects associated with ship communication;

[0009] Measure the parameter set of wireless communication in the target sea area, obtain the measured values corresponding to the sensitive features according to the parameter set, and construct an evaluation data set;

[0010] Use the evaluation data set to evaluate each model to be evaluated and obtain an evaluation result.

[0011] In the second aspect of this application, a device for evaluating the signal strength prediction model of marine wireless communication is provided. The device includes a determination module, a construction module, and an evaluation module; where

[0012] The determining module is configured to determine a machine learning model set to be evaluated and a linear model set to be evaluated, obtain a plurality of models to be evaluated, and the outputs of the plurality of models to be evaluated are prediction evaluation values of signal strength.

[0013] The determining module is further configured to determine, from a first initial feature set, the top n sensitive features with the highest correlation degree with the output of the model to be evaluated according to the parameters of each model to be evaluated, where the sensitive features include features of different objects associated with ship communication.

[0014] The constructing module is configured to measure a parameter set of wireless communication in a target sea area, obtain measurement values corresponding to the sensitive features according to the parameter set, and construct an evaluation data set.

[0015] The evaluating module is configured to evaluate each model to be evaluated by using the evaluation data set to obtain an evaluation result.

[0016] The signal strength prediction model evaluation method and device for maritime wireless communication provided by the present application can, through a set of standardized metrics, evaluate different types of models, such as linear and machine learning models, on the same scale and according to the same standard. Thus, the same set of metrics is used to comprehensively evaluate the performance of different types of models, more comprehensively evaluate the performance of different models in specific tasks, and help find the most suitable model type for solving the signal strength prediction of maritime wireless communication. Second, measuring the parameter set of wireless communication in the target sea area and obtaining the measurement values corresponding to the sensitive features to construct the evaluation data set can ensure that the evaluation is based on real actual data. The evaluation results obtained in this way are more reliable and practical, can reflect the performance of the model in the actual application scenario, and since the evaluation data set is constructed based on sensitive features, it can more specifically evaluate the model's ability to process important features associated with ship communication. This helps ensure the effectiveness of the model in the actual ship communication scenario. Third, using a unified evaluation data set to evaluate multiple models to be evaluated can objectively compare the performance of different models, clarify the advantages and disadvantages of each model, and provide a basis for selecting the best model. Description of the Drawings

[0017] Figure 1 It is a flowchart of the first embodiment of the signal strength prediction model evaluation method for maritime wireless communication provided by the present application;

[0018] Figure 2 It is a flowchart of the second embodiment of the signal strength prediction model evaluation method for maritime wireless communication provided by the present application;

[0019] Figure 3 It is a flowchart of the third embodiment of the signal strength prediction model evaluation method for maritime wireless communication provided by the present application;

[0020] Figure 4 FIG. 1 is a schematic structural diagram of the first embodiment of the signal strength prediction model evaluation device for maritime wireless communication provided by the present application. Detailed implementation manners

[0021] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application.

[0022] The terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the present application are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0023] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0024] Specific embodiments are given below to introduce the technical solutions of the present application in detail.

[0025] Figure 1 FIG. 2 is a flowchart of the first embodiment of the signal strength prediction model evaluation method for maritime wireless communication provided by the present application. Please refer to Figure 1 , the method provided in this embodiment may include:

[0026] S101. Determine a set of machine learning models to be evaluated and a set of linear models to be evaluated, and obtain a plurality of models to be evaluated, where the outputs of the plurality of models to be evaluated are prediction evaluation values of signal strength.

[0027] Specifically, the method for evaluating the signal strength prediction model of maritime wireless communication provided in this embodiment is applied to the ground-to-ship scenario. The signal strength prediction model of maritime wireless communication is usually used in aspects such as ship communication system planning and optimization, navigation safety guarantee, maritime operation management, and marine scientific research. It can predict the communication signal strength at sea and is crucial for promoting the establishment of a maritime communication system. Currently, there are methods for constructing prediction models based on various learning methods. Since different methods and data are used in the construction of different signal strength prediction models of maritime wireless communication, the evaluation indicators of different signal strength prediction models of maritime wireless communication are inconsistent. Therefore, when selecting a signal strength prediction model of maritime wireless communication, it is impossible to intuitively compare the differences between various signal strength prediction models of maritime wireless communication.

[0028] Furthermore, the machine learning model set to be evaluated includes a decision tree regression model, a random forest regression model, an adaptive boosting regression model, a support vector regression model, a multi-layer perceptron regression model, a kernel ridge regression model, a Gaussian process regression model, and a K-nearest neighbor regression model. The linear model set to be evaluated includes a least absolute shrinkage and selection operator regression model, a linear regression model, a Bayesian ridge regression model, and an elastic net regression model. The inputs of the machine learning models and linear models are data related to the signal strength of maritime wireless communication, which can specifically include geographical location (such as geographical distance GeoDis, line-of-sight distance GeoLos, reflection distance GeoRef, etc.), environmental factors (atmospheric pressure, temperature, humidity, etc.), ship motion-related (ship speed ShipSpeed, three-dimensional ship angular acceleration ShipAccX, ShipAccY, ShipAccZ, three-dimensional ship angular velocity ShipAngVelX, ShipAngVelY, ShipAngVelZ, three-dimensional tilt angle ShipAngX, ShipAngY, ShipAngZ), and other aspects of data, and the output is the predicted evaluation value of the maritime signal strength.

[0029] It can be understood that the models in the machine learning model set to be evaluated and the linear model set to be evaluated are models to be evaluated. The number of models to be evaluated is determined according to actual needs and is not limited in this embodiment. For example, in a possible implementation, all the models in the machine learning model set to be evaluated and the linear model set to be evaluated are determined as the models to be evaluated; for another example, in another possible implementation, some models are selected from the machine learning model set to be evaluated and the linear model set to be evaluated as the models to be evaluated. Specifically, when implementing, the models can be screened based on the model complexity. When the computing resources are limited or the real-time requirement for the model is high, models with lower complexity can be preferentially selected. For example, in a maritime communication scenario, some embedded devices have limited computing power and cannot run complex models. At this time, simple linear regression models or K-nearest neighbor regression models may be more suitable as the models to be evaluated because of their low computational complexity and fast running speed; the models can also be screened based on the data characteristics obtained currently. For example, when the data volume is small, models with low requirements for the data volume, such as kernel ridge regression models, can be selected, which can also fit and predict well in the case of small samples; if there are complex non-linear relationships between the data characteristics, machine learning models that can capture non-linear relationships, such as multi-layer perceptron regression models, should be considered first. Selecting models based on data characteristics in this way can improve the adaptability of the model to the data and make the evaluation results more valuable for reference.

[0030] Further, multiple models to be evaluated are used to predict the signal strength of maritime wireless communication. The inputs of the multiple models to be evaluated are data related to the signal strength of maritime wireless communication, such as geographical location, environmental factors, time, etc., and the outputs of the multiple models to be evaluated are predicted evaluation values of the signal strength. It should be noted that in order to ensure that the processing capabilities of different models for the same influencing factors can be evaluated under a unified standard, when evaluating multiple models to be evaluated, the input content of the multiple models to be evaluated is the same.

[0031] S102. Determine the sensitive features ranked top n in terms of the association degree with the output of the model to be evaluated from the first initial feature set according to the parameters of each model to be evaluated. The sensitive features include the features of different objects associated with ship communication.

[0032] Specifically, the first initial feature set at least includes geographical feature types, ship feature types, and environmental feature types. The specific steps for determining the first initial feature set include:

[0033] (1) Determine multiple associated objects of maritime wireless communication.

[0034] Specifically, maritime wireless communication involves multiple related objects, mainly including the communication environment, shore-based base stations, ship-based base stations, etc. The communication environment encompasses factors such as atmospheric pressure, temperature, and humidity, which affect the propagation characteristics of signals in the air. As the signal transmitter, the transmitting power, antenna height, and position of the shore-based base station have an important impact on the signal strength. The ship-based base station is the signal receiver, and its receiving sensitivity, antenna direction, and height also affect the signal reception effect.

[0035] (2) Determine the key features of each related object under each type.

[0036] Specifically, the key features refer to the features of the related object itself that affect the signal strength. For example, for the communication environment, temperature has a significant impact on the signal strength. In a high-temperature environment, the thermal motion of air molecules intensifies, and the scattering and absorption during signal propagation increase, resulting in signal attenuation. In a high-humidity environment, water vapor will scatter and absorb the signal, affecting signal propagation. Atmospheric pressure also affects signal propagation. In a low-pressure area, the air density is small, and the signal propagation loss is relatively small, but it will change the signal propagation speed, affecting signal stability.

[0037] For the shore-based base station, its transmitting power is an important feature. The higher the transmitting power, the farther the signal can propagate and the relatively greater the strength. For the ship-based base station, the receiving sensitivity reflects the ability of the ship-based base station to receive weak signals. A base station with high receiving sensitivity can receive weaker signals, ensuring signal strength. At the same time, the height of the ship-based base station is also crucial. Increasing the height can reduce the signal occlusion by surrounding objects and improve the signal reception strength.

[0038] (3) Determine the collaborative key features among the objects under each type.

[0039] Specifically, the collaborative key features refer to the features between two associated objects that affect the signal strength. For example, for the communication environment and the shore base station, the temperature and the transmitting power of the shore base station interact to affect the signal strength. In a high-temperature environment, if the transmitting power of the shore base station remains unchanged, the signal attenuation will increase. At this time, it is necessary to appropriately increase the transmitting power to ensure the signal strength. There is also a collaborative relationship between atmospheric pressure and antenna height. In a low-pressure environment, appropriately increasing the antenna height can reduce the loss of the signal during propagation and improve the signal strength. For the communication environment and the ship base station, humidity and the receiving sensitivity of the ship base station jointly affect signal reception. In a high-humidity environment, the receiving sensitivity of the ship base station will decrease. At this time, it is necessary to adjust the receiving sensitivity parameters to ensure stable signal reception. The change of the communication environment will also affect the adjustment of the antenna direction of the ship base station. For example, in strong wind weather, the ship shakes, and it is necessary to adjust the antenna direction in real time according to the wind direction and the change of the signal strength to ensure the signal reception effect. For the shore base station and the ship base station, the transmitting power of the shore base station and the receiving sensitivity of the ship base station need to be matched. If the transmitting power of the shore base station is too high and the receiving sensitivity of the ship base station is limited, it may lead to signal saturation and affect the signal quality. On the contrary, if the transmitting power is too low, even if the receiving sensitivity is high, it is impossible to receive a signal with sufficient strength. The relative position and antenna direction of the shore base station and the ship base station are also closely related. Adjusting the antenna direction according to their relative position can effectively improve the signal transmission efficiency and strength.

[0040] Specifically, the parameters of each model to be evaluated may include position, time, wind speed, temperature, ship speed, RSRP (Reference Signal Received Power), RSSI (Received Signal Strength Indication), etc. By cleaning and sorting the parameters of the model to be evaluated, the accuracy and integrity of the parameters are ensured, and feature extraction is performed on the processed parameters to obtain all the features related to the parameters of the model to be evaluated. For the detailed process of feature extraction, please refer to the description in the related technology and will not be elaborated here.

[0041] Further, the specific implementation steps for determining the top n sensitive features with the highest correlation degree with the output of the model to be evaluated from the first initial feature set according to the parameters of each model to be evaluated include:

[0042] (1) Screening the features in the first initial feature set based on multiple screening methods to obtain a screening result.

[0043] Specifically, the number of screening methods is set according to actual needs and is not limited in this embodiment. For example, in one embodiment, three screening methods are used to screen the features of the model to be evaluated. These three screening methods are the Pearson correlation coefficient, the P-value of the feature, and the feature score, and these three methods are used to screen the features simultaneously.

[0044] Further, in combination with the above description, first, the features are screened according to the Pearson correlation coefficient of the features. The Pearson correlation coefficient is used to measure the linear correlation degree between the feature and the signal strength, and its value range is between -1 and 1. If the Pearson correlation coefficient of the feature meets the first threshold, the feature is retained; if the Pearson correlation coefficient of the feature does not meet the first threshold, the feature is deleted. It should be noted that the first threshold is set according to actual needs and is not limited in this embodiment.

[0045] Similarly, the features are screened using the P-value of the features. The P-value is used to judge whether the null hypothesis in the statistical hypothesis test is rejected. In this embodiment, the null hypothesis is "the feature has nothing to do with the signal strength". If the P-value of the feature is less than the second threshold, it means that the feature is related to the signal strength and the feature is retained; if the P-value of the feature is greater than the second threshold, it means that the feature has nothing to do with the signal strength and the feature is deleted. It should be noted that the second threshold is set according to actual needs and is not limited in this embodiment.

[0046] Similarly, the features are screened using the feature score. The feature score is a measure of the importance of the feature. If the feature score of the feature is greater than the third threshold, the feature is retained; if the feature score of the feature is less than the third threshold, the feature is deleted. It should be noted that the third threshold is set according to actual needs and is not limited in this embodiment. And the first threshold, the second threshold, and the third threshold are all different.

[0047] It should be noted that when multiple screening methods are used to screen the features in the first initial feature set, each screening method screens the features in the first initial feature set simultaneously. In this way, the consistency of the screening process can be maintained, treating all models equally, not being affected by the differences in model types, being able to comprehensively consider the relationship between each feature and the signal strength, and screening out the features with a relatively high correlation with the outputs of all models as a whole from the overall perspective, providing a unified and comprehensive feature set for subsequent evaluation and facilitating fair comparison between different models.

[0048] Further, in combination with the above description, the union of the features screened by the three methods of the Pearson correlation coefficient, the P-value of the feature, and the feature score is taken, and this feature union is used as the screening result.

[0049] (2)Sort the screening results according to the sensitivity of the multiple screening methods.

[0050] Specifically, for each screening method, calculate its sensitivity respectively, and based on the sensitivity of each screening method obtained by calculation, use the weighted average method to synthesize the sensitivity of each screening method to obtain the comprehensive sensitivity. Further, based on the order from large to small of the comprehensive sensitivity, sort the features in the screening results to obtain the sorting result. The process of calculating the sensitivity of the screening method can refer to the description in the related technology and will not be elaborated here.

[0051] (3)Determine the top n features in the sorting result as sensitive features.

[0052] Specifically, the value of n is set according to actual needs and is not limited in this embodiment. For example, in a possible implementation, the first initial feature set includes 19 features such as geographical distance, line-of-sight distance, reflection distance, phase difference between line-of-sight and reflection, angle between the ship and the base station in the horizontal direction, angle between the ship and the base station in the vertical direction, ship speed, three-dimensional ship angular acceleration (ShipAccX, ShipAccY, ShipAccZ), three-dimensional ship angular velocity (ShipAngVelX, ShipAngVelY, ShipAngVelZ), three-dimensional tilt angle (ShipAngX, ShipAngY, ShipAngZ), atmospheric pressure, temperature and humidity of the environment, so as to capture the influence of various factors that may affect the channel characteristics, such as line-of-sight, reflection, scattering, diffraction, refraction, etc. For example, the distance and angle between the ship and the base station reflect the line-of-sight condition and path loss; the ship speed and angular acceleration indicate the Doppler shift and small-scale fading; the tilt angle and weather information imply the refraction and attenuation of the signal.

[0053] After screening by multiple screening methods, remove the features with weak correlation and retain the key features with high correlation with the signal strength prediction. In the actual screening process, when screening features by methods such as Pearson correlation coefficient, P value of the feature, and feature score, finally retain 10 features that play an important role in prediction, namely geographical distance, line-of-sight distance, reflection distance, angle between the ship and the base station in the horizontal direction, angle between the ship and the base station in the vertical direction, ship speed, three-dimensional tilt angle (ShipAngZ), atmospheric pressure, temperature and humidity of the environment. This screening change can make the subsequent model training and evaluation focus on more critical factors and improve the accuracy and efficiency of model prediction.

[0054] S103. Measure the parameter set of the wireless communication in the target sea area, obtain the measured values corresponding to the sensitive features according to the parameter set, and construct an evaluation data set.

[0055] Specifically, the specific implementation process of this step includes:

[0056] (1) Determine the measurement parameters and sensitive features, and construct the parameter set according to the measurement parameters.

[0057] Specifically, the sensitive features are features that have a significant impact, a high degree of correlation, or an important role on the model prediction results. The sensitive features may include: geographical distance GeoDis, line-of-sight distance GeoLos, reflection distance GeoRef, phase difference GeoPhaseDiff between line-of-sight and reflection, angles GeoThetaX and GeoTheteY between the ship and the base station in the horizontal and vertical directions, ship speed (ShipSpeed), three-dimensional ship angular acceleration (ShipAccX, ShipAccY, ShipAccZ), three-dimensional ship angular velocity (ShipAngVelX, ShipAngVelY, ShipAngVelZ), three-dimensional tilt angles (ShipAngX, ShipAngY, ShipAngZ), atmospheric pressure, temperature, and humidity of the surrounding environment, etc.

[0058] Specifically, in combination with the above description, after determining the measurement parameters, a maritime 5G terminal operating in the 5G band is used for parameter measurement. This terminal is designed specifically for maritime applications and is equipped with a four-element antenna array. This antenna array can provide an omnidirectional radiation pattern and an antenna gain of up to 6.35 dBi. A standard 5G module is integrated into the terminal, and the transmit power is 23 dBm. Further, the terminal is arranged on a fishery administration ship to receive the reference signal power transmitted by the 5G base station along the coastline. The total height of the ship is about 9 meters, and the 5G terminal is placed at a position about 7 meters above the sea surface. The 5G network provided by China Mobile operates in the frequency bands of 2.496 GHz - 2.690 GHz and 4.4 GHz - 5.0 GHz. The ship's traveling speed is up to 14.40 kilometers per hour, and the total voyage is about 53.74 kilometers. During the measurement process, the terminal automatically obtains data including RSRP and RSRQ (Reference Signal Received Quality) from the 5G module. These are key indicators of the 5G mobile network signal level and quality. At the same time, 9-axis motion sensor data and the ship's positioning data are also collected.

[0059] Further, RSSI can be calculated according to the following formula:

[0060] ;

[0061] Wherein, N is the maximum number of resource blocks;

[0062] is the measured value of the reference signal received power;

[0063] is the measured value of the reference signal received quality.

[0064] Furthermore, preprocess the measured parameters to solve problems such as data format mismatch, data anomalies, and inconsistent data granularity in the original measurement data, and eliminate interference features with low contribution to model prediction. Determine the preprocessed measurement parameters as the parameter set.

[0065] As an optional embodiment, the preprocessing of the measured parameters includes: determining the RSRQ value of each measurement parameter, screening the parameters to be preprocessed based on the RSRQ value. Among them, for parameters with poor reception quality, due to poor reception quality, data is prone to anomalies such as reception interruptions and data loss. On this basis, select parameters with a lower RSRQ value as the parameters to be preprocessed. The RSRQ value can be compared with the preprocessing quality threshold to determine the parameters with a lower RSRQ value. Determine the reference parameter corresponding to the parameter to be preprocessed. Among the measured parameters, determine the reference parameter from the parameters other than the parameter to be preprocessed. The RSRQ of the reference parameter needs to be greater than the preprocessing quality threshold, and at the same time, it has the same time as the corresponding parameter to be preprocessed, adjacent or the same transceiver device, and the same communication topic. Perform anomaly marking on the parameter to be preprocessed according to the reference parameter. The anomaly marking includes data loss marking, data error marking, data format anomaly marking, etc. The marked position is the specific position where the anomaly occurs in the data, such as the nth digit of the data. Perform anomaly processing on the parameter to be preprocessed based on the anomaly marking. The anomaly processing is to process the content marked by the anomaly marking. For data loss and data error, use the mean value of the adjacent data of the anomaly marking for substitution. For data format anomaly, change the parameter at this position to the target format.

[0066] Take the data after the first preprocessing as the target data. Establish a relationship graph between each target data according to the data reception time, sending device, and sending topic. In the relationship graph, the target data is the data node, and the association relationship between the data reception time, sending device, and sending topic is the connection edge. Establish the matching relationship of the target data based on the relationship graph. Determine the abnormal data in the target data according to the matching relationship. Specifically, take the two groups of data with the highest correlation as the matching data. Identify the change trend of the sending topic according to the data values between the two matching data. Take the matching data with conflicting change trends between the two groups of data as the abnormal data. Correct the abnormal data based on the relationship graph. Specifically, use the associated data with the second highest correlation in the relationship graph, add it to the matching data, form three data pairs, and use the two groups of data with the same change trend as the reference objects to correct the third group of data.

[0067] The method provided by the present invention first corrects the fine-grained errors inside the data based on preprocessing, that is, the first correction is for several bits in a piece of data. After the correction is completed, on the basis of ensuring that the data itself is correct, the correction is performed based on the correlation relationship between the data, that is, the change trend between the data is corrected. The smallest unit of the data corrected the second time is a piece of data itself. Thus, through data corrections at two different levels, the accuracy of the data itself and the change trend between the data is ensured, the accuracy of the received data is improved, and further the interference features with low contribution to the model prediction are eliminated.

[0068] (2)According to the determined sensitive feature, search for the measured value corresponding to the sensitive feature from the parameter set.

[0069] (3)Sort out the measured values and construct the evaluation data set according to the measured values.

[0070] Specifically, in combination with the above description, directly search for the measured value corresponding to the sensitive feature in the parameter set, and sort and classify the measured values, thereby constructing the evaluation data set.

[0071] S104. Use the evaluation data set to evaluate each of the models to be evaluated, and obtain the evaluation results.

[0072] Specifically, the evaluation data set is a set of data used to measure the performance of the models to be evaluated. By using the evaluation data set to evaluate the above-mentioned multiple models to be evaluated, the above-mentioned multiple models to be evaluated can be quantitatively described to select the best model from them. For the detailed process of using the evaluation data set to evaluate each of the models to be evaluated, please refer to the description in the related technology and will not be elaborated here.

[0073] For the signal strength prediction model evaluation method of maritime wireless communication provided in this embodiment, on the one hand, by determining the set of machine learning models to be evaluated and the set of linear models to be evaluated, various different types of models can be covered, so as to more comprehensively evaluate the performance of different models in a specific task, which helps to find the most suitable model type for solving the signal strength prediction of maritime wireless communication. On the other hand, measuring the parameter set of wireless communication in the target sea area and obtaining the measured values corresponding to the sensitive features to construct the evaluation data set can ensure that the evaluation is based on real actual data. In this way, the obtained evaluation results are more reliable and practical, can reflect the performance of the model in the actual application scenario, and since the evaluation data set is constructed based on the sensitive features, it can more specifically evaluate the processing ability of the model for the important features associated with ship communication. This helps to ensure the effectiveness of the model in the actual ship communication scenario. On the third hand, using the unified evaluation data set to evaluate multiple models to be evaluated can objectively compare the performance of different models, can clarify the advantages and disadvantages of each model, and provide a basis for selecting the best model.

[0074] Figure 2 This is the flowchart of the second embodiment of the method for evaluating the signal strength prediction model of maritime wireless communication provided by this application. Please refer to Figure 2 , based on the above embodiment, the step of using the evaluation dataset to evaluate each of the models to be evaluated and obtaining an evaluation result includes:

[0075] S201. Evaluate the prediction performance of each of the models to be evaluated according to the first evaluation index set, and obtain a first evaluation result. The values corresponding to the first evaluation index set of the models in the model set to be evaluated for linear models are inferior to those of the models in the model set to be evaluated for machine learning models.

[0076] Specifically, the first evaluation index set includes multiple evaluation indexes, which are used for the preliminary evaluation of the prediction performance of each model to be evaluated.

[0077] Optionally, in a possible implementation manner, the first evaluation index set at least includes the following first evaluation indexes: mean absolute error, mean square error, root mean square error, mean absolute percentage error, median absolute error, maximum error, coefficient of determination, explained variance, and running time.

[0078] Further, when using the first evaluation index set to evaluate each model to be evaluated, the evaluation dataset is used as input and input into the model to be evaluated. The model to be evaluated outputs a prediction result, and then the prediction result is compared with the true label or target value in the evaluation dataset, and the values of each evaluation index are calculated according to the comparison result.

[0079] Further, the values corresponding to the first evaluation index set of the models in the model set to be evaluated for linear models are inferior to those of the models in the model set to be evaluated for machine learning models, indicating that in the current task and evaluation dataset, the machine learning model performs better in terms of prediction performance. This may be because the relationship between the signal strength in ship communication and various features is relatively complex, and the machine learning model can better capture these complex relationships, while the linear model is limited by its linear assumption and cannot fully fit the data.

[0080] In this step, the prediction performance of each model to be evaluated is evaluated according to the first evaluation index set. The indexes in this index set are quantitative values, which can quantitatively evaluate the difference between the model prediction value and the true value as a whole, and reflect the accuracy and precision of the model prediction. The mean absolute error, mean square error, and root mean square error can directly measure the average deviation degree, the mean square of the deviation, and the square root of the mean square of the deviation between the prediction value and the true value. The smaller the value, the closer the model prediction value is to the true value, and the higher the prediction accuracy. The coefficient of determination and the explained variance are used to evaluate the goodness of fit of the model and reflect the model's ability to explain the data. The closer the value is to 1, the better the model fits the data and the more variation in the data it can explain. The running time reflects the training efficiency of the model and is an important indicator to measure the practical application feasibility of the model.

[0081] S202. According to the first evaluation result, screen the target models to be evaluated from the models to be evaluated to form a target set of models to be evaluated.

[0082] The specific implementation steps include:

[0083] (1) Calculate the index scores of each model to be evaluated.

[0084] Specifically, each index in the first evaluation index set is used to evaluate the model to be evaluated, and the index scores corresponding to multiple indexes are obtained. The sum of the index scores corresponding to multiple indexes is the index score of the model to be evaluated in the first evaluation index set.

[0085] (2) Sort each model to be evaluated according to the index scores to obtain the first index sorting result.

[0086] Specifically, multiple models to be evaluated are sorted in descending order of index scores, and the sorting result is recorded as the first index sorting result.

[0087] (3) Determine the top K models to be evaluated in the first index sorting result as the target models to be evaluated, and the K target models to be evaluated form the target set of models to be evaluated.

[0088] In step S202, the prediction performance is evaluated according to the prediction result curve characteristics of each target model to be evaluated. The prediction result curve characteristics are mainly the RSRP line-of-sight distance curve and the RSSI line-of-sight distance curve. This evaluation method starts from an intuitive graphical perspective and evaluates the model by observing and analyzing the characteristics of the curve. By observing the degree of proximity between the curves of different models and the true data curve, the accuracy of the model can be judged; by analyzing the smoothness and stability of the curve, the reliability of the model can be evaluated; by studying whether the trend of the curve conforms to the actual situation, the rationality of the model can be measured. If the RSRP line-of-sight distance curve of a certain model highly coincides with the trend of the true data curve and the curve is smooth without violent fluctuations, it indicates that the model has high accuracy and good stability in predicting the change of RSRP with the line-of-sight distance. Different from the quantitative indicators in S201, the curve characteristic evaluation pays more attention to the performance of the model in the overall trend and change law, and can intuitively show the change of the model's prediction of the signal strength at different line-of-sight distances.

[0089] S203. Evaluate the prediction performance of each of the target models to be evaluated according to the prediction result curve characteristics of each of the target models to be evaluated, and obtain a second evaluation result.

[0090] Optionally, in a possible implementation manner, the prediction result curve characteristics are: the RSRP line-of-sight distance curve and the RSSI line-of-sight distance curve.

[0091] Specifically, further evaluate each target model to be evaluated through the prediction result curve characteristics. The prediction result curve characteristics can provide more information about the model performance and can capture the long-term trend of the model more deeply. By observing and analyzing the prediction result curve characteristics of each target model to be evaluated, the prediction performance of the model can be evaluated from different angles. For example, in one embodiment, the proximity between the curves of different models and the true data curve can be compared to judge the accuracy of the model; the smoothness and stability of the curve can be observed to evaluate the reliability of the model; the trend of the curve can be analyzed to see if it conforms to the actual situation to measure the rationality of the model.

[0092] Furthermore, the second evaluation result is a conclusion drawn based on the evaluation of the prediction result curve characteristics of the target model to be evaluated. It can complement the first evaluation result and provide a more comprehensive basis for finally selecting the most suitable model.

[0093] S204. Determine the optimal signal strength prediction model according to the second evaluation result.

[0094] In specific implementation, the model to be evaluated with the highest prediction performance evaluation score in the second evaluation result is determined as the optimal signal strength prediction model.

[0095] The method provided in this embodiment evaluates the model to be evaluated through the first evaluation index set and the characteristics of the prediction result curve, considering the prediction performance of the model to be evaluated from different perspectives. The first evaluation index set usually covers quantitative indexes such as accuracy and precision, while the curve feature evaluation observes the trends, stability, etc. of the model from an intuitive graphical perspective. This multi-dimensional evaluation can comprehensively understand the performance of the model and avoid the one-sidedness that may be brought by a single evaluation method. In addition, first evaluating all the models to be evaluated using the first evaluation index set can quickly screen out a set of models with relatively good performance. Since the values of the models in the set of linear models to be evaluated corresponding to the first evaluation index set are inferior to those of the models in the set of machine learning models to be evaluated, this step can initially exclude some poorly performing linear models and reduce the workload of subsequent evaluations. After screening out the target set of models to be evaluated from the models to be evaluated, then evaluate according to the characteristics of the prediction result curves of each target model to be evaluated. This can concentrate the evaluation resources on more potential models and improve the evaluation efficiency. At the same time, the further evaluation of the target set of models to be evaluated can be more in-depth and detailed, which helps to discover those models that perform well in the first evaluation but still have some minor problems.

[0096] Figure 3 This is the flowchart of the third embodiment of the signal strength prediction model evaluation method for maritime wireless communication provided by this application. Please refer to Figure 3 , on the basis of the above embodiment, evaluating the prediction performance of each of the target models to be evaluated according to the characteristics of the prediction result curves of each of the target models to be evaluated, and obtaining a second evaluation result; including:

[0097] S301. Calculate the predicted sample coverage area and the true sample coverage area corresponding to the characteristics of the prediction result curves of each of the target models to be evaluated.

[0098] Specifically, taking the line-of-sight distance as the abscissa and the characteristics of the prediction result curve as the ordinate, taking (line-of-sight distance, prediction result) as a set of data, and inputting it into the line-of-sight distance - prediction result coordinate system, the characteristics of the prediction result curves of each target model to be evaluated are obtained.

[0099] Furthermore, establish a blank image, determine the number of pixels in the image, draw the predicted samples into the blank image, traverse each pixel point in the image, check whether the pixel point is occupied by the predicted sample points, and count the number of pixels of the predicted samples in the blank image, which is the predicted sample coverage area.

[0100] Similarly, when calculating the true sample coverage area, draw the true samples into the blank image, traverse each pixel point in the image, check whether the pixel point is occupied by the true sample points, and count the number of pixels of the true samples in the blank image, which is the true sample coverage area.

[0101] S302. Screen the target model set to be evaluated according to the predicted sample coverage area to obtain a first target model set to be evaluated.

[0102] Optionally, the predicted sample coverage area of each target model to be evaluated in the first target model set to be evaluated is greater than the true sample coverage area.

[0103] Specifically, retain the models in the first target model set to be evaluated whose predicted sample coverage area is greater than the true sample coverage area as the first target model set to be evaluated.

[0104] S303. For the models in the first target model set to be evaluated, construct a prediction result regression coordinate with the true sample as the abscissa and the predicted sample as the ordinate.

[0105] Specifically, in order to further select the signal strength prediction model with the best performance, construct a prediction result regression coordinate with the true sample as the abscissa and the predicted sample as the ordinate. The purpose of constructing the prediction result regression coordinate is to intuitively analyze and evaluate the prediction performance of the models in the first target model set to be evaluated. By taking the true sample as the abscissa and the predicted sample as the ordinate, the relationship between the predicted value and the true value of the model can be observed.

[0106] S304. Calculate the regression degree of the prediction result of the target model to be evaluated according to the prediction result regression coordinate.

[0107] Specifically, in combination with the above description, it can be understood that in an ideal situation, the predicted sample output by the signal strength prediction model should be the same as the value of the true sample. Therefore, in the prediction result regression coordinate, the line where the points with the same predicted sample and true sample values are located is used as the regression line, and (true sample, predicted sample) is used as a point in the prediction result regression coordinate. By fitting the points in the prediction result regression coordinate through a linear regression model, it can be understood that the closer the prediction result regression degree is to 1, the better the prediction performance of the target model to be evaluated.

[0108] S305. Sort the target models to be evaluated according to the regression degree, and determine the sorting result as the second sorting result.

[0109] Specifically, sort the target models to be evaluated in descending order of the regression degree, and this sorting result is the second sorting result.

[0110] The method provided in this embodiment can intuitively understand the coverage of the model for the sample space by calculating the coverage area of the predicted samples and the coverage area of the true samples. If the difference between the coverage area of the predicted samples and the coverage area of the true samples of a model is too large, it indicates that the model may have large deviations or instabilities. These less reliable models can be excluded through screening, thereby improving the accuracy of subsequent evaluations. Further, by constructing a prediction result regression coordinate system with the true samples as the abscissa and the predicted samples as the ordinate, the relationship between the prediction results of the model and the true values can be analyzed more comprehensively. The calculation of the regression degree can quantify the tightness of this relationship, making the evaluation of the model performance more accurate.

[0111] Corresponding to the foregoing embodiment of the method for evaluating the signal strength prediction model of maritime wireless communication, the present application also provides an embodiment of an apparatus for evaluating the signal strength prediction model of maritime wireless communication.

[0112] Figure 4 It is a schematic structural diagram of the first embodiment of the apparatus for evaluating the signal strength prediction model of maritime wireless communication provided by the present application. Please refer to Figure 4 The apparatus provided in this embodiment includes a determination module 410, a construction module 420, and an evaluation module 430; wherein,

[0113] The determination module 410 is configured to determine a set of machine learning models to be evaluated and a set of linear models to be evaluated, obtain a plurality of models to be evaluated, and the outputs of the plurality of models to be evaluated are predicted evaluation values of the signal strength;

[0114] The determination module 410 is further configured to determine, from the first initial feature set, the top n sensitive features with the highest correlation degree with the output of the model to be evaluated according to the parameters of each model to be evaluated, and the sensitive features include features of different objects associated with ship communication;

[0115] The construction module 420 is configured to measure a parameter set of wireless communication in the target sea area, obtain measured values corresponding to the sensitive features according to the parameter set, and construct an evaluation data set;

[0116] The evaluation module 430 is configured to evaluate each model to be evaluated by using the evaluation data set to obtain an evaluation result.

[0117] The apparatus in this embodiment can be used to execute Figure 1 the steps of the method embodiment shown, and the specific implementation principle and implementation process are similar, and will not be elaborated here.

[0118] For the specific implementation process of the functions and roles of each unit in the above apparatus, please refer to the implementation process of the corresponding steps in the above method, which will not be elaborated here.

[0119] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the descriptions of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this application. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0120] The above are only the preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A signal strength prediction model evaluation method for marine wireless communication, characterized in that: The method comprises: Determine a set of machine learning models to be evaluated and a set of linear models to be evaluated, and obtain a plurality of models to be evaluated, wherein outputs of the plurality of models to be evaluated are predicted evaluation values ​​of signal strength; Determining the top n sensitive features ranked by output relevance of the model to be evaluated from the first initial feature set according to the parameters of each model to be evaluated, wherein the sensitive features include features of different objects associated with ship communication; The first initial feature set includes at least a geographical feature type, a ship feature type, and an environmental feature type. The specific steps of determining the first initial feature set include: Identify multiple associated objects for maritime wireless communications; Determine the key characteristics of each associated object under each type; Determine the key features of collaboration between objects of each type; Wherein, determining the top n sensitive features in terms of output relevance to the model to be evaluated from the first initial feature set according to the parameters of each model to be evaluated; comprising: Screening the features in the first initial feature set based on multiple screening methods to obtain screening results; sorting the screening results according to the sensitivity of the multiple screening methods; The top n features in the sorting results are determined as sensitive features; Measuring a parameter set of wireless communication in the target sea area, obtaining a measurement value corresponding to the sensitive feature according to the parameter set, and constructing an evaluation data set; The parameter set of the wireless communication of the measurement target sea area, obtaining the measurement value corresponding to the sensitive feature according to the parameter set, and constructing an evaluation data set; includes: Determining measurement parameters and sensitive features, and constructing the parameter set according to the measurement parameters; According to the determined sensitive feature, searching for a measurement value corresponding to the sensitive feature from the parameter set; Arranging the measured values ​​and constructing the evaluation data set based on the measured values; The evaluation data set is used to evaluate each of the models to be evaluated to obtain an evaluation result.

2. The method according to claim 1, characterized in that The step of evaluating each of the models to be evaluated by using the evaluation data set to obtain an evaluation result comprises: Evaluate the prediction performance of each of the models to be evaluated according to the first evaluation indicator set to obtain a first evaluation result, wherein the value corresponding to the first evaluation indicator set of the model in the linear model set to be evaluated is inferior to the model in the machine learning model set to be evaluated; According to the first evaluation result, selecting a target set of models to be evaluated from the models to be evaluated; Evaluate the prediction performance of each target model to be evaluated according to the prediction result curve characteristics of each target model to be evaluated, and obtain a second evaluation result; An optimal signal strength prediction model is determined according to the second evaluation result.

3. The method according to claim 2, characterized in that The first evaluation indicator set includes at least the following first evaluation indicators: mean absolute error, mean square error, root mean square error, mean absolute percentage error, median absolute error, maximum error, determination coefficient, explained variance and running time.

4. The method according to claim 2, characterized in that: According to the first evaluation result, a target set of models to be evaluated is selected from the models to be evaluated; including: Calculating the index score of each model to be evaluated; Sorting each of the models to be evaluated according to the indicator score to obtain a first indicator ranking result; The top K models to be evaluated in the first indicator ranking result are determined as the target models to be evaluated, and the K target models to be evaluated form the target model set to be evaluated.

5. The method according to claim 2, characterized in that: The prediction result curve features include: RSRP line-of-sight distance curve and RSSI line-of-sight distance curve.

6. The method according to claim 2, characterized in that The step of evaluating the prediction performance of each target model to be evaluated according to the prediction result curve characteristics of each target model to be evaluated to obtain a second evaluation result comprises: Calculate the predicted sample coverage area and the real sample coverage area corresponding to the predicted result curve characteristics of each target model to be evaluated; The target model set to be evaluated is screened according to the predicted sample coverage area to obtain a first target model set to be evaluated; For the models in the first target to-be-evaluated model set, construct a regression coordinate of the prediction result with the real sample as the horizontal coordinate and the predicted sample as the vertical coordinate; Calculating the regression degree of the prediction result of the target model to be evaluated according to the prediction result regression coordinates; The target models to be evaluated are sorted according to the regression degree, and the sorting result is determined as the second sorting result.

7. The method according to claim 6, characterized in that The predicted sample coverage area of ​​each of the target models to be evaluated in the first target model set to be evaluated is greater than the true sample coverage area.

8. A signal strength prediction model evaluation device for marine wireless communication, characterized in that: The device comprises a determination module, a construction module and an evaluation module; wherein, The determination module is used to determine a set of machine learning models to be evaluated and a set of linear models to be evaluated, and obtain multiple models to be evaluated, wherein the outputs of the multiple models to be evaluated are predicted evaluation values ​​of signal strength; The determination module is further used to determine the top n sensitive features in terms of output relevance ranking with the model to be evaluated from the first initial feature set according to the parameters of each model to be evaluated, wherein the sensitive features include features of different objects associated with ship communication; The first initial feature set includes at least a geographical feature type, a ship feature type, and an environmental feature type. The specific steps of determining the first initial feature set include: Identify multiple associated objects for maritime wireless communications; Determine the key characteristics of each associated object under each type; Determine the key features of collaboration between objects of each type; Wherein, determining the top n sensitive features in terms of output relevance to the model to be evaluated from the first initial feature set according to the parameters of each model to be evaluated; comprising: Screening the features in the first initial feature set based on multiple screening methods to obtain screening results; sorting the screening results according to the sensitivity of the multiple screening methods; The top n features in the sorting results are determined as sensitive features; The construction module is used to measure the parameter set of wireless communication in the target sea area, obtain the measurement value corresponding to the sensitive feature according to the parameter set, and construct an evaluation data set; The parameter set of the wireless communication of the measurement target sea area, obtaining the measurement value corresponding to the sensitive feature according to the parameter set, and constructing an evaluation data set; includes: Determining measurement parameters and sensitive features, and constructing the parameter set according to the measurement parameters; According to the determined sensitive feature, searching for a measurement value corresponding to the sensitive feature from the parameter set; Arranging the measured values ​​and constructing the evaluation data set based on the measured values; The evaluation module is used to evaluate each of the models to be evaluated using the evaluation data set to obtain an evaluation result.

Images