Method and device for evaluating signal intensity prediction model of maritime wireless communication

Abstract

The invention provides a signal intensity prediction model evaluation method and device for maritime wireless communication. The method provided by the invention comprises the steps that a to-be-evaluated machine learning model set and a to-be-evaluated linear model set are determined, a plurality of to-be-evaluated models are obtained, and the outputs of the plurality of to-be-evaluated models are predicted evaluation values of signal intensity; according to the parameters of each to-be-evaluated model, sensitive features which are ranked to the first n of the output association degrees of the to-be-evaluated models are determined from the first initial feature set, wherein the sensitive features comprise features of different objects associated with ship communication; measuring a parameter set of wireless communication of the target sea area, obtaining a measurement value corresponding to the sensitive feature according to the parameter set, and constructing an evaluation data set; and evaluating each to-be-evaluated model by using the evaluation data set to obtain an evaluation result. According to the method and the device for evaluating the signal strength prediction model of the maritime wireless communication, provided by the invention, the problems that evaluation indexes are inconsistent and performance comparison cannot be intuitively carried out on the prediction model can be solved.

Description

Technical Field

[0001] The present application relates to the technical field of marine channel measurement, and in particular to a method and device for evaluating a signal strength prediction model for marine wireless communications. Background Art

[0002] Maritime wireless channel modeling is very important in promoting maritime communication systems and is an important supporting technology for the digitalization of the ocean. With the development of technology, maritime wireless channel modeling technology has evolved from traditional modeling methods to artificial intelligence-based modeling methods.

[0003] Due to different modeling methods, different evaluation indicators are used when evaluating different marine signal prediction models. The inconsistency of the evaluation indicators makes it difficult to intuitively compare and analyze the prediction capabilities of different marine signal prediction models, and it is difficult to determine which model's prediction results are the most accurate and reliable in a specific scenario. Summary of the invention

[0004] In view of this, the present application provides a signal strength prediction model evaluation method and device for marine wireless communications, which are used to solve the problem of inconsistent evaluation indicators and inability to intuitively compare the performance of prediction models.

[0005] Specifically, the present application is implemented through the following technical solutions: The first aspect of the present application provides a signal strength prediction model evaluation method for marine wireless communication, the method comprising: Determine a set of machine learning models to be evaluated and a set of linear models to be evaluated, and obtain a plurality of models to be evaluated, wherein outputs of the plurality of models to be evaluated are predicted evaluation values ​​of signal strength; Determining the top n sensitive features ranked by output relevance of the model to be evaluated from the first initial feature set according to the parameters of each model to be evaluated, wherein the sensitive features include features of different objects associated with ship communication; Measuring a parameter set of wireless communication in the target sea area, obtaining a measurement value corresponding to the sensitive feature according to the parameter set, and constructing an evaluation data set; The evaluation data set is used to evaluate each of the models to be evaluated to obtain an evaluation result.

[0006] The second aspect of the present application provides a signal strength prediction model evaluation device for marine wireless communication, the device comprising a determination module, a construction module and an evaluation module; wherein: The determination module is used to determine a set of machine learning models to be evaluated and a set of linear models to be evaluated, and obtain multiple models to be evaluated, wherein the outputs of the multiple models to be evaluated are predicted evaluation values ​​of signal strength; The determination module is further used to determine the top n sensitive features in terms of output relevance ranking with the model to be evaluated from the first initial feature set according to the parameters of each model to be evaluated, wherein the sensitive features include features of different objects associated with ship communication; The construction module is used to measure the parameter set of wireless communication in the target sea area, obtain the measurement value corresponding to the sensitive feature according to the parameter set, and construct an evaluation data set; The evaluation module is used to evaluate each of the models to be evaluated using the evaluation data set to obtain an evaluation result.

[0007] The signal strength prediction model evaluation method and device for maritime wireless communication provided in the present application can evaluate different types of linear and machine learning models at the same scale and standard through a set of standardized indicators, so that the same set of indicators can be used for all-round performance evaluation of different types of models, and the performance of different models on specific tasks can be more comprehensively evaluated, which is helpful to find the most suitable model type for solving the signal strength prediction of maritime wireless communication. On the second hand, the parameter set of wireless communication in the target sea area is measured and the measurement values ​​corresponding to the sensitive features are obtained to construct an evaluation data set, which can ensure that the evaluation is based on real actual data. The evaluation results obtained in this way are more reliable and practical, and can reflect the performance of the model in actual application scenarios, and because the evaluation data set is constructed based on sensitive features, it can more specifically evaluate the model's processing capabilities for important features associated with ship communications. This helps to ensure the effectiveness of the model in actual ship communication scenarios. On the third hand, using a unified evaluation data set to evaluate multiple models to be evaluated can objectively compare the performance of different models, clarify the advantages and disadvantages of each model, and provide a basis for selecting the best model. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A flowchart of a first embodiment of a signal strength prediction model evaluation method for marine wireless communications provided by the present application; Figure 2 A flowchart of a second embodiment of a signal strength prediction model evaluation method for marine wireless communications provided by the present application; Figure 3 A flowchart of Embodiment 3 of the signal strength prediction model evaluation method for marine wireless communication provided by the present application; Figure 4 This is a structural diagram of Embodiment 1 of the signal strength prediction model evaluation device for marine wireless communications provided in the present application. DETAILED DESCRIPTION

[0009] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0010] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items.

[0011] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0012] Specific embodiments are given below to introduce the technical solution of the present application in detail.

[0013] Figure 1 This is a flowchart of the first embodiment of the signal strength prediction model evaluation method for marine wireless communication provided by this application. Please refer to Figure 1 , the method provided in this embodiment may include: S101. Determine a set of machine learning models to be evaluated and a set of linear models to be evaluated, and obtain multiple models to be evaluated, wherein outputs of the multiple models to be evaluated are predicted evaluation values ​​of signal strength.

[0014] Specifically, the signal strength prediction model evaluation method for marine wireless communication provided in this embodiment is applied to the ground-to-ship scenario. The signal strength prediction model for marine wireless communication is usually used in ship communication system planning and optimization, navigation safety, marine operation management and marine science research. It can predict the communication signal strength at sea, which is crucial to promoting the establishment of marine communication systems. At present, there are methods for constructing prediction models based on multiple learning methods. Since different marine wireless communication signal strength prediction models use different methods and data when constructing, the evaluation indicators of different marine wireless communication signal strength prediction models are inconsistent. Therefore, when selecting a marine wireless communication signal strength prediction model, it is impossible to intuitively compare the differences between the marine wireless communication signal strength prediction models.

[0015] Furthermore, the machine learning models to be evaluated include decision tree regression model, random forest regression model, adaptive boosting regression model, support vector regression model, multilayer perceptron regression model, kernel ridge regression model, Gaussian process regression model and K nearest neighbor regression model; the linear models to be evaluated include least absolute shrinkage and selection operator regression model, linear regression model, Bayesian ridge regression model and elastic network regression model; the input of the machine learning model and the linear model is data on the strength of wireless communication signals at sea, which may include geographical location (such as geographical distance GeoDis, line of sight distance GeoLos, reflection distance GeoRef, etc.), environmental factors (atmospheric pressure, temperature, humidity, etc.), ship motion related data (ship speed ShipSpeed, three-dimensional ship angular acceleration ShipAccX, ShipAccY, ShipAccZ, three-dimensional ship angular velocity ShipAngVelX, ShipAngVelY, ShipAngVelZ, three-dimensional inclination angle ShipAngX, ShipAngY, ShipAngZ) and other data; the output is the predicted evaluation value of the signal strength at sea.

[0016] It can be understood that the models in the set of machine learning models to be evaluated and the set of linear models to be evaluated are both models to be evaluated, wherein the number of models to be evaluated is determined according to actual needs, and is not limited in this embodiment. For example, in one possible implementation, all models in the set of machine learning models to be evaluated and the set of linear models to be evaluated are determined as models to be evaluated; for another example, in another possible implementation, some models are selected from the set of machine learning models to be evaluated and the set of linear models to be evaluated as models to be evaluated. In specific implementation, the models can be screened based on the complexity of the models. When computing resources are limited or the real-time requirements for the models are high, models with lower complexity can be given priority. For example, in maritime communication scenarios, some embedded devices have limited computing power and cannot run complex models. At this time, a simple linear regression model or K nearest neighbor regression model may be more suitable as the model to be evaluated because of its low computational complexity and fast running speed. Models can also be screened based on the currently acquired data features. For example, when the amount of data is small, models that do not require a high amount of data can be selected, such as the kernel ridge regression model, which can also perform good fitting and prediction in small sample conditions. If there are complex nonlinear relationships between data features, machine learning models that can capture nonlinear relationships, such as the multilayer perceptron regression model, are given priority. Selecting a model based on data characteristics can improve the adaptability of the model to the data and make the evaluation results more valuable for reference.

[0017] Furthermore, multiple models to be evaluated are used to predict the signal strength of wireless communication at sea, and the inputs of the multiple models to be evaluated are data related to the signal strength of wireless communication at sea, such as geographical location, environmental factors, time, etc., and the outputs of the multiple models to be evaluated are predicted evaluation values ​​of the signal strength. It should be noted that in order to ensure that the processing capabilities of different models for the same influencing factors can be evaluated under a unified standard, when evaluating multiple models to be evaluated, the input contents of the multiple models to be evaluated are the same.

[0018] S102: determining the top n sensitive features in terms of output relevance to the model to be evaluated from a first initial feature set according to the parameters of each model to be evaluated, wherein the sensitive features include features of different objects associated with ship communication.

[0019] Specifically, the first initial feature set includes at least a geographical feature type, a ship feature type, and an environmental feature type. The specific steps of determining the first initial feature set include: (1) Identify multiple associated objects of maritime wireless communications.

[0020] Specifically, wireless communication at sea involves multiple related objects, including communication environment, shore base stations, ship base stations, etc. The communication environment covers factors such as atmospheric pressure, temperature, and humidity, which affect the propagation characteristics of the signal in the air. As the transmitter of the signal, the shore base station has an important influence on the signal strength through its transmission power, antenna height, and position. The ship base station is the receiver of the signal, and its receiving sensitivity, antenna direction, and height also affect the signal reception effect.

[0021] (2) Determine the key features of each associated object under each type.

[0022] Specifically, key features refer to the features of the associated object itself that affect the signal strength. For example, for the communication environment, temperature has a significant impact on signal strength. In a high temperature environment, the thermal motion of air molecules intensifies, and the scattering and absorption during signal propagation increase, resulting in signal attenuation. In a high humidity environment, water vapor will scatter and absorb the signal, affecting signal propagation. Atmospheric pressure also affects signal propagation. The air density in low-pressure areas is small, and the signal propagation loss is relatively small, but it will cause the signal propagation speed to change, affecting the stability of the signal.

[0023] For shore base stations, the transmission power is an important feature of shore base stations. The higher the transmission power, the longer the signal propagates and the greater the strength. For ship base stations, the receiving sensitivity reflects the ability of ship base stations to receive weak signals. Base stations with high receiving sensitivity can receive weaker signals and ensure signal strength. At the same time, the height of ship base stations is also critical. Increasing the height can reduce the obstruction of signals by surrounding objects and improve the signal reception strength.

[0024] (3) Determine the key features of collaboration between objects of each type.

[0025] Specifically, the collaborative key feature refers to the feature between two associated objects that affects the signal strength. For example, for the communication environment and the shore base station, the temperature and the shore base station transmission power interact to affect the signal strength. In a high temperature environment, if the shore base station transmission power remains unchanged, the signal attenuation is aggravated. At this time, the transmission power needs to be appropriately increased to ensure the signal strength. There is also a collaborative relationship between atmospheric pressure and antenna height. In a low pressure environment, appropriately increasing the antenna height can reduce the loss of the signal during propagation and improve the signal strength; for the communication environment and the ship base station, humidity and the receiving sensitivity of the ship base station synergistically affect the signal reception. In a high humidity environment, the receiving sensitivity of the ship base station will decrease. At this time, the receiving sensitivity parameter needs to be adjusted to ensure stable signal reception. Changes in the communication environment will also affect the adjustment of the antenna direction of the ship base station. For example, in windy weather, the ship shakes, and the antenna direction needs to be adjusted in real time according to the wind direction and signal strength changes to ensure the signal reception effect; for the shore base station and the ship base station, the transmitting power of the shore base station and the receiving sensitivity of the ship base station need to match. If the transmission power of the shore base station is too high and the receiving sensitivity of the ship base station is limited, it may cause signal saturation and affect the signal quality; conversely, if the transmission power is too low, the receiving sensitivity is high and it is not possible to receive a signal of sufficient strength. The relative position and antenna direction of the shore base station and the ship base station are also closely related. Adjusting the antenna direction according to the relative position of the two can effectively improve the signal transmission efficiency and strength.

[0026] Specifically, the parameters of each model to be evaluated may include location, time, wind speed, temperature, ship speed, RSRP (Reference Signal Received Power), RSSI (Received Signal Strength Indication), etc. The parameters of the model to be evaluated are cleaned and sorted to ensure the accuracy and completeness of the parameters, and feature extraction is performed on the processed parameters to obtain all features related to the parameters of the model to be evaluated. For the detailed process of feature extraction, please refer to the description in the relevant technology, which will not be repeated here.

[0027] Furthermore, the specific implementation steps of determining the top n sensitive features in terms of output relevance to the model to be evaluated from the first initial feature set according to the parameters of each model to be evaluated include: (1) Screening the features in the first initial feature set based on multiple screening methods to obtain screening results.

[0028] Specifically, the number of screening methods is set according to actual needs, and is not limited in this embodiment. For example, in one embodiment, three screening methods are used to screen the features of the model to be evaluated, and the three screening methods are Pearson correlation coefficient, feature P value and feature score, which are used to screen the features at the same time.

[0029] Further, in combination with the above description, the features are first screened according to the Pearson correlation coefficient of the features. The Pearson correlation coefficient is used to measure the linear correlation between the feature and the signal strength, and its value range is between -1 and 1. If the Pearson correlation coefficient of the feature meets the first threshold, the feature is retained. If the Pearson correlation coefficient of the feature does not meet the first threshold, the feature is deleted. It should be noted that the first threshold is set according to actual needs and is not limited in this embodiment.

[0030] Similarly, the features are screened using the P value of the feature, and the P value is used to determine whether the null hypothesis in the statistical hypothesis test is rejected. In this embodiment, the null hypothesis is "the feature is independent of the signal strength". If the P value of the feature is less than the second threshold, it means that the feature is related to the signal strength and the feature is retained. If the P value of the feature is greater than the second threshold, it means that the feature is independent of the signal strength and the feature is deleted. It should be noted that the second threshold is set according to actual needs and is not limited in this embodiment.

[0031] Similarly, the feature score is used to screen the features. The feature score is a measure of the importance of the feature. If the feature score of the feature is greater than the third threshold, the feature is retained. If the feature score of the feature is less than the third threshold, the feature is deleted. It should be noted that the third threshold is set according to actual needs and is not limited in this embodiment. In addition, the first threshold, the second threshold and the third threshold are different.

[0032] It should be noted that when multiple screening methods are used to screen the features in the first initial feature set, each screening method screens the features in the first initial feature set at the same time. In this way, the consistency of the screening process can be maintained, all models are treated equally, and are not affected by differences in model types. The relationship between each feature and the signal strength can be fully considered, and features with a high correlation with all model outputs can be screened out as a whole, providing a unified and comprehensive feature set for subsequent evaluation, facilitating fair comparison between different models.

[0033] Further, in combination with the above description, the features filtered by the three methods of Pearson correlation coefficient, feature P value and feature score are combined, and the feature combination is used as the screening result.

[0034] (2) Sorting the screening results according to the sensitivity of the multiple screening methods.

[0035] Specifically, for each screening method, its sensitivity is calculated respectively, and based on the calculated sensitivity of each screening method, the sensitivity of each screening method is integrated by weighted average to obtain a comprehensive sensitivity. Further, based on the order of comprehensive sensitivity from large to small, the features in the screening results are sorted to obtain a sorting result. For the process of calculating the sensitivity of the screening method, please refer to the description in the relevant technology, which will not be repeated here.

[0036] (3) The top n features in the sorting results are determined as sensitive features.

[0037] Specifically, the value of n is set according to actual needs, and is not limited in this embodiment. For example, in a possible implementation, the first initial feature set includes 19 features, including geographic distance, line-of-sight distance, reflection distance, phase difference between line-of-sight and reflection, angle between ship and base station in horizontal direction, angle between ship and base station in vertical direction, ship speed, three-dimensional ship angular acceleration (ShipAccX, ShipAccY, ShipAccZ), three-dimensional ship angular velocity (ShipAngVelX, ShipAngVelY, ShipAngVelZ), three-dimensional tilt angle (ShipAngX, ShipAngY, ShipAngZ), atmospheric pressure, temperature and humidity of the environment, so as to capture the influence of various factors that may affect channel characteristics, such as line-of-sight, reflection, scattering, diffraction, refraction, etc. For example, the distance and angle between the ship and the base station reflect the line-of-sight condition and path loss; the ship speed and angular acceleration indicate Doppler shift and small-scale fading; the tilt angle and weather information imply the refraction and attenuation of the signal.

[0038] After screening by multiple screening methods, the features with weak correlation are removed, and the key features with high correlation with signal strength prediction are retained. In the actual screening process, when screening features by methods such as Pearson correlation coefficient, feature P value and feature score, the 10 features that play an important role in prediction are finally retained, including geographic distance, line of sight distance, reflection distance, angle between the ship and the base station in the horizontal direction, angle between the ship and the base station in the vertical direction, ship speed, three-dimensional tilt angle (ShipAngZ), atmospheric pressure, temperature and humidity of the environment. This screening change can enable subsequent model training and evaluation to focus on more critical factors, improving the accuracy and efficiency of model prediction.

[0039] S103, measuring a parameter set of wireless communication in the target sea area, obtaining measurement values ​​corresponding to the sensitive features according to the parameter set, and constructing an evaluation data set.

[0040] Specifically, the specific implementation process of this step includes: (1) Determine measurement parameters and sensitive features, and construct the parameter set according to the measurement parameters.

[0041] Specifically, sensitive features are features that have a significant impact, high correlation or important role on the model prediction results. Sensitive features may include: geographic distance GeoDis, line-of-sight distance GeoLos, reflection distance GeoRef, phase difference between line-of-sight and reflection GeoPhaseDiff, angles GeoThetaX and GeoTheteY between the ship and the base station in the horizontal and vertical directions, ship speed (ShipSpeed), three-dimensional ship angular acceleration (ShipAccX, ShipAccY, ShipAccZ), three-dimensional ship angular velocity (ShipAngVelX, ShipAngVelY, ShipAngVelZ), three-dimensional tilt angle (ShipAngX, ShipAngY, ShipAngZ), atmospheric pressure, temperature and humidity of the surrounding environment, etc.

[0042] Specifically, combined with the above description, after determining the measurement parameters, a maritime 5G terminal operating in the 5G frequency band is used for parameter measurement. The terminal is designed for maritime applications and has a four-element antenna array. The antenna array can provide an omnidirectional radiation pattern and an antenna gain of up to 6.35dBi. The standard 5G module is integrated into the terminal with a transmission power of 23dBm. Further, the terminal is arranged on a fishery ship to receive the reference signal power transmitted by the 5G base station along the coastline. The total height of the ship is about 9 meters, and the 5G terminal is placed about 7 meters from the sea surface. The 5G network provided by China Mobile operates in the 2.496GHz - 2.690GHz and 4.4GHz - 5.0GHz frequency bands. The ship travels at a speed of up to 14.40 kilometers per hour, and the total voyage is about 53.74 kilometers. During the measurement process, the terminal automatically obtains data including RSRP and RSRQ (ReferenceSignal Received Quality) from the 5G module, which are key indicators of the signal level and quality of the 5G mobile network. 9-axis motion sensor data and the vessel’s positioning data are also collected.

[0043] Furthermore, RSSI can be calculated according to the following formula: ; in, N is the maximum number of resource blocks; is the measured value of the received power of the reference signal; is a measurement of the quality of the reference signal reception.

[0044] Furthermore, the measured parameters are preprocessed to solve the problems of data format mismatch, data anomaly and inconsistent data granularity in the original measurement data, and to eliminate the interference features that have a low contribution to the model prediction, and the preprocessed measurement parameters are determined as a parameter set.

[0045] As an optional embodiment, the preprocessing of the measured parameters includes: determining the RSRQ value of each measured parameter, screening the parameters to be preprocessed based on the RSRQ value, wherein the parameters with poor reception quality are prone to abnormalities such as reception interruption and data loss due to poor reception quality. On this basis, the parameters with lower RSRQ values ​​are screened out as the parameters to be preprocessed, and the RSRQ value can be compared with the preprocessing quality threshold to determine the parameters with lower RSRQ values. Determine the reference parameter corresponding to the parameter to be preprocessed, and determine the reference parameter from the measured parameters other than the parameter to be preprocessed. The RSRQ of the reference parameter needs to be greater than the preprocessing quality threshold, and at the same time, the time is the same as the corresponding parameter to be preprocessed, the transceiver equipment is adjacent or the same, and the subject of the communication is the same. The parameters to be preprocessed are marked for exceptions according to the reference parameters. The exception marks include data missing marks, data error marks, data format exception marks, etc. The marked position is the specific position where the exception occurs in the data, such as the number of the data. Based on the exception marks, the parameters to be preprocessed are subjected to exception processing. The exception processing is to process the content marked by the exception mark. Data missing and data error are replaced by the mean of the data adjacent to the exception mark. Data format exception is to change the position parameter to the target format.

[0046] The data after the first preprocessing is used as the target data, and a relationship map between each target data is established according to the data receiving time, the sending device and the sending subject. In the relationship map, the target data is the data node, and the association relationship between the data receiving time, the sending device and the sending subject is the connecting edge. A matching relationship of the target data is established based on the relationship map, and abnormal data in the target data is determined according to the matching relationship. Specifically, the two groups of data with the highest correlation are used as matching data, and the change trend of the sending subject is identified according to the data value between the two matching data. The matching data with conflicting change trends of the two groups of data is used as abnormal data, and the abnormal data is corrected based on the relationship map. Specifically, the associated data with the second highest correlation in the relationship map is used to add the matching data to form three data pairs, and the two groups of data with the same change trend are used as reference to correct the third group of data.

[0047] The method provided by the present invention first corrects the fine-grained errors within the data based on preprocessing, that is, the first correction is of certain bits in a data. After the correction is completed, on the basis of ensuring that the data itself is correct, correction is performed based on the correlation between the data, that is, the change trend between the data is corrected. The minimum unit of the data for the second correction is one data itself. Therefore, after two different levels of data correction, the accuracy of the data itself and the change trend between the data is guaranteed, the accuracy of the received data is improved, and then the interference features with low contribution to model prediction are eliminated.

[0048] (2) Based on the determined sensitive feature, searching for a measurement value corresponding to the sensitive feature from the parameter set.

[0049] (3) Arranging the measurement values ​​and constructing the evaluation data set based on the measurement values.

[0050] Specifically, combined with the above description, the measured values ​​corresponding to the sensitive features are directly searched in the parameter set, and the measured values ​​are sorted and classified to construct an evaluation data set.

[0051] S104: using the evaluation data set to evaluate each of the models to be evaluated to obtain an evaluation result.

[0052] Specifically, the evaluation data set is a set of data used to measure the performance of the model to be evaluated. By evaluating the above-mentioned multiple models to be evaluated through the evaluation data set, the above-mentioned multiple models to be evaluated can be quantitatively described to select the best model from them. For the detailed process of evaluating each of the models to be evaluated using the evaluation data set, please refer to the description in the relevant technology, which will not be repeated here.

[0053] The signal strength prediction model evaluation method for maritime wireless communication provided in this embodiment, on the one hand, by determining the machine learning model set to be evaluated and the linear model set to be evaluated, it can cover a variety of different types of models, so as to more comprehensively evaluate the performance of different models on specific tasks, which is helpful to find the model type that is most suitable for solving the signal strength prediction of maritime wireless communication. On the second hand, measuring the parameter set of wireless communication in the target sea area and obtaining the measured values ​​corresponding to the sensitive features to construct an evaluation data set can ensure that the evaluation is based on real actual data. The evaluation results obtained in this way are more reliable and practical, and can reflect the performance of the model in actual application scenarios, and because the evaluation data set is constructed based on sensitive features, it can more specifically evaluate the model's processing capabilities for important features associated with ship communications. This helps to ensure the effectiveness of the model in actual ship communication scenarios. On the third hand, using a unified evaluation data set to evaluate multiple models to be evaluated can objectively compare the performance of different models, clarify the advantages and disadvantages of each model, and provide a basis for selecting the best model.

[0054] Figure 2 This is a flowchart of the second embodiment of the signal strength prediction model evaluation method for marine wireless communication provided by this application. Figure 2 Based on the above embodiment, the step of evaluating each of the models to be evaluated using the evaluation data set to obtain the evaluation result includes: S201. Evaluate the prediction performance of each of the models to be evaluated according to the first evaluation indicator set to obtain a first evaluation result, wherein the value corresponding to the first evaluation indicator set of the model in the linear model set to be evaluated is inferior to that of the model in the machine learning model set to be evaluated.

[0055] Specifically, the first evaluation indicator set includes multiple evaluation indicators, which are used to perform a preliminary evaluation of the prediction performance of each model to be evaluated.

[0056] Optionally, in a possible implementation, the first evaluation indicator set includes at least the following first evaluation indicators: mean absolute error, mean square error, root mean square error, mean absolute percentage error, median absolute error, maximum error, determination coefficient, explained variance and running time.

[0057] Furthermore, when each model to be evaluated is evaluated using the first evaluation indicator set, the evaluation data set is used as input and input into the model to be evaluated. The model to be evaluated outputs a prediction result, and then the prediction result is compared with the true label or target value in the evaluation data set, and the value of each evaluation indicator is calculated according to the comparison result.

[0058] Furthermore, the values ​​corresponding to the first evaluation index set of the models in the set of linear models to be evaluated are inferior to those of the models in the set of machine learning models to be evaluated, indicating that the machine learning model performs better in terms of prediction performance under the current task and evaluation data set. This may be because the relationship between the signal strength and various features in ship communications is relatively complex, and the machine learning model can better capture these complex relationships, while the linear model is limited by its linear assumption and cannot fully fit the data.

[0059] In this step, the prediction performance of each model to be evaluated is evaluated according to the first evaluation index set. The index in this index set is a quantitative value, which can quantitatively evaluate the difference between the model prediction value and the true value as a whole, reflecting the accuracy and precision of the model prediction. The mean absolute error, mean square error and root mean square error can directly measure the average deviation degree between the predicted value and the true value, the square mean of the deviation and the square root of the square mean of the deviation. The smaller the value, the closer the model prediction value is to the true value, and the higher the prediction accuracy. The determination coefficient and explained variance are used to evaluate the goodness of fit of the model, reflecting the model's ability to explain the data. The closer the value is to 1, the better the model fits the data and can explain more variations in the data. The running time reflects the efficiency of model training and is an important indicator for measuring the feasibility of the model in practical application.

[0060] S202: Filter a target set of models to be evaluated from the models to be evaluated according to the first evaluation result.

[0061] The specific implementation steps include: (1) Calculate the index score of each model to be evaluated.

[0062] Specifically, each indicator in the first evaluation indicator set is used to evaluate the model to be evaluated, and indicator scores corresponding to multiple indicators are obtained. The indicator scores corresponding to the multiple indicators are added together, and the sum of the indicator scores is the indicator score of the model to be evaluated in the first evaluation indicator set.

[0063] (2) Sort the models to be evaluated according to the indicator scores to obtain a first indicator ranking result.

[0064] Specifically, the multiple models to be evaluated are sorted in descending order of the indicator scores, and the sorting result is recorded as the first indicator sorting result.

[0065] (3) The top K models to be evaluated in the ranking result of the first indicator are determined as the target models to be evaluated, and the K target models to be evaluated form the target model set to be evaluated.

[0066] In step S202, the prediction performance is evaluated according to the characteristics of the prediction result curves of each target model to be evaluated. The characteristics of the prediction result curves are mainly the RSRP line-of-sight distance curve and the RSSI line-of-sight distance curve. This evaluation method evaluates the model from an intuitive graphical perspective by observing and analyzing the characteristics of the curves. The accuracy of the model can be judged by observing the closeness of the curves of different models to the real data curves; the reliability of the model can be evaluated by analyzing the smoothness and stability of the curves; the rationality of the model can be measured by studying whether the trend of the curve is consistent with the actual situation. If the RSRP line-of-sight distance curve of a model is highly consistent with the trend of the real data curve, and the curve is smooth without drastic fluctuations, it means that the model has high accuracy and good stability in predicting the change of RSRP with line-of-sight distance. Different from the quantitative indicators of S201, the curve feature evaluation pays more attention to the performance of the model in the overall trend and change law, and can intuitively show the changes in the model's prediction of signal strength at different line-of-sight distances.

[0067] S203 . Evaluate the prediction performance of each target model to be evaluated according to the prediction result curve characteristics of each target model to be evaluated, and obtain a second evaluation result.

[0068] Optionally, in a possible implementation, the prediction result curve characteristics are: an RSRP line-of-sight distance curve and an RSSI line-of-sight distance curve.

[0069] Specifically, each target model to be evaluated is further evaluated through the prediction result curve characteristics. The prediction result curve characteristics can provide more information about the model performance and can capture the long-term trend of the model in more depth. By observing and analyzing the prediction result curve characteristics of each target model to be evaluated, the prediction performance of the model can be evaluated from different angles. For example, in one embodiment, the degree of closeness of the curves of different models to the real data curve can be compared to judge the accuracy of the model; the smoothness and stability of the curve can be observed to evaluate the reliability of the model; and the trend of the curve can be analyzed to see whether it is consistent with the actual situation to measure the rationality of the model.

[0070] Furthermore, the second evaluation result is a conclusion drawn based on the evaluation of the prediction result curve characteristics of the target model to be evaluated. It can complement the first evaluation result and provide a more comprehensive basis for the final selection of the most suitable model.

[0071] S204: Determine an optimal signal strength prediction model according to the second evaluation result.

[0072] In specific implementation, the model to be evaluated with the highest prediction performance evaluation score in the second evaluation result is determined as the optimal signal strength prediction model.

[0073] The method provided in this embodiment evaluates the model to be evaluated through the first evaluation index set and the prediction result curve characteristics, and considers the prediction performance of the model to be evaluated from different angles. The first evaluation index set usually covers quantitative indicators such as accuracy and precision, while the curve feature evaluation observes the trend, stability and other aspects of the model from an intuitive graphical perspective. This multi-dimensional evaluation can more comprehensively understand the performance of the model and avoid the one-sidedness that may be caused by a single evaluation method. In addition, all models to be evaluated are first evaluated using the first evaluation index set, and a model set with relatively good performance can be quickly screened out. Since the model in the linear model set to be evaluated has a value corresponding to the first evaluation index set that is inferior to the model in the machine learning model set to be evaluated, this step can preliminarily exclude some poorly performing linear models, reduce the workload of subsequent evaluations, and screen out the target model set to be evaluated from the model to be evaluated, and then evaluate it according to the prediction result curve characteristics of each target model to be evaluated. In this way, evaluation resources can be concentrated on models with more potential, improving evaluation efficiency. At the same time, further evaluation of the target model set to be evaluated can be more in-depth and detailed, which helps to find models that perform well in the first evaluation but still have some subtle problems.

[0074] Figure 3 This is a flowchart of Embodiment 3 of the signal strength prediction model evaluation method for marine wireless communication provided by this application. Figure 3 Based on the above embodiment, the prediction performance of each target model to be evaluated is evaluated according to the prediction result curve characteristics of each target model to be evaluated to obtain a second evaluation result; including: S301, calculating the predicted sample coverage area and the real sample coverage area corresponding to the predicted result curve characteristics of each target model to be evaluated.

[0075] Specifically, with the sight distance as the horizontal coordinate and the prediction result curve characteristics as the vertical coordinate, (sight distance, prediction result) is taken as a set of data and input into the sight distance-prediction result coordinate system to obtain the prediction result curve characteristics of each target model to be evaluated.

[0076] Furthermore, a blank image is created, the number of pixels in the image is determined, the predicted samples are drawn into the blank image, each pixel in the image is traversed, and it is checked whether the pixel is occupied by the predicted sample point. The number of pixels of the predicted samples in the blank image is counted, which is the predicted sample coverage area.

[0077] Similarly, when calculating the true sample coverage area, the true sample is drawn into the blank image, and each pixel in the image is traversed to check whether the pixel is occupied by the true sample point. The number of pixels of the true sample in the blank image is counted, which is the true sample coverage area.

[0078] S302: Screen the target model set to be evaluated according to the predicted sample coverage area to obtain a first target model set to be evaluated.

[0079] Optionally, the predicted sample coverage area of ​​each of the target models to be evaluated in the first target model set to be evaluated is greater than the true sample coverage area.

[0080] Specifically, the models in the first target model set to be evaluated whose predicted sample coverage area is larger than the actual sample coverage area are retained as the first target model set to be evaluated.

[0081] S303: For the models in the first target model set to be evaluated, construct a regression coordinate of the prediction result with the real sample as the horizontal coordinate and the predicted sample as the vertical coordinate.

[0082] Specifically, in order to further select the signal strength prediction model with the best performance, the prediction result regression coordinates are constructed with the real samples as the horizontal coordinates and the predicted samples as the vertical coordinates. The purpose of constructing the prediction result regression coordinates is to intuitively analyze and evaluate the prediction performance of the model in the first target model set to be evaluated. By using the real samples as the horizontal coordinates and the predicted samples as the vertical coordinates, the relationship between the predicted value and the true value of the model can be observed.

[0083] S304: Calculate the regression degree of the prediction result of the target model to be evaluated according to the prediction result regression coordinates.

[0084] Specifically, combined with the above description, it can be understood that, ideally, the predicted samples output by the signal strength prediction model should have the same value as the true sample. Therefore, in the prediction result regression coordinates, the line where the predicted sample has the same value as the true sample is used as the regression line, and (true sample, predicted sample) is used as a point in the prediction result regression coordinates. The points in the prediction result regression coordinates are fitted by a linear regression model. It can be understood that the closer the prediction result regression degree is to 1, the better the prediction performance of the target model to be evaluated.

[0085] S305. Sort the target models to be evaluated according to the regression degree, and determine the sorting result as a second sorting result.

[0086] Specifically, the target models to be evaluated are sorted in descending order of regression degree, and the sorting result is the second sorting result.

[0087] The method provided in this embodiment can intuitively understand the model's coverage of the sample space by calculating the predicted sample coverage area and the real sample coverage area. If the predicted sample coverage area of ​​a model differs too much from the real sample coverage area, it means that the model may have a large deviation or instability. These less reliable models can be excluded by screening, thereby improving the accuracy of subsequent evaluations. Furthermore, by constructing a regression coordinate of the prediction result with the real sample as the horizontal coordinate and the predicted sample as the vertical coordinate, the relationship between the prediction result of the model and the true value can be more comprehensively analyzed. The calculation of the regression degree can quantify the closeness of this relationship, making the evaluation of model performance more accurate.

[0088] Corresponding to the aforementioned embodiment of a signal strength prediction model evaluation method for maritime wireless communication, the present application also provides an embodiment of a signal strength prediction model evaluation device for maritime wireless communication.

[0089] Figure 4 This is a schematic diagram of the structure of the first embodiment of the signal strength prediction model evaluation device for marine wireless communication provided by this application. Figure 4 The device provided in this embodiment includes a determination module 410, a construction module 420 and an evaluation module 430; wherein, The determination module 410 is used to determine a set of machine learning models to be evaluated and a set of linear models to be evaluated, and obtain multiple models to be evaluated, wherein the outputs of the multiple models to be evaluated are predicted evaluation values ​​of signal strength; The determination module 410 is further used to determine the top n sensitive features in terms of output relevance to the model to be evaluated from the first initial feature set according to the parameters of each model to be evaluated, wherein the sensitive features include features of different objects associated with ship communication; The construction module 420 is used to measure the parameter set of wireless communication in the target sea area, obtain the measurement value corresponding to the sensitive feature according to the parameter set, and construct an evaluation data set; The evaluation module 430 is used to evaluate each of the models to be evaluated using the evaluation data set to obtain an evaluation result.

[0090] The device of this embodiment can be used to perform Figure 1 The steps, specific implementation principles and implementation processes of the method embodiment shown are similar and will not be repeated here.

[0091] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0092] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0093] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A signal strength prediction model evaluation method for marine wireless communication, characterized in that: The method comprises: Determine a set of machine learning models to be evaluated and a set of linear models to be evaluated, and obtain a plurality of models to be evaluated, wherein outputs of the plurality of models to be evaluated are predicted evaluation values ​​of signal strength; Determining the top n sensitive features ranked by output relevance of the model to be evaluated from the first initial feature set according to the parameters of each model to be evaluated, wherein the sensitive features include features of different objects associated with ship communication; Measuring a parameter set of wireless communication in the target sea area, obtaining a measurement value corresponding to the sensitive feature according to the parameter set, and constructing an evaluation data set; The evaluation data set is used to evaluate each of the models to be evaluated to obtain an evaluation result.

2. The method according to claim 1, characterized in that The step of evaluating each of the models to be evaluated by using the evaluation data set to obtain an evaluation result comprises: Evaluate the prediction performance of each of the models to be evaluated according to the first evaluation indicator set to obtain a first evaluation result, wherein the value corresponding to the first evaluation indicator set of the model in the linear model set to be evaluated is inferior to the model in the machine learning model set to be evaluated; According to the first evaluation result, selecting a target set of models to be evaluated from the models to be evaluated; Evaluate the prediction performance of each target model to be evaluated according to the prediction result curve characteristics of each target model to be evaluated, and obtain a second evaluation result; An optimal signal strength prediction model is determined according to the second evaluation result.

3. The method according to claim 2, characterized in that The first evaluation indicator set includes at least the following first evaluation indicators: mean absolute error, mean square error, root mean square error, mean absolute percentage error, median absolute error, maximum error, determination coefficient, explained variance and running time.

4. The method according to claim 2, characterized in that: According to the first evaluation result, a target set of models to be evaluated is selected from the models to be evaluated; including: Calculating the index score of each model to be evaluated; Sorting each of the models to be evaluated according to the indicator score to obtain a first indicator ranking result; The top K models to be evaluated in the first indicator ranking result are determined as the target models to be evaluated, and the K target models to be evaluated form the target model set to be evaluated.

5. The method according to claim 2, characterized in that: The prediction result curve features include: RSRP line-of-sight distance curve and RSSI line-of-sight distance curve.

6. The method according to claim 2, characterized in that The step of evaluating the prediction performance of each target model to be evaluated according to the prediction result curve characteristics of each target model to be evaluated to obtain a second evaluation result comprises: Calculate the predicted sample coverage area and the real sample coverage area corresponding to the predicted result curve characteristics of each target model to be evaluated; The target model set to be evaluated is screened according to the predicted sample coverage area to obtain a first target model set to be evaluated; For the models in the first target to-be-evaluated model set, construct a regression coordinate of the prediction result with the real sample as the horizontal coordinate and the predicted sample as the vertical coordinate; Calculating the regression degree of the prediction result of the target model to be evaluated according to the prediction result regression coordinates; The target models to be evaluated are sorted according to the regression degree, and the sorting result is determined as the second sorting result.

7. The method according to claim 6, characterized in that The predicted sample coverage area of ​​each of the target models to be evaluated in the first target model set to be evaluated is greater than the true sample coverage area.

8. The method according to claim 1, characterized in that Determining the top n sensitive features ranked by output relevance of the model to be evaluated from the first initial feature set according to the parameters of each model to be evaluated; including: Screening the features in the first initial feature set based on multiple screening methods to obtain screening results; sorting the screening results according to the sensitivity of the multiple screening methods; The top n features in the sorting results are determined as sensitive features.

9. The method according to claim 1, characterized in that: Measuring a parameter set of wireless communication in the target sea area, obtaining a measurement value corresponding to the sensitive feature according to the parameter set, and constructing an evaluation data set; including: Determining measurement parameters and sensitive features, and constructing the parameter set according to the measurement parameters; According to the determined sensitive feature, searching for a measurement value corresponding to the sensitive feature from the parameter set; The measured values ​​are collated, and the evaluation data set is constructed based on the measured values.

10. A signal strength prediction model evaluation device for marine wireless communication, characterized in that: The device comprises a determination module, a construction module and an evaluation module; wherein, The determination module is used to determine a set of machine learning models to be evaluated and a set of linear models to be evaluated, and obtain multiple models to be evaluated, wherein the outputs of the multiple models to be evaluated are predicted evaluation values ​​of signal strength; The determination module is further used to determine the top n sensitive features in terms of output relevance ranking with the model to be evaluated from the first initial feature set according to the parameters of each model to be evaluated, wherein the sensitive features include features of different objects associated with ship communication; The construction module is used to measure the parameter set of wireless communication in the target sea area, obtain the measurement value corresponding to the sensitive feature according to the parameter set, and construct an evaluation data set; The evaluation module is used to evaluate each of the models to be evaluated using the evaluation data set to obtain an evaluation result.

Images