A data cleaning method for Wi-Fi geolocation benchmark points in IP positioning technology

Abstract

The present invention provides a data cleaning method for Wi-Fi geolocation reference points in IP positioning technology, which relates to the technical field of data processing. The method includes: obtaining reference point data, and extracting Wi-Fi point data from the reference point data; determining a search radius according to the Wi-Fi point data, and determining candidate information points and candidate information point names according to the search radius; calculating the semantic similarity between the candidate information point name and the Wi-Fi point user name; determining the semantic matching confidence between the reference point data and the candidate information point according to the similarity; determining the signal strength confidence of each candidate information point according to the signal strength and distance relationship of the Wi-Fi point; determining the final matching information point data according to the semantic matching confidence and the signal strength confidence; and verifying and cleaning the reference point data according to the final matching information point data. The present invention realizes accurate matching of Wi-Fi information points through efficient data cleaning to improve the accuracy and reliability of data.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a data cleaning method for Wi-Fi geographic positioning reference points in IP positioning technology. Background Art

[0002] At present, the sources of benchmark data points are wide-ranging, including network user location information, Wi-Fi location information, and information contained in existing geographic data databases. Most of these data are often inaccurate, and a major factor is the difficulty of accurately matching IP addresses and geographic locations. In addition, the randomness of IP allocation by some Internet service providers (ISPs), the impact of network data being affected by VPNs, etc., may all lead to inaccurate benchmark data.

[0003] In order to obtain the most accurate benchmark data, some current data cleaning methods use multi-source comparison, geographic location feature judgment, IP feature judgment and other methods to filter out erroneous data. However, for current methods, it is often difficult to obtain accurate results through a single comparison due to the dynamic nature of IP, IP address allocation rules and geographic location information are not always accurately corresponding, which may make it difficult to distinguish multi-source data. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a data cleaning method for Wi-Fi geographic positioning reference points in IP positioning technology, so as to improve positioning accuracy and enhance user experience.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] In a first aspect, a method for cleaning data of Wi-Fi geolocation reference points in IP positioning technology, the method comprising:

[0007] Obtain benchmark point data, and extract Wi-Fi point data from the benchmark point data;

[0008] Determine a search radius based on the Wi-Fi point data, and determine candidate information points and names of the candidate information points based on the search radius;

[0009] Calculate the semantic similarity between the candidate information point name and the Wi-Fi point user name;

[0010] According to the similarity, the semantic matching confidence between the reference point data and the candidate information point is determined;

[0011] According to the relationship between the signal strength and distance of the Wi-Fi point, the signal strength confidence of each candidate information point is determined;

[0012] Determine the final matching information point data according to the semantic matching confidence and signal strength confidence;

[0013] Based on the final matched information point data, the benchmark point data is verified and cleaned.

[0014] Further, a search radius is determined according to the Wi-Fi point data, and candidate information points and names of the candidate information points are determined according to the search radius, including:

[0015] Get the user name of Wi-Fi point data, the latitude and longitude of the positioning reference point, and the signal;

[0016] Set an initial search radius based on Wi-Fi signal coverage, building density, and geographic location factors;

[0017] Taking the location of the Wi-Fi point as the center and based on the latitude and longitude of the positioning reference point, the initial search radius is used to determine the candidate information points in the data set;

[0018] Analyze the number and distribution density of Wi-Fi points within the initial search radius;

[0019] According to the distribution density, the search radius is adjusted, within the adjusted search radius, candidate information points are determined, and the names of the candidate information points are extracted.

[0020] Furthermore, the calculation formula for the semantic similarity between the candidate information point name and the Wi-Fi point user name is:

[0021]

[0022] Among them, SS represents the semantic similarity between the candidate information point name and the Wi-Fi point user name; Indicates the TF-IDF value of the i-th element in the candidate information point name; represents the TF-IDF value of the i-th element in the Wi-Fi point user name; n represents the total number of terms; i represents the index variable; N represents the candidate information point name; s represents the Wi-Fi point user name.

[0023] Furthermore, according to the relationship between the signal strength and the distance provided by the Wi-Fi point, the signal strength confidence of each candidate information point is determined, including:

[0024] According to the relationship between the signal strength and distance provided by the Wi-Fi point, the free path loss model is used to obtain the path loss, and the actual distance from each information point in the Wi-Fi area to the Wi-Fi point and the measured actual signal strength are calculated;

[0025] Estimate the theoretical signal strength received at each information point based on the signal attenuation model and the actual distance from each information point to the Wi-Fi point;

[0026] Comparing the theoretical signal strength with the actual signal strength to obtain a difference between the theoretical signal strength and the actual signal strength;

[0027] According to the difference between the theoretical signal strength and the actual signal strength, the signal strength confidence of each candidate information point is obtained.

[0028] Furthermore, the calculation formula of the free path loss model is:

[0029]

[0030] Where PL(d) represents the path loss; d represents the distance the signal propagates; f represents the frequency of the signal; represents a constant; c represents the speed of light; 20log 10 Indicates signal strength.

[0031] Furthermore, according to the semantic matching confidence and the signal strength confidence, the final matching information point data is determined, including:

[0032] Compare the semantic matching confidence and signal strength confidence of the candidate information point to obtain a comparison result;

[0033] According to the comparison result and the preset threshold, the weights of the semantic matching confidence and the signal strength confidence are adjusted to obtain adjusted weights;

[0034] According to the adjusted weight, the comprehensive confidence of each candidate information point is calculated;

[0035] Sort the candidate information points according to the comprehensive confidence, and compare the comprehensive confidence of each candidate information point to obtain the final candidate information point;

[0036] According to the final candidate information points, the final matching information point data is determined.

[0037] Further, according to the comparison result and the preset threshold, the weights of the semantic matching confidence and the signal strength confidence are adjusted to obtain adjusted weights, including:

[0038] Obtaining semantic matching confidence weight and signal strength confidence weight;

[0039] According to the semantic matching confidence weight and signal strength confidence weight, Make adjustments to obtain adjusted weights;

[0040] Among them, Ws (n+1) represents the confidence weight of the semantic match at time n+1; W s (n) represents the confidence weight of semantic matching at time n; α represents the learning rate; C s Indicates the confidence of semantic matching; C g Indicates the signal strength confidence.

[0041] In a second aspect, a data cleaning system for Wi-Fi geolocation reference points in IP positioning technology includes:

[0042] The acquisition module is used to acquire the benchmark point data and extract the Wi-Fi point data from the benchmark point data; determine the search radius according to the Wi-Fi point data, determine the candidate information point and the candidate information point name according to the search radius; calculate the semantic similarity between the candidate information point name and the Wi-Fi point user name; and determine the semantic matching confidence between the benchmark point data and the candidate information point according to the similarity;

[0043] The processing module is used to determine the signal strength confidence of each candidate information point according to the relationship between the signal strength and distance of the Wi-Fi point; determine the final matching information point data according to the semantic matching confidence and the signal strength confidence; and verify and clean the benchmark point data according to the final matching information point data.

[0044] According to a third aspect, a computing device includes:

[0045] one or more processors;

[0046] The storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method described.

[0047] In a fourth aspect, a computer-readable storage medium stores a program, and when the program is executed by a processor, the method described is implemented.

[0048] The above solution of the present invention includes at least the following beneficial effects:

[0049] Extracting Wi-Fi point data directly from benchmark point data improves the efficiency and accuracy of data processing, laying a solid foundation for subsequent positioning and cleaning work. By setting the search radius to determine candidate information points, this method narrows the search range, makes positioning more accurate, and reduces unnecessary data processing. By calculating the semantic similarity between the candidate information point name and the Wi-Fi point user name, the accuracy of data matching is enhanced, and mismatching caused by name differences is effectively avoided. Combining semantic matching confidence and signal strength confidence to determine the final matching information point data, this multi-dimensional evaluation method improves the reliability and stability of data matching. Verifying and cleaning the benchmark point data based on the final matching information point data effectively eliminates errors and abnormal data, improves data quality, and provides more accurate positioning results for subsequent positioning services. Users can obtain more accurate and reliable positioning services, providing users with higher quality positioning services. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flow chart of a method for cleaning data of Wi-Fi geographic positioning reference points in an IP positioning technology provided by an embodiment of the present invention.

[0051] Figure 2 It is a schematic diagram of a data cleaning system for Wi-Fi geographic positioning reference points in an IP positioning technology provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0053] like Figure 1 As shown, an embodiment of the present invention provides a data cleaning method for Wi-Fi geolocation reference points in IP positioning technology, and the method includes the following steps:

[0054] Step 1, obtaining benchmark point data, and extracting Wi-Fi point data from the benchmark point data;

[0055] Step 2: determine the search radius according to the Wi-Fi point data, and determine the candidate information points and the names of the candidate information points according to the search radius;

[0056] Step 3, calculating the semantic similarity between the candidate information point name and the Wi-Fi point user name;

[0057] Step 4: Determine the semantic matching confidence between the reference point data and the candidate information point based on the similarity;

[0058] Step 5: Determine the signal strength confidence of each candidate information point based on the relationship between the signal strength and distance of the Wi-Fi point;

[0059] Step 6, determining the final matching information point data according to the semantic matching confidence and the signal strength confidence;

[0060] Step 7: Verify and clean the reference point data based on the final matched information point data.

[0061] In the embodiment of the present invention, the benchmark point data is first obtained, which usually contains detailed information of the geographic location and the Wi-Fi point data associated therewith. The relevant information of the Wi-Fi point is further extracted from the benchmark point data to prepare for subsequent processing. Next, the system determines a search radius based on the extracted Wi-Fi point data. This search radius is used to define a range in the geographic space, so as to determine the candidate information points and their names that may be associated with this Wi-Fi point. The semantic similarity between each candidate information point name and the Wi-Fi point user name is calculated. This usually involves natural language processing technology, which is used to evaluate the similarity between two text strings. According to the calculated similarity, a semantic matching confidence is assigned to the matching relationship between the benchmark point data and each candidate information point. This confidence reflects the degree of association between the two. Considering that the strength of the Wi-Fi signal usually weakens with increasing distance, this feature is used to determine the signal strength confidence of each candidate information point. This helps to further screen and verify the candidate information points that are closer to the actual location of the Wi-Fi point. Combining the semantic matching confidence and signal strength confidence, the system can determine the candidate information point data that ultimately matches the Wi-Fi point best. Finally, based on the final matching information point data, the system verifies and cleans the original benchmark point data, including eliminating erroneous data, correcting inaccurate information, and updating outdated records to ensure the accuracy and completeness of the data.

[0062] By accurately extracting and verifying Wi-Fi point data, the accuracy of benchmark data can be significantly improved, and errors and misleading information can be reduced. The setting of search radius and signal strength considerations can help to more accurately locate Wi-Fi points and related information points, thereby optimizing the quality of location services. Through the automated data cleaning process, erroneous data can be efficiently eliminated, reducing the need for manual intervention, saving time and costs. Combining semantic similarity and signal strength confidence for multi-dimensional data verification enhances the reliability and robustness of matching results. Accurate benchmark data and optimized positioning services can enhance the user experience when using location-based services (LBS), such as navigation, map queries, etc.

[0063] In a preferred embodiment of the present invention, a search radius is determined according to Wi-Fi point data, and candidate information points and names of the candidate information points are determined according to the search radius, including:

[0064] Get the user name of Wi-Fi point data, the latitude and longitude of the positioning reference point, and the signal;

[0065] Set an initial search radius based on Wi-Fi signal coverage, building density, and geographic location factors;

[0066] Taking the location of the Wi-Fi point as the center and based on the latitude and longitude of the positioning reference point, the initial search radius is used to determine the candidate information points in the data set;

[0067] Analyze the number and distribution density of Wi-Fi points within the initial search radius;

[0068] According to the distribution density, the search radius is adjusted, within the adjusted search radius, candidate information points are determined, and the names of the candidate information points are extracted.

[0069] In an embodiment of the present invention, Wi-Fi point data is obtained, including user name, longitude and latitude of the positioning reference point, and signal strength. These data are the basis for subsequent processing and analysis. Based on factors such as the coverage of the Wi-Fi signal, building density, and geographical location, the system will set an initial search radius. This radius will serve as the starting point for subsequent searches. Centered on the location of the Wi-Fi point and combined with the longitude and latitude of the positioning reference point, the system uses the initial search radius to search in the data set to determine a batch of candidate information points. Analyze the number and distribution density of Wi-Fi points within the initial search radius. Based on the distribution density of the Wi-Fi points, the system will adjust the search radius. Within the adjusted search radius, the system re-determines the candidate information points and extracts the names of these points.

[0070] By dynamically adjusting the search radius according to the distribution density of Wi-Fi points, it can adapt to different environments and scenarios more flexibly, improving the accuracy and efficiency of the search. Reasonable search radius settings can reduce unnecessary search space, thereby speeding up the search and improving data processing efficiency. By adjusting the search radius by analyzing the number and distribution density of Wi-Fi points, candidate information points associated with Wi-Fi points can be more accurately locked, improving the accuracy of data matching. Determining candidate information points within the adjusted search radius and extracting their names helps to eliminate irrelevant or erroneous information related to Wi-Fi points, retaining accurate and useful data, and thus improving data quality. This method takes into account a variety of factors (such as Wi-Fi signal coverage, building density, etc.), making it suitable not only for densely populated urban areas, but also for different geographical environments such as suburbs or rural areas.

[0071] In a preferred embodiment of the present invention, the calculation formula for the semantic similarity between the candidate information point name and the Wi-Fi point user name is:

[0072]

[0073] Among them, SS represents the semantic similarity between the candidate information point name and the Wi-Fi point user name; Represents the TF-IDF value of the i-th term in the candidate information point name; represents the TF-IDF value of the i-th term in the Wi-Fi point user name; n represents the total number of terms; i represents the index variable; N represents the candidate information point name; s represents the Wi-Fi point user name.

[0074] In an embodiment of the present invention, first, the TF-IDF value of each term in the candidate information point name N and the Wi-Fi point user name s is calculated. TF (term frequency) indicates the frequency of a word appearing in a text, while IDF (inverse document frequency) reflects the rarity of the word in all document sets. Combined, the TF-IDF value can represent the importance and uniqueness of a word in a specific text. These TF-IDF values ​​are used to construct two vectors, representing the candidate information point name N and the Wi-Fi point user name s, respectively. Each dimension of the vector corresponds to a term i, and the value of the dimension is the TF-IDF value of the term. Then, the cosine similarity of the two vectors is calculated. This similarity value is between 0 and 1, and the closer the value is to 1, the more semantically similar the two texts are. Finally, the system evaluates the semantic matching similarity SS between the candidate information point name N and the Wi-Fi point user name s based on the calculated similarity value.

[0075] By combining TF-IDF weighting and cosine similarity, the semantic similarity between the candidate information point name N and the Wi-Fi point user name s can be measured more accurately, taking into account the word frequency and word importance, thereby improving the accuracy of matching. It can flexibly handle texts of different lengths without truncating or padding the text, maintaining the integrity of the original text information. By calculating the semantic similarity, the candidate information points that are highly related to the Wi-Fi point user name can be quickly screened out, reducing unnecessary data comparisons and improving the efficiency of data matching. In the data cleaning process, the candidate information points that do not semantically match the Wi-Fi point user name can be accurately identified and eliminated, thereby improving the effect and quality of data cleaning.

[0076] In a preferred embodiment of the present invention, the signal strength confidence of each candidate information point is determined according to the relationship between the signal strength and the distance provided by the Wi-Fi point, including:

[0077] According to the relationship between the signal strength and distance provided by the Wi-Fi point, the free path loss model is used to obtain the path loss, and the actual distance from each information point in the Wi-Fi area to the Wi-Fi point and the measured actual signal strength are calculated;

[0078] Estimate the theoretical signal strength received at each information point based on the signal attenuation model and the actual distance from each information point to the Wi-Fi point;

[0079] Comparing the theoretical signal strength with the actual signal strength to obtain a difference between the theoretical signal strength and the actual signal strength;

[0080] According to the difference between the theoretical signal strength and the actual signal strength, the signal strength confidence of each candidate information point is obtained.

[0081] In an embodiment of the present invention, the path loss is calculated by using the relationship between the signal strength and distance provided by the Wi-Fi point and combining the free path loss model. This model takes into account the attenuation of the signal when it propagates in space. Subsequently, the system calculates the actual distance from each information point to the Wi-Fi point in the Wi-Fi area based on the location information of the Wi-Fi point, and measures the actual signal strength at these points. The signal attenuation model is used to estimate the signal strength that should be received theoretically at each information point in combination with the actual distance from each information point to the Wi-Fi point. This estimation process is based on the law of signal attenuation with distance. The system then compares the estimated theoretical signal strength with the actually measured signal strength and calculates the difference between the two. This difference value reflects the degree of deviation between the actual signal and the theoretical expectation. Finally, based on the difference between the theoretical signal strength and the actual signal strength, the system assigns a signal strength confidence value to each candidate information point. This confidence value is used to quantify the reliability of the actual signal strength. The smaller the difference, the higher the confidence.

[0082] By comparing the theoretical signal strength with the actual signal strength, the location accuracy of each candidate information point can be more accurately evaluated, thereby improving the indoor positioning accuracy based on Wi-Fi signals. Signal strength confidence provides a quantitative way to evaluate the reliability of Wi-Fi signals, which helps to identify and eliminate signal anomalies caused by multipath effects, interference or other factors. In complex indoor environments, signal strength may be affected by many factors. By calculating the signal strength confidence, the system can handle these uncertainties more robustly and improve the stability and reliability of the system. Accurate signal strength confidence evaluation can help the system better select reliable Wi-Fi signals for connection, thereby improving user satisfaction and experience when using Wi-Fi services. Signal strength confidence can be used as an important input parameter of intelligent decision-making systems, such as in applications such as path planning, resource allocation or fault prediction, to help the system make more intelligent decisions.

[0083] In a preferred embodiment of the present invention, the calculation formula of the free path loss model is:

[0084]

[0085] Where PL(d) represents the path loss; d represents the distance the signal propagates; f represents the frequency of the signal; represents a constant; c represents the speed of light; 20log 10 Indicates signal strength.

[0086] In an embodiment of the present invention, when the signal propagation distance d propagates in space, its intensity decreases as the distance increases. This is because electromagnetic waves diffuse during the propagation process, covering larger and larger areas, resulting in a decrease in energy per unit area. In the free path loss model, the signal propagation distance is one of the main factors affecting the path loss. The longer the distance, the greater the signal attenuation. The signal frequency f will also affect its propagation loss in space. Generally speaking, the higher the frequency of the signal, the faster it attenuates during the propagation process. This is because high-frequency signals have shorter wavelengths and are more easily affected by obstacles. At the same time, media such as air also absorb high-frequency signals more strongly. Therefore, at the same propagation distance, high-frequency signals usually experience greater path loss than low-frequency signals. Constant factor Related to the speed of light c, it reflects the basic characteristics of signal propagation in space. The speed of light is the upper limit of signal propagation speed and is a constant value in a vacuum. It reflects the geometric characteristics of signal diffusion in space, which is related to the spherical diffusion of electromagnetic waves. Specifically, as the signal propagates outward from the point source, its intensity decreases in proportion to the increase in the surface area of ​​the sphere. Path loss PL(d) refers to the attenuation of the signal intensity during the process of propagating from the transmitting point to the receiving point. It is a relative value. Path loss is proportional to the logarithm of the signal propagation distance, the logarithm of the signal frequency, and the logarithm of the constant factor. 20log 10 Indicates signal strength. This is because the strength range of wireless signals is very large, and logarithms can more conveniently represent and process these values.

[0087] The free path loss model can accurately predict the strength attenuation of wireless signals on a straight propagation path. By inputting parameters such as signal propagation distance and signal frequency, the model can calculate the path loss, thereby helping engineers understand the attenuation of signals during propagation. Designers of wireless communication systems can better plan network layouts. By predicting the path loss at different distances and frequencies, the optimal locations of base stations and receivers can be determined to maximize signal coverage and transmission efficiency. The model provides important guidance for parameter settings of wireless communication systems. For example, when selecting a signal frequency, the model can be used to predict the path loss at different frequencies, so that the most appropriate frequency can be selected to avoid excessive signal attenuation. By understanding the path loss, targeted measures can be taken to reduce signal attenuation, such as increasing the signal transmission power, adjusting the antenna direction, or using signal enhancement equipment. These measures help improve communication quality and ensure stable signal transmission.

[0088] In a preferred embodiment of the present invention, the final matched information point data is determined according to the semantic matching confidence and the signal strength confidence, including:

[0089] Compare the semantic matching confidence and signal strength confidence of the candidate information point to obtain a comparison result;

[0090] According to the comparison result and the preset threshold, the weights of the semantic matching confidence and the signal strength confidence are adjusted to obtain adjusted weights;

[0091] According to the adjusted weight, the comprehensive confidence of each candidate information point is calculated;

[0092] Sort the candidate information points according to the comprehensive confidence, and compare the comprehensive confidence of each candidate information point to obtain the final candidate information point;

[0093] According to the final candidate information points, the final matching information point data is determined.

[0094] In an embodiment of the present invention, the semantic matching confidence and signal strength confidence of each candidate information point are compared. According to the comparison result and the preset threshold, the system adjusts the weights of the semantic matching confidence and the signal strength confidence. If the semantic matching confidence of a candidate information point is much higher than its signal strength confidence, or vice versa, the system will adjust the weights of the two accordingly to reflect the importance of different confidences in the comprehensive evaluation. Using the adjusted weights, the system calculates the comprehensive confidence of each candidate information point. This comprehensive confidence is a weighted average that combines the confidence of semantic matching and signal strength to provide a comprehensive reliability assessment for each candidate information point. According to the comprehensive confidence, the system sorts the candidate information points. By comparing the comprehensive confidence of each candidate information point, the system determines the final candidate information point. This point is considered to be the most reliable information point after comprehensive consideration of semantic matching and signal strength. Based on the final candidate information point, the system determines the final matching information point data. These data will be used for subsequent applications or services, such as indoor navigation, location services, etc.

[0095] By comprehensively considering the semantic matching confidence and signal strength confidence, this method can match information points more accurately and reduce the possibility of mismatching. Accurate matching can provide a better user experience, such as providing more accurate route guidance in indoor navigation. Comprehensive confidence can provide an important basis for intelligent decision-making systems, such as helping the system make more reasonable choices in applications such as selecting the best path and recommending points of interest. By adjusting the weights of semantic matching confidence and signal strength confidence, this method can adapt to different scenarios and needs, improving the flexibility and applicability of the system.

[0096] In a preferred embodiment of the present invention, the weights of the semantic matching confidence and the signal strength confidence are adjusted according to the comparison result and the preset threshold value to obtain the adjusted weights, including:

[0097] Obtaining semantic matching confidence weight and signal strength confidence weight;

[0098] According to the semantic matching confidence weight and signal strength confidence weight, Make adjustments to obtain adjusted weights;

[0099] Among them, W s (n+1) represents the confidence weight of the semantic match at time n+1; W s (n) represents the confidence weight of semantic matching at time n; α represents the learning rate; C s Indicates the confidence of semantic matching; C g Indicates the signal strength confidence.

[0100] In this embodiment of the present invention, the semantic matching confidence C is obtained.s and signal strength confidence C g The two confidences represent the accuracy of semantic matching and the reliability of signal strength respectively. The semantic matching confidence weight W is dynamically adjusted using the given formula s (n). This formula takes into account the weight of n at the current moment, the learning rate α, and the semantic matching confidence C s and signal strength confidence C g Through continuous iterative updates, the weights will gradually tend to reflect the relative importance of semantic matching and signal strength. Specifically, if the semantic matching confidence is higher than the signal strength confidence, the fractional part in the formula will be positive, thereby increasing the semantic matching confidence weight C s On the contrary, if the signal strength confidence is higher, the semantic matching confidence weight C will be reduced. s The learning rate α controls the magnitude of weight adjustment to prevent the weights from changing too quickly or too slowly.

[0101] By continuously adjusting the weights, the system can dynamically adapt to the changes in the importance of semantic matching and signal strength in different environments. This helps the system maintain high performance in various complex environments. By comprehensively considering semantic matching and signal strength, the system can more accurately evaluate the credibility and importance of information. This helps reduce misjudgments and improve the accuracy of decisions. The adjustability of the learning rate α enables the system to adapt to environmental changes quickly or slowly according to actual needs. This increases the flexibility and configurability of the system.

[0102] For example, in specific applications, the method of data cleaning based on Wi-Fi geolocation benchmark points is mainly to collect benchmark point data and extract Wi-Fi point information from it; calculate the matching degree between Wi-Fi points and information points through feature semantic matching of Wi-Fi points and information point data, and determine a list of candidate information points that may match Wi-Fi; select the information point that best matches the Wi-Fi point from the list of candidate information points; and use the data of the best matching information point to verify and clean the benchmark point data. In this way, the accuracy of the benchmark point data can be improved. The specific implementation steps are as follows:

[0103] 1) Collect and organize benchmark point data, which mainly includes {IP, Wi-Fi information, Address, longitude and latitude of the positioning point}; considering the dynamic nature of IP and the irregularity of its correspondence with the geographic location, we extract {Wi-Fi name SSID, signal, longitude and latitude of the positioning point} as Wi-Fi point data, and match the corresponding geographic location based on the characteristics of Wi-Fi point data.

[0104] 2) Call multiple maps, set map interface parameters, use the reference point location as the retrieval center, adaptively set the circular area radius r according to the Wi-Fi point data signal strength, search for information points around the Wi-Fi point in the circular area, and if no information point is found, the return value is None, retain the original data, and determine the searched information point as a candidate information point. Select the surrounding candidate information points that match the Wi-Fi point in the map, and store their {information point name, geographic location (longitude and latitude), category} in the candidate information point list.

[0105] In this process, the main signal strength and distance relationship model we use is the Free-Space Path Loss (FSPL) model, so that the specified area can contain most of the information points covered by the Wi-Fi signal within the circular area.

[0106] 3) For Wi-Fi point and information point data, the most direct feature is the name of both. Therefore, in order to select the most matching information point, the semantic similarity between SSID and information point name is a major criterion. Traverse the candidate information points, collect SSIDs in Wi-Fi point data, collect information point names, and pre-process SSIDs and information point names, such as removing special characters, converting to lowercase, etc. Select an appropriate pre-trained BERT model, and load the model and the corresponding tokenizer. If there are no candidate information points, keep the original data.

[0107] 4) In this process, the data needs to be processed in a way that is suitable for the BERT model. The extracted SSID and information point name are encoded through the BERT word segmenter and converted into an input format acceptable to the model. The SSID and information point name are encoded through the BERT word segmenter and converted into an input format acceptable to the model. BERT is used to calculate the semantic similarity between the SSID and the information point name, and the cosine similarity is used for measurement. The calculated similarity is used as the value of the semantic matching confidence.

[0108] 5) At the same time, for the Wi-Fi point data of the information point, it includes the Wi-Fi signal strength. The signal strength often has a corresponding relationship with the distance, so the signal strength confidence also becomes an important metric. Traverse the Wi-Fi point data and information point data, collect the Wi-Fi point data, including the signal strength and location of each Wi-Fi point, and collect the location data of the information point. According to the free path loss model (FSPL) formula and the distance d from each information point to the Wi-Fi point, according to the formula Calculate the path loss PL(d), and connect to other known points for the Wi-Fi to obtain the signal strength and estimate the Wi-Fi transmission power P t , then according to the formula P r (d) = P t -PL(d) calculates the signal strength P of the Wi-Fi signal received at each information point r (d) Compare the calculated signal strength P r (d) The signal strength confidence level is determined by comparing the actual received signal strength.

[0109] For Wi-Fi points and information point data, the semantic matching confidence of the name is the most intuitive measurement method; while the signal strength confidence is a reliable measurement method. When selecting the information point that best matches the Wi-Fi point, two aspects need to be fully considered, so our method chooses to combine the two measurement methods. We selected an adaptive method to adjust the weights of the two. According to the two confidences obtained, if the semantic matching confidence is higher, its weight is increased; if the signal strength confidence is higher, its weight is increased, and the weights of the two are adaptively adjusted. The advantage of this method is that it can dynamically optimize the matching results, comprehensively consider multiple factors, improve the matching accuracy, and enhance the robustness of the system. Its strong adaptability enables it to adapt to different application scenarios and needs, and it can work effectively in both high-density urban environments and open suburban environments. In addition, by improving the accuracy and robustness of matching, users can obtain more reliable and accurate positioning services, thereby optimizing the overall user experience.

[0110] 6) Traverse the list of candidate information points of the Wi-Fi point, and add the confidence of each information point we finally obtain to its list; if the list is not empty, select the information point with the highest confidence as the most matching information point. If the list is empty, keep the original reference point data corresponding to the Wi-Fi point.

[0111] 7) Compare the geographical location of the Wi-Fi corresponding information point with the original reference point data to determine whether the reference point is accurate. If inaccurate, modify the data based on the information point.

[0112] Due to the propagation characteristics of Wi-Fi signals and the attenuation law of signal strength, Wi-Fi geolocation is more accurate. If the Wi-Fi point has the best matching information point, the result we get is considered accurate. The positioning information corresponding to this Wi-Fi in the corresponding benchmark point data is compared with the information point data. If they are the same, it means that the benchmark point data is accurate, if they are different, it is inaccurate. In addition, if the Wi-Fi point does not have the best matching information point, it means that the specific positioning cannot be obtained using Wi-Fi geolocation, which may be through network positioning and other methods. We regard the original benchmark point data as accurate data.

[0113] The present invention is further described in conjunction with a real data set. Table 1 below shows the collected benchmark data.

[0114] Collect and organize the benchmark point data, and extract the Wi-Fi point data. Table 1 is the benchmark point data, and we extract the Wi-Fi SSID, signal, Lon, and Lat from it to get Table 2.

[0115] The third-party map positioning module is called to search for the Wi-Fi points in Table 2, search for information point data around the Wi-Fi points, obtain different information point data from different sources, and sort them into Table 3, which shows the matching information point data.

[0116] The signal strength confidence of Wi-Fi points and information points is calculated by performing feature semantic matching confidence calculation through Wi-Fi points and information point data, and the final confidence is added to the candidate information point list, as shown in Table 4. According to Table 4, we obtain the semantic matching confidence and signal strength confidence of the three candidate information points respectively.

[0117] Select the information point that best matches the Wi-Fi point from the candidate information point list. Based on the semantic matching confidence and signal strength confidence, we select the information point with the highest comprehensive confidence as the best match. Therefore, for the Wi-Fi point, "A Park" has the highest semantic matching confidence and signal strength confidence. Therefore, for this benchmark point, we select "A Park" as the best match information point.

[0118] The data of the most matching information point is used to verify and clean the benchmark point data. According to the comparison between the most matching information point and the benchmark data, we cleaned the data of the benchmark point and obtained the final Table 5.

[0119] After the above steps, we obtained the benchmark point data in Table 5. Through the data cleaning method of Wi-Fi geolocation benchmark points, we used the surrounding more matching information points (the second data) to clean the benchmark point data, filter out inaccurate parts, and improve the accuracy of the data.

[0120] Table 1. Benchmark data

[0121]

[0122] Table 2. Wi-Fi point data

[0123]

[0124]

[0125] Table 3. Information point data

[0126] poiname Lon Lat Weiha stir-fried vegetables 15 24 Park A 16 21 Square B 10 24

[0127] Table 4. Candidate point information table

[0128]

[0129] Table 5. Final benchmark data

[0130] IP Address Wi-Finame Lon Lat 2.2.2.2 Park A Weihaigongyuan 16 21

[0131] like Figure 2 As shown, an embodiment of the present invention further provides a data cleaning system 20 for Wi-Fi geographic positioning reference points in IP positioning technology, comprising:

[0132] The acquisition module 21 is used to acquire the benchmark point data and extract the Wi-Fi point data from the benchmark point data; determine the search radius according to the Wi-Fi point data, determine the candidate information point and the candidate information point name according to the search radius; calculate the semantic similarity between the candidate information point name and the Wi-Fi point user name; and determine the semantic matching confidence between the benchmark point data and the candidate information point according to the similarity;

[0133] The processing module 22 is used to determine the signal strength confidence of each candidate information point according to the relationship between the signal strength and distance of the Wi-Fi point; determine the final matching information point data according to the semantic matching confidence and the signal strength confidence; and verify and clean the benchmark point data according to the final matching information point data.

[0134] It should be noted that the system is a system corresponding to the above method, and all implementation methods in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0135] The embodiment of the present invention further provides a computing device, comprising: a processor, a memory storing a computer program, wherein when the computer program is executed by the processor, the method described above is executed. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0136] The embodiment of the present invention also provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the method described above. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0137] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0138] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0139] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0140] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0142] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0143] In addition, it should be noted that in the apparatus and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. Moreover, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order, and some steps can be performed in parallel or independently of each other. For those of ordinary skill in the art, it is understood that all or any steps or components of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or a network of computing devices in hardware, firmware, software or a combination thereof, which can be achieved by those of ordinary skill in the art using basic programming skills after reading the description of the present invention.

[0144] Therefore, the purpose of the present invention can also be achieved by running a program or a group of programs on any computing device. The computing device can be a well-known general device. Therefore, the purpose of the present invention can also be achieved by simply providing a program product containing a program code that implements the method or device. That is to say, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any well-known storage medium or any storage medium developed in the future. It should also be pointed out that in the device and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. In addition, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order. Some steps can be performed in parallel or independently of each other.

[0145] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for cleaning data of Wi-Fi geographic positioning reference points in IP positioning technology, characterized in that: The method comprises: Obtain benchmark point data, and extract Wi-Fi point data from the benchmark point data; Determine a search radius based on the Wi-Fi point data, and determine candidate information points and names of the candidate information points based on the search radius; Calculate the semantic similarity between the candidate information point name and the Wi-Fi point user name; According to the similarity, the semantic matching confidence between the reference point data and the candidate information point is determined; According to the relationship between the signal strength and distance of the Wi-Fi point, the signal strength confidence of each candidate information point is determined; Determine the final matching information point data according to the semantic matching confidence and signal strength confidence; Verify and clean the benchmark point data based on the final matched information point data; According to the semantic matching confidence and signal strength confidence, the final matching information point data is determined, including: Compare the semantic matching confidence and signal strength confidence of the candidate information point to obtain a comparison result; According to the comparison result and the preset threshold, the weights of the semantic matching confidence and the signal strength confidence are adjusted to obtain adjusted weights; According to the adjusted weight, the comprehensive confidence of each candidate information point is calculated; Sort the candidate information points according to the comprehensive confidence, and compare the comprehensive confidence of each candidate information point to obtain the final candidate information point; Determine the final matching information point data according to the final candidate information points; According to the comparison result and the preset threshold, the weights of the semantic matching confidence and the signal strength confidence are adjusted to obtain adjusted weights, including: Obtaining semantic matching confidence weight and signal strength confidence weight; According to the semantic matching confidence weight and signal strength confidence weight, Make adjustments to obtain adjusted weights; in, Represents the confidence weight of semantic matching at time n+1; Represents the confidence weight of semantic matching at time n; represents the learning rate; Indicates the confidence of semantic matching; Indicates the signal strength confidence.

2. The data cleaning method of Wi-Fi geographic positioning reference point in IP positioning technology according to claim 1, characterized in that: According to the Wi-Fi point data, determine the search radius, and according to the search radius, determine the candidate information points and the names of the candidate information points, including: Get the user name of Wi-Fi point data, the latitude and longitude of the positioning reference point, and the signal; Set an initial search radius based on Wi-Fi signal coverage, building density, and geographic location factors; Taking the location of the Wi-Fi point as the center and based on the latitude and longitude of the positioning reference point, the initial search radius is used to determine the candidate information points in the data set; Analyze the number and distribution density of Wi-Fi points within the initial search radius; According to the distribution density, the search radius is adjusted, within the adjusted search radius, candidate information points are determined, and the names of the candidate information points are extracted.

3. The data cleaning method of Wi-Fi geographic positioning reference point in IP positioning technology according to claim 2 is characterized in that: The calculation formula for the semantic similarity between the candidate information point name and the Wi-Fi point user name is: ； in Indicates the semantic similarity between the candidate information point name and the Wi-Fi point user name; Indicates the first TF-IDF value of elements; Indicates the first TF-IDF value of elements; Indicates the total number of terms; Represents an index variable; Indicates the name of the candidate information point; s indicates the Wi-Fi point user name.

4. The data cleaning method of Wi-Fi geographic positioning reference points in the IP positioning technology according to claim 3 is characterized in that: Based on the relationship between the signal strength and distance provided by the Wi-Fi point, the signal strength confidence of each candidate information point is determined, including: According to the relationship between the signal strength and distance provided by the Wi-Fi point, the free path loss model is used to obtain the path loss, and the actual distance from each information point in the Wi-Fi area to the Wi-Fi point and the measured actual signal strength are calculated; Estimate the theoretical signal strength received at each information point based on the signal attenuation model and the actual distance from each information point to the Wi-Fi point; Comparing the theoretical signal strength with the actual signal strength to obtain a difference between the theoretical signal strength and the actual signal strength; According to the difference between the theoretical signal strength and the actual signal strength, the signal strength confidence of each candidate information point is obtained.

5. The data cleaning method of Wi-Fi geographic positioning reference point in IP positioning technology according to claim 4 is characterized in that: The calculation formula of the free path loss model is: ； in, represents the path loss; d represents the distance the signal propagates; Indicates the frequency of the signal; represents a constant; represents the speed of light; Indicates signal strength.

6. A data cleaning system for Wi-Fi geolocation reference points in IP positioning technology, characterized in that: include: An acquisition module, used to acquire reference point data and extract Wi-Fi point data from the reference point data; Determine a search radius based on the Wi-Fi point data, and determine candidate information points and names of the candidate information points based on the search radius; Calculate the semantic similarity between the candidate information point name and the Wi-Fi point user name; based on the similarity, determine the semantic matching confidence between the benchmark point data and the candidate information point; A processing module, used to determine the signal strength confidence of each candidate information point according to the relationship between the signal strength and distance of the Wi-Fi point; Determine the final matching information point data based on the semantic matching confidence and signal strength confidence; verify and clean the reference point data based on the final matching information point data; According to the semantic matching confidence and signal strength confidence, the final matching information point data is determined, including: Compare the semantic matching confidence and signal strength confidence of the candidate information point to obtain a comparison result; According to the comparison result and the preset threshold, the weights of the semantic matching confidence and the signal strength confidence are adjusted to obtain adjusted weights; According to the adjusted weight, the comprehensive confidence of each candidate information point is calculated; Sort the candidate information points according to the comprehensive confidence, and compare the comprehensive confidence of each candidate information point to obtain the final candidate information point; Determine the final matching information point data according to the final candidate information points; According to the comparison result and the preset threshold, the weights of the semantic matching confidence and the signal strength confidence are adjusted to obtain adjusted weights, including: Obtaining semantic matching confidence weight and signal strength confidence weight; According to the semantic matching confidence weight and signal strength confidence weight, Make adjustments to obtain adjusted weights; in, Represents the confidence weight of semantic matching at time n+1; Represents the confidence weight of semantic matching at time n; represents the learning rate; Indicates the confidence of semantic matching; Indicates the signal strength confidence.

7. A computing device, characterized in that include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method as claimed in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, which implements the method according to any one of claims 1 to 5 when executed by a processor.

Images