An intelligent indexing method for the full optical fiber path

Abstract

The present invention discloses an intelligent indexing method for the full path of an optical fiber, which relates to the technical field of optical link indexing. The present invention includes: Step101: Construct a sampling channel in a real-time incremental trajectory database, dock data items, and perform node collection. Among them, the data items include: object identifier, timestamp, and spatial location; manage the latest trajectory nodes of all objects with a hash table; Step102: Adopt a data cleaning method to correct the recognizable errors in the data obtained from the sampling channel. By improving the indexing algorithm of the existing R-tree, the present invention realizes the complete and ordered indexing of nodes, analyzes the interconnection relationship, connection order, and dependency relationship between nodes according to the path, geographical information, and various constraint conditions, realizes the indexing of optical link nodes, can greatly improve the query effect, remove error data, improve the accuracy of indexing values, adapt to the continuous update and iteration of data, and meet the market demand.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical link indexing, and particularly relates to an intelligent indexing method for the full path of optical fibers. Background Art

[0002] An optical link is a link that uses optical fiber communication technology to transmit voice, image, and data signals. Its principle is to use various optical methods to achieve network interconnection levels, and various free-space optical interconnection networks with different topological structures can be obtained. Generally, it consists of an optical transmitter, optical fiber, optical receiver, and other necessary optical devices. An R-tree is a tree-like data structure used for storing spatial data. For example, it can create indexes for multi-dimensional data such as geographical locations, rectangles, and polygons, and can be used to store spatial information on maps. Polygons used to construct streets, buildings, lake edges, and coastlines on maps can also be created using an R-tree. The R-tree can also be used to accelerate the nearest neighbor search using various distance metrics including the great circle distance;

[0003] However, it still has the following drawbacks in actual use:

[0004] 1. Existing optical link indexing algorithms have limited management capabilities for spatio-temporal data. As the update speed of big data becomes faster and faster, the efficiency of index queries becomes lower and lower, the content is not rich, and it is difficult to meet market demands;

[0005] 2. Existing optical link indexing algorithms are difficult to solve the problem of high maintenance costs brought by real-time updates and occupy a large amount of storage space.

[0006] Therefore, the existing ones cannot meet the needs in actual use, so there is an urgent need for improved technologies in the market to solve the above problems. Summary of the Invention

[0007] The purpose of the present invention is to provide an intelligent indexing method for the full path of optical fibers. By improving the indexing algorithm of the existing R-tree, complete and ordered indexing of nodes is achieved, and the mutual connection relationships, connection orders, and dependency relationships between nodes are analyzed according to the path, geographical information, and various constraint conditions, realizing the indexing of optical link nodes, regularly releasing the update cache, and solving the problems that existing optical link indexing algorithms have limited management capabilities for spatio-temporal data. As the update speed of big data becomes faster and faster, the efficiency of index queries becomes lower and lower, the content is not rich, it is difficult to meet market demands, it is difficult to solve the problem of high maintenance costs brought by real-time updates, and it occupies a large amount of storage space.

[0008] To solve the above technical problems, the present invention is realized through the following technical solutions:

[0009] The present invention is an intelligent indexing method for the full path of optical fibers, including:

[0010] Step101: Construct a sampling channel in the real-time incremental trajectory database, dock data items, and perform node collection. Among them, the data items include: object identifier, timestamp, and spatial location.

[0011] Manage the latest trajectory nodes of all objects with a hash table.

[0012] Step102: Adopt a data cleaning method to correct the recognizable errors in the data obtained from the sampling channel. Among them, it includes: checking the reasonable value range and mutual relationship of each variable, checking whether the data meets the requirements, and finding data that exceeds the normal range, is logically unreasonable, or is contradictory; estimating, deleting the whole case, deleting variables, and pairwise deleting the invalid values and missing values in the data.

[0013] Step103: Concentrate the continuous sampling points of the obtained objects and store them in groups on the trajectory nodes.

[0014] Step104: Set the sampling threshold of the trajectory node and judge whether the continuous sampling points on the trajectory node exceed the threshold during the real-time update.

[0015] Step105: If it does not exceed the threshold, continue sampling according to the predetermined settings.

[0016] Step106: If it exceeds the threshold, insert an R-tree on the trajectory node that exceeds the threshold, and build a new trajectory node to receive new sampling points.

[0017] Step107: Perform pre-sorting of the index records according to the path, geographical information, and constraint conditions of the object.

[0018] Step108: Analyze the mutual connection relationship, connection order, and dependency relationship between the sampling nodes. According to the analysis results of the sampling nodes, update the acquisition method of the sampling nodes to complete the final sorting of the index records.

[0019] Step109: Set a time threshold in the R-tree. After the set time threshold is triggered, clean the R-tree cache data and release the node data that has not been accessed for the time threshold since the current time from the cache.

[0020] Furthermore, the trajectory nodes in the step Step101 only store the continuous sampling points of a single object.

[0021] Furthermore, the trajectory nodes in the step Step101 serve as the leaf nodes of the R-tree. Adopt a new node insertion algorithm to insert its index items into the upper layer of the leaf node layer, and optimize the R-tree structure by using node selection and node splitting sub-algorithms.

[0022] Further, the leaf nodes are used to record object identifiers and continuous trajectory sampling points, while the non-leaf nodes record child node information, including: the spatio-temporal range of the child nodes.

[0023] Further, the process of inserting into the R-tree in step Step106 includes: node selection and node splitting;

[0024] Among them, the number of operations for node selection is equal to the number of leaf nodes, and the number of operations for node splitting is equal to the number of non-leaf nodes.

[0025] Further, the calculation formula for the evaluation value EVAL of the R-tree in step Step106 is:

[0026]

[0027] That is, the spatio-temporal evaluation value EVAL: the product of the average value of the intervals of the node on the N spatial coordinate axes and the interval T of the time coordinate axis. The product of the average value of the intervals of the node on the N spatial coordinate axes and the interval T of the time coordinate axis.

[0028] Further, the implementation method for the final sorting of the index records in step Step108 includes the following steps:

[0029] Step201: Extract the key features of the trajectory nodes, perform screening after extraction, and send them to the temporary storage area after compression;

[0030] Step202: Scan according to the object attributes, extract the trajectory nodes in the temporary storage area, and perform docking;

[0031] Step203: Divide the extracted trajectory nodes into time slices, and create corresponding R-trees according to each time slice.

[0032] Step204: Scan all the trajectory node data, improve the R-tree construction process, and finally construct the data index data.

[0033] Further, the system composition of the data cleaning in step Step102 includes:

[0034] An import module, used to receive all imported data and perform format conversion;

[0035] A marking module, used to analyze the recognizable error values in the data and perform marking;

[0036] An extraction module, used to extract the marked error values and perform estimation;

[0037] A processing module, configured to remove error values, estimated invalid values, and missing values from data;

[0038] An export module, configured to adapt the format of the cleaned values and then export them to a sending end.

[0039] The present invention has the following beneficial effects:

[0040] 1. By improving the index algorithm of the existing R-tree, the present invention realizes a complete and ordered index of nodes, analyzes the mutual connection relationships, connection orders, and dependency relationships between nodes according to paths, geographical information, and various constrained conditions, realizes the indexing of optical link nodes, can greatly improve the query effect, remove error data, improve the accuracy of index values, adapt to the continuous update and iteration of data, and meet the market demand.

[0041] 2. By setting a time threshold, after the set time threshold is triggered, the cache data is cleared, and the updated cache is regularly released, reducing the originally occupied storage space, improving the data throughput, and effectively solving the problem of high maintenance costs brought by real-time updates. Description of the Drawings

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0043] FIG. 1 is a schematic flow chart of an intelligent indexing method for an optical fiber full path;

[0044] FIG. 2 is a schematic flow chart of an implementation method for the final sorting of index records in the present invention;

[0045] FIG. 3 is a schematic framework diagram of a data cleaning system in the present invention;

[0046] FIG. 4 is a schematic architecture demonstration diagram of an R-tree in the present invention;

[0047] FIG. 5 is a schematic comparison diagram of the improved query efficiency and the existing query efficiency in the present invention.

[0048] In the drawings, the component lists represented by the reference numerals are as follows:

[0049] 1. An import module; 2. A marking module; 3. An extraction module; 4. A processing module; 5. An export module. Detailed Embodiments

[0050] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention.

[0051] Embodiment 1

[0052] An intelligent indexing method for the full path of an optical fiber, as shown in FIGS. 1 and 4, includes:

[0053] Step101: Construct a sampling channel in the real-time incremental trajectory database, dock data items, and expand node collection. Among them, the data items include: object identifier, timestamp, and spatial location;

[0054] Manage the latest trajectory nodes of all objects with a hash table;

[0055] Step102: Adopt a data cleaning method to correct the recognizable errors in the data obtained from the sampling channel. Among them, it includes: checking the reasonable value range and mutual relationship of each variable, checking whether the data meets the requirements, and finding data that exceeds the normal range, is logically unreasonable, or is contradictory; estimating, deleting the whole case, deleting variables, and pairwise deleting the invalid values and missing values in the data;

[0056] Step103: Concentrate the continuous sampling points of the obtained objects and store them in groups on the trajectory nodes;

[0057] Step104: Set the sampling threshold of the trajectory node, and judge whether the continuous sampling points on the trajectory node exceed the threshold during the real-time update process;

[0058] Step105: If the threshold is not exceeded, continue sampling according to the predetermined settings;

[0059] Step106: If the threshold is exceeded, insert an R-tree on the trajectory node that exceeds the threshold, and build a new trajectory node to receive new sampling points;

[0060] Step107: Perform pre-sorting of the index records according to the path, geographical information, and constraint conditions of the object;

[0061] Step108: Analyze the mutual connection relationship, connection order, and dependency relationship between the sampling nodes. According to the analysis results of the sampling nodes, update the acquisition method of the sampling nodes to complete the final sorting of the index records;

[0062] Step109: Set a time threshold in the R-tree. After the set time threshold is triggered, clean the R-tree cache data and release the nodes that have not been accessed for the time threshold from the current time from the cache

[0063] data.

[0064] The trajectory nodes in the step Step101 only store consecutive sampling points of a single object.

[0065] The trajectory nodes in the step Step101 serve as the leaf nodes of the R-tree. Using the new node insertion algorithm, their index entries are inserted into the layer above the leaf node layer, and the R-tree structure is optimized using the node selection and node splitting sub-algorithms.

[0066] The leaf nodes are used to record object identifiers and consecutive trajectory sampling points, while non-leaf nodes record child node information, including: the spatio-temporal range of the child nodes.

[0067] The process of inserting into the R-tree in the step Step106 includes: node selection and node splitting;

[0068] Among them, the number of operations of node selection is equal to the number of leaf nodes, and the number of operations of node splitting is equal to the number of non-leaf nodes.

[0069] The calculation formula for the evaluation value EVAL of the R-tree in the step Step106 is:

[0070]

[0071] That is, the spatio-temporal evaluation value EVAL: the product of the average value of the node in the intervals of N spatial coordinate axes and the interval T of the time coordinate axis.

[0072] In this embodiment, by improving the indexing algorithm of the existing R-tree, a complete and ordered index of nodes is realized, and according to the path, geographical information and various constraint conditions, the interconnection relationship, connection order, and dependence relationship between nodes are analyzed, realizing the indexing of optical link nodes, which can greatly improve the query effect, adapt to the continuous update and iteration of data, and meet the market demand;

[0073] By setting a time threshold, after the set time threshold is triggered, the cache data is cleared, and the updated cache is regularly released, reducing the originally occupied storage space, improving the data throughput, and effectively solving the problem of high maintenance costs brought by real-time updates.

[0074] Embodiment 2

[0075] In this example, as shown in Figure 2, the implementation method for the final sorting of the index records in the step Step108 includes the following steps:

[0076] Step201: Extract the key features of the trajectory nodes, perform screening after extraction, and send them to the temporary storage area after compression;

[0077] Step202: Scan according to object attributes, extract the trajectory nodes in the temporary storage area, and perform docking;

[0078] Step203: Divide the extracted trajectory nodes into time slices, and create corresponding R-trees according to each time slice;

[0079] Step204: Scan all trajectory node data, improve the construction process of the R-tree, and finally construct the data index data.

[0080] This embodiment can achieve the complete sorting of optical link nodes, facilitate subsequent indexing, reduce the loading response time, and improve the usage effect.

[0081] Embodiment 3

[0082] In this example, as shown in Figure 3, the system composition of data cleaning in the step Step102 includes:

[0083] Import module 1, which is used to receive all imported data and perform format conversion;

[0084] Marking module 2, which is used to analyze the recognizable error values in the data and mark them;

[0085] Extraction module 3, which is used to extract the marked error values and perform estimation;

[0086] Processing module 4, which is used to remove the error values, estimated invalid values, and missing values in the data;

[0087] Export module 5, which is used to adapt the format of the cleaned values and then export them to the sending end. In the specific implementation of this embodiment, all imported data is received by the import module 1 for format conversion, and then submitted to the marking module 2 to analyze the recognizable error values in the data and mark them. Then, the marked error values are extracted by the extraction module 3 and estimated. Next, the error values, estimated invalid values, and missing values in the data are removed by the processing module 4. Finally, the export module 5 adapts the format of the cleaned values and then exports them to the sending end;

[0088] It can effectively remove the error data, estimated invalid values, and missing values, and improve the accuracy of the indexed values.

[0089] Operating performance test

[0090] As shown in Figure 5:

[0091]

[0092] In summary, the present invention is based on the idea of spatial geographical segmentation hashing to study the intelligent indexing technology of the entire optical fiber path. A complete node system is used to represent the entire optical link, and the entire node system is indexed;

[0093] The present invention improves the indexing algorithm of the existing R-tree to achieve a complete and ordered index of nodes, and analyzes the mutual connection relationship, connection order, dependency relationship, etc. between nodes according to the path, geographical information and various constraint conditions, realizing the indexing of optical link nodes, which can greatly improve the query effect, remove incorrect data, improve the accuracy of index values, adapt to the continuous update and iteration of data, and meet the market demand;

[0094] By setting a time threshold, after the set time threshold is triggered, the cache data is cleared, and the updated cache is regularly released, reducing the originally occupied storage space, improving the data throughput, and effectively solving the problem of high maintenance costs brought by real-time updates.

[0095] The above are only the preferred embodiments of the present invention and do not limit the present invention. Any modification of the technical solutions recorded in the foregoing embodiments, any equivalent replacement of some technical features, and any modification, equivalent replacement, and improvement made all fall within the protection scope of the present invention.

[0096] ​

Claims

1. An intelligent indexing method for the full path of an optical fiber, characterized in that, it includes: Step101: Construct a sampling channel in the real-time incremental trajectory database, dock data items, and carry out node collection. Among them, the data items include: object identifier, timestamp, and spatial location; Manage the latest trajectory nodes of all objects with a hash table; Step102: Adopt a data cleaning method to correct the recognizable errors in the data obtained from the sampling channel. Among them, it includes: checking the reasonable value range and mutual relationship of each variable, checking whether the data meets the requirements, and finding data that exceeds the normal range, is logically unreasonable, or is contradictory; estimating, deleting the whole case, deleting variables, and pairwise deleting the invalid values and missing values in the data; Step103: Concentrate the continuous sampling points of the obtained objects and store them in groups on the trajectory nodes; Step104: Set the sampling threshold of the trajectory node, and judge whether the continuous sampling points on the trajectory node exceed the threshold during the real-time update process; Step105: If it does not exceed the threshold, continue sampling according to the predetermined settings; Step106: If it exceeds the threshold, insert an R-tree on the trajectory node that exceeds the threshold, and build a new trajectory node to receive new sampling points; Step107: Perform pre-sorting of the index records according to the path, geographical information, and constraint conditions of the object; Step108: Analyze the mutual connection relationship, connection order, and dependence relationship between the sampling nodes. According to the analysis results of the sampling nodes, update the acquisition method of the sampling nodes to complete the final sorting of the index records; Step109: Set a time threshold in the R-tree. After the set time threshold is triggered, clean the cache data of the R-tree and release the node data that has not been accessed for the time threshold since the current time from the cache.

2. The intelligent indexing method for the full path of an optical fiber according to claim 1, characterized in that, the trajectory nodes in step Step101 only store the continuous sampling points of a single object.

3. The intelligent indexing method for the full path of an optical fiber according to claim 1, characterized in that, the trajectory nodes in step Step101 are used as the leaf nodes of the R-tree. Adopt a new node insertion algorithm to insert its index items into the upper layer of the leaf node layer, and optimize the R-tree structure by using node selection and node splitting sub-algorithms.

4. The intelligent indexing method for the full path of an optical fiber according to claim 3, characterized in that, the leaf nodes are used to record the object identifier and continuous trajectory sampling points, and the non-leaf nodes record the child node information, among which, it includes: the spatio-temporal range of the child nodes.

5. The intelligent indexing method for the full path of an optical fiber according to claim 1, characterized in that, the process of inserting into the R-tree in step Step106 includes: node selection and node splitting; wherein, the number of operations of node selection is equal to the number of leaf nodes, and the number of operations of node splitting is equal to the number of non-leaf nodes.

6. The intelligent indexing method for the full path of an optical fiber according to claim 1, characterized in that, The calculation formula for the evaluation value EVAL of the R-tree in step Step106 is as follows: ； That is, the spatio-temporal evaluation value EVAL: the interval of the node on the N spatial coordinate axes, is the product of the average value and the interval T of the time coordinate axis.

7. An intelligent indexing method for the full optical fiber path according to claim 1, characterized in that The implementation method for the final sorting of the index records in step Step108 includes the following steps: Step201: Extract the key features of the trajectory nodes, perform screening after extraction, and send them to the temporary storage area after compression; Step202: Scan according to the object attributes, extract the trajectory nodes in the temporary storage area, and perform docking; Step203: Divide the extracted trajectory nodes into time slices, and create corresponding R-trees according to each time slice; Step204: Scan all the trajectory node data, improve the R-tree construction process, and finally construct the data index data.

8. An intelligent indexing method for the full optical fiber path according to claim 1, characterized in that The system composition of the data cleaning in step Step102 includes: An import module for receiving all imported data and performing format conversion; A marking module for analyzing the recognizable error values in the data and marking them; An extraction module for extracting the marked error values and performing estimation; A processing module for removing the error values, the estimated invalid values and the missing values in the data; An export module for adapting the format of the cleaned values and then exporting them to the sending end.

Images