Data indexing method and system for view modeling

Abstract

The invention belongs to the technical field of system modeling, and particularly relates to a data indexing method and system for view modeling, and the method comprises the steps: constructing a view to generate a distributed index database, configuring an enhanced call model into the database, building parallel search mapping with each storage node, and obtaining the real-time storage space of the nodes; secondly, configuring a view modeling decomposition model to decompose a view modeling demand, obtaining a single-class demand data set, inputting the single-class demand data set into an enhanced calling model, searching and calling data of a required type in parallel by utilizing parallel search mapping, and dynamically adjusting a data transmission rate according to a real-time storage amount; and finally, carrying out error data discrimination on the called data, and feeding back the discriminated data to a view modeling tool to carry out view modeling. According to the method, the data transmission rate and an automatic error data correction mechanism are dynamically adjusted, so that the data search efficiency and accuracy in the view modeling process are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of system modeling, and in particular, relates to a data indexing method and system for view modeling. Background Art

[0002] With the development of information technology, system modeling, as an effective method for organizing and managing complex systems, has been widely used in military, commercial, scientific research and other fields. However, the design of existing system modeling tools is often based on traditional concepts and technical specifications, resulting in users facing a high learning curve when using these tools. On the one hand, the operation process of the tool itself is complicated, requiring users to undergo long-term professional training to master its functions; on the other hand, even if the user has mastered the use of the tool, if it is not used frequently, it will need to be relearned due to memory decay. In addition, when dealing with reference data in the same view, existing system modeling tools often lack an intuitive display mechanism, and users have to spend a lot of time to remember and find relevant reference data, which not only increases the cognitive burden of users, but also easily introduces errors, thus affecting the speed and quality of modeling.

[0003] For example, the patent with publication number CN111563283A discloses a method for quickly referencing data in system modeling. This method for quickly referencing data analyzes and organizes the main reference data types that can be used in each view according to system modeling practice, the system architecture framework DoDAF2.0 standard and the description language UPDM2.0 standard, and defines a reference data list for each view. Then, based on the data generated during the modeling process, the referenced data of the view is automatically displayed in the reference data part of the view. When modeling, users can directly drag and drop the required referenced data from the reference data list.

[0004] For example, a patent with authorization announcement number CN117271530B discloses a method and device for modeling pragmatic relationships of digital objects in a digital network, the method comprising: respectively determining the digital object types of the starting digital object and the end digital object; defining the digital object relationship based on the digital object type of the starting digital object, the digital object type of the end digital object and the pragmatic relationship between the digital objects to obtain the digital object relationship, the pragmatic relationship comprising: reference relationship, usage relationship, version inheritance relationship; saving the digital object relationship as metadata together with the metadata of the starting digital object in a relationship registry.

[0005] The above existing technologies have the following problems: traditional data retrieval methods are often inefficient due to problems such as uneven data distribution and multi-node query. Especially when the data volume is large and widely distributed, it becomes particularly difficult to find and retrieve the required data. For this reason, the present invention provides a data indexing method and system for view modeling. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention proposes a data indexing method and system for view modeling, which generates a distributed index database by constructing a view, configures a trained enhanced call model into the database, and establishes a parallel search mapping with each storage node to obtain the real-time storage capacity of the corresponding type of data of each storage node; secondly, configures a view modeling decomposition model, and inputs the view modeling requirements into the model for decomposition to obtain a single-class requirement data set for view modeling; at the same time, the single-class requirement data set is input into the enhanced call model, and the required type of view modeling data is searched and retrieved in parallel using the parallel search mapping, and the data transmission rate is dynamically adjusted according to the real-time storage capacity; finally, the retrieved data is input into the discrimination model for erroneous data discrimination, the erroneous data is stored back to the original location and retrieved for a second time, and the correct data is directly fed back to the view modeling tool for view modeling; the present invention significantly improves the data processing efficiency and accuracy in the view modeling process by dynamically adjusting the data transmission rate and the secondary retrieval mechanism of erroneous data, and is suitable for view modeling application scenarios that require efficient processing of large-scale data.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A data indexing method for view modeling, comprising:

[0009] S1. Build a view to generate a distributed index database, configure the trained reinforcement call model into the distributed index database, and establish a parallel search mapping with each storage node in the distributed index database to obtain the real-time storage capacity of the corresponding type of data for each storage node;

[0010] S2. Configure a view modeling decomposition model, and input the view modeling requirements into the view modeling decomposition model for decomposition, to obtain a view modeling single-class requirement data set;

[0011] S3. Input the single-class requirement data set of view modeling into the enhanced call model, and search and retrieve the required type of view modeling data in parallel through the constructed parallel search mapping. At the same time, according to the real-time storage capacity of the corresponding type of data of each storage node, construct the transmission rate adjustment factor corresponding to each mapping in the parallel search mapping, and dynamically adjust the data transmission rate corresponding to each mapping in the parallel search mapping;

[0012] S4. Input the retrieved view modeling data of the required type into the discrimination model to perform error data discrimination, store the found error data in the original storage node location, and perform secondary retrieval of the corresponding error single type requirement data, and at the same time directly feed back the correct discrimination data to the view modeling tool for view modeling.

[0013] Specifically, the steps of constructing a view-generated distributed index database include:

[0014] S101, constructing a distributed index database framework in combination with a tree-like database, and setting upper-layer concurrent index nodes and lower-layer storage nodes in the distributed index database;

[0015] S102, obtaining view modeling data, and inputting the obtained data into the configured classification model to obtain classified view modeling data, and storing each type of classified view modeling data in a storage node, and analyzing the data table stored in the same storage node to obtain all common fields and non-common fields of the data table in the same storage node;

[0016] S103, based on the tree database principle, the common fields are stored in the first-level common child nodes in the storage node, and each non-common field is stored in a one-to-one correspondence in the second-level non-common child nodes mounted on the first-level common child nodes;

[0017] S104. Mark the classification labels in S102 on each storage node, and at the same time, based on the relationship between the storage node and the first-level shared child nodes, establish an index mapping between the classification labels and the first-level shared child nodes in the same storage node, an index mapping between the first-level shared child nodes and all second-level non-shared child nodes, and an index mapping between the classification labels and all second-level non-shared child nodes, and save the three constructed index mappings to the upper-level concurrent index node corresponding to the current storage node.

[0018] Specifically, the steps of constructing a view-generated distributed index database also include:

[0019] S105, obtaining the corresponding data type of the concurrent query index of the historical view modeling, and inputting it into the concurrent connection model constructed by the support vector machine for training, obtaining the trained concurrent connection model, and building the obtained trained concurrent connection model into the upper-layer concurrent index node;

[0020] S106, inputting the view modeling requirement into the view modeling decomposition model to obtain the requirement index data type;

[0021] S107, input the acquired demand index data type into the concurrent connection model, and construct a concurrent query index between storage nodes corresponding to the current demand index data type according to the storage node classification label stored in the upper-layer concurrent index node;

[0022] S108. When the indexing is completed, the constructed concurrent query index is backed up and maintained in the upper concurrent index node. When there is a data index of the same storage node, the concurrent query index is directly called to perform a query.

[0023] Specifically, the step of constructing the transmission rate adjustment factor in S3 includes:

[0024] S301, assuming that the current view modeling requirement is decomposed through the view modeling decomposition model, and m types of view modeling single-category requirement data are obtained;

[0025] S302: m types of view modeling single-category requirement data are matched with the classified data stored in the storage nodes in the view generation distributed index database through a matching algorithm to obtain the classified data stored in K storage nodes that meet the view modeling requirements among the N storage nodes, and output the real-time storage capacity of the corresponding type of data of the K storage nodes;

[0026] S303, calculating a node transmission rate adjustment factor Tk corresponding to each storage node according to the real-time storage capacity of the corresponding type of data of K storage nodes that currently meet the view modeling requirements;

[0027] S304. Obtain the maximum transmission rate Rmax of the transmission device, and obtain the data transmission rate set Rk corresponding to the currently matched storage node based on the obtained Tk and Rmax, where k=1…K, and Rk represents the data transmission rate corresponding to the kth storage node that meets the view modeling requirements.

[0028] Specifically, the steps of strengthening the call model construction in S1 include:

[0029] S111. According to the current view modeling requirements, the state data of the view-generated distributed index database and the network transmission state data, the current input state st = (m, Ns, X, Rmax) is constructed, wherein Ns represents the data corresponding to the K storage nodes that meet the single-category requirements of m view modeling obtained in the S302 process, and X represents the three corresponding index mappings in each storage node stored in the upper-layer concurrent index node;

[0030] S112, setting the execution action trigger information corresponding to the current state according to the current input state, specifically:

[0031] Assuming that the current view modeling requirements include m types of view modeling single-class requirement data, which are matched to K storage nodes through the S302 process, the concurrent connection model is called through the enhanced call model, and a K-concurrent query index is established according to the classification labels corresponding to the K storage nodes, and the K storage nodes are queried simultaneously;

[0032] According to the maximum transmission rate Rmax of the transmission device, the data transmission rate set Rk, k=1...K corresponding to the concurrent query index of K storage nodes is obtained through the S301-S304 process, and each obtained data transmission rate set Rk, k=1...K is configured to the corresponding query index to perform data query and data transmission on the corresponding storage node.

[0033] Specifically, the steps of simultaneously querying K storage nodes include:

[0034] Configure the index parsing model to parse each query index in the K-concurrent query index, and obtain the common field query index and non-common field query index contained in each query index;

[0035] According to the common field query index and non-common field query index contained in each query index and the three corresponding index mappings in each storage node in the upper-level concurrent index node, the data in the corresponding storage node is indexed twice concurrently to obtain the data corresponding to the view modeling requirements in each storage node;

[0036] When a new view modeling requirement appears, determine whether the new view modeling requirement belongs to the current view modeling. If so, determine whether the view modeling single-class requirement data type corresponding to the new view modeling requirement is included in the K-concurrent query index. If so, directly parse the common field query index and non-common field query index in the new view modeling requirement through the index parsing model and configure the corresponding data transmission rate. At the same time, perform secondary concurrent indexing based on the parsing results to obtain the query index data corresponding to the new view modeling requirement.

[0037] If it belongs to the current view modeling but is not included in the K-concurrent query index, the K-concurrent query index is expanded through the concurrent connection model according to the required data type corresponding to the new view modeling requirement and the corresponding data transmission rate is configured. At the same time, secondary concurrent indexing is performed according to the parsing result of the expanded query index to obtain the query index data corresponding to the new view modeling requirement;

[0038] If it does not belong to the current view modeling but is included in the K-concurrent query index, the current K-concurrent query index is directly called to perform secondary concurrent indexing;

[0039] If it does not belong to the current view modeling and is not included in the K-concurrent query index, the process of S111-S112 is repeated to construct a new K-concurrent query index and perform a new view modeling requirement index.

[0040] Specifically, the steps of secondary concurrent indexing include:

[0041] According to the index resolution model, the common field query index and the non-common field query index are obtained and mapped to three indexes corresponding to each storage node;

[0042] If the query index corresponding to the current k-th storage node contains a common field query index and a non-common field query index, then the index mapping between the classification label and the first-level common child node, the index mapping between the first-level common child node and all the second-level non-common child nodes, and the index mapping between the classification label and all the second-level non-common child nodes corresponding to the k-th storage node are called simultaneously;

[0043] The common fields are queried using the index mapping between the classification label and the first-level common child nodes. At the same time, the non-common fields are queried in parallel using the index mapping between the classification label and all the second-level non-common child nodes under the k-th storage node. At the same time, the queried common fields and non-common fields are merged correspondingly using the index mapping between the first-level common child nodes and all the second-level non-common child nodes to obtain the view modeling requirement data queried by the k-th storage node.

[0044] Specifically, the steps of the secondary concurrent indexing also include:

[0045] If the query index corresponding to the current k-th storage node contains a common field query index or a non-common field query index, then the index mapping between the classification label corresponding to the k-th storage node and the first-level common child nodes or the index mapping between the classification label and all second-level non-common child nodes is called simultaneously;

[0046] When only the common field query index is included, the classification label corresponding to the k-th storage node and the index mapping query of the first-level common child node are called to obtain the corresponding common field;

[0047] When the query index only contains non-common fields, the index mapping query between the classification label corresponding to the k-th storage node and all secondary non-common child nodes is called to obtain the corresponding non-common fields.

[0048] A data indexing system for view modeling, comprising: an index building module, a demand decomposition module and a data retrieval module;

[0049] The index building module is used to build a distributed index database for view generation, configure the enhanced call model into the database, establish a parallel search mapping with each storage node, and obtain the real-time storage capacity of the corresponding type of data of each storage node;

[0050] A requirement decomposition module is used to configure a view modeling decomposition model, input the view modeling requirement into the view modeling decomposition model for decomposition, and obtain a view modeling single-class requirement data set;

[0051] The data retrieval module is used to input the single-class requirement data set of view modeling into the enhanced call model, and search and retrieve the required type of view modeling data in parallel. At the same time, the data transmission rate in the parallel search mapping is dynamically adjusted according to the real-time storage volume.

[0052] A computer-readable storage medium stores computer instructions, which, when executed, execute a data indexing method for view modeling.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] In view of the deficiencies in the prior art, the present invention constructs a view to generate a distributed index database and combines the enhanced call model with the concurrent connection model, thereby effectively solving the problem of low efficiency of traditional data retrieval methods caused by uneven data distribution and multi-node query. Specifically, a distributed index database framework is constructed using the tree database principle, and view modeling data is stored in different storage nodes on demand through a classification model. At the same time, a three-level index mapping mechanism is established to ensure efficient retrieval and rapid positioning of data; secondly, the data transmission rate is dynamically adjusted through the enhanced call model, so that nodes with large data volumes can obtain higher transmission rates, thereby speeding up data retrieval; in addition, the present invention also introduces a concurrent query index mechanism, which can quickly determine and call appropriate secondary concurrent indexes when receiving new view modeling requirements, directly query data, and eliminate the process of querying the upper-level index, thereby further improving query efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flow chart of a data indexing method for view modeling according to Embodiment 1 of the present invention;

[0056] Figure 2 This is a flow chart of obtaining a transmission rate adjustment factor according to Embodiment 1 of the present invention;

[0057] Figure 3 This is a module diagram of a data index system for view modeling according to Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0058] Example 1

[0059] See also Figure 1 , an embodiment of the present invention provides: a data indexing method for view modeling, comprising the following steps:

[0060] S1. Build a view to generate a distributed index database, configure the trained reinforcement call model into the distributed index database, and establish a parallel search mapping with each storage node in the distributed index database to obtain the real-time storage capacity of the corresponding type of data for each storage node;

[0061] Furthermore, in this embodiment, the steps of constructing a view-generated distributed index database include:

[0062] S101, constructing a distributed index database framework in combination with a tree-like database, and setting upper-layer concurrent index nodes and lower-layer storage nodes in the distributed index database;

[0063] S102, obtaining view modeling data, and inputting the obtained data into the configured classification model to obtain classified view modeling data, and storing each type of classified view modeling data in a storage node, and analyzing the data table stored in the same storage node to obtain all common fields and non-common fields of the data table in the same storage node;

[0064] S103. Based on the tree database principle, the common fields are stored in the first-level common child nodes in the storage node, and each non-common field is stored in a one-to-one correspondence in the second-level non-common child nodes mounted by the first-level common child nodes; further, each first-level common child node is mounted with a corresponding number of second-level non-common child nodes according to the number of non-common fields, for storing non-common field data;

[0065] S104, marking the classification label in S102 on each storage node, and at the same time, establishing an index mapping between the classification label and the first-level common sub-node in the same storage node, an index mapping between the first-level common sub-node and all second-level non-common sub-nodes, and an index mapping between the classification label and all second-level non-common sub-nodes according to the relationship between the storage node and the first-level common sub-node, and saving the three constructed index mappings to the upper-level concurrent index node corresponding to the current storage node;

[0066] S105, obtaining the corresponding data type of the concurrent query index of the historical view modeling, and inputting it into the concurrent connection model constructed by the support vector machine for training, obtaining the trained concurrent connection model, and building the obtained trained concurrent connection model into the upper-layer concurrent index node;

[0067] S106, inputting the view modeling requirement into the view modeling decomposition model to obtain the requirement index data type;

[0068] S107, input the acquired demand index data type into the concurrent connection model, and construct a concurrent query index between storage nodes corresponding to the current demand index data type according to the storage node classification label stored in the upper-layer concurrent index node;

[0069] S108. When the indexing is completed, the constructed concurrent query index is backed up and maintained in the upper concurrent index node. When there is a data index of the same storage node, the concurrent query index is directly called to perform a query.

[0070] This process ensures efficient retrieval and rapid positioning of data by classifying and storing view modeling data in different storage nodes and establishing a three-level index mapping (classification labels and first-level common child nodes, first-level common child nodes and second-level non-common child nodes, classification labels and second-level non-common child nodes); in addition, by training the concurrent connection model and building it into the upper-level concurrent index node, it is possible to quickly build a concurrent query index when receiving new view modeling requirements, further improving query efficiency.

[0071] S2. Configure a view modeling decomposition model, and input the view modeling requirements into the view modeling decomposition model for decomposition, and obtain a view modeling single-category requirement data set; further, the type data corresponding to the view modeling in this embodiment is defined by those skilled in the art according to the data corresponding to the requirements in the view modeling process, such as person image data, various types of object image data, video data, text data, and numerical sequence data;

[0072] Furthermore, the view modeling decomposition model in this embodiment is constructed by a random forest algorithm, which is used to decompose the overall data of the view modeling requirements into data requirements of a single type;

[0073] S3. Input the single-class requirement data set of view modeling into the enhanced call model, and search and retrieve the required type of view modeling data in parallel through the constructed parallel search mapping. At the same time, according to the real-time storage capacity of the corresponding type of data of each storage node, construct the transmission rate adjustment factor corresponding to each mapping in the parallel search mapping, and dynamically adjust the data transmission rate corresponding to each mapping in the parallel search mapping;

[0074] For further information, see Figure 2 In this embodiment, the steps of constructing the transmission rate adjustment factor include:

[0075] S301, assuming that the current view modeling requirement is decomposed through the view modeling decomposition model, and m types of view modeling single-category requirement data are obtained;

[0076] S302, match the m types of view modeling single-class requirement data with the classified saved data stored in the storage nodes in the view generation distributed index database through a matching algorithm, obtain the classified data stored in K storage nodes that meet the view modeling requirements among the N storage nodes, and output the real-time storage capacity of the corresponding type data of the K storage nodes; in this embodiment, N is the total number of storage nodes in the view generation distributed index database; K is the number of storage nodes that meet the view modeling requirements obtained after matching in S302;

[0077] S303: According to the real-time storage capacity of the corresponding type of data of the K storage nodes that currently meet the view modeling requirements, the node transmission rate adjustment factor Tk corresponding to each storage node is calculated, which is specifically:

[0078]

[0079] Among them, Tk represents the node transmission rate adjustment factor corresponding to the k-th storage node that currently meets the view modeling requirements, dk represents the real-time storage capacity of data corresponding to the k-th storage node that currently meets the view modeling requirements, and gt represents the proportional adjustment coefficient of the maximum transmission rate of the overall data at the current time t;

[0080] S304, obtaining the maximum transmission rate Rmax of the transmission device, and obtaining the data transmission rate set Rk corresponding to the currently matched storage node according to the obtained Tk and Rmax, where k=1…K, specifically, Rk=Rmax×Tk; wherein Rk represents the data transmission rate corresponding to the kth storage node that meets the view modeling requirements;

[0081] This process dynamically adjusts the transmission rate according to the real-time storage capacity of the storage node by calculating the node transmission rate adjustment factor, making data transmission more efficient. Secondly, the data transmission speed is optimized by obtaining the maximum transmission rate of the transmission device and adjusting it according to the node transmission rate adjustment factor. Finally, by constructing a parallel search mapping, fast search and retrieval of different types of data is achieved, greatly improving data processing efficiency.

[0082] S4. Input the retrieved view modeling data of the required type into the discrimination model to perform error data discrimination, store the found error data in the original storage node location, and perform secondary retrieval of the corresponding error single type requirement data, and at the same time directly feed back the correct discrimination data to the view modeling tool for view modeling.

[0083] Furthermore, in this embodiment, the discriminant model is constructed by a support vector machine;

[0084] Furthermore, in this embodiment, the step of performing discriminant feedback through the discriminant model includes:

[0085] Each type of data called by the query is input into the discriminant model to determine whether it is the same as the view modeling single-class requirement data type obtained by decomposing the view modeling decomposition model. If they are the same, it is fed back to the view modeling process. If they are not the same, it is returned to the corresponding storage node, and the call error information is fed back to the enhanced call model to make a secondary call to the corresponding type of data.

[0086] Furthermore, in this embodiment, the steps of strengthening the call model construction include:

[0087] S111. According to the current view modeling requirements, the state data of the view-generated distributed index database and the network transmission state data, the current input state st = (m, Ns, X, Rmax) is constructed, wherein Ns represents the data corresponding to the K storage nodes that meet the single-category requirements of m view modeling obtained in the S302 process, and X represents the three corresponding index mappings in each storage node stored in the upper-layer concurrent index node;

[0088] S112, setting the execution action trigger information corresponding to the current state according to the current input state, specifically:

[0089] Assuming that the current view modeling requirements include m types of view modeling single-class requirement data, which are matched to K storage nodes through the S302 process, the concurrent connection model is called through the enhanced call model, and a K-concurrent query index is established according to the classification labels corresponding to the K storage nodes, and the K storage nodes are queried simultaneously;

[0090] According to the maximum transmission rate Rmax of the transmission device, the data transmission rate set Rk, k=1...K corresponding to the concurrent query index of K storage nodes is obtained through the process S301-S304, and each obtained data transmission rate set Rk, k=1...K is configured to the corresponding query index, and data query and data transmission are performed on the corresponding storage node;

[0091] Furthermore, in this embodiment, the step of simultaneously querying K storage nodes includes:

[0092] Configure the index parsing model to parse each query index in the K-concurrent query index, and obtain the common field query index and non-common field query index contained in each query index;

[0093] According to the common field query index and non-common field query index contained in each query index and the three corresponding index mappings in each storage node in the upper-level concurrent index node, the data in the corresponding storage node is indexed twice concurrently to obtain the data corresponding to the view modeling requirements in each storage node;

[0094] When a new view modeling requirement appears, determine whether the new view modeling requirement belongs to the current view modeling. If so, determine whether the view modeling single-class requirement data type corresponding to the new view modeling requirement is included in the K-concurrent query index. If so, directly parse the common field query index and non-common field query index in the new view modeling requirement through the index parsing model and configure the corresponding data transmission rate. At the same time, perform secondary concurrent indexing based on the parsing results to obtain the query index data corresponding to the new view modeling requirement.

[0095] If it belongs to the current view modeling but is not included in the K-concurrent query index, the K-concurrent query index is expanded through the concurrent connection model according to the required data type corresponding to the new view modeling requirement and the corresponding data transmission rate is configured. At the same time, secondary concurrent indexing is performed according to the parsing result of the expanded query index to obtain the query index data corresponding to the new view modeling requirement;

[0096] If it does not belong to the current view modeling but is included in the K-concurrent query index, the current K-concurrent query index is directly called to perform secondary concurrent indexing;

[0097] If it does not belong to the current view modeling and is not included in the K-concurrent query index, repeat the process of S111-S112 to build a new K-concurrent query index and perform a new view modeling requirement index;

[0098] Furthermore, the specific steps of secondary concurrent indexing in this embodiment include:

[0099] 1) Parsing the common field query index and the non-common field query index obtained by the index parsing model and mapping them to three indexes corresponding to each storage node;

[0100] 2) If the query index corresponding to the current k-th storage node includes a common field query index and a non-common field query index, then the index mapping between the classification label corresponding to the k-th storage node and the first-level common child node, the index mapping between the first-level common child node and all the second-level non-common child nodes, and the index mapping between the classification label and all the second-level non-common child nodes are called simultaneously;

[0101] The common fields are queried using the index mapping between the classification label and the first-level common child nodes, and the non-common fields are queried in parallel using the index mapping between the classification label and all the second-level non-common child nodes under the k-th storage node. At the same time, the queried common fields and non-common fields are correspondingly merged using the index mapping between the first-level common child nodes and all the second-level non-common child nodes to obtain the view modeling requirement data queried by the k-th storage node;

[0102] If the query index corresponding to the current k-th storage node contains a common field query index or a non-common field query index, then the index mapping between the classification label corresponding to the k-th storage node and the first-level common child nodes or the index mapping between the classification label and all second-level non-common child nodes is called simultaneously;

[0103] When only the common field query index is included, the classification label corresponding to the k-th storage node and the index mapping query of the first-level common child node are called to obtain the corresponding common field;

[0104] When the query index only contains non-shared fields, the index mapping query between the classification label corresponding to the k-th storage node and all secondary non-shared child nodes is called to obtain the corresponding non-shared fields;

[0105] 3) According to the maximum transmission rate Rmax of the transmission device, the data transmission rate set Rk of the K storage nodes corresponding to the concurrent query index is obtained through the process S301-S304, and each obtained data transmission rate set Rk is configured to the corresponding query index, and data query and data transmission are performed on the corresponding storage node;

[0106] S113. Construct the current execution action according to the execution action trigger information, and construct the enhanced call model through the SAC algorithm and input the constructed current input state, execution action trigger information and current execution action into the enhanced call model for training, obtain the trained enhanced call model, and build it into the upper-level concurrent index node to query and call the corresponding data.

[0107] This process achieves fast search and call of storage node data by strengthening the calling model. In particular, when dealing with new view modeling requirements, it can quickly determine whether the new requirements belong to the current view modeling, and decide whether to directly use the existing K-concurrent query index to accelerate data retrieval or expand the K-concurrent query index to adapt to new requirements. In addition, this process realizes parallel query and merging of common and non-common fields through the secondary concurrent indexing strategy, which greatly improves the speed and efficiency of data processing. Combined with the dynamically adjusted data transmission rate, this method not only reduces query latency, but also optimizes the data transmission process, thereby significantly improving the system's response speed and user satisfaction. By continuously training and strengthening the calling model, the system can continuously adapt to changing needs and maintain a high-performance data service status.

[0108] Example 2

[0109] See also Figure 3 ,Another embodiment provided by the present invention: A data indexing system for view modeling, comprising: an index building module, a demand decomposition module, a data retrieval module and a discrimination feedback module;

[0110] The index building module is used to build a view to generate a distributed index database, configure the enhanced call model into the database, establish a parallel search mapping with each storage node, and obtain the real-time storage capacity of the corresponding type of data of each storage node;

[0111] The requirement decomposition module is used to configure the view modeling decomposition model, and input the view modeling requirements into the view modeling decomposition model for decomposition, so as to obtain a view modeling single-category requirement data set;

[0112] The data retrieval module is used to input the single-class requirement data set of view modeling into the enhanced call model, and search and retrieve the required type of view modeling data in parallel. At the same time, the data transmission rate in the parallel search mapping is dynamically adjusted according to the real-time storage capacity;

[0113] The discrimination feedback module is used to input the retrieved view modeling data into the discrimination model to perform erroneous data discrimination, store the erroneous data back to the original location, and perform secondary retrieval, and directly feed back the correct data to the view modeling tool for view modeling.

[0114] Example 3

[0115] A computer-readable storage medium stores computer instructions, which, when executed, execute a data indexing method for view modeling.

[0116] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements a data indexing method for view modeling when executing the computer program.

[0117] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in the field may also change, modify, replace and modify the above-mentioned embodiments without departing from the scope of protection of the purpose of the present invention and the claims, and all of these are within the protection of the present invention.

[0118] If the disclosed technical solution involves personal information, the product using the disclosed technical solution has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the disclosed technical solution involves sensitive personal information, the product using the disclosed technical solution has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. A data indexing method for view modeling, characterized in that: include: S1. Build a view to generate a distributed index database, configure the trained reinforcement call model into the distributed index database, and establish a parallel search mapping with each storage node in the distributed index database to obtain the real-time storage capacity of the corresponding type of data of each storage node; S2. Configure a view modeling decomposition model, and input the view modeling requirements into the view modeling decomposition model for decomposition, to obtain a view modeling single-class requirement data set; S3. Input the single-class requirement data set of view modeling into the enhanced call model, and search and retrieve the required type of view modeling data in parallel through the constructed parallel search mapping. At the same time, according to the real-time storage capacity of the corresponding type of data of each storage node, construct the transmission rate adjustment factor corresponding to each mapping in the parallel search mapping, and dynamically adjust the data transmission rate corresponding to each mapping in the parallel search mapping; S4. Input the retrieved view modeling data of the required type into the discrimination model to perform error data discrimination, store the found error data in the original storage node location, and perform secondary retrieval of the corresponding error single type requirement data, and at the same time directly feed back the correct discrimination data to the view modeling tool for view modeling.

2. A data indexing method for view modeling according to claim 1, characterized in that: The steps of constructing the view generation distributed index database include: S101, constructing a distributed index database framework in combination with a tree-like database, and setting upper-layer concurrent index nodes and lower-layer storage nodes in the distributed index database; S102, obtaining view modeling data, and inputting the obtained data into the configured classification model to obtain classified view modeling data, and storing each type of classified view modeling data in a storage node, and analyzing the data table stored in the same storage node to obtain all common fields and non-common fields of the data table in the same storage node; S103, based on the tree database principle, the common fields are stored in the first-level common child nodes in the storage node, and each non-common field is stored in a one-to-one correspondence in the second-level non-common child nodes mounted on the first-level common child nodes; S104. Mark the classification labels in S102 on each storage node, and at the same time, based on the relationship between the storage node and the first-level shared child nodes, establish an index mapping between the classification labels and the first-level shared child nodes in the same storage node, an index mapping between the first-level shared child nodes and all second-level non-shared child nodes, and an index mapping between the classification labels and all second-level non-shared child nodes, and save the three constructed index mappings to the upper-level concurrent index node corresponding to the current storage node.

3. A data indexing method for view modeling as claimed in claim 2, characterized in that: The step of constructing the view generation distributed index database also includes: S105, obtaining the corresponding data type of the concurrent query index of the historical view modeling, and inputting it into the concurrent connection model constructed by the support vector machine for training, obtaining the trained concurrent connection model, and building the obtained trained concurrent connection model into the upper-layer concurrent index node; S106, inputting the view modeling requirement into the view modeling decomposition model to obtain the requirement index data type; S107, input the acquired demand index data type into the concurrent connection model, and construct a concurrent query index between storage nodes corresponding to the current demand index data type according to the storage node classification label stored in the upper-layer concurrent index node; S108. When the indexing is completed, the constructed concurrent query index is backed up and maintained in the upper concurrent index node. When there is a data index of the same storage node, the concurrent query index is directly called to perform a query.

4. A data indexing method for view modeling as claimed in claim 3, characterized in that: The step of constructing the transmission rate adjustment factor in S3 includes: S301, assuming that the current view modeling requirement is decomposed through the view modeling decomposition model, and m types of view modeling single-category requirement data are obtained; S302: m types of view modeling single-category requirement data are matched with the classified data stored in the storage nodes in the view generation distributed index database through a matching algorithm to obtain the classified data stored in K storage nodes that meet the view modeling requirements among the N storage nodes, and output the real-time storage capacity of the corresponding type of data of the K storage nodes; S303, calculating a node transmission rate adjustment factor Tk corresponding to each storage node according to the real-time storage capacity of the corresponding type of data of K storage nodes that currently meet the view modeling requirements; S304. Obtain the maximum transmission rate Rmax of the transmission device, and obtain the data transmission rate set Rk corresponding to the currently matched storage node based on the obtained Tk and Rmax, where k=1…K, and Rk represents the data transmission rate set corresponding to the kth storage node that meets the view modeling requirements.

5. A data indexing method for view modeling as claimed in claim 4, characterized in that: The steps of strengthening the call model construction in S1 include: S111. According to the current view modeling requirements, the state data of the view-generated distributed index database and the network transmission state data, the current input state st = (m, Ns, X, Rmax) is constructed, wherein Ns represents the data corresponding to the K storage nodes that meet the single-category requirements of m view modeling obtained in the S302 process, and X represents the three corresponding index mappings in each storage node stored in the upper-layer concurrent index node; S112, setting the execution action trigger information corresponding to the current state according to the current input state, specifically: Assuming that the current view modeling requirements include m types of view modeling single-class requirement data, which are matched to K storage nodes through the S302 process, the concurrent connection model is called through the enhanced call model, and a K-concurrent query index is established according to the classification labels corresponding to the K storage nodes, and the K storage nodes are queried simultaneously; According to the maximum transmission rate Rmax of the transmission device, the data transmission rate set Rk, k=1...K corresponding to the concurrent query index of K storage nodes is obtained through the S301-S304 process, and each obtained data transmission rate set Rk, k=1...K is configured to the corresponding query index to perform data query and data transmission on the corresponding storage node.

6. A data indexing method for view modeling as claimed in claim 5, characterized in that: The step of simultaneously querying K storage nodes comprises: Configure the index parsing model to parse each query index in the K-concurrent query index, and obtain the common field query index and non-common field query index contained in each query index; According to the common field query index and non-common field query index contained in each query index and the three corresponding index mappings in each storage node in the upper-level concurrent index node, the data in the corresponding storage node is indexed twice concurrently to obtain the data corresponding to the view modeling requirements in each storage node; When a new view modeling requirement appears, determine whether the new view modeling requirement belongs to the current view modeling. If so, determine whether the view modeling single-class requirement data type corresponding to the new view modeling requirement is included in the K-concurrent query index. If so, directly parse the common field query index and non-common field query index in the new view modeling requirement through the index parsing model and configure the corresponding data transmission rate. At the same time, perform secondary concurrent indexing based on the parsing results to obtain the query index data corresponding to the new view modeling requirement. If it belongs to the current view modeling but is not included in the K-concurrent query index, the K-concurrent query index is expanded through the concurrent connection model according to the required data type corresponding to the new view modeling requirement and the corresponding data transmission rate is configured. At the same time, secondary concurrent indexing is performed according to the parsing result of the expanded query index to obtain the query index data corresponding to the new view modeling requirement; If it does not belong to the current view modeling but is included in the K-concurrent query index, the current K-concurrent query index is directly called to perform secondary concurrent indexing; If it does not belong to the current view modeling and is not included in the K-concurrent query index, the process of S111-S112 is repeated to construct a new K-concurrent query index and perform a new view modeling requirement index.

7. A data indexing method for view modeling according to claim 6, characterized in that: The specific steps of the secondary concurrent indexing include: According to the index resolution model, the common field query index and the non-common field query index are obtained and mapped to three indexes corresponding to each storage node; If the query index corresponding to the current k-th storage node contains a common field query index and a non-common field query index, then the index mapping between the classification label and the first-level common child node, the index mapping between the first-level common child node and all the second-level non-common child nodes, and the index mapping between the classification label and all the second-level non-common child nodes corresponding to the k-th storage node are called simultaneously; The common fields are queried using the index mapping between the classification label and the first-level common child nodes. At the same time, the non-common fields are queried in parallel using the index mapping between the classification label and all the second-level non-common child nodes under the k-th storage node. At the same time, the queried common fields and non-common fields are merged correspondingly using the index mapping between the first-level common child nodes and all the second-level non-common child nodes to obtain the view modeling requirement data queried by the k-th storage node.

8. A data indexing method for view modeling as claimed in claim 7, characterized in that: The specific steps of the secondary concurrent indexing also include: If the query index corresponding to the current k-th storage node contains a common field query index or a non-common field query index, then the index mapping between the classification label corresponding to the k-th storage node and the first-level common child nodes or the index mapping between the classification label and all second-level non-common child nodes is called simultaneously; When only the common field query index is included, the classification label corresponding to the k-th storage node and the index mapping query of the first-level common child node are called to obtain the corresponding common field; When the query index only contains non-common fields, the index mapping query between the classification label corresponding to the k-th storage node and all secondary non-common child nodes is called to obtain the corresponding non-common fields.

9. A data indexing system for view modeling, which is implemented based on a data indexing method for view modeling according to any one of claims 1 to 8, characterized in that: include: Index building module, demand decomposition module and data retrieval module; The index building module is used to build a view to generate a distributed index database, configure the enhanced call model into the database, establish a parallel search mapping with each storage node, and obtain the real-time storage capacity of the corresponding type of data of each storage node; The requirement decomposition module is used to configure the view modeling decomposition model, and input the view modeling requirements into the view modeling decomposition model for decomposition, so as to obtain a view modeling single-category requirement data set; The data retrieval module is used to input the view modeling single-class requirement data set into the enhanced call model, and search and retrieve the required type of view modeling data in parallel; at the same time, dynamically adjust the data transmission rate in the parallel search mapping according to the real-time storage capacity.

10. A computer-readable storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed, a data indexing method for view modeling as described in any one of claims 1 to 8 is executed.

Images