Multi-source data acquisition method and system based on adaptive interface

Abstract

The invention provides a multi-source data acquisition method and system based on an adaptive interface, relates to the field of data resource acquisition in the water conservancy industry, and solves the technical problem of low water conservancy data acquisition efficiency caused by dependence on manual interface configuration during water conservancy data acquisition in the prior art. The method comprises the following steps: taking a water conservancy data source of a known protocol type as a first data source; constructing a protocol recognition model based on the convolutional neural network; training a protocol identification model through a preset loss function; acquiring a protocol type of the first data source through a protocol identification model; key feature information of the protocol type is extracted through a principal component analysis technology; constructing a decision tree based on the key feature information to generate a corresponding adaptation rule; accessing a second data source; presetting a similarity threshold, and calculating the similarity between the second data source and the adaptation rule; and if the similarity is not less than a preset similarity threshold, marking the second data source and the first data source as similar data sources. The method and the device are used in a multi-source data integration process.

Description

Technical Field

[0001] This application relates to the field of data resource acquisition in the water conservancy industry, and in particular to a multi-source data acquisition method and system based on an adaptive interface. Background Technology

[0002] In the water conservancy industry, data sources are extensive and complex, with different data sources having different interface types and communication protocols. Traditional data acquisition methods often rely on manual configuration of data interfaces and protocol conversion rules. For each new sensor introduced, it takes 2-3 days to complete parameter surveys, program writing, and testing. For the hundreds of sensor types in the water conservancy industry, the accumulated working hours are extremely large. At the same time, when the sensor protocol or parameters change, the program needs to be modified again, which may lead to system downtime. In the current water conservancy data acquisition process, the technology relies heavily on manual configuration of interfaces and protocol conversion rules. For water conservancy data sources that are diverse and have complex interfaces and protocols, manual operation is not only time-consuming and labor-intensive, but also requires rewriting adaptation code every time a new data source is connected, resulting in technical problems such as low efficiency and difficulty in adapting water conservancy data acquisition. Summary of the Invention

[0003] This application provides a multi-source data acquisition method and system based on adaptive interfaces, which solves the technical problems of low efficiency and difficulty in adapting water conservancy data acquisition caused by the reliance on manual configuration of interfaces and protocol conversion rules in the existing water conservancy data acquisition process. For water conservancy data sources with wide range of sources and complex interfaces and protocols, manual operation is not only time-consuming and labor-intensive, but also requires rewriting adaptation code every time a new data source is connected.

[0004] To achieve the above objectives, this application adopts the following technical solution: Firstly, a multi-source data acquisition method based on an adaptive interface is provided, including: acquiring several water conservancy data sources with known protocol types, and using the several water conservancy data sources as the first data source; A convolutional neural network is invoked to construct a protocol recognition model based on the convolutional neural network; a preset loss function is used to train the protocol recognition model; the protocol type of the first data source is obtained through the trained protocol recognition model; key feature information of the protocol type is extracted through principal component analysis; a decision tree is constructed based on the key feature information to generate corresponding adaptation rules, and several adaptation rules are stored in the rule base in the form of rule feature vectors; Connect to the second data source; preset a similarity threshold and calculate the similarity between the second data source and the adaptation rule; if the similarity is not less than the preset similarity threshold, mark the second data source and the first data source as the same type of data source, and set the same type of data source to use the same adaptation rule; wherein, the second data source is the water conservancy data source to be matched with the adaptation rule.

[0005] In conjunction with the first aspect above, in one possible implementation, the protocol identification model includes: an input layer, a convolutional layer, a pooling layer, and a fully connected layer; The input layer is used to retrieve the first data source and integrate the first data source into a data sequence X=[ ];in, Let L represent the i-th element in the data sequence, and L be the length of the data sequence. The convolutional layer is used to preset the j-th convolutional kernel as... And the size of each convolutional kernel is m; where, It is a positive integer; Through formula The convolution result of the j-th convolution kernel at position k is calculated. ;in, It is the bias term corresponding to the j-th convolution kernel; This represents the weight value of the i-th convolution result in the j-th convolution kernel; Through convolution operations, several feature mapping sets are obtained. ; The pooling layer is used to set the pooling window length to s, using the formula... The result of pooling the j-th feature map at position k is calculated. ;in, This represents the global position of the i-th element within the k-th window; The set of feature vectors is obtained through convolution operation. ; The fully connected layer is used to connect several feature vector sets output by the pooling layer, and to map the feature vector sets to the protocol category space to obtain the protocol category probability distribution vector.

[0006] It should be noted that each convolution result Corresponding feature mapping set An element on.

[0007] In conjunction with the first aspect above, in one possible implementation, the step of mapping the feature vector set to the protocol category space through a fully connected layer to obtain the protocol category probability distribution vector includes: Let W be the weight matrix of the fully connected layer and b be the bias vector; The output formula for a fully connected layer is: ;in, The feature vector output by the pooling layer. is the preset activation function, o is the protocol category probability distribution vector; and each element in the protocol category probability distribution vector corresponds to the probability of a protocol type.

[0008] In conjunction with the first aspect above, in one possible implementation, the method for obtaining the preset loss function includes: Obtain the first historical data source as training set samples ,in, For the nth historical first data source sample, This represents the corresponding protocol category label; n is a positive integer; Through formula The loss function is calculated. Where N is the total number of data samples, and C is the total number of protocol categories. This represents the true label of the nth historical training set sample on the c-th protocol. The protocol identification model predicts the probability that the nth sample index belongs to the cth protocol class; where the true labels include 0 and 1.

[0009] It should be noted that the loss function is used to calculate the error between the protocol identification model's prediction and the true label; protocol category label. It's a general term, referring to the actual labels. It is the encoding form of the protocol category label; the true label is 1 when the samples in the training set belong to the same protocol category, and 0 when they do not belong to the same protocol category.

[0010] In conjunction with the first aspect above, in one possible implementation, the step of training the protocol recognition model through a preset loss function includes: Formula based on backpropagation algorithm The weight matrix of the fully connected layer after the protocol recognition model is updated is calculated. ;in, The learning rate preset for the protocol recognition model. This represents the weight matrix of the fully connected layer before the update. This is the feature vector output after the nth sample is processed by the pooling layer. express Transpose of; Invoke the preset loss function and obtain the updated loss function based on the weight matrix of the fully connected layer; When the updated loss function is minimized, the protocol recognition model is marked as having completed training.

[0011] In conjunction with the first aspect above, in one possible implementation, obtaining the protocol type of the first data source through the trained protocol identification model includes: The first data source is input into the trained protocol recognition model. The model outputs a protocol category probability distribution vector, and the category with the highest probability value in the vector is selected as the identified protocol type. ,in, , This represents the probability of the c-th type of protocol during the inference process of the protocol identification model.

[0012] It should be noted that, because the model focuses on the single input sample during inference, the sample index n is omitted in single-sample inference scenarios. The essence is .

[0013] In conjunction with the first aspect above, in one possible implementation, the extraction of key feature information of the protocol type using principal component analysis includes: Principal component analysis was used to analyze the data format, interface parameters, and data transmission frequency of the first data source to obtain several features; The matrix form of the first data source is set as follows: Where m is the number of the first data sources and n is the number of features; standardization of the first data sources yields a mean of 0 and a variance of 1 for each feature; ] represents a matrix containing the b-th feature of the a-th sample; Through formula The covariance matrix of the standardized data was calculated. ;in, It is a standardized data matrix; Perform eigenvalue decomposition on the covariance matrix Σ to obtain the eigenvalues. eigenvectors corresponding to eigenvalues ;in, ; Based on the preset principal component contribution rate threshold, select the eigenvectors corresponding to the top k eigenvalues. As principal components, the first data source is projected onto the principal components for dimensionality reduction, resulting in the dimensionality-reduced feature vector. and the feature vector Mark as key feature information.

[0014] In conjunction with the first aspect above, in one possible implementation, the step of constructing a decision tree based on key feature information to generate corresponding adaptation rules includes: S110: For any feature A, label the value of feature A as... The training dataset D is divided into n subsets based on feature A. ; S120: Let the training dataset of the first data source be D, containing n classes of adaptation rule types, and the number of samples in the i-th class be... Through formula Information entropy was calculated ; Through formula The information gain of feature A on D is calculated. ; S130: Through formula The information gain ratio was calculated. The decision tree algorithm is invoked to select the feature with the largest information gain ratio as the splitting attribute of the current node; S140: Starting from the root node of the decision tree, split the tree by the feature with the largest information gain ratio to form child nodes. Repeat steps S110-S130 for each child node until all samples belong to the same type of adaptation rule. Mark the end of the repetition process and mark the leaf nodes of the decision tree at this time as the corresponding adaptation rule.

[0015] In conjunction with the first aspect above, in one possible implementation, calculating the similarity between the second data source and the adaptation rule includes: The rule feature vector of the first data source is set as follows: The regular feature vector of the second data source is ; Through formula Calculate the similarity .

[0016] Secondly, this application provides a multi-source data acquisition system based on an adaptive interface, including: a communication unit and a processing unit; The communication unit is used to acquire several water conservancy data sources of known protocol types, use the several water conservancy data sources as the first data source, and access the second data source; The processing unit is configured to invoke a convolutional neural network to construct a protocol recognition model; preset a loss function to train the protocol recognition model; obtain the protocol type of the first data source through the trained protocol recognition model; extract key feature information of the protocol type using principal component analysis; construct a decision tree based on the key feature information to generate corresponding adaptation rules, and store several adaptation rules in the form of rule feature vectors in a rule base; and preset a similarity threshold to calculate the similarity between the second data source and the adaptation rules; if the similarity is not less than the preset similarity threshold, the second data source and the first data source are marked as similar data sources, and the same adaptation rule is set for similar data sources; wherein, the second data source is a water conservancy data source to be matched with the adaptation rule.

[0017] This application provides a multi-source data acquisition method and system based on adaptive interfaces. Through deep learning and machine learning algorithms, the system can quickly and accurately identify various communication protocols and interface types. When encountering new data sources, regardless of whether they employ novel protocols or have unique interface characteristics, the system can generate corresponding adaptation rules through protocol identification and feature analysis, achieving effective data acquisition from new data sources. This solves the problem of difficulty in adapting to new data sources in traditional technologies, and offers strong scalability. It also lowers the technical threshold and operational difficulty: traditional data acquisition methods require high levels of expertise from technicians, necessitating familiarity with various protocols and interface configurations. The automated process of this patent eliminates the need for complex manual operations, reducing the technical requirements for operators, minimizing operational errors caused by human factors, and making data acquisition easier to conduct.

[0018] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0019] Figure 1 A system architecture diagram of a multi-source data acquisition system based on an adaptive interface is provided in this application embodiment; Figure 2 A flowchart illustrating a multi-source data acquisition method based on an adaptive interface, provided in an embodiment of this application; Figure 3 A flowchart illustrating another multi-source data acquisition method based on an adaptive interface provided in this application embodiment; Detailed Implementation

[0020] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0021] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0022] To address the technical problem of low efficiency in water conservancy data acquisition due to reliance on manual interface configuration in existing technologies, this application provides a multi-source data acquisition method based on an adaptive interface, such as... Figure 1 As shown, the method includes: Obtain several water conservancy data sources with known protocol types, and use these water conservancy data sources as the first data source; A convolutional neural network is invoked to construct a protocol recognition model based on the convolutional neural network; a preset loss function is used to train the protocol recognition model; the protocol type of the first data source is obtained through the trained protocol recognition model; key feature information of the protocol type is extracted through principal component analysis; a decision tree is constructed based on the key feature information to generate corresponding adaptation rules, and several adaptation rules are stored in the rule base in the form of rule feature vectors; The system connects to a second data source; a preset similarity threshold is used to calculate the similarity between the second data source and the adaptation rule; if the similarity is not less than the preset similarity threshold, the second data source and the first data source are marked as similar data sources, and the same adaptation rule is set for similar data sources; wherein, the second data source is a water conservancy data source to be matched with the adaptation rule; based on this, the invention realizes the automation and intelligence of water conservancy multi-source data collection, eliminating the need for manual configuration of interfaces and protocol conversion rules in traditional methods. When a new data source is connected, there is no need to rewrite the adaptation code; the system automatically completes operations such as protocol identification and rule generation, greatly shortening the preparation time for data collection; for similar data sources, the system can directly call the adaptation rules in the rule base to quickly complete the collection, significantly improving data collection efficiency and reducing labor and time costs.

[0023] like Figure 2 As shown in the figure, an embodiment of this application provides a multi-source data acquisition method based on an adaptive interface, comprising: Key feature information of the protocol type was extracted using principal component analysis. In some implementations, principal component analysis is used to analyze the data format, interface parameters, and data transmission frequency of the first data source to obtain several features; The matrix form of the first data source is set as follows: Where m is the number of the first data sources and n is the number of features; the first data sources are standardized to obtain a mean of 0 and a variance of 1 for each feature; ] represents a matrix containing the b-th feature of the a-th sample; Through formula The covariance matrix of the standardized data was calculated. ;in, It is a standardized data matrix; Perform eigenvalue decomposition on the covariance matrix Σ to obtain the eigenvalues. eigenvectors corresponding to eigenvalues ;in, ; Based on the preset principal component contribution rate threshold, select the eigenvectors corresponding to the top k eigenvalues. As principal components, the first data source is projected onto the principal components for dimensionality reduction, resulting in the dimensionality-reduced feature vector. and the feature vector Mark as key feature information.

[0024] For example, after identifying the communication protocol, further analysis is performed on the data source's data format, interface parameters, data transmission frequency, and other interface type-related features; key feature information is extracted using feature extraction algorithms in machine learning.

[0025] Principal component analysis was used to analyze the data format, interface parameters, data transmission frequency and other interface type-related features of the data source, and to extract key feature information.

[0026] (1) Data standardization: Assume the original data matrix is Where m is the number of samples and n is the number of features. The data is standardized so that the mean of each feature is 0 and the variance is 1.

[0027] The standardized formula is: ,in, It is the mean of the i-th feature. It is the variance of the b-th feature.

[0028] (2) Calculate the covariance matrix: Calculate the covariance matrix Σ of the standardized data, using the formula: ,in, It is a standardized data matrix; (3) Eigenvalue decomposition: Perform eigenvalue decomposition on the covariance matrix Σ to obtain the eigenvalues. and the corresponding feature vector ; (4) Selecting principal components: Based on the set principal component contribution rate threshold of 85% (i.e., retaining principal components with a cumulative contribution rate of 85% or higher), select the eigenvectors corresponding to the first k eigenvalues. These are the principal components. The original data is projected onto these principal components to obtain the dimensionality-reduced feature vectors. Y represents the extracted key feature information, which is used for subsequent adaptation rule generation.

[0029] like Figure 3 As shown in the embodiments of this application, another multi-source data acquisition method based on an adaptive interface includes: Decision trees are constructed based on key feature information to generate corresponding adaptation rules; In some implementations, S110: For any feature A, the value of feature A is marked as... The training dataset D is divided into n subsets based on feature A. ; S120: Let the training dataset of the first data source be D, containing n classes of adaptation rule types, and the number of samples in the i-th class be... Through formula Information entropy was calculated ; Through formula The information gain of feature A on D is calculated. ; S130: Through formula The information gain ratio was calculated. The decision tree algorithm is invoked to select the feature with the largest information gain ratio as the splitting attribute of the current node; S140: Starting from the root node of the decision tree, split the tree by the feature with the largest information gain ratio to form child nodes. Repeat steps S110-S130 for each child node until all samples belong to the same type of adaptation rule. Mark the end of the repetition process and mark the leaf nodes of the decision tree at this time as the corresponding adaptation rule.

[0030] For example, a decision tree algorithm is used to generate adaptation rules for the data source. The construction of the decision tree is based on existing experience and rule templates for water conservancy data acquisition adaptation. By judging the characteristic information of the data source and selecting branches, specific adaptation rules are generated, including data parsing rules, data transformation rules and communication connection rules.

[0031] (1) Basis for the selection and construction of decision tree algorithm

[0032] The C4.5 decision tree algorithm was selected. This algorithm can handle both continuous and discrete features and can select the optimal splitting attribute based on the information gain ratio, making it suitable for scenarios with complex water conservancy data source characteristics. The construction of the decision tree is based on existing experience and rule templates for water conservancy data acquisition and adaptation. Data source feature information such as protocol type, data format, and interface parameters are used as decision attributes, and adaptation rules are used as leaf nodes.

[0033] (2) Decision tree construction and rule generation

[0034] 1) Calculate information entropy

[0035] Let the training dataset be D, containing k classes of adaptation rule types, and the number of samples in the i-th class be... Then the information entropy of D is: .

[0036] 2) Calculate the information gain ratio

[0037] For feature A, its value is... Divide D into n subsets based on A. Then the information gain of feature A on D is: The information gain ratio is: ,in The feature with the largest information gain ratio is selected as the splitting attribute of the current node.

[0038] 3) Recursively construct the decision tree

[0039] Starting from the root node, split the feature with the highest information gain ratio to form child nodes, and repeat the above process for each child node until all samples belong to the same class or there are no features to split. At this point, the leaf node is the generated adaptation rule.

[0040] Another multi-source data acquisition method based on an adaptive interface provided in this application includes: Connect to the second data source; preset a similarity threshold and calculate the similarity between the second data source and the adaptation rule; if the similarity is not less than the preset similarity threshold, mark the second data source and the first data source as the same type of data source, and set the same type of data source to use the same adaptation rule; wherein, the second data source is the water conservancy data source to be matched with the adaptation rule.

[0041] For example, the generated adaptation rules are stored in a rule base, and database technology is used for storage management. The rule base supports querying, updating, and deleting rules, and can dynamically maintain the adaptation rules according to changes in the data source.

[0042] When a new data source of the same type is connected, the corresponding adaptation rules are quickly called from the rule base, and the data source is collected, parsed and transformed according to the rules. The collected data is then converted into a unified format and transmitted to the data aggregation platform. Data sources of the same type are identified by comparing the similarity between the protocol type and feature information of newly accessed data sources and the feature information stored in the rule base. Cosine similarity is used for calculation, and the feature vector of the data source is set as follows: The feature vector of a certain rule in the rule base is When S≥0.8, it is determined to be a data source of the same type.

[0043] Based on the above technical solution, this invention employs a protocol recognition model, trained on a large number of known protocol-type water conservancy data samples, enabling it to accurately identify various water conservancy data communication protocols such as Modbus and MQTT. The model preprocesses the data, including cleaning and normalization, then extracts protocol feature vectors and matches them with stored feature patterns, achieving automatic protocol recognition and eliminating reliance on manual protocol identification. Using the protocol recognition results and interface feature information obtained from feature analysis as input, a decision tree algorithm is used to generate adaptation rules. The decision tree, based on existing water conservancy data acquisition adaptation experience and rule templates, generates specific rules for data parsing, transformation, and communication connection through the judgment of feature information, achieving automated generation of adaptation rules. The rule base uses database technology to store adaptation rules, supporting operations such as rule querying, updating, and deletion, and can be dynamically maintained according to changes in data sources. When similar data sources are accessed again, adaptation rules in the rule base can be quickly invoked, achieving rule reuse and avoiding the process of repeatedly generating rules.

[0044] The above primarily describes the solutions of the embodiments of this application from the perspective of device implementation. It is understood that each device, for example, a multi-source data acquisition system based on an adaptive interface, includes at least one of the hardware structures and software modules corresponding to the execution of each function in order to achieve the above-mentioned functions. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software-driven manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0045] This application embodiment can divide a multi-source data acquisition system based on an adaptive interface into functional units according to the above method example. For example, each function can be divided into separate functional units, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0046] The units in the embodiments of this application may also be called modules. For example, a processing unit may be called a processing module.

[0047] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0048] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0049] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A multi-source data acquisition method based on an adaptive interface, characterized in that, include: Obtain several water conservancy data sources with known protocol types, and use these water conservancy data sources as the first data source; A convolutional neural network is invoked to construct a protocol recognition model; a preset loss function is used to train the protocol recognition model; the protocol type of the first data source is obtained through the trained protocol recognition model; key feature information of the protocol type is extracted using principal component analysis; based on... Key feature information is used to construct a decision tree to generate corresponding adaptation rules, and several adaptation rules are stored in the rule base in the form of rule feature vectors; Connect to a second data source; A preset similarity threshold is used to calculate the similarity between the second data source and the adaptation rule. If the similarity is not less than the preset similarity threshold, the second data source and the first data source are marked as the same type of data source, and the same type of data source is set to use the same adaptation rule. The second data source is the water conservancy data source to be matched with the adaptation rule.

2. The multi-source data acquisition method based on an adaptive interface according to claim 1, characterized in that, The protocol recognition model includes: an input layer, a convolutional layer, a pooling layer, and a fully connected layer; The input layer is used to retrieve the first data source and integrate the first data source into a data sequence X=[ ];in, Let L represent the i-th element in the data sequence, and L be the length of the data sequence. The convolutional layer is used to preset the j-th convolutional kernel as... And the size of each convolutional kernel is m; where, It is a positive integer; Through formula The convolution result of the j-th convolution kernel at position k is calculated. ;in, It is the bias term corresponding to the j-th convolution kernel; This represents the weight value of the i-th convolution result in the j-th convolution kernel; Through convolution operations, several feature mapping sets are obtained. ; The pooling layer is used to set the pooling window length to s, using the formula... The result of pooling the j-th feature map at position k is calculated. ;in, This represents the global position of the i-th element within the k-th window; The set of feature vectors is obtained through convolution operation. ; The fully connected layer is used to connect several feature vector sets output by the pooling layer, and to map the feature vector sets to the protocol category space to obtain the protocol category probability distribution vector.

3. The multi-source data acquisition method based on an adaptive interface according to claim 2, characterized in that, The process of mapping the feature vector set to the protocol category space through a fully connected layer to obtain the protocol category probability distribution vector includes: Let W be the weight matrix of the fully connected layer and b be the bias vector; The output formula for a fully connected layer is: ;in, The set of feature vectors output by the pooling layer. is the preset activation function, o is the protocol category probability distribution vector; and each element in the protocol category probability distribution vector corresponds to the probability of a protocol type.

4. The multi-source data acquisition method based on an adaptive interface according to claim 1, characterized in that, The method for obtaining the preset loss function includes: Obtain the first historical data source as training set samples ,in, For the nth historical first data source sample, This represents the corresponding protocol category label; n is a positive integer; Through formula The loss function is calculated. Where N is the total number of data samples, and C is the total number of protocol categories. This represents the true label of the nth historical training set sample on the c-th protocol. The protocol identification model predicts the probability that the nth sample index belongs to the cth protocol class; where the true labels include 0 and 1.

5. The multi-source data acquisition method based on an adaptive interface according to claim 4, characterized in that, The method of training the recognition model using a preset loss function includes: Formula based on backpropagation algorithm The weight matrix of the fully connected layer after the protocol recognition model is updated is calculated. ;in, The learning rate preset for the protocol recognition model. This represents the weight matrix of the fully connected layer before the update. This is the feature vector output after the nth historical training set sample is processed by the pooling layer. express Transpose of; Invoke the preset loss function and obtain the updated loss function based on the weight matrix of the fully connected layer; When the updated loss function is minimized, the protocol recognition model is marked as having completed training.

6. The multi-source data acquisition method based on an adaptive interface according to claim 1, characterized in that, The step of obtaining the protocol type of the first data source through the trained protocol recognition model includes: The first data source is input into the trained protocol recognition model. The model outputs a protocol category probability distribution vector, and the category with the highest probability value in the vector is selected as the identified protocol type. ,in, , This represents the probability of the c-th type of protocol during the inference process of the protocol identification model.

7. The multi-source data acquisition method based on an adaptive interface according to claim 1, characterized in that, The extraction of key feature information of the protocol type using principal component analysis includes: Principal component analysis was used to analyze the data format, interface parameters, and data transmission frequency of the first data source to obtain several features; The matrix form of the first data source is set as follows: Where m is the number of the first data sources and n is the number of features; the first data sources are standardized to obtain a mean of 0 and a variance of 1 for each feature; ] represents a matrix containing the b-th feature of the a-th sample; Through formula The covariance matrix of the standardized data was calculated. ;in, It is a standardized data matrix; Perform eigenvalue decomposition on the covariance matrix Σ to obtain the eigenvalues. eigenvectors corresponding to eigenvalues ;in, ; Based on the preset principal component contribution rate threshold, select the eigenvectors corresponding to the top k eigenvalues. As principal components, the first data source is projected onto the principal components for dimensionality reduction, resulting in the dimensionality-reduced feature vector. and the feature vector Mark as key feature information.

8. The multi-source data acquisition method based on an adaptive interface according to claim 1, characterized in that, The process of constructing a decision tree based on key feature information to generate corresponding adaptation rules includes: S110: For any feature A, label the value of feature A as... The training dataset D is divided into n subsets based on feature A. ; S120: Let the training dataset of the first data source be D, containing n classes of adaptation rule types, and the number of samples in the i-th class be... Through formula Information entropy was calculated ; Through formula The information gain of feature A on D is calculated. ; S130: Through formula The information gain ratio was calculated. The decision tree algorithm is invoked to select the feature with the largest information gain ratio as the splitting attribute of the current node; S140: Starting from the root node of the decision tree, split the tree by the feature with the largest information gain ratio to form child nodes. Repeat steps S110-S130 for each child node until all samples belong to the same type of adaptation rule. Mark the end of the repetition process and mark the leaf nodes of the decision tree at this time as the corresponding adaptation rule.

9. The multi-source data acquisition method based on an adaptive interface according to claim 1, characterized in that, The calculation of the similarity between the second data source and the adaptation rules includes: The rule feature vector of the first data source is set as follows: The regular feature vector of the second data source is ; Through formula Calculate the similarity .

10. A multi-source data acquisition system based on an adaptive interface, characterized in that, include: Communication unit and processing unit; The communication unit is used to acquire several water conservancy data sources of known protocol types, use the several water conservancy data sources as the first data source, and access the second data source; The processing unit is used to invoke a convolutional neural network to construct a protocol recognition model based on the convolutional neural network; preset a loss function to train the protocol recognition model; obtain the protocol type of the first data source through the trained protocol recognition model; and extract key feature information of the protocol type using principal component analysis (PCA) technology. Key feature information is used to construct a decision tree to generate corresponding adaptation rules, and several adaptation rules are stored in the rule base in the form of rule feature vectors; And a preset similarity threshold is set to calculate the similarity between the second data source and the adaptation rule; if the similarity is not less than the preset similarity threshold, the second data source and the first data source are marked as the same type of data source, and the same type of data source is set to use the same adaptation rule; wherein, the second data source is the water conservancy data source to be matched with the adaptation rule.

Images