42 results about "Complex data structures" patented technology

The invention relates to the technical field of data warehouses, and particularly provides a statistical aperture generation method and device, a storage medium and computer equipment. The large aperture generation model can automatically understand the complex data structure of the data warehouse through the physical table building statement of the data warehouse. When the initial statistical caliber is obtained, mutual exclusion verification, grammar compliance verification and data source matching verification can be performed on the initial statistical caliber so as to verify whether the statistical caliber generated by the large model has the problems of logic conflict, grammar non-compliance or data source inconsistency and the like which influence the reliability of the statistical result; and judging whether the statistical caliber generated by the large model can be directly put into use or not. And if yes, setting a final statistical caliber corresponding to the target data index according to the initial statistical caliber. Therefore, by introducing the large model technology, the technical threshold is greatly reduced, the generation efficiency of the statistical caliber is improved, and the reliability of data analysis can also be ensured.

The invention discloses a data model field mapping method and device, a storage medium and a processor. According to the scheme, metadata of a source model and metadata of a target model are obtained; constructing a candidate field mapping set based on the metadata of the source model and the metadata of the target model; traversing each pair of candidate field mappings in the candidate field mapping set, calculating a comprehensive similarity value of each pair of candidate field mappings in the candidate field mapping set, and determining a mapping rule corresponding to each pair of candidate field mappings based on the comprehensive similarity value of each pair of candidate field mappings, and converting each pair of candidate field mapping from the source model to the target model based on the determined mapping rule. Compared with the problems of low efficiency, high error rate and the like of a manual maintenance mode in the prior art for a large number of fields and complex data structures, the method has obvious advantages.

An artificial intelligence (AI) language model is trained using input data from one or more original data sources and a selected training algorithm that includes data extraction and condensing techniques. The trained AI model generates self-written code to create an executable query script for extracting data and an executable load script for loading the extracted data into a graph database. The extracted data is stored as newly created nodes within the graph database. The AI model transforms the loaded data into a condensed data structure that represents the full architecture of the data in a natural language format. The AI model is further trained to respond to user input in natural human language using the condensed data structure. Upon receiving a user query, the AI model processes the input, generates a structured data query, retrieves relevant information from the graph database, and outputs a response in natural language.

The invention relates to a large-scale education data migration method based on dynamic service routing, and the method comprises the steps: carrying out the optimization of a ClickHouse batch writing operation, employing a transaction control and distributed lock mechanism in the data conversion and writing process, effectively preventing the data from being repeated or lost, enabling the converted data to be efficiently written into a ClickHouse target table through a batch loading mode, and enabling the data to be more stable and reliable. Dynamic table name adaptation and complex data structure conversion are supported, efficient and reliable migration of ten-million-level education data is achieved through asynchronous parallel processing, fragmentation task scheduling, a multi-dimensional duplicate removal mechanism and an abnormal retry strategy, and the throughput and consistency of data writing are remarkably improved.

The invention relates to a data synchronization and comparison method and device, and aims to efficiently and accurately solve the problem of data consistency of a source end and a target end. The method comprises the following steps: (1) data partitioning: partitioning data of a source end and a target end according to a preset rule; (2) hash calculation: generating a hash value for each data block; (3) hash comparison: rapidly screening inconsistent data blocks by comparing hash values; (4) recording and comparing: comparing inconsistent data blocks item by item and field by field to determine specific differences; and (5) data restoration: restoring the target end data according to the comparison result. According to the method, the comparison data volume is reduced through data partitioning and Hash calculation, and the comparison efficiency is remarkably improved; hash comparison and record comparison are combined, so that the comparison accuracy is ensured; the scheme does not depend on external factors, is suitable for various complex data structures, and has wide applicability.

The invention belongs to the related technical field of runoff forecasting, and discloses a runoff forecasting method and system based on an improved generative adversarial network, and the method comprises the steps: building an initial sample set according to runoff historical data; training a variational auto-encoder model by using the initial sample set, the trained variational auto-encoder model being used for generating a reconstructed sample according to an initial sample in the initial sample set, and combining the initial sample and the reconstructed sample to form a reconstructed sample set; constructing a forecasting model based on the generative adversarial network, and training the forecasting model by using the reconstructed sample set to obtain a trained forecasting model; and forecasting the runoff by using the trained forecasting model. According to the method, the data are reconstructed by constructing the variational auto-encoder, the original data are merged and input into the subsequent forecasting model, the understanding ability of the model for a complex data structure is enhanced, the forecasting model is trained based on the reconstructed sample set, the training difficulty is reduced, and the forecasting precision of the trained forecasting model is improved.

The invention discloses a multi-source complex table-oriented trusted question and answer agent construction method and system, a question and answer agent is constructed based on a ReAct framework, at least a code interpreter tool and a document reading tool are used in the ReAct framework, and the multi-source complex table-oriented trusted question and answer agent construction method comprises the following steps: constructing a multi-mode complex table analysis tool; the analysis module is used for analyzing a non-editable unstructured table file into an editable structured table text; constructing a single-mode complex table cleaning tool for performing abstract modeling, identification extraction and structure reconstruction on a complex data structure of the structured table text to generate a normalized table which is simpler than an original table in structure; and constructing an illusion detection and repair tool for performing real-time illusion verification on the output content of the question and answer agent, and triggering a fallback regeneration mechanism when illusion is detected. According to the invention, through tool set expansion and multi-tool cooperation, an efficient and credible table question and answer agent is constructed.

This invention belongs to the technical field of runoff forecasting, and discloses a runoff forecasting method and system based on an improved generative adversarial network (GAN). The method includes: establishing an initial sample set based on historical runoff data; training a variational autoencoder (VA) model using the initial sample set; using the trained VVA model to generate reconstructed samples from the initial samples in the initial sample set; combining the initial samples and reconstructed samples to form a reconstructed sample set; constructing a forecasting model based on a GAN; training the forecasting model using the reconstructed sample set to obtain a trained forecasting model; and using the trained forecasting model to predict runoff. This invention enhances the model's ability to understand complex data structures by constructing a variational autoencoder to reconstruct data and merging the original data into the subsequent forecasting model. Training the forecasting model based on the reconstructed sample set helps reduce training difficulty and improves the forecasting accuracy of the trained forecasting model.

This invention discloses an FPGA-based image classification method and system in the field of image classification technology. It aims to solve the problems of high computational load and complex data structures in existing deep neural network modules, which increase the difficulty of FPGA deployment and result in low operating efficiency. The method includes steps such as system initialization, loading model weights, acquiring image data, generating feature matrices, and displaying results. The image classification method provided by this invention can fully utilize hardware resources, linearly reduce the computational load of basic convolutional modules, and achieve the goal of improving operating efficiency.

The invention concerns an OPC-UA (Open Protocol Communications Unified Architecture) system and its use in creating light digital cognitive twins for use in industrial automation, especially in edge IoT devices. OPC UA clients can retrieve real-time data updates and historical data for analysis and interaction with complex data structures. Cognitive Digital Twins (CDTs) represent a significant evolution in digital twin technology, integrating cognitive abilities into the digital representations of physical assets. The lightweight cognitive and interlinked digital twin embedded system a) is connected to digital assets provided by PLC server (or relevant edge IoT-enabled architecture) and b) is powered by a lightweight Open Protocol Communications Unified Architecture (OPC-UA) to enable seamless and continuous generation and update of the digital twin.

This invention relates to a quantum-enhanced clustering method based on spheres, belonging to the field of quantum computing technology. This method addresses the problems of high computational complexity, difficulty in capturing intrinsic geometric features, and sensitivity to noise in traditional clustering algorithms when processing large-scale, high-dimensional, nonlinear data. The technical solution includes obtaining a compressed set of spheres from the original dataset and extracting the center vector; encoding the center vector into a quantum state through quantum feature mapping; constructing a quantum kernel matrix using quantum circuits to calculate fidelity; calculating cohesion based on the quantum kernel matrix to identify and remove noisy spheres; performing hierarchical clustering on the core sphere set; and mapping the results back to the original data points. This invention improves clustering accuracy, robustness, and the ability to handle complex data structures.

This invention relates to the field of software development technology and discloses a method, apparatus, device, and storage medium for interface iteration risk alarm. The method includes: acquiring data returned by the interface after an iterative update as comparison data, and acquiring pre-stored data returned by the interface before the iterative update as reference data; converting the comparison data and reference data into a structured data model in a preset format using a nested structure to generate a first structured data representation and a second structured data representation; performing a multi-dimensional comparison based on the first and second structured data representations to obtain a multi-dimensional comparison result; and determining whether to trigger an alarm notification based on the business impact of the multi-dimensional comparison result. Applying the technical solution of this invention can solve the problems of inefficiently handling complex data structure comparisons during interface iteration and the imperfect alarm mechanism in the prior art.

The present invention discloses an AI-driven efficient data redundancy detection and cleaning method and system, which relates to the field of data processing technology. It includes constructing an AI-driven fragment recognition architecture based on the categories of different paragraphs in the target data chain, and the fragment recognition architecture is used to locate the location of duplicate data in the target data chain; and the duplicate data is used as the positioning reference. The present invention performs structured analysis on the target data chain through the fragment recognition architecture, combines structural features, semantic features and functional features, and realizes the positioning and recognition of duplicate data in different contexts and different data types. It is suitable for diversified data scenarios such as text sequences, data streams, multimedia images, etc. In addition, by constructing a feature mapping relationship table, it is possible to construct a fragment recognition framework based on the feature content of different paragraphs, and adopt AI recognition logic matching, so that it has higher adaptability and recognition accuracy when facing complex data structures and implicit redundancy problems.

An early fault positioning method for a complex data structure of a large hydraulic turbine set belongs to the technical field of hydraulic turbine set fault detection, and comprises the following steps: collecting 52 key operation parameters, covering multiple types of data such as rotating speed, power, vibration and the like, classifying the complex data structure according to the collection position, and carrying out abnormal value processing, vacancy value filling and normalization preprocessing to obtain a complex data structure; dimension differences are eliminated, and data quality is improved. The Pearson's correlation coefficient is calculated by using a sliding window, the parameter correlation change is monitored in real time, a dynamic threshold value is set based on the historical data standard deviation, and the abnormality of the unit is accurately judged. And after abnormality is detected, constructing fault feature vectors containing correlation variable quantity and abnormal moment parameter values, and dynamically adjusting fault position scores according to pointing weights of a plurality of feature vectors so as to realize accurate fault positioning. According to the method, the multi-source data processing problem is systematically solved, the fault early warning and positioning accuracy of the unit is remarkably improved, and safe and stable operation of the large hydraulic turbine unit is powerfully guaranteed.

The invention relates to the technical field of railway signal safety control, and provides an interlocking product data analysis method, system and equipment, and the method comprises the steps: analyzing program language data in interlocking software, and obtaining a structural body type definition and a structural body variable; a multi-way tree is constructed according to the structural body type definition, and a multi-way tree model is optimized through node replacement; the optimized multi-way tree is adopted to carry out assignment analysis on the structural body variables, and structural body variable values are obtained; and verifying the obtained structural body variable value and input data, and outputting a verification result. According to the interlocking product data analysis method, system and equipment, structured analysis of interlocking program language data is realized by constructing the multi-way tree model and carrying out node replacement optimization, a complex data structure is efficiently processed, unanalyzed nodes are ensured to be correctly supplemented through a queue traversal and type replacement mechanism, and the data analysis efficiency is improved. And the integrity and accuracy of data analysis are improved.

The invention relates to the technical field of electric digital data processing, in particular to a multi-source heterogeneous data fusion method and system, and the method comprises the steps: connecting a distributed data collection platform to obtain a to-be-fused data set, constructing a hierarchical processing path, carrying out data format conversion and authority verification, and optimizing a decision to generate a fusion strategy. By introducing an error model, a calling probability set and a risk mapping mechanism, the accuracy, efficiency and safety of data fusion are improved. According to the method, the problems of low processing efficiency, insufficient flexibility and limited data security guarantee of a complex data structure can be effectively solved, and the reliability and adaptability of data fusion are remarkably improved.

The invention discloses a coding module for an optical neural network, an input port and a three-dimensional input port. The coding module comprises a plane wave generator; an information encoder; the plane wave generator is arranged at the upstream of the information encoder along the light path propagation direction; the plane wave generator is used for converting the received light beams into plane waves, and the information encoder is used for carrying out information encoding on the plane waves; wherein the information encoder is a reconfigurable metasurface, and the reconfigurable metasurface comprises at least two reconfigurable micro-nano units. According to the method, the light beam is converted into the plane wave before information coding, so that the precision of information coding is improved, and the accuracy of processing a light calculation task is further improved; moreover, the adjustable metasurface serves as an information encoder, so that the number of information which can be borne in a unit area is larger, the input information density of the optical neural network is improved, a data structure with higher dimensionality or more complexity can be processed, and the optical neural network can process a large-scale optical calculation task.

An embodiment of the present application provides a data structure for multi-party call records, and a call record aggregation method based on the above data structure. The multi-party call record adds a summary field on the basis of the existing two-party call records. This field can record summary information of the entire call record. Therefore, when aggregating multi-party call records, the electronic device can determine whether the compared call records meet the aggregation conditions by comparing the summaries in the call records. Implementing the method provided by the embodiment of the present application can achieve the aggregated display effect of multi-party call records. At the same time, the improved multi-party call record data structure can also make the aggregation process more efficient, thereby solving the problems of lengthy logic, complex data structure, low efficiency, etc. in the process of aggregating multi-party call records using the two-party call record aggregation method.

The application discloses a large-scale water turbine set complex data structure-oriented early fault positioning method, belongs to the technical field of water turbine set fault detection, and comprises the following steps: collecting 52 kinds of key operation parameters, covering multiple types of data such as rotating speed, power and vibration; classifying the complex data structure according to collection positions; and performing preprocessing such as abnormal value processing, missing value filling and normalization, so as to eliminate dimension differences and improve data quality. A sliding window is used to calculate a Pearson correlation coefficient, real-time monitoring of parameter correlation change is performed, a dynamic threshold is set based on historical data standard deviation, and unit abnormalities are accurately judged. After detecting abnormalities, a fault characteristic vector containing a correlation change amount and an abnormal time parameter value is constructed, a fault position score is dynamically adjusted according to the pointing weight of multiple characteristic vectors, and accurate fault positioning is realized. The method systematically solves the problem of multi-source data processing, significantly improves the accuracy of unit fault early warning and positioning, and effectively guarantees the safe and stable operation of large-scale water turbine sets.

The invention discloses a quick arrangement method, device and system for an aerial photography flight data record table. The method comprises the steps of creating a structured table, and generating a self-adaptive flight data record table framework; obtaining multi-source data and extracting key parameters from the multi-source data, wherein the multi-source data comprises aerial image EXIF data and aerial photography POS data; aligning and associating the multi-source data according to the extracted key parameters; judging and marking the position of the missed sheet through a rapid detection method of the missed sheet; and inputting the multi-source data of the first route in the structured table, so that the system automatically fills the residual data in the structured table according to the multi-source data of the first route, the multi-source data alignment association result of each route and the missed sheet mark to automatically generate a complete flight data record table. According to the method, after the single-lens or multi-lens aerial photography data is obtained, the complex data structure and data content can be conveniently and rapidly sorted and concluded, and a flight data record document is generated.

The utility model discloses a coding module for an optical neural network, an input port and a three-dimensional input port. The coding module comprises a plane wave generator; an information encoder; the plane wave generator is arranged at the upstream of the information encoder along the light path propagation direction; the plane wave generator is used for converting the received light beams into plane waves, and the information encoder is used for carrying out information encoding on the plane waves; wherein the information encoder is a reconfigurable metasurface, and the reconfigurable metasurface comprises at least two reconfigurable micro-nano units. According to the method, the light beam is converted into the plane wave before information coding, so that the precision of information coding is improved, and the accuracy of processing a light calculation task is further improved; moreover, the adjustable metasurface serves as an information encoder, so that the number of information which can be borne in a unit area is larger, the input information density of the optical neural network is improved, a data structure with higher dimensionality or more complexity can be processed, and the optical neural network can process a large-scale optical calculation task.

The invention provides an interaction method and device supporting infinite nesting data structure visualization, and relates to the technical field of data structure visualization and man-machine interaction collaborative modeling. The method comprises the steps that recursive rendering of type perception is executed in response to a data structure loading event, and a nested graph with a clear hierarchical topology is generated; providing a double-input mode of an expression mode and a fixed value mode for basic type nodes, and realizing accurate binding of array variables through a special binding button, an intelligent array selector and a node type label identification and single-selection constraint mechanism under the expression mode; and starting a double-layer recursive verification process driven by a root node during submitting, and performing upward aggregation propagation and visual feedback on a verification result along a parent path. According to the method, the rendering performance and interactive user experience of the deep nested data structure are improved, real-time infinite recursion operation is supported, and the problems that topology is not visual, binding is prone to errors, verification is low in efficiency and the like in complex data structure visualization of a traditional tool are effectively solved.

The invention discloses a text generation method and device, equipment, a storage medium and a program product, and the method comprises the steps: representing input data as a graph structure, carrying out the embedded learning of the graph structure through employing a graph neural network, capturing the global information (i.e., global latent variables) and local information (i.e., local latent variables) between the input data, a complex data structure can be more effectively organized and utilized, information can be extracted on different levels, and the logic continuity and the structural property of the model during text generation are improved; the global planning module is used for generating sentence topics and main content representations, then the refining generation module is used for generating content representations of sentences, information can be captured and integrated on different levels, and inter-sentence coherence and consistency of generated texts are remarkably improved; and the expression mode of each sentence is adjusted according to the local latent variables, so that diversified expression of the sentences is realized, coherent and consistent texts with various styles can be generated, and the requirements on diversified texts in practical application are met.

The embodiment of the invention belongs to the technical field of data processing, and relates to an information aggregation transmission method, which comprises the following steps: receiving a protocol generation instruction carrying protocol information, and generating a target protocol corresponding to the protocol generation instruction according to the protocol information; formulating a corresponding application layer protocol according to the target protocol; expanding the preset class according to a preset expansion method to obtain an expanded target class; outputting a target event according to the application layer protocol and the target class; according to the target protocol, event enumeration is defined, and when the target event occurs, the target event is transmitted to the corresponding consumer. The invention further provides an information aggregation transmission device, computer equipment and a storage medium. According to the method and the device, the event can be flexibly and reliably transmitted along the response chain through an event transmission mechanism based on a protocol, a hard coding parameter transmission mode is not needed, the maintenance problem caused by transmission of a complex data structure is avoided, and thus the information aggregation transmission efficiency is improved.

The application provides a program processing method for reducing the difficulty of data structure formal verification, which comprises the following steps: firstly, the complex data structure is described by a nested shape graph; then, the shape definition of the final nested shape graph is obtained by analyzing the shape definition of the main data structure and the secondary data structure; then, the predicate and macro definition containing the universal quantification assertion in the assertion are expanded, all the assertions are traversed and matched based on the deductive reasoning, and the new assertion is obtained by splitting and splicing according to the matching result. The application adopts the splitting and splicing mode for the complex data structure program with the universal quantification assertion, greatly reduces the complexity of the formal verification, effectively reduces the difficulty of the data structure program verification, and improves the verification efficiency.

The invention discloses an assertion script reconstruction method and related equipment, and the method comprises the steps: obtaining an original assertion script and a user neglect rule, executing a script record field dynamic track to generate a system neglect rule, and carrying out the semantic-level marking of an assertion node of a field needing to be neglected in an abstract syntax tree through the combination of the two types of rules, and reconstructing and generating a target script based on various neglect strategies. By automatically identifying the runtime dynamic field and fusing the user-defined rule, the adaptability to the dynamic scene is remarkably improved. Meanwhile, automatic positioning of the neglected field in the nested structure is achieved by means of the abstract syntax tree, tedious manual modification operation is replaced, adaptation can be completed only by updating a strategy when field features change, the test and maintenance cost is greatly reduced, and repeated labor and human errors are reduced. In addition, different dynamic fields are precisely processed through a strategy, the stability of assertion logic under a complex data structure is guaranteed, and the test accuracy is effectively improved.

This invention discloses a method and system for predicting metabolite structures based on a BERT Siamese neural network model. The invention identifies metabolite structures by constructing a BERT Siamese neural network model and performing feature extraction, similarity calculation, and network training optimization. Compared to traditional methods, the use of deep learning improves feature extraction capabilities, better handles complex data structures, and thus improves the accuracy of similarity calculation. The application of this invention enables faster and more accurate metabolite identification, especially on large-scale, high-dimensional datasets, and has significant theoretical and practical value. Validation results show that this method exhibits high prediction accuracy on both publicly available and real-world datasets, providing a more reliable tool for metabolomics data analysis.

The present application relates to the technical field of electric digital data processing, and more particularly to a multi-source heterogeneous data fusion method and system, which comprises connecting a distributed data acquisition platform to obtain a data set to be fused, constructing a hierarchical processing path, performing data format conversion and permission verification, and optimizing decision-making to generate a fusion strategy. By introducing an error model, calling a probability set, and a risk mapping mechanism, the accuracy, efficiency, and security of data fusion are improved. The present application can effectively solve the problems of low processing efficiency, insufficient flexibility, and limited data security protection for complex data structures, significantly improving the reliability and adaptability of data fusion.

The application discloses a kind of multi-source complex table-oriented trusted question and answer agent construction method, system, question and answer agent is based on ReAct framework construction, at least code interpreter tool and document reading tool are used in ReAct framework, multi-source complex table-oriented trusted question and answer agent construction method includes: constructing multimodal complex table parsing tool, for the uneditable unstructured table file is parsed into editable structured table text;Single-mode complex table cleaning tool is constructed, for the complex data structure of structured table text is abstractly modeled, identified extraction and structure reconstruction, generates the normalized table of simpler original table structure;Construct illusion detection and repair tool, for the output content of question and answer agent is checked in real time illusion, and when detecting illusion, trigger fallback regeneration mechanism.The application expands through tool set and multi-tool cooperation, constructs efficient trusted table question and answer agent.

The invention belongs to the technical field of artificial intelligence, and relates to a data check rule generation and optimization method, which comprises the following steps: constructing machine learning classifier features based on a business mode and a normal range, training a machine learning classifier after feature construction through pre-processed historical check data, and training an anomaly detection model in parallel; obtaining a decision condition of the trained classification model and a threshold value of the anomaly detection model, and generating an executable initial check rule; applying the initial check rule to real-time check data to obtain a classification result and an abnormal score; and feedback information is obtained according to the classification result and the abnormal score, and the check rule is optimized according to the feedback information. The invention further provides a data checking rule generation and optimization system and device and a storage medium. The method can be applied to business management program systems of financial science and technology and the like, and can update the rules in time under the conditions of complex data structure and changeable business logic.