32 results about "Complex data structures" patented technology

The invention relates to the technical field of data warehouses, and particularly provides a statistical aperture generation method and device, a storage medium and computer equipment. The large aperture generation model can automatically understand the complex data structure of the data warehouse through the physical table building statement of the data warehouse. When the initial statistical caliber is obtained, mutual exclusion verification, grammar compliance verification and data source matching verification can be performed on the initial statistical caliber so as to verify whether the statistical caliber generated by the large model has the problems of logic conflict, grammar non-compliance or data source inconsistency and the like which influence the reliability of the statistical result; and judging whether the statistical caliber generated by the large model can be directly put into use or not. And if yes, setting a final statistical caliber corresponding to the target data index according to the initial statistical caliber. Therefore, by introducing the large model technology, the technical threshold is greatly reduced, the generation efficiency of the statistical caliber is improved, and the reliability of data analysis can also be ensured.

The invention discloses a data model field mapping method and device, a storage medium and a processor. According to the scheme, metadata of a source model and metadata of a target model are obtained; constructing a candidate field mapping set based on the metadata of the source model and the metadata of the target model; traversing each pair of candidate field mappings in the candidate field mapping set, calculating a comprehensive similarity value of each pair of candidate field mappings in the candidate field mapping set, and determining a mapping rule corresponding to each pair of candidate field mappings based on the comprehensive similarity value of each pair of candidate field mappings, and converting each pair of candidate field mapping from the source model to the target model based on the determined mapping rule. Compared with the problems of low efficiency, high error rate and the like of a manual maintenance mode in the prior art for a large number of fields and complex data structures, the method has obvious advantages.

The invention relates to a large-scale education data migration method based on dynamic service routing, and the method comprises the steps: carrying out the optimization of a ClickHouse batch writing operation, employing a transaction control and distributed lock mechanism in the data conversion and writing process, effectively preventing the data from being repeated or lost, enabling the converted data to be efficiently written into a ClickHouse target table through a batch loading mode, and enabling the data to be more stable and reliable. Dynamic table name adaptation and complex data structure conversion are supported, efficient and reliable migration of ten-million-level education data is achieved through asynchronous parallel processing, fragmentation task scheduling, a multi-dimensional duplicate removal mechanism and an abnormal retry strategy, and the throughput and consistency of data writing are remarkably improved.

The invention discloses a multi-source complex table-oriented trusted question and answer agent construction method and system, a question and answer agent is constructed based on a ReAct framework, at least a code interpreter tool and a document reading tool are used in the ReAct framework, and the multi-source complex table-oriented trusted question and answer agent construction method comprises the following steps: constructing a multi-mode complex table analysis tool; the analysis module is used for analyzing a non-editable unstructured table file into an editable structured table text; constructing a single-mode complex table cleaning tool for performing abstract modeling, identification extraction and structure reconstruction on a complex data structure of the structured table text to generate a normalized table which is simpler than an original table in structure; and constructing an illusion detection and repair tool for performing real-time illusion verification on the output content of the question and answer agent, and triggering a fallback regeneration mechanism when illusion is detected. According to the invention, through tool set expansion and multi-tool cooperation, an efficient and credible table question and answer agent is constructed.

This invention belongs to the technical field of runoff forecasting, and discloses a runoff forecasting method and system based on an improved generative adversarial network (GAN). The method includes: establishing an initial sample set based on historical runoff data; training a variational autoencoder (VA) model using the initial sample set; using the trained VVA model to generate reconstructed samples from the initial samples in the initial sample set; combining the initial samples and reconstructed samples to form a reconstructed sample set; constructing a forecasting model based on a GAN; training the forecasting model using the reconstructed sample set to obtain a trained forecasting model; and using the trained forecasting model to predict runoff. This invention enhances the model's ability to understand complex data structures by constructing a variational autoencoder to reconstruct data and merging the original data into the subsequent forecasting model. Training the forecasting model based on the reconstructed sample set helps reduce training difficulty and improves the forecasting accuracy of the trained forecasting model.

This invention discloses an FPGA-based image classification method and system in the field of image classification technology. It aims to solve the problems of high computational load and complex data structures in existing deep neural network modules, which increase the difficulty of FPGA deployment and result in low operating efficiency. The method includes steps such as system initialization, loading model weights, acquiring image data, generating feature matrices, and displaying results. The image classification method provided by this invention can fully utilize hardware resources, linearly reduce the computational load of basic convolutional modules, and achieve the goal of improving operating efficiency.

This invention relates to a quantum-enhanced clustering method based on spheres, belonging to the field of quantum computing technology. This method addresses the problems of high computational complexity, difficulty in capturing intrinsic geometric features, and sensitivity to noise in traditional clustering algorithms when processing large-scale, high-dimensional, nonlinear data. The technical solution includes obtaining a compressed set of spheres from the original dataset and extracting the center vector; encoding the center vector into a quantum state through quantum feature mapping; constructing a quantum kernel matrix using quantum circuits to calculate fidelity; calculating cohesion based on the quantum kernel matrix to identify and remove noisy spheres; performing hierarchical clustering on the core sphere set; and mapping the results back to the original data points. This invention improves clustering accuracy, robustness, and the ability to handle complex data structures.

This invention relates to the field of software development technology and discloses a method, apparatus, device, and storage medium for interface iteration risk alarm. The method includes: acquiring data returned by the interface after an iterative update as comparison data, and acquiring pre-stored data returned by the interface before the iterative update as reference data; converting the comparison data and reference data into a structured data model in a preset format using a nested structure to generate a first structured data representation and a second structured data representation; performing a multi-dimensional comparison based on the first and second structured data representations to obtain a multi-dimensional comparison result; and determining whether to trigger an alarm notification based on the business impact of the multi-dimensional comparison result. Applying the technical solution of this invention can solve the problems of inefficiently handling complex data structure comparisons during interface iteration and the imperfect alarm mechanism in the prior art.

The present invention discloses an AI-driven efficient data redundancy detection and cleaning method and system, which relates to the field of data processing technology. It includes constructing an AI-driven fragment recognition architecture based on the categories of different paragraphs in the target data chain, and the fragment recognition architecture is used to locate the location of duplicate data in the target data chain; and the duplicate data is used as the positioning reference. The present invention performs structured analysis on the target data chain through the fragment recognition architecture, combines structural features, semantic features and functional features, and realizes the positioning and recognition of duplicate data in different contexts and different data types. It is suitable for diversified data scenarios such as text sequences, data streams, multimedia images, etc. In addition, by constructing a feature mapping relationship table, it is possible to construct a fragment recognition framework based on the feature content of different paragraphs, and adopt AI recognition logic matching, so that it has higher adaptability and recognition accuracy when facing complex data structures and implicit redundancy problems.

The invention relates to the technical field of railway signal safety control, and provides an interlocking product data analysis method, system and equipment, and the method comprises the steps: analyzing program language data in interlocking software, and obtaining a structural body type definition and a structural body variable; a multi-way tree is constructed according to the structural body type definition, and a multi-way tree model is optimized through node replacement; the optimized multi-way tree is adopted to carry out assignment analysis on the structural body variables, and structural body variable values are obtained; and verifying the obtained structural body variable value and input data, and outputting a verification result. According to the interlocking product data analysis method, system and equipment, structured analysis of interlocking program language data is realized by constructing the multi-way tree model and carrying out node replacement optimization, a complex data structure is efficiently processed, unanalyzed nodes are ensured to be correctly supplemented through a queue traversal and type replacement mechanism, and the data analysis efficiency is improved. And the integrity and accuracy of data analysis are improved.

The invention relates to the technical field of electric digital data processing, in particular to a multi-source heterogeneous data fusion method and system, and the method comprises the steps: connecting a distributed data collection platform to obtain a to-be-fused data set, constructing a hierarchical processing path, carrying out data format conversion and authority verification, and optimizing a decision to generate a fusion strategy. By introducing an error model, a calling probability set and a risk mapping mechanism, the accuracy, efficiency and safety of data fusion are improved. According to the method, the problems of low processing efficiency, insufficient flexibility and limited data security guarantee of a complex data structure can be effectively solved, and the reliability and adaptability of data fusion are remarkably improved.

The application discloses a large-scale water turbine set complex data structure-oriented early fault positioning method, belongs to the technical field of water turbine set fault detection, and comprises the following steps: collecting 52 kinds of key operation parameters, covering multiple types of data such as rotating speed, power and vibration; classifying the complex data structure according to collection positions; and performing preprocessing such as abnormal value processing, missing value filling and normalization, so as to eliminate dimension differences and improve data quality. A sliding window is used to calculate a Pearson correlation coefficient, real-time monitoring of parameter correlation change is performed, a dynamic threshold is set based on historical data standard deviation, and unit abnormalities are accurately judged. After detecting abnormalities, a fault characteristic vector containing a correlation change amount and an abnormal time parameter value is constructed, a fault position score is dynamically adjusted according to the pointing weight of multiple characteristic vectors, and accurate fault positioning is realized. The method systematically solves the problem of multi-source data processing, significantly improves the accuracy of unit fault early warning and positioning, and effectively guarantees the safe and stable operation of large-scale water turbine sets.

The invention discloses a quick arrangement method, device and system for an aerial photography flight data record table. The method comprises the steps of creating a structured table, and generating a self-adaptive flight data record table framework; obtaining multi-source data and extracting key parameters from the multi-source data, wherein the multi-source data comprises aerial image EXIF data and aerial photography POS data; aligning and associating the multi-source data according to the extracted key parameters; judging and marking the position of the missed sheet through a rapid detection method of the missed sheet; and inputting the multi-source data of the first route in the structured table, so that the system automatically fills the residual data in the structured table according to the multi-source data of the first route, the multi-source data alignment association result of each route and the missed sheet mark to automatically generate a complete flight data record table. According to the method, after the single-lens or multi-lens aerial photography data is obtained, the complex data structure and data content can be conveniently and rapidly sorted and concluded, and a flight data record document is generated.

The utility model discloses a coding module for an optical neural network, an input port and a three-dimensional input port. The coding module comprises a plane wave generator; an information encoder; the plane wave generator is arranged at the upstream of the information encoder along the light path propagation direction; the plane wave generator is used for converting the received light beams into plane waves, and the information encoder is used for carrying out information encoding on the plane waves; wherein the information encoder is a reconfigurable metasurface, and the reconfigurable metasurface comprises at least two reconfigurable micro-nano units. According to the method, the light beam is converted into the plane wave before information coding, so that the precision of information coding is improved, and the accuracy of processing a light calculation task is further improved; moreover, the adjustable metasurface serves as an information encoder, so that the number of information which can be borne in a unit area is larger, the input information density of the optical neural network is improved, a data structure with higher dimensionality or more complexity can be processed, and the optical neural network can process a large-scale optical calculation task.

The invention provides an interaction method and device supporting infinite nesting data structure visualization, and relates to the technical field of data structure visualization and man-machine interaction collaborative modeling. The method comprises the steps that recursive rendering of type perception is executed in response to a data structure loading event, and a nested graph with a clear hierarchical topology is generated; providing a double-input mode of an expression mode and a fixed value mode for basic type nodes, and realizing accurate binding of array variables through a special binding button, an intelligent array selector and a node type label identification and single-selection constraint mechanism under the expression mode; and starting a double-layer recursive verification process driven by a root node during submitting, and performing upward aggregation propagation and visual feedback on a verification result along a parent path. According to the method, the rendering performance and interactive user experience of the deep nested data structure are improved, real-time infinite recursion operation is supported, and the problems that topology is not visual, binding is prone to errors, verification is low in efficiency and the like in complex data structure visualization of a traditional tool are effectively solved.

The invention discloses a text generation method and device, equipment, a storage medium and a program product, and the method comprises the steps: representing input data as a graph structure, carrying out the embedded learning of the graph structure through employing a graph neural network, capturing the global information (i.e., global latent variables) and local information (i.e., local latent variables) between the input data, a complex data structure can be more effectively organized and utilized, information can be extracted on different levels, and the logic continuity and the structural property of the model during text generation are improved; the global planning module is used for generating sentence topics and main content representations, then the refining generation module is used for generating content representations of sentences, information can be captured and integrated on different levels, and inter-sentence coherence and consistency of generated texts are remarkably improved; and the expression mode of each sentence is adjusted according to the local latent variables, so that diversified expression of the sentences is realized, coherent and consistent texts with various styles can be generated, and the requirements on diversified texts in practical application are met.

The application provides a program processing method for reducing the difficulty of data structure formal verification, which comprises the following steps: firstly, the complex data structure is described by a nested shape graph; then, the shape definition of the final nested shape graph is obtained by analyzing the shape definition of the main data structure and the secondary data structure; then, the predicate and macro definition containing the universal quantification assertion in the assertion are expanded, all the assertions are traversed and matched based on the deductive reasoning, and the new assertion is obtained by splitting and splicing according to the matching result. The application adopts the splitting and splicing mode for the complex data structure program with the universal quantification assertion, greatly reduces the complexity of the formal verification, effectively reduces the difficulty of the data structure program verification, and improves the verification efficiency.

The invention discloses an assertion script reconstruction method and related equipment, and the method comprises the steps: obtaining an original assertion script and a user neglect rule, executing a script record field dynamic track to generate a system neglect rule, and carrying out the semantic-level marking of an assertion node of a field needing to be neglected in an abstract syntax tree through the combination of the two types of rules, and reconstructing and generating a target script based on various neglect strategies. By automatically identifying the runtime dynamic field and fusing the user-defined rule, the adaptability to the dynamic scene is remarkably improved. Meanwhile, automatic positioning of the neglected field in the nested structure is achieved by means of the abstract syntax tree, tedious manual modification operation is replaced, adaptation can be completed only by updating a strategy when field features change, the test and maintenance cost is greatly reduced, and repeated labor and human errors are reduced. In addition, different dynamic fields are precisely processed through a strategy, the stability of assertion logic under a complex data structure is guaranteed, and the test accuracy is effectively improved.

This invention discloses a method and system for predicting metabolite structures based on a BERT Siamese neural network model. The invention identifies metabolite structures by constructing a BERT Siamese neural network model and performing feature extraction, similarity calculation, and network training optimization. Compared to traditional methods, the use of deep learning improves feature extraction capabilities, better handles complex data structures, and thus improves the accuracy of similarity calculation. The application of this invention enables faster and more accurate metabolite identification, especially on large-scale, high-dimensional datasets, and has significant theoretical and practical value. Validation results show that this method exhibits high prediction accuracy on both publicly available and real-world datasets, providing a more reliable tool for metabolomics data analysis.

The present application relates to the technical field of electric digital data processing, and more particularly to a multi-source heterogeneous data fusion method and system, which comprises connecting a distributed data acquisition platform to obtain a data set to be fused, constructing a hierarchical processing path, performing data format conversion and permission verification, and optimizing decision-making to generate a fusion strategy. By introducing an error model, calling a probability set, and a risk mapping mechanism, the accuracy, efficiency, and security of data fusion are improved. The present application can effectively solve the problems of low processing efficiency, insufficient flexibility, and limited data security protection for complex data structures, significantly improving the reliability and adaptability of data fusion.

The application discloses a kind of multi-source complex table-oriented trusted question and answer agent construction method, system, question and answer agent is based on ReAct framework construction, at least code interpreter tool and document reading tool are used in ReAct framework, multi-source complex table-oriented trusted question and answer agent construction method includes: constructing multimodal complex table parsing tool, for the uneditable unstructured table file is parsed into editable structured table text;Single-mode complex table cleaning tool is constructed, for the complex data structure of structured table text is abstractly modeled, identified extraction and structure reconstruction, generates the normalized table of simpler original table structure;Construct illusion detection and repair tool, for the output content of question and answer agent is checked in real time illusion, and when detecting illusion, trigger fallback regeneration mechanism.The application expands through tool set and multi-tool cooperation, constructs efficient trusted table question and answer agent.

The invention belongs to the technical field of artificial intelligence, and relates to a data check rule generation and optimization method, which comprises the following steps: constructing machine learning classifier features based on a business mode and a normal range, training a machine learning classifier after feature construction through pre-processed historical check data, and training an anomaly detection model in parallel; obtaining a decision condition of the trained classification model and a threshold value of the anomaly detection model, and generating an executable initial check rule; applying the initial check rule to real-time check data to obtain a classification result and an abnormal score; and feedback information is obtained according to the classification result and the abnormal score, and the check rule is optimized according to the feedback information. The invention further provides a data checking rule generation and optimization system and device and a storage medium. The method can be applied to business management program systems of financial science and technology and the like, and can update the rules in time under the conditions of complex data structure and changeable business logic.

The application provides a kind of operating system kernel complex data structure formal verification method, first based on the definition of in-block existence operation, the operation of data structure in kernel source program is mapped to abstract memory, the type, byte size and other information contained in the data structure object of program level are carried to memory model, the corresponding property of abstract memory level is obtained;Then according to the reasoning rule based on memory state semantics, the assertion of different states in the process of memory operation is obtained;Combined with the processing of complex structure description semantic model defined for abstract program layer, the extended Hall logic is obtained, and the reasoning is carried out through symbolic program logic;Then the logic reasoning and memory state assertion are handed over to automatic prover for proof, whether the related properties of complex data structure meet the expectation is verified automatically.The application realizes efficient and accurate formal verification of operating system kernel complex data structure.

The invention discloses an API interface test data processing method and device and a computer readable medium, relates to the technical field of software engineering and testing, and provides a test data cleaning and verifying method and system which are based on a dynamic rule engine and support complex JSON path analysis, dynamic encryption and decryption, multi-round session binding and agent collaborative verification. According to the method, the maintenance cost is greatly reduced only by adjusting a configuration file or a use case file, a dynamic encryption and decryption framework and a complex JSON path analyzer are built in, security data and complex data structures in a modern API interface can be efficiently processed, multiple rounds of interaction behaviors of a terminal user are accurately simulated through MAC, Group and Rotation triple binding and a context persistence mechanism, end-to-end testing is achieved, and the service life of the terminal user is prolonged. According to the method, user sessions are truly simulated, traditional deterministic assertion and AI agent uncertainty evaluation are fused, the verification range and capacity of automatic testing are greatly expanded, and man-machine cooperation intelligent verification can help developers to quickly locate the specific turns and reasons of defect occurrence and accurately locate faults.

The invention discloses a heterogeneous database synchronization method, electronic equipment and a computer readable storage medium, the heterogeneous database synchronization method is applied to a second database, and comprises the following steps: detecting whether a database table structure of a first database is changed or not; when it is detected that the database table structure of the first database is changed, generating ES mapping corresponding to the current database table structure of the first database; and the ES mapping is used for replacing the historical ES mapping which is stored locally. According to the method, whether data synchronization is needed or not is judged by detecting the change of the database table structure of the first database, asynchronous data updating is achieved by asynchronously updating ES mapping, the data updating efficiency is improved, it is ensured that a complex data structure is correctly stored in ES, and the consistency and accuracy of change operation and mapping updating are ensured.

The present invention relates to the field of digitally accessing, modeling and optimizing scattered ‘big-data’ from various assets having complex and dynamic physical relationships. The present disclosure also relates to a system and method for supporting version-of-the-truth from multiple data sources and for multiple disciplines. It helps organizations that possess various physical assets that are been utilized by multiple user groups and through a period of time to optimize operations, conduct surveillance and manage complex business processes related to their valuable assets and equipment. The invention is built with an open architecture to connect to various corporate databases combining real-time and relational data. The system employs exception-based surveillance, sophisticated queries, and no-code methods to automatically detect deviations in asset performance from optimal conditions, and flag them to the right users, at the right time, and in the right way. The present invention also allows monitoring the status and performance of equipment and facilities by presenting its findings through intuitive dashboards and automated workflows. In addition, a mobile version of the present invention is developed to help users execute day to day work as well as manage and maintain equipment and assets more efficiently.

The present disclosure relates to the technical field of data processing, and provides a business data desensitization method and device, electronic equipment and a readable storage medium. The method comprises: performing meta-information analysis processing on to-be-processed business data to obtain meta-information data corresponding to the to-be-processed business data; in the case that the meta-information data comprises a class level sensitive annotation mark, performing rule dynamic recursive desensitization processing on the to-be-processed business data based on the meta-information data and scene parameters to obtain a target desensitization field; and performing reflection setting processing on the target desensitization field based on the to-be-processed business data to obtain target desensitization business data. In this way, the data structure support degree is enhanced, the dynamic adaptability of the scene is improved, the complex data structure processing efficiency is improved, the automation degree of data desensitization is improved, the flexibility and scalability of the desensitization rule are improved, the scene-based dynamic rule adaptation is realized, and the coupling degree of the business code and the desensitization logic is reduced.

The invention discloses an Electron-based version update log generation method and device and a medium, belongs to the technical field of front-end engineering and Electron application development, and is used for solving the technical problems that in the existing Electron project application publishing process, a version log only supports a character string form, cannot express a complex data structure and is difficult to retain structured data features. The method comprises the following steps of: defining and generating structured log data based on a standardized JSON (JavaScript Object Notation) file; carrying out related YAML format content conversion processing on the analyzed log data, so as to obtain a release-notes. Md file; the method comprises the following steps of: carrying out packaging processing on a release-notes. Md file; a latest.yml file is identified and read through the post hook script after packaging processing and based on the resource storage path; the content data in the latest.yml file is converted and repaired, and the repaired JSON data is obtained; and carrying out write-back processing on the repaired JSON data, and generating a new laste.yml file.

The invention discloses a WebAssessment type inference method and device based on an enhanced semantic map, and a storage medium. Firstly, deep memory access semantics are extracted from WebAssembly binary codes, then an enhanced semantic graph containing access mode clustering, dependency chain analysis and type propagation paths is constructed, finally, the semantic graph is converted into high-precision type definition and data structure layout, and accurate inference of a complex data structure is achieved. Specifically, deep semantic association can be found from a seemingly irrelevant memory access sequence, and a logic structure of an original program is reconstructed. According to the method, through the construction of the enhanced semantic map, a complete definition including a nested structure, a complex, an array and other complex data types can be accurately identified and reconstructed, and the reverse analysis precision of WebAssembly codes is remarkably improved.

The invention relates to the technical field of power system automation, and provides an edge gateway modeling method and system based on data format fusion, and the method comprises the steps: fusing the data format of multi-source heterogeneous protocol data into a JSON format; the equipment is accessed to the edge gateway based on the data format of the JSON format and the plug-and-play technology; and carrying out identification, fusion and simulation verification on the data format of the protocol data of the equipment accessed to the edge gateway, and completing modeling after the verification is passed. The model file in the JSON format can represent a complex data structure and various data types, and is suitable for different devices and communication protocols. Therefore, the configuration of the edge gateway machine has high flexibility. The model file in the JSON format has readability, and adjustment and optimization of equipment parameters and communication protocols are facilitated.