1055 results about "Data type" patented technology

The invention discloses a mineral resource intelligent prediction method and system based on multi-source heterogeneous data fusion and deep learning, and the method comprises the steps: collecting and preprocessing multi-source heterogeneous data, and carrying out the standardization processing to form a structured data set; multi-source heterogeneous data fusion: realizing data layer space registration and feature layer weight dynamic allocation through an attention mechanism multi-modal fusion module, and outputting a high-dimensional metallogenic feature vector; constructing a CNN-LSTM mixed deep learning model and completing initialization training, and outputting an initial mineralization probability graph; and establishing a dynamic updating engine, performing model increment training based on transfer learning, correcting the mineralization probability through positive and negative sample reinforcement learning in combination with a newly added data type, and outputting a time sequence dynamic mineralization probability graph. According to the method, mineralization probability dynamic evaluation and risk quantitative updating are realized, the prediction precision and the model updating efficiency are improved, the method is adaptive to a multi-stage exploration scene, and accurate real-time support is provided for exploration decision making.

The invention discloses a method for simulating and forecasting flood in a cold highland area based on hydrological and hydrodynamic coupling, and belongs to the technical field of disaster forecasting. The method specifically comprises the following steps: S1, multi-source basic data collection and preprocessing: collecting multi-type and multi-scale basic data for a target cold and cold mountainous area drainage basin; and S2, deep learning correction and fusion of the satellite rainfall data: aiming at the local overestimation and underestimation problems of the satellite rainfall data, a deep learning algorithm is adopted to carry out hour scale correction and fusion. Four types of core data of satellite remote sensing, reanalysis, ground observation and geographic space are collected, total factors of'rainfall-runoff-terrain-underlying surface 'required by flood simulation in the cold and cold mountainous area are covered, simulation one-sidedness caused by lack of data types in traditional modeling is avoided, rainfall and runoff abnormal values are eliminated by adopting a 3-sigma criterion, data formats and spatial-temporal scales are unified, and the modeling efficiency is improved. A standardized data set is formed, and interference of abnormal values, format incompatibility and space-time mismatching on subsequent model input is avoided.

The invention discloses a privacy protection data sharing and cooperative training method and system based on federated learning. The method comprises the steps of receiving software development log data, adaptively judging the sensitivity degree according to a data type, dynamically adjusting noise disturbance intensity according to the sensitivity degree to perform data desensitization, and generating a sensitivity index; selecting a feature extraction strategy, extracting time sequence correlation features from the desensitization data, constructing a dynamic graph structure with a weight, and obtaining a time sequence feature vector through iterative fusion; calculating the time sequence correlation of the time sequence feature vector to obtain a data quality score, and setting a contribution weight based on the quality score to perform parameter aggregation; combining sensitivity indexes with data quality scores to construct a security sharing domain, decoupling global training parameters into knowledge fragments in the domain, formulating a recombination rule, and selectively acquiring the required knowledge fragments by all parties for local training. According to the method, deep collaboration is realized on the premise of protecting data privacy, and the collaboration training effect is improved.

The invention provides a self-adaptive semantic-driven data set field matching method and system, and the system comprises a data preprocessing module which is used for carrying out the cleaning, standardization and preliminary analysis of an input data set, and extracting a field name, a data type, a field description and a data sample; the deep semantic representation modeling module is used for constructing a field-level semantic representation vector; the multi-level similarity calculation module is used for comprehensively calculating the grammatical similarity, the semantic similarity and the statistical similarity among the fields, dynamically adjusting the weight of the similarity of each level by adopting a weighted fusion algorithm, and generating a comprehensive similarity matrix; and the matching result management and application module is used for generating a field matching mapping table and a fusion suggestion according to the comprehensive similarity matrix. According to the method, high-precision automatic matching of data set fields is realized by fusing deep semantic understanding, multi-dimensional similarity calculation and incremental adaptive learning, and the efficiency and accuracy of data set fusion are remarkably improved.

The invention discloses a scene and compliance fused dynamic hierarchical data encryption method and system, and the method comprises the steps: obtaining multi-source data, and carrying out the preprocessing of the multi-source data, and obtaining to-be-encrypted data; key words in the to-be-encrypted data are extracted, and weight coefficients are distributed to the key words according to semantic roles and / or industry risk coefficients; according to the data type, determining a characteristic value corresponding to each keyword; determining a scene coefficient and a compliance coefficient; based on the weight coefficient, the feature value, the scene coefficient and the compliance coefficient, determining a data sensitivity score and a data sensitivity level corresponding to the to-be-encrypted data; and based on the data sensitivity level, encrypting the to-be-encrypted data by adopting the encryption strategy of the corresponding level. According to the method provided by the embodiment of the invention, the security of the high-sensitivity data is ensured, the resource waste caused by excessive encryption is avoided, the whole process is automatic, the labor cost and the overall cost of security protection are greatly reduced, and the comprehensive efficiency of data processing and security protection is improved.

The invention discloses a system and method for ensuring data consistency of a low-voltage power distribution network, and the method comprises the following steps: obtaining multi-source operation data uploaded by all collection devices in the low-voltage power distribution network, and converging the multi-source operation data to a unified processing channel; a timestamp dynamic alignment and mapping mechanism is adopted, sampling frequency and time offset are detected and corrected, and a unified time axis is generated; executing unit conversion and field unification operation according to the data type label, and constructing a standard field structure; carrying out prediction compensation on missing data by adopting a time sequence sliding window interpolation model; a sliding window anomaly rate detection strategy is applied to identify numerical value abrupt change and execute anomaly replacement; and performing structured output on the data set subjected to the consistency processing. According to the method, fine alignment, standardized processing and high-reliability correction of multi-source heterogeneous data can be realized, and the data consistency level and decision support capability of the low-voltage power distribution network in operation monitoring, analysis and control scenes are improved.

The invention provides a time sequence data storage method and system, electronic equipment and a storage medium, and the method comprises the steps: obtaining time sequence data and metadata information of the time sequence data, the metadata information comprising a first point location identifier, a data source and a data type corresponding to the time sequence data; according to the data source and the data type, determining a storage strategy of the time series data, including a target storage period and a target storage position; according to the storage strategy and the first point position identification, a target storage instance corresponding to the time sequence data is matched from a storage instance set, all storage instances in the storage instance set are combined and constructed according to different storage strategies and the point position identification, and all the storage instances are parallel storage instances; and storing the time sequence data to a time sequence database through the target storage instance. By determining a dynamic storage strategy driven by metadata and a parallel storage instance, the problem of performance bottleneck during high-concurrency writing of massive time series data is solved, and the real-time performance and reliability of time series data storage are ensured.

The invention discloses a stacking-based steel hot-rolled product mechanical property prediction method, and relates to the technical field of steel product quality prediction. The method comprises the following steps: constructing a steel hot rolling actual production data set and carrying out missing value interpolation on the data set; performing normalization processing on the complete data set, and dividing a part of data from the normalized data set as a training set; establishing a stacking ensemble learning model based on a regression chain; taking the yield strength, the tensile strength and the elongation in the training set as training labels, taking other data types as training features, and training the learning model by using the training set to obtain a steel hot-rolled product mechanical property prediction model; and the steel hot-rolled product mechanical property prediction model is practically applied to perform real-time prediction on the mechanical property of the steel hot-rolled product. According to the method, the coupling relation among the multiple target variables can be effectively processed, and the accuracy and stability of mechanical property prediction of the steel hot-rolled product can be effectively improved.

The invention belongs to the field of software testing and defect detection, focuses on reliability verification of a physical simulation engine, is suitable for scenes depending on physical simulation, such as robot simulation, automatic driving testing, unmanned aerial vehicle control, education and scientific research, and particularly relates to a variation-based physical simulation engine defect detection method. Potential defects of core modules of collision detection, mechanical calculation, dynamics and the like are efficiently found through a variation test method. According to the method, a feature library is constructed through static library API analysis and AST / IR code analysis, test code expansion is performed by utilizing a large model, a test driver adaptive to a specific data type is generated, a physical semantic constraint mutation seed is combined, a test is executed through libFuzzer, defect and coverage rate data are analyzed, a use case selection strategy is optimized, and the test efficiency is improved. Defects of core modules such as collision detection and dynamic solution are efficiently detected, and the test pertinence and efficiency are improved.

The invention relates to a tensor data processing method, a tensor data processing device, electronic equipment and a computer readable storage medium. The method for processing the tensor data comprises the steps that a first tensor data block is obtained, and the first tensor data block comprises at least one scalar element; determining, based on the first tensor data block, information indicating a type domain of a second tensor data block, the type domain indicating a data type of a private element of the second tensor data block; and converting the first tensor data block into the second tensor data block based on the information indicating the type domain of the second tensor data block.

Methods, systems, apparatuses, and computer program products are described. A system may obtain a set of metadata associated with multiple entities of a datastore, the multiple entities including at least two different data types. The system may augment, using a generative artificial intelligence (AI) model, the set of metadata resulting in a set of augmented metadata for the multiple entities of the datastore. The system may generate, using the set of augmented metadata, a set of entity objects for the multiple entities of the datastore and in accordance with a reference schema. The system may identify, using the generative AI model and the set of entity objects as inputs into the generative AI model, first relationships between two or more first entities of the multiple entities of the datastore. Based on the identified relationships and a reference schema relationship model, the system may generate a schema relationship model.

The invention relates to the technical field of agent construction, in particular to an agent construction method based on a large language model, and the method comprises the steps: S1, configuring a scenarized execution specification which comprises the steps: receiving constraint information of an execution component in a scene according to a preset scene; converting the constraint information into a structured cue language which can be understood by a large language model, and generating a standard verification rule base which can be analyzed by an execution component; and storing the structured prompt words and the standard verification rule base. According to the method, by configuring the scenarized execution specification, the constraint information of the execution component is converted into a structured cue which can be understood by a large language model and a specification verification rule base which can be analyzed by the execution component, and the problems of non-explicit constraint such as inconsistent parameter naming and non-standardized data type are solved.

One or more computing devices, systems, and / or methods for privacy data augmentation are provided. An augmentation pipeline is selected to process data based upon a data type of the data. The augmentation pipeline processes the data to generate information that is input into a machine learning model. The machine learning model processes the information and privacy laws to determine a subset of the data to mask. In this way, the subset of the data is masked to create augmented data that complies with the privacy laws.

The invention discloses a semantic matching-based field name standard abbreviation mapping method and system, and particularly relates to the technical field of data governance, a source field set to be subjected to abbreviation and a standard field set in a standard field library are acquired, field names and field descriptions are normalized, and field text representation is generated; using the pre-training language model to generate a semantic vector and recalling a candidate mapping pair set; calculating character string similarity, synonym expansion matching score, data type consistency score and sample value pattern consistency score for the candidate mapping pairs; generating a neighbor confusion risk index based on semantic neighbor density, entity category distribution deviation, synonym expansion and context anti-verification, performing risk gating correction, and judging a score in combination with a difficult case; and applying table-level consistency constraints in the same service table or entity domain to carry out joint consistency solution to obtain a table-level optimal mapping scheme, generating confirmation confidence, and outputting an automatic confirmation or manual recheck mapping result.

The invention discloses an automatic interlocking test method and system based on a RobotFrameWork framework, and belongs to the technical field of interlocking tests.The automatic interlocking test method comprises the steps that on the basis of a preset rule and a general test case, function logic decoupling is conducted on a test script in combination with the RobotFrameWork framework, and function keywords are obtained; obtaining a configuration file according to the interlocking function and the test data type, and performing feature extraction on the to-be-tested data and the interlocking table based on the configuration file to obtain associated test data; and according to the test requirements and the associated test data, performing testing in combination with the function keywords. The method comprises the following steps: performing function decoupling on a test script through a keyword driving characteristic of a RobotFrameWork framework to obtain a function keyword which only encapsulates single function logic, and extracting associated test data related to a test from to-be-tested data and an interlocking table through an interlocking function defined by a configuration file and a mapping rule of a data type. The technical problems that test efficiency is difficult to improve and script maintenance is difficult in the prior art are solved.

The invention discloses a silicon nitride ceramic thermal conductivity prediction method based on machine learning coding-free feature processing. The method comprises the following steps: acquiring a data set; declaring a chemical character string field of the sintering aid as a disordered data type, and preprocessing other numerical characteristics at the same time; keeping the field of the sintering aid as a disordered data type, and preprocessing other numerical values; selecting a gradient lifting decision tree model supporting native processing category features as a prediction model and performing optimization; a to-be-predicted formula-process parameter combination is input into the optimized prediction model, a thermal conductivity prediction value is output in real time, and the to-be-predicted formula-process parameter combination is directly input into the model through an original chemical formula character string. The invention further discloses a device for implementing the method, computer equipment and a readable storage medium. According to the invention, the original chemical formula of the sintering aid can be directly utilized, and the thermal conductivity can be predicted within millisecond response time without any code conversion.

The invention discloses an RAG knowledge base construction method and device and a medium, and the method comprises the steps: carrying out the analysis of the format, structure and semantic features of input data through a multi-level priority judgment logic, and automatically recognizing the specific data type. And calling a corresponding adaptive segmentation strategy for processing according to the identified data type, and generating data slices with semantic coherence. And asynchronously calling the large language model to generate a keyword set for each data slice, and uniformly storing the data slices, the keyword sets and associated multi-modal resource links by adopting a mixed storage mode to construct an enhanced RAG knowledge base. In the retrieval stage, query keywords are generated for user query according to the same logic, and related results are accurately retrieved from the knowledge base through a keyword matching mechanism. According to the method, the semantic segmentation problem is effectively avoided through intelligent segmentation with adaptive types, multi-modal data access is supported, and unified management and efficient retrieval of heterogeneous resources are realized.

The invention discloses a data coding method and device, electronic equipment, a processor and a storage medium, and relates to the field of large language models. The method comprises the following steps: by taking a group consisting of a plurality of continuous elements in a channel as a unit, performing grouping quantization on reasoning parameters of a neural network model in a reasoning process to obtain a plurality of groups of quantized data, mapping a first group of quantized data into first cache data based on a first adjustment coefficient, and mapping a second group of quantized data into second cache data based on a second adjustment coefficient; the data distribution can be changed due to the first adjustment coefficient, so that the method can adapt to various data types, a flexible mapping mode is provided, and the data distribution of the mapped first cache data can fit the data distribution of the first group of quantized data before quantization. And the cached first cache data is the data after the reasoning parameters are compressed, so that the memory resources required by the reasoning parameter cache, such as weight, key data and value data, are reduced, and the reasoning speed of the large language model is further improved.

The invention discloses a bridge engineering knowledge base construction method and system, and the method comprises the steps: classifying the multi-modal data of each project in the bridge engineering field according to the data type, carrying out the structural conversion, forming a structural data document corresponding to each project, and constructing a modular parallel workflow; the data types comprise texts, images and voices; splitting each structured data document into document segments according to semantic features, and generating document segment vectors by using an embedded model; forming double indexes by using all document fragment vectors and an entity-relationship-attribute triple of the bridge knowledge graph, and constructing a structured data document library-document fragment vector library-knowledge graph three-layer knowledge base; and training a bridge large model based on implicit rule coding, explicit engineering constraint and a rule activation mechanism, and embedding the trained bridge large model into the three-layer knowledge base to complete construction of the bridge engineering knowledge base. Data are fully utilized, the coverage range of the knowledge base is expanded, a multi-dimensional evidence source is provided, and question and answer credibility is improved.

The invention relates to the technical field of image recognition, in particular to a building plane element recognition model construction method based on a convolutional neural network, and the method comprises the following steps: obtaining an input APN sample; elements in the APN sample are labeled, and a labeled file is generated; converting the annotation file into a format required by YOLO, and normalizing the annotation file; performing data enhancement on the APN sample, expanding the training data volume, and outputting to obtain an element detection result and a feature vector; installing dependency and carrying out data configuration; setting command line model parameters and customizing training scripts; and carrying out model training and model reasoning. According to the method, the plane layout is abstracted based on the machine learning method, and the vector data type has high flexibility and deformation capability, so that the vector output which keeps the original plane image form unchanged can be converted into various objects according to the purpose of a user. And classifying and identifying the plane elements by using the GNN.

The invention relates to the technical field of artificial intelligence, and particularly provides a heterogeneous data preprocessing method, system and device and a storage medium, and the method comprises the steps: obtaining multi-source heterogeneous original data, and analyzing the multi-source heterogeneous original data into intermediate format sample data; calling a large language model to generate a field semantic tag; converting the original field into a standard field in a standard field library through a field mapping rule; cleaning abnormal values by using a machine learning model to obtain a primary sample data set; performing semantic analysis on the unstructured text fields to generate structured feature vectors, and adding the structured feature vectors into a data set; and finally, performing feature derivation on the structured field according to the data type and the semantic tag, and outputting a final data set. According to the method, automatic processing from the original data to the training ready data set is realized, and the data preprocessing efficiency, the automation degree and the reliability are remarkably improved.

A method includes obtaining defect data and context data in association with a substrate, and providing the defect data and the context data to a first trained machine learning model as input. The method further includes obtaining output from the first trained machine learning model based on the defect data and the context data. The output is indicative of a predicted root cause in association with the defect data. The method further includes performing a corrective action in view of the output.

The invention discloses an intelligent inspection and fault diagnosis method for power equipment and related equipment, and the method comprises the steps: carrying out the preprocessing of multi-source data, and obtaining an adaptive diagnosis model and a trend prediction model through the matching of a diagnosis-prediction model according to the specific model, working condition and data type of the current equipment through the matching of a diagnosis model and a prediction model; performing anomaly diagnosis on the multi-source data by using the matched diagnosis model to obtain abnormal data; and finally, deep fusion with multi-dimensional information such as the historical state, the real-time load and the external weather of the equipment is carried out, and a matched prediction model is driven to carry out comprehensive analysis, so that sequential and personalized prediction of fault evolution is realized, the crossing from post-event alarm to pre-event early warning is completed, and the reliability of fault evolution is improved. And an operation and maintenance closed loop from accurate diagnosis to advanced prediction is also formed, and the adaptive capability of the system to different power transmission, power transformation and power distribution scenes is comprehensively enhanced.

The invention discloses a demand development method, device and equipment. The method comprises the following steps: calling a demand analysis agent to analyze a natural language demand into a structured function demand unit comprising an input specification, an output specification and an execution constraint; the calling code generation agent generates source codes corresponding to the development requirements according to data type definitions and function calling specifications in the structured function requirement unit and the local project code library; calling a code test agent to generate and execute a test case based on the source code and the structured function demand unit; and after determining that a software deployment condition is met based on a test result, calling a code deployment agent to generate a deployment script and configuration information, and submitting the deployment script and the configuration information to a deployment execution engine to complete automatic deployment. According to the application, end-to-end automatic development from natural language requirements to application deployment can be realized.

The invention relates to the field of medical data processing, and particularly provides a medical standard value code dynamic generation method and system, a terminal and a medium, and the method comprises the steps: receiving original data containing medical terms, and carrying out the feature analysis of the original data through a scheduling classification model, and generating an analysis result carrying a data type label; according to the data type label, an analysis result is routed to a target domain agent; calling a multi-modal large model through a target domain agent, and converting an analysis result into a unified multi-modal vector; dynamically activating at least one expert sub-network based on a hybrid expert model MoE gating mechanism, and processing the multi-modal vector through the expert sub-network to generate a standard value code candidate set in a candidate code space; and performing logic rule verification on the standard value code candidate set according to a pre-constructed medical knowledge graph, and outputting a final target standard value code. According to the invention, the efficiency and precision of medical standard value code conversion are improved, and the expandability is improved.

The invention discloses a hand-eye cooperative robot control system and method based on dynamic operator arrangement, an upper computer planning layer operates a master control computer to periodically trigger task scheduling, a joint controller of a lower computer execution layer receives an instruction through a redundant bus to perform servo control, hand-eye camera data triggers a visual assembly line through an interrupt event, and a visual assembly line is controlled through a control interface. According to the method, pressure is calculated through visual processing, motion planning and joint control in a load sharing mode, a double-buffering mechanism is adopted to enable current frame visual processing and previous frame motion control to be executed in an overlapping mode, real-time scheduling and parallel computing of tasks are completed, hot data are cached in a memory database, cold data are archived to an HDFS and migrated through an LRU strategy, and the real-time scheduling and parallel computing of the tasks are completed. Data type conversion is automatically derived and performed based on a feature type registry, and conditional branch execution is triggered according to real-time sensor data. According to the control system, cross-hardware plug and play, algorithm flow configurability and data flow real-time sharing are achieved, and the requirement for improving the efficiency of complex operation tasks is met.

The invention belongs to the technical field of data processing, and particularly discloses a multi-source heterogeneous data acquisition access method and system based on intelligent mapping and a dynamic interface, and the method comprises the steps: registering an input data source class, generating a data source identifier corresponding to the data type, and storing the data source identifier; respectively executing meta-model analysis operation to generate a standardized meta-model associated with the data source identifier and a container instance template; generating a mapping rule set through a twin neural network architecture with multi-modal feature fusion and a preset mapping rule; a middle layer API interface is constructed through interface logic, and real-time conversion and access between input and output data are achieved; and realizing containerized parallel access based on the container instance template and the containerized interface service instance. According to the method, data source change is dynamically adapted, the real-time performance and accuracy of the mapping rule are ensured, and meanwhile, multi-source heterogeneous data can be accessed efficiently in real time in combination with containerized parallel processing and a data source change monitoring mechanism.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling data access. One of the methods includes receiving, from a user device associated with an account, a request for data; accessing data from one or more data sources that includes, for each of a plurality of data types, corresponding data values; selecting, from the plurality of data types and using permissions data for the account, one or more data types that the account has permission to access; and providing, to the user device and for each of the one or more data types that the account has permission to access, the corresponding data values.

The invention discloses a column storage-based big data compression and query optimization method, which relates to the technical field of big data compression and query optimization, and comprises the following steps of: performing sliding window division on original data of column storage to obtain a plurality of data windows; calculating a feature vector based on the value distribution characteristics in the multi-data window, wherein the feature vector is used for representing a local statistical rule of the data; performing dynamic merging according to the similarity of the feature vectors of the adjacent data windows to form a compressed block; selecting a corresponding compression coding mode based on the data type and the distribution characteristic of the compression block, and generating a metadata index containing a minimum value, a maximum value, a coding type and a compression size; after receiving a query request, mapping a predicate condition to a compressed coding space, and executing non-decompressed filtering judgment on a compressed block by utilizing the metadata index and a compressed coding type; and determining a decompression range based on a filtering result of the compressed blocks, and only performing local decompression and data reduction on the possibly hit compressed blocks.

The invention discloses a water conservancy digital twin data fusion method, system and device based on multi-source heterogeneous data dynamic alignment and a medium, and belongs to the technical field of data fusion. Performing cleaning processing, format standardization processing and timestamp alignment processing based on a time window on the data to obtain preprocessed data; performing sliding window division processing on the preprocessed data, performing regional grid numbering processing according to a space division rule, and enabling each data point to correspond to a divided time window and a space grid to obtain time-space mapping data; and performing fusion processing on the data subjected to space-time mapping according to the data type. According to the method, the fusion strategy is designed by distinguishing the data types in the fusion stage, and it is guaranteed that the fusion result can still keep high reliability and stability when facing data of different qualities and different sources.