54 results about "Type conversion" patented technology

The invention relates to the technical field of data processing, in particular to an LLM-based multi-source heterogeneous data intelligent fusion and self-adaptive processing method, which comprises the following steps of: receiving multi-source heterogeneous data, matching character sequences of mailbox fields and identity card number fields in records one by one by adopting a preset regular expression model, according to a preset key value pair mapping dictionary, items are uniformly converted, data records which cannot pass format verification, type conversion and value domain mapping are marked as doubt, and data records which cannot be processed or are marked as suspicious are separated. According to the method, a hierarchical processing flow is constructed, and deterministic format verification, type conversion and value domain mapping are arranged at the front end of a processing link, so that most simple data quality problems can be quickly processed with low calculation overhead. Then, numeric statistics and character string comparison are used to carry out fuzzy correction on the data, and spelling errors and numeric abnormities which cannot be covered by rules are further solved.

The invention relates to a data synchronization method and system for intelligent dirty data detection and restoration based on DataX. According to the method, an intelligent dirty data processing engine is embedded between a Reader plug-in and a Writer plug-in of DataX, type matching, format verification, constraint conflict pre-detection and abnormal value recognition are completed online, and type conversion, format standardization, abnormal value replacement or flexible writing are carried out on dirty data according to a preset or self-adaptive strategy. And finally, the Writer executes insertion, updating, skipping or log recording only according to an instruction, and a traceable governance log is generated in the whole process. According to the method, the synchronization-cleaning capability is embedded into the DataX framework, so that the synchronization success rate and the data quality are remarkably improved, the external ETL dependence and the artificial script cost are reduced, and the method has the advantages of high intelligence, high throughput and flexible configuration.

The invention relates to the technical field of data processing, and particularly provides a multi-source data-based conversion method and system, comprising: acquiring original data from a plurality of data sources, the data sources including but not limited to a structured data source, a semi-structured data source and an unstructured data source; performing format unification on the cleaned data according to a preset standardization rule, wherein the standardization rule comprises a coding format, field mapping and data type conversion; carrying out encryption processing on the standardized data, and setting a data access permission to ensure the security and privacy of the data; the standardized multi-source data are fused, redundant information is eliminated, and consistency and integrity of the data are guaranteed; storing the fused data into a target database, and supporting dynamic expansion and data version management; standardized data services are provided for external applications through an API interface, and real-time query and analysis of data are supported. The method has the effects of protecting the data security and encrypting the data.

One or more embodiments of the specification provide a data processing method and device applied to a database system. After triggering a data type conversion task, the data processing method obtains a to-be-converted floating point number of the data type conversion task, converts the to-be-converted floating point number into a first floating point number represented by an extended precision, and then calculates a second floating point number scaled to a preset numerical interval through multiple loop integer calculations to perform accurate rounding. According to whether the decimal part of the intermediate value of the last loop integer calculation and the distance between the upper interval and the lower interval of the last loop integer calculation are both greater than the error value of the last loop integer calculation, in the case of satisfying the error value of the last loop integer calculation, the second floating point number is converted into a fixed point number according to the integer part of the intermediate value of each loop integer calculation and the rounding result of the last loop integer calculation, to serve as the execution result of the data type conversion task.

This invention relates to a reliable translation method for synchronous dataflow languages ​​based on the 8114 criterion, belonging to the field of code generation. The invention obtains the abstract syntax tree of the Ctemp intermediate language through lexical and syntactic analysis and translation of Lustre source code; it traverses the Ctemp abstract syntax tree and designs translation methods for control flow statements and assignment statements within the tree structure; it traverses expressions within statements and designs translation methods that add type casting to ensure that statements are semantically correct while avoiding implicit type conversions; it extends the theorem on type relations in Cltype for verifying type operations; and it extends the theorem on type casting and binary operations in Ctemp semantics to achieve semantic formal verification. This invention significantly improves the compliance of C code generated by code generation tools with the GJB 8114 specification.

The application relates to the technical field of computer programming, and discloses a Go language structure body automatic conversion method and system. The method first receives the definitions of a source structure body and a target structure body as input, automatically matches the fields of the two structure bodies according to preset rules through a field mapping rule engine, and generates a field mapping relationship; then, based on the field mapping relationship, a code generator is used to automatically generate conversion code containing field assignment and type conversion logic; finally, the generated conversion code is compiled into an executable conversion function, which can convert and fill the data in the source structure body instance into the target structure body instance according to the field mapping relationship during program running. The method realizes the automation of structure body conversion, avoids the error-prone problem of manually writing conversion code, improves the maintainability of the code, significantly improves the conversion efficiency by replacing the runtime reflection mode with code generation, and simultaneously supports batch conversion and concurrent processing, and can meet the performance requirements of large-scale data processing.

The application relates to the technical field of deep learning compilation, and provides a method, device, equipment and medium for converting an MLIR file into a PyTorch file. The method comprises the following steps: obtaining an MLIR file, extracting data information and operation information in the MLIR file; retaining original identifiers of the data information and converting data types of the data information into Tensor data types in a PyTorch file; constructing a dependency relationship between operations according to a front-back order relationship between the operations; converting operation types in the MLIR file into operation types in the PyTorch file according to a conversion relationship between the PyTorch file and the MLIR file; and forming the PyTorch file according to the converted data types, the dependency relationship between the operations and the converted operation types. The method can equivalently convert the MLIR file into the corresponding PyTorch file.

This application discloses an automatic code generation method and system for SOME / IP data types, relating to the field of vehicle system development. The method includes: S1, data dictionary reading: obtaining the data dictionary for SOME / IP protocol communication; S2, data type classification, identification, and processing: opening the Excel file from step S1, classifying each row of data in the worksheet, and performing specific parsing processing for different types; S3, multi-round iterative dependency resolution: using a multi-round scanning strategy to resolve nested dependencies between various data types; S4, naming standardization processing: standardizing the names of each parsed data type according to preset naming rules; S5, standard code generation and output: converting all data types that have undergone naming standardization processing into type definition code conforming to C language specifications; simultaneously converting the value range, precision, unit, and Chinese description metadata in the data dictionary of the Excel file into code comments, generating a C language header file that can be included in the compilation project.

The present disclosure provides an operation unit, related devices and methods. The operation unit comprises: a first format conversion unit configured to convert data in a source data format into an intermediate data format according to a first rule; and a second format conversion unit configured to convert data in the intermediate data format into data in a target data format according to a second rule. The intermediate data format comprises all fields of the source data format and all fields of the target data format, and the number of bits in each field of the intermediate data format is not less than the number of bits in the corresponding field of either the source data format or the target data format. The embodiments of the present disclosure improve the versatility of hardware implementation of data type conversion and reduce hardware resource consumption caused by data type conversion.

This invention discloses a method for implementing a multi-source heterogeneous data transformation pipeline for time-series databases, comprising: parsing, transforming, and modeling the original data according to configuration rules to obtain a streaming transformation pipeline including a parsing stage, a transformation stage, and a modeling stage connected in sequence; in the parsing stage, parsing various data formats of the original data and converting the fields extracted after parsing into data in a unified columnar storage format; in the transformation stage, filtering, mapping, calculating, and type conversion operations are performed on the data processed in the parsing stage; in the modeling stage, the data processed in the transformation stage is mapped to the super table and sub-table structure of the time-series database according to the configuration rules; the modeling process includes: dynamic table name generation based on template expressions, extraction of label fields and data fields, and batch data assembly; the configuration rules specifically include loading, validating, and hot-updating the configuration file of the streaming transformation pipeline.

Systems and methods for generating synthetic training data are disclosed. A method may include: (1) receiving user speech from a user; (2) generating an input file comprising text of the user speech; (3) extracting entities from the text in the input file; (4) creating an input data structure for a data structure for the entities, wherein the input data structure comprises a plurality of columns, a column name for each column, a data attribute for each column, and a data type for each column, and a number of records based on a volume parameter; (5) converting the data type for each column to an ANSI SQL-standard data type; (6) generating a database agnostic data structure having the column names and the ANSI SQL-standard data type; (7) generating synthetic data for the database agnostic data structure; and (8) outputting an output file comprising the synthetic data.

The invention provides a task execution method based on a deep learning model, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, in particular to the fields of deep learning, model compression, model quantification, generative models, pre-training, large models, large language models and the like. The specific implementation scheme is as follows: in response to a task execution instruction, acquiring input data and quantization parameters for a quantized previous model in a plurality of models; according to the quantization parameter of the previous model and the input data of the previous model, performing data type conversion to obtain converted input data; according to output data of the previous model, calibration data of the current model in the multiple models are determined, and the output data of the previous model are obtained by inputting the converted input data into the quantized previous model; based on the calibration data of the current model, the calculation unit determines a quantization parameter of the current model and stores the quantization parameter of the current model to the storage unit.

The embodiment of the application discloses a model data dequantization method, a matrix operation method and related equipment, wherein the dequantization method comprises the following steps: determining first floating point data to be dequantized in a model; regarding the first floating point data as first integer data with the same bit width as the first floating point data, performing first type conversion processing from the first bit width to the second bit width to generate intermediate floating point data; performing second type conversion processing on the intermediate floating point data to maintain the second bit width to generate second integer data; and performing format correction processing on the second integer data to generate second floating point data after dequantization. The application can simplify the dequantization process, improve the calculation efficiency, adapt to some chips with instruction set restrictions and lack of support for special computing instructions for low-precision floating point data, and improve the overall performance of artificial intelligence model training.

The invention provides a method and a system for performing bidirectional data type derivation in graphical programming, and relates to the technical field of graphical programming, and the method comprises the following steps: S1, analyzing graphical programming data into a graph structure and a tree structure; s2, data type derivation from outside to inside is executed, specifically, on the basis of the tree structure, all graph structures are traversed according to the depth priority sequence, internal nodes of the currently traversed graph structure are traversed according to the topological sequence of a directed acyclic graph, and the data type of downstream nodes is derived according to the data type of upstream nodes in the data flow direction; s3, data type derivation from inside to outside is executed, specifically, on the basis of the tree structure, all graph structures are traversed according to a subsequent sequence, for the currently traversed graph structure, the data type of an output boundary of the currently traversed graph structure is determined according to the data type of internal nodes of the currently traversed graph structure, and the data type is reversely propagated to a corresponding node in a parent graph of the currently traversed graph structure; and S4, automatically generating a type conversion code according to a predefined data type processing rule.

The invention relates to the technical field of compressible turbulence intensity parameter prediction, in particular to a compressible turbulence intensity parameter prediction method based on full-connection neural network five-fold cross validation, and the method comprises the steps: constructing a two-dimensional input feature vector comprising a transmission distance, label data, beam width and interaction features; performing robust scaling and standardization on the transmission distance, the beam width and the interaction characteristics, performing normalization processing on the label data, converting the processed transmission distance, the processed label data, the processed beam width and the processed interaction characteristics into a tensor format, and converting the type of a tensor element into a floating point type; constructing a self-defined full-connection neural network model; dividing a data set through five-fold cross validation, and dynamically storing model parameters with minimum validation loss; carrying out model training; multiple models are integrated to predict a test set, the value of the final compressible turbulence intensity parameter C2 is output through mean value calculation and reverse normalization, the compressible turbulence intensity parameter C2 can be efficiently predicted, and the prediction result is high in precision.

This application provides a data processing method, which includes: determining source data to be printed, the source data including multiple sets of variable data, with different sets of variable data having different change attributes; extracting each set of variable data from the source data; performing data type conversion processing on each set of variable data to obtain target variable data corresponding to each set of variable data, and assigning different target variable data to different identification information; identifying the target variable data based on the identification information, and performing matching processing on the target variable data corresponding to different sets of variable data, and performing fusion processing on the target variable data that meet the matching conditions to obtain multiple fused data; and segmenting the fused data to obtain multiple target printing data for printing. In this way, the fusion and printing of multiple sets of variable data with different change attributes can be realized, thereby better meeting the printing needs of users. This application also provides a digital printing method and system.

The invention provides a data synchronization configuration optimization method and device based on metadata automatic discovery and electronic equipment, and relates to the technical field of data optimizing.The method comprises the steps that metadata obtaining logic of heterogeneous data sources is abstracted through a uniform interface, and table structures, field types and primary key constraints of the data sources are automatically connected and analyzed; based on a declarative configuration engine, automatically generating a field mapping rule and a target end DDL statement according to a source library name, a table name and a target end address input by a user; a source end DDL event is monitored in real time by integrating a change data capture technology, and a target end table structure is updated by adopting a dual-stage submission protocol; and constructing a cross-data source type mapping knowledge base, and dynamically adapting a type conversion rule in combination with semantic reasoning. According to the method, the metadata acquisition logic of the heterogeneous data source is abstracted through the uniform interface, the zero configuration access of a traditional database, a big data component and an API interface is supported, the compatibility is improved, and the tedious process of manually compiling field mapping rules and type conversion scripts is eliminated.

The application discloses a conversion method and system from RSML ‑e model to Luster synchronous data flow language. ‑e The method converts RSML ‑e model into Luster, which involves type conversion, Boolean expression conversion, and-or table conversion, variable conversion, macro conversion and function conversion. Type conversion handles undefined values by creating enumerations and structs. Boolean expression conversion includes truth judgment, logical negation, previous state reference and equality judgment. and-or table conversion constructs sub-expressions by columns and connects them with logical or. Variable conversion is converted into Luster nodes, and ordinary variables and input variables are processed with initial values and conditions respectively. Macro and function conversion are converted into nodes with input and return values, and the contained logic is converted into Boolean expressions. The overall process realizes the mapping of RSML ‑e logic to Luster data flow, so that the safety properties of the system can be verified using model checking tools, improving system security.

The application discloses an annotation-driven business object change recognition and log generation system and method, and the system is based on a four-layer architecture design and comprises an annotation-driven module, a type strategy module, a reflection processing module and an ID conversion module.The annotation-driven module is configured to compare annotation-related parameters and support log output and field comparison.The type strategy module is configured to realize a strategy mode through comparison enumeration, and each data type corresponds to a comparison function interface.The reflection processing module is configured to obtain all fields of a current class and parent classes through recursive scanning, support complete field scanning of inheritance relationships, avoid repeated reflection operations through a class field cache mechanism, and the ID conversion module is configured to define a unified ID conversion specification through a functional conversion interface, provide a plurality of conversion implementation classes, and configure a conversion interface singleton cache mechanism.The scheme realizes automatic recognition of business field changes, type conversion and log generation, supports unified processing of a plurality of data types, and significantly improves development efficiency and system maintainability.

The invention relates to the technical field of data input and processing, and discloses a context sensing type intelligent input system and method supporting multi-modal input, and the system comprises a multi-modal input processing module which is used for receiving texts, codes, expressions or graphic data input by a user, carrying out the format recognition and structure verification of the input data, and outputting a multi-modal input result; uniformly converting the data into structured input representation, and outputting the structured input representation to an intention recognition module; the intention recognition module is used for receiving the structured input representation, recognizing the fixed text content, the dynamic field reference and the expression content in the structured input representation and generating a corresponding structured label, and the structured label is used for the type conversion module to carry out data type analysis. By adopting the technical scheme of multi-modal input unified identification and structure verification, the technical effect of automatically normalizing input data in various forms of texts, graphs, codes and expressions is achieved, and the problems of poor compatibility, low efficiency and difficulty in extension are solved.

A converter for data type conversion includes a first conversion stage and a second conversion stage. The first conversion stage is configured to receive first type data and first descriptive information about the first type data, and according to the first descriptive information, convert the first type data into an intermediate result. The second conversion stage is configured to receive second descriptive information about second type data, and according to the second descriptive information, convert the intermediate result into the second type data. A method for data type conversion includes: receiving first type data and first descriptive information about the first type data, and according to the first descriptive information, converting the first type data into an intermediate result; and receiving second descriptive information about second type data, and according to the second descriptive information, converting the intermediate result into the second type data.

Systems and methods for migrating data are provided. A migration application communicates with a source database and a target database, and is configured to migrate data therebetween. It automatically detects that, for a source data field in the source database that is to be migrated to a target data field in the target database, the source data field is of a first data type and the target data field is of a second data type that is different from the first data type. The migration application automatically obtains a conversion rule in relation to the first data type and the second data type. The migration application automatically executes the conversion rule to convert data in the source data field to converted data that is of the second data type. The migration application automatically migrates the converted data to the target database.

A set of processing core resources of a database system is operable to obtain the query statement regarding a set of data values of a dataset. The set is further operable to obtain source data type information regarding source data type of the set of data values and target data type information regarding target data type regarding resulting data of the query statement. When the source data type does not match the target data type, the set is operable to access a data type conversion path list to identify a set of entries. The set is operable to determine, for the set of entries, whether an entry of the set of entries includes a source data type that corresponds to the source data type information. When the entry includes the source data type, the set is operable to output a representation of data conversion path information of the entry.

The application discloses a processing method and an electronic device, the method comprises a processing method applied to a first data party and a processing method applied to a second data party, the processing method applied to the first data party comprises: obtaining second-type conversion data sets and position mapping information of the second data party, and obtaining the second-type conversion data sets of the first data party according to the position mapping information; determining whether second-type conversion data, which is consistent in identification between the second-type conversion data sets of the two data parties, satisfies a matching condition; and determining, as target data objects that need to be cleaned by the first data party, data objects corresponding to second-type conversion data that does not satisfy the matching condition in the second-type conversion data sets of the first data party. The two data parties respectively have data objects comprising identification and label data, and the second-type conversion data sets of the two data parties respectively comprise results obtained by converting label data corresponding to consistent identification between the two data parties according to a second-type conversion rule.

The present application relates to the field of big data, and discloses a data type conversion method, device, equipment and storage medium.The method comprises the following steps: obtaining a first storage data table corresponding to a target data object, and traversing the first storage data table to obtain a plurality of key-value pairs; encoding the key field in each key-value pair to obtain first PB field information corresponding to each key-value pair, and converting the value field in each key-value pair according to the key field to obtain first PB data information corresponding to each key-value pair; and combining the first PB field information and the first PB data information to obtain a first PB data set corresponding to the target data object.The present application realizes automatic conversion of PB data, reduces the number of compiled files, and reduces the complexity of PB data maintenance.

The invention discloses a zero-intrusion-oriented compiling period type conversion generation method. The method comprises the steps that a type conversion annotation in a source code is scanned and analyzed, a generation task is constructed for the type conversion annotation, and the generation task indicates annotated element information and a target type expression; for each generation task, respectively constructing a source type descriptor and a target type descriptor for the annotated element and the target type expression, the source type descriptor and the target type descriptor being based on the same data model; performing consistency verification on the source type descriptor and the target type descriptor in each generation task; and for the generation task which can be subjected to safety conversion according to the consistency verification judgment, a converter class code with a safe type is generated, safety enhancement logic based on a type conversion annotation is embedded in the converter class code, and safe type conversion is realized when the converter class code is called. According to the embodiment of the invention, on the premise of not changing service logic, the converter class code with the safe type can be automatically generated.

The invention discloses a geographic entity construction method and device based on DLG data analysis, and relates to the technical field of data analysis. Analyzing the DLG data in the graphic file format into space vector data, generating a vector data set, completing element type conversion, and synchronously extracting text attributes associated with elements; traversing the vector data set, and performing original structured classification on the element objects based on a preset rule; selecting a data extraction method to extract geographic entity elements according to the difference between the original structured classification and the geometrical morphology to be generated in the mapping table; and according to casscodes carried by the elements converted into the geographic coordinate system, performing contrast mapping with classification codes of basic geographic entity elements, and respectively outputting extraction results to corresponding element data sets according to geometric types. The problems that an existing conversion method is poor in universality, and most conversion tools cannot independently complete batch processing of graphs and are not closed are solved.

The application provides a kind of SQL sentence dynamic construction method and device based on FlinkCDC, applied in deserializer, comprising: receiving the acquisition data sent by collector, and parsing the acquisition data into general data;Based on the type of write-in library, obtain corresponding type parser and SQL builder;Based on field type, use type parser to type conversion on the field value before change and / or field value after change in general data, obtain the converted field value before change and / or field value after change;Sentence operation type in general data, and the field value before change and / or field value after change after type conversion are input into SQL builder, and SQL sentence that can be normally executed in write-in library is constructed.The application can automatically type conversion and restore executable SQL according to different write-in library types, solve the problem of data conversion failure or precision loss, and has good expansibility and reusability.

The application discloses a model conversion method based on Flowable workflow nested JSON, and relates to the technical field of computer software; JSON structure analysis is carried out: the hierarchical structure of the nested JSON is recursively traversed, the paths of all fields are extracted, the JSON fields are corresponded with the attributes of the target data model based on predefined field mapping rules, Flowable node configuration is carried out: the JSON conversion node is embedded in the Flowable process, which is used for carrying out the field mapping rule configuration and performing the conditional judgment of the dynamic script for the data type conversion, the nested JSON is converted into the target data model through the JSON conversion node according to the mapping rule and the script logic, and the conversion result is transmitted to the subsequent node through the Flowable process variable or is persisted to the database.