100 results about "Type conversion" patented technology

The invention relates to the technical field of computer programming, and discloses a Go language structural body automatic conversion method and system. The method comprises the following steps: firstly, receiving definitions of a source structure body and a target structure body as input, automatically matching fields of the two structure bodies through a field mapping rule engine according to a preset rule, and generating a field mapping relationship; then, based on the field mapping relation, a code generator is used for automatically generating a conversion code containing field assignment and type conversion logic; finally, the generated conversion codes are compiled into an executable conversion function, and the function can convert data in the source structure instance according to the field mapping relation during program running and fill the data into the target structure instance. According to the method, automation of structural body conversion is achieved, the problem that errors are prone to occurring when conversion codes are written manually is solved, and the maintainability of the codes is improved; the mode of reflection during operation is replaced by code generation, so that the conversion efficiency is remarkably improved; meanwhile, the method supports batch conversion and concurrent processing, and can meet the performance requirements of large-scale data processing.

The invention relates to an optimization method of a data type conversion operator, computer equipment, a readable storage medium and a computer program product. The method comprises the following steps: allocating memory resources for target data; establishing a read mapping relationship between each logic index in the logic structure of the source data and the memory address of the source data, and establishing a write mapping relationship between each logic index in the logic structure and the memory address of the target data; traversing the logic structure, aiming at any traversed logic index, determining a read address corresponding to the logic index based on the read mapping relationship, performing data reading from the memory resource corresponding to the source data based on the read address, converting the read data element from the first data type to the second data type, and storing the converted data element in the memory resource. And writing the data elements subjected to data type conversion into memory resources corresponding to the target data according to the write mapping relationship. By adopting the method, the calculation capability of the data type conversion operator can be improved.

The invention discloses an equipment data collection method, device and system based on multi-source heterogeneity and a storage medium. The method comprises the following steps: collecting device data of a plurality of devices; according to the data format and characteristics of each device, configuring a corresponding data analysis code, and analyzing the collected device data based on the data analysis code to obtain device analysis data; the attribute and the point address of each device are extracted from the device analysis data, and function types or physical quantity information contained in the point addresses are identified and mapped to corresponding device data items; extracting a data value corresponding to the point address; filtering illegal data from the extracted data according to a preset rule, and performing type conversion on legal data; organizing the target data subjected to verification and type conversion into a standard and unified JSON data structure; and analyzing the generated JSON data structure, and writing the analyzed data into a target database for data analysis and equipment operation state monitoring.

The invention relates to the technical field of data processing, in particular to an LLM-based multi-source heterogeneous data intelligent fusion and self-adaptive processing method, which comprises the following steps of: receiving multi-source heterogeneous data, matching character sequences of mailbox fields and identity card number fields in records one by one by adopting a preset regular expression model, according to a preset key value pair mapping dictionary, items are uniformly converted, data records which cannot pass format verification, type conversion and value domain mapping are marked as doubt, and data records which cannot be processed or are marked as suspicious are separated. According to the method, a hierarchical processing flow is constructed, and deterministic format verification, type conversion and value domain mapping are arranged at the front end of a processing link, so that most simple data quality problems can be quickly processed with low calculation overhead. Then, numeric statistics and character string comparison are used to carry out fuzzy correction on the data, and spelling errors and numeric abnormities which cannot be covered by rules are further solved.

The embodiment of the invention discloses an inverse quantization method of model data, a matrix operation method and related equipment, and the method comprises the steps: determining first floating point data to be inversely quantized in a model; regarding the first floating point data as first integer data with the same bit width as the first floating point data, and performing first type conversion processing of expanding the first bit width to a second bit width to generate intermediate floating point data; performing second type conversion processing for maintaining a second bit width on the intermediate floating point data to generate second integer data; performing format correction processing on the second integer data to generate inversely quantized second floating point data; according to the method, the inverse quantization process can be simplified, and the calculation efficiency can be improved, so that chips which are partially limited by instruction sets and lack special calculation instruction support for low-precision floating point data are adapted, and the overall performance of artificial intelligence model training is improved.

The invention discloses a model training method and device, equipment, a storage medium and a program product, and the method comprises the steps: carrying out the type conversion of an input tensor of a second data type based on a zooming parameter corresponding to a linear layer in a forward process, and obtaining an input tensor of a first data type; after the calculation of the linear layer is completed, converting the output tensor of the first data type into an output tensor of a second data type; in a reverse process, performing type conversion on the model parameter of the first data type based on a scaling parameter corresponding to a linear layer to obtain a model parameter of a second data type; determining a gradient of the second data type based on the model parameter of the second data type and the loss value of the second data type; and updating model parameters of the model by using the gradient of the second data type to obtain a trained model. Therefore, under the condition that the model precision is guaranteed, the model training efficiency is improved, and the requirements for computing power and storage during model training are reduced.

The invention relates to a data synchronization method and system for intelligent dirty data detection and restoration based on DataX. According to the method, an intelligent dirty data processing engine is embedded between a Reader plug-in and a Writer plug-in of DataX, type matching, format verification, constraint conflict pre-detection and abnormal value recognition are completed online, and type conversion, format standardization, abnormal value replacement or flexible writing are carried out on dirty data according to a preset or self-adaptive strategy. And finally, the Writer executes insertion, updating, skipping or log recording only according to an instruction, and a traceable governance log is generated in the whole process. According to the method, the synchronization-cleaning capability is embedded into the DataX framework, so that the synchronization success rate and the data quality are remarkably improved, the external ETL dependence and the artificial script cost are reduced, and the method has the advantages of high intelligence, high throughput and flexible configuration.

The subject technology receives a first semi-structured object. The subject technology iterates through a list of fields specified by a target object type. The subject technology, for each field, determines whether a field with a same name is present in the first semi-structured object. The subject technology, in response to the field being found in the first semi-structured object, converts a value of the field to a target field type according to defined type conversion rules. The subject technology stores the converted value in a unified representation comprising a data structure that stores both structured and semi-structured data types. The subject technology processes a query using the unified representation.

A data processing device. The data processing device includes: an input interface configured to receive input data and extract raw data and information about a data structure according to which the raw data are structured; a generator configured to, for at least one data type used by the data structure, select a source code generation plugin, assigned to the data type, generate, using the selected source code generation plugin, conversion source code for converting the data type to a target format, combine the conversion source codes for converting the data types to the target format to form conversion source code for converting the data structure to the target format, and generate a conversion plugin for converting the data structure to the target format; and a converter configured to execute the conversion plugin and thus to convert the raw data to the target format.

The invention relates to the technical field of data processing, and particularly provides a multi-source data-based conversion method and system, comprising: acquiring original data from a plurality of data sources, the data sources including but not limited to a structured data source, a semi-structured data source and an unstructured data source; performing format unification on the cleaned data according to a preset standardization rule, wherein the standardization rule comprises a coding format, field mapping and data type conversion; carrying out encryption processing on the standardized data, and setting a data access permission to ensure the security and privacy of the data; the standardized multi-source data are fused, redundant information is eliminated, and consistency and integrity of the data are guaranteed; storing the fused data into a target database, and supporting dynamic expansion and data version management; standardized data services are provided for external applications through an API interface, and real-time query and analysis of data are supported. The method has the effects of protecting the data security and encrypting the data.

The invention discloses a method and device for transmitting big data between a C language and a Flutter, and relates to the technical field of big data.The method for transmitting the big data between the C language and the Flutter comprises the steps that a structural body is declared in the C language, a class consistent with the data type of the structural body is defined in the Flutter, and the structural body in the C language and the class defined in the Flutter are mapped; configuring an external function interface package in the Flutter, wherein the external function interface package is used for enabling the Flutter application to call a native code library written by a C language; calling a C function in the Flutter, and obtaining a pointer address returned by the function; the obtained pointer address is converted into a corresponding Pointer type in a Flutter; and converting the Pointer type into a mapping class type defined in the Flutter, so as to obtain the representation of the C structural body in the Flutter. The problem that in the prior art, the efficiency of big data transmission between the C language and the Flutter is low is solved, and efficient and stable data interaction and sharing are achieved.

One or more embodiments of the specification provide a data processing method and device applied to a database system. After triggering a data type conversion task, the data processing method obtains a to-be-converted floating point number of the data type conversion task, converts the to-be-converted floating point number into a first floating point number represented by an extended precision, and then calculates a second floating point number scaled to a preset numerical interval through multiple loop integer calculations to perform accurate rounding. According to whether the decimal part of the intermediate value of the last loop integer calculation and the distance between the upper interval and the lower interval of the last loop integer calculation are both greater than the error value of the last loop integer calculation, in the case of satisfying the error value of the last loop integer calculation, the second floating point number is converted into a fixed point number according to the integer part of the intermediate value of each loop integer calculation and the rounding result of the last loop integer calculation, to serve as the execution result of the data type conversion task.

A matrix multiplier includes an input vector scaler configured to generate a first scaled input vector based on a first input vector and a plurality of quantization scale coefficients, a first data type converter configured to generate a first fixed-point scaled input vector based on the first scaled input vector, a processing element array including a first processing element configured to generate a first fixed-point output element based on the first fixed-point scaled input vector and first plurality of quantization sign values and a second processing element configured to generate a second fixed-point output element based on the first fixed-point scaled input vector and second plurality of quantization sign values, and a second data type converter configured to generate first and second output elements by converting data type of the first and second fixed-point output elements, and to output a first output vector including the first and second output elements.

The present application discloses a device data collection method, apparatus, system, and storage medium based on multi-source heterogeneity. The method includes: collecting device data from multiple devices; configuring corresponding data parsing code according to the data format and characteristics of each device, and parsing the collected device data based on the data parsing code to obtain device parsed data; extracting the attributes and point addresses of each device from the device parsed data, identifying the functional type or physical quantity information contained in the point address, and mapping it to the corresponding device data item; extracting the data value corresponding to the point address; filtering illegal data according to preset rules for the extracted data, and performing type conversion on legal data; organizing the target data after verification and type conversion into a standardized and unified JSON data structure; parsing the generated JSON data structure, and writing the parsed data into a target database for data analysis and device operation status monitoring.

The invention provides a data processing method and device, equipment, a storage medium and a program product, and can be applied to the technical field of databases and the technical field of financial science and technology. The method comprises the steps that first to-be-processed data used when a target system is used for being connected with a target database is obtained, the target system is achieved in the environment of a first programming language, and the data type of the first to-be-processed data is the first data type of the first programming language; converting the first to-be-processed data into first intermediate data through a data type conversion component, wherein the data type of the first intermediate data is a second data type of a second programming language; calling a dynamic processing tool through the data type conversion component to enable the dynamic processing tool to dynamically process the first intermediate data to obtain first intermediate processing data; converting the first intermediate processing data into target data of a first data type through a data type conversion component; and controlling the target system to connect the target database according to the target data.

This invention relates to a reliable translation method for synchronous dataflow languages ​​based on the 8114 criterion, belonging to the field of code generation. The invention obtains the abstract syntax tree of the Ctemp intermediate language through lexical and syntactic analysis and translation of Lustre source code; it traverses the Ctemp abstract syntax tree and designs translation methods for control flow statements and assignment statements within the tree structure; it traverses expressions within statements and designs translation methods that add type casting to ensure that statements are semantically correct while avoiding implicit type conversions; it extends the theorem on type relations in Cltype for verifying type operations; and it extends the theorem on type casting and binary operations in Ctemp semantics to achieve semantic formal verification. This invention significantly improves the compliance of C code generated by code generation tools with the GJB 8114 specification.

A matrix multiplier includes an input vector scaler configured to generate a first scaled input vector based on a first input vector and a plurality of quantization scale coefficients, a first data type converter configured to generate a first fixed-point scaled input vector based on the first scaled input vector, a processing element array including a first processing element configured to generate a first fixed-point output element based on the first fixed-point scaled input vector and first plurality of quantization sign values and a second processing element configured to generate a second fixed-point output element based on the first fixed-point scaled input vector and second plurality of quantization sign values, and a second data type converter configured to generate first and second output elements by converting data type of the first and second fixed-point output elements, and to output a first output vector including the first and second output elements.

The application relates to the technical field of computer programming, and discloses a Go language structure body automatic conversion method and system. The method first receives the definitions of a source structure body and a target structure body as input, automatically matches the fields of the two structure bodies according to preset rules through a field mapping rule engine, and generates a field mapping relationship; then, based on the field mapping relationship, a code generator is used to automatically generate conversion code containing field assignment and type conversion logic; finally, the generated conversion code is compiled into an executable conversion function, which can convert and fill the data in the source structure body instance into the target structure body instance according to the field mapping relationship during program running. The method realizes the automation of structure body conversion, avoids the error-prone problem of manually writing conversion code, improves the maintainability of the code, significantly improves the conversion efficiency by replacing the runtime reflection mode with code generation, and simultaneously supports batch conversion and concurrent processing, and can meet the performance requirements of large-scale data processing.

The application relates to the technical field of deep learning compilation, and provides a method, device, equipment and medium for converting an MLIR file into a PyTorch file. The method comprises the following steps: obtaining an MLIR file, extracting data information and operation information in the MLIR file; retaining original identifiers of the data information and converting data types of the data information into Tensor data types in a PyTorch file; constructing a dependency relationship between operations according to a front-back order relationship between the operations; converting operation types in the MLIR file into operation types in the PyTorch file according to a conversion relationship between the PyTorch file and the MLIR file; and forming the PyTorch file according to the converted data types, the dependency relationship between the operations and the converted operation types. The method can equivalently convert the MLIR file into the corresponding PyTorch file.

This application discloses an automatic code generation method and system for SOME / IP data types, relating to the field of vehicle system development. The method includes: S1, data dictionary reading: obtaining the data dictionary for SOME / IP protocol communication; S2, data type classification, identification, and processing: opening the Excel file from step S1, classifying each row of data in the worksheet, and performing specific parsing processing for different types; S3, multi-round iterative dependency resolution: using a multi-round scanning strategy to resolve nested dependencies between various data types; S4, naming standardization processing: standardizing the names of each parsed data type according to preset naming rules; S5, standard code generation and output: converting all data types that have undergone naming standardization processing into type definition code conforming to C language specifications; simultaneously converting the value range, precision, unit, and Chinese description metadata in the data dictionary of the Excel file into code comments, generating a C language header file that can be included in the compilation project.

The present disclosure provides an operation unit, related devices and methods. The operation unit comprises: a first format conversion unit configured to convert data in a source data format into an intermediate data format according to a first rule; and a second format conversion unit configured to convert data in the intermediate data format into data in a target data format according to a second rule. The intermediate data format comprises all fields of the source data format and all fields of the target data format, and the number of bits in each field of the intermediate data format is not less than the number of bits in the corresponding field of either the source data format or the target data format. The embodiments of the present disclosure improve the versatility of hardware implementation of data type conversion and reduce hardware resource consumption caused by data type conversion.

The present application relates to an optimization method, computer device, readable storage medium, and computer program product for a data type conversion operator. The method comprises: allocating memory resources for target data; establishing a read mapping relationship between each logical index in the logical structure of the source data and the memory address of the source data, and establishing a write mapping relationship between each logical index in the logical structure and the memory address of the target data; traversing the logical structure, for any traversed logical index, determining the read address corresponding to the logical index based on the read mapping relationship, and reading data from the memory resource corresponding to the source data based on the read address, converting the read data element from a first data type to a second data type, and then writing the data element after the data type conversion to the memory resource corresponding to the target data according to the write mapping relationship. The use of this method can improve the computing power of the data type conversion operator.

This invention discloses a method for implementing a multi-source heterogeneous data transformation pipeline for time-series databases, comprising: parsing, transforming, and modeling the original data according to configuration rules to obtain a streaming transformation pipeline including a parsing stage, a transformation stage, and a modeling stage connected in sequence; in the parsing stage, parsing various data formats of the original data and converting the fields extracted after parsing into data in a unified columnar storage format; in the transformation stage, filtering, mapping, calculating, and type conversion operations are performed on the data processed in the parsing stage; in the modeling stage, the data processed in the transformation stage is mapped to the super table and sub-table structure of the time-series database according to the configuration rules; the modeling process includes: dynamic table name generation based on template expressions, extraction of label fields and data fields, and batch data assembly; the configuration rules specifically include loading, validating, and hot-updating the configuration file of the streaming transformation pipeline.

The invention provides a production process abnormity recognition method and device, and the method comprises the steps: receiving a target demand inputted by a user on an operation interface, and enabling the target demand to represent a text demand of a preset type of products under a preset detection item; a characteristic data set of the preset type of products is obtained, the characteristic data set comprises multiple pieces of characteristic data, and each piece of characteristic data indicates non-numeric data, under a preset detection item, of a target product collected at a detection point of industrial equipment used for producing the preset type of products; for each piece of characteristic data, carrying out demand identification on the characteristic data to determine a numeric type conversion result of the characteristic data; according to the numeric type conversion result, a characteristic statistical chart is constructed, the characteristic statistical chart comprises a plurality of data points, and each data point represents the numeric type conversion result of the target product under the preset detection item. According to the method and the device, the management and control requirements for the non-numeric data are realized.

Systems and methods for generating synthetic training data are disclosed. A method may include: (1) receiving user speech from a user; (2) generating an input file comprising text of the user speech; (3) extracting entities from the text in the input file; (4) creating an input data structure for a data structure for the entities, wherein the input data structure comprises a plurality of columns, a column name for each column, a data attribute for each column, and a data type for each column, and a number of records based on a volume parameter; (5) converting the data type for each column to an ANSI SQL-standard data type; (6) generating a database agnostic data structure having the column names and the ANSI SQL-standard data type; (7) generating synthetic data for the database agnostic data structure; and (8) outputting an output file comprising the synthetic data.

The invention provides a task execution method based on a deep learning model, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, in particular to the fields of deep learning, model compression, model quantification, generative models, pre-training, large models, large language models and the like. The specific implementation scheme is as follows: in response to a task execution instruction, acquiring input data and quantization parameters for a quantized previous model in a plurality of models; according to the quantization parameter of the previous model and the input data of the previous model, performing data type conversion to obtain converted input data; according to output data of the previous model, calibration data of the current model in the multiple models are determined, and the output data of the previous model are obtained by inputting the converted input data into the quantized previous model; based on the calibration data of the current model, the calculation unit determines a quantization parameter of the current model and stores the quantization parameter of the current model to the storage unit.

The embodiment of the application discloses a model data dequantization method, a matrix operation method and related equipment, wherein the dequantization method comprises the following steps: determining first floating point data to be dequantized in a model; regarding the first floating point data as first integer data with the same bit width as the first floating point data, performing first type conversion processing from the first bit width to the second bit width to generate intermediate floating point data; performing second type conversion processing on the intermediate floating point data to maintain the second bit width to generate second integer data; and performing format correction processing on the second integer data to generate second floating point data after dequantization. The application can simplify the dequantization process, improve the calculation efficiency, adapt to some chips with instruction set restrictions and lack of support for special computing instructions for low-precision floating point data, and improve the overall performance of artificial intelligence model training.

The invention provides a method and a system for performing bidirectional data type derivation in graphical programming, and relates to the technical field of graphical programming, and the method comprises the following steps: S1, analyzing graphical programming data into a graph structure and a tree structure; s2, data type derivation from outside to inside is executed, specifically, on the basis of the tree structure, all graph structures are traversed according to the depth priority sequence, internal nodes of the currently traversed graph structure are traversed according to the topological sequence of a directed acyclic graph, and the data type of downstream nodes is derived according to the data type of upstream nodes in the data flow direction; s3, data type derivation from inside to outside is executed, specifically, on the basis of the tree structure, all graph structures are traversed according to a subsequent sequence, for the currently traversed graph structure, the data type of an output boundary of the currently traversed graph structure is determined according to the data type of internal nodes of the currently traversed graph structure, and the data type is reversely propagated to a corresponding node in a parent graph of the currently traversed graph structure; and S4, automatically generating a type conversion code according to a predefined data type processing rule.

According to the atomic data storage method and device, the equipment and the storage medium provided by the invention, whether the data type of the to-be-stored data is the atomic data type or not is judged by responding to the database storage instruction; if the to-be-stored data is an atomic data type, converting the data type of the to-be-stored data into a preset data type through a type processor to obtain target stored data; and storing the target storage data in a database. According to the method, the value of the atomic data type is automatically extracted and converted into the proper common data type, so that the problems of human errors and type mismatching are avoided, the consistency and integrity of the data are ensured, the requirement of a developer for manually writing a conversion code is reduced, and the development efficiency is improved.

The invention provides a data processing method and device, electronic equipment and a storage medium, relates to the technical field of data processing, in particular to the field of memory compression and the like, and can be applied to an intelligent recommendation scene. According to the specific implementation scheme, for initial data loaded into a target container, a target type is determined from a plurality of candidate data types according to a data value of the initial data; wherein the target container comprises a basic storage area used for storing the mark and an extended storage area used for storing the initial data, and the mark represents the address of the initial data in the extended storage area; the method comprises the following steps: converting initial data from an initial type to a target type to obtain converted data; the data length of the converted data is smaller than that of the initial data; and in response to detection that the capacity difference between the first demand storage area and the basic storage area determined according to the target type meets a first preset condition, storing the converted data to an available storage area except the basic storage area.