69results about "Semi-structured data indexing" patented technology

The invention discloses a multi-type XML (Extensible Markup Language) equipment data processing method and device based on Netty. The method comprises the following steps: constructing a TCP long connection server through a Netty framework to receive a device coding data message; decoding and decompressing the message to obtain XML format data; analyzing the XML data to extract a type identification field; routing the data to a corresponding service processing strategy according to the type identification field; packaging the data into a processing task, and delivering the processing task to a corresponding thread pool for asynchronous processing based on the priority; and finally, converting the processed data into a unified data structure and storing the unified data structure in a database. High-concurrency receiving and processing of equipment data are achieved, multi-type XML message analysis is supported through dynamic routing, key data processing timeliness is guaranteed based on priority scheduling, the data processing throughput is improved by adopting an asynchronous mechanism, and high-concurrency receiving and processing of equipment data are achieved through a unified data conversion standard storage structure. And the efficiency, the real-time performance and the expandability of data processing are remarkably improved.

A computer-implemented method of predicting, simulating, or forecasting values of one or more specified subject-related attributes during a clinical trial comprises: receiving input data comprising: a medical history of a subject, the medical history comprising values of a plurality of subject-related attributes of a subject; and data specifying a requested output, the data comprising: the one or more specified subject-related attributes of the subject and a time frame; and applying a trained generative machine-learning model to the received input data, the trained generative machine-learning model configured to generate output data based on the input data, the output data comprising: respective values of the one or more specified subject-related attributes of the subject in the specified time frame.

A method (300) of storing semi-structured data (12U) includes receiving user data (12) comprising semi-structured user data from a user (10) of a query system (150). The method includes receiving an indication (14) that the semi-structured user data fails to include a fixed schema. In response, the method further includes parsing the semi-structured user data into a plurality of data paths (210) and extracting a data type (220) associated with each respective data path of the plurality of data paths. The method additionally includes storing the semi-structured user data as a row entry in a table (204) of a database in communication with the query system, wherein each column value associated with the row entry corresponds to a respective one of the plurality of data paths and the data type associated with the respective data path.

The application discloses a database analysis and index optimization method for semi-structured data of signal intelligence, which comprises the following steps: creating at least one physical column for storing parsed data in a database table storing original semi-structured data fields; creating a trigger configured to be activated before inserting or updating a data row into the database table; when the trigger is activated, automatically parsing target data from the semi-structured data field of the data row to be inserted or updated according to a predefined path and writing the target data into the physical column corresponding to the same data row; and creating a database index based on the data in the physical column. The system is used for executing the method. The application realizes the improvement of query performance by automatically and real-timely parsing key fields of semi-structured data into structured columns and establishing indexes in the database, and guarantees zero modification of application layer code, data strong consistency and automation of data processing procedures.

The application provides a power equipment wiring diagram data display method, device and storage medium. The method comprises the following steps: selecting a current data updating mode from a plurality of preset data updating modes; when a trigger condition required by the current data updating mode is met, updating an initialization xml file according to measurement point data, so as to display a graph element corresponding to the measurement point data in a browser. According to the embodiment of the application, when the trigger condition required by the current data updating mode is met, the initialization xml file can be updated according to the measurement point data, so as to display the graph element of different shapes and colors corresponding to the measurement point data in the browser. In addition, the graph element type can be customized according to requirements, and the display mode is flexible.

Processing structured data to improve the accuracy of generative models. [Solution] The structured data processing method is characterized by including an acquisition step of acquiring a document, a conversion step of converting the acquired document into structured data, a formatting step of shaping the converted structured data into data that can be read by a generative model, and an output step of outputting the formatted structured data. For example, if the structured data includes image data, the conversion step incorporates text data converted by OCR processing into the structured data.

Among other things, we describe a method of receiving a portion of metadata from a data source, the portion of metadata describing nodes and edges; generating instances of a data structure representing the portion of metadata, at least one instance of the data structure including an identification value that identifies a corresponding node, one or more property values representing respective properties of the corresponding node, and one or more pointers to respective identification values, each pointer representing an edge associated with a node identified by the corresponding respective identification value; storing the instances of the data structure in random access memory; receiving a query that includes an identification of at least one particular element of data; and using at least one instance of the data structure to cause a display of a computer system to display a representation of lineage of the particular element of data.

The invention discloses a structural circuit integrated manufacturing-oriented H-AMF multi-material model digital expression method, a storage medium and equipment, and the method comprises the steps: constructing a file header based on an XML architecture, defining a metadata region and a resource region of a structural circuit integrated model in the file header, and building a material library definition in the resource region; a compact ASCII data block is constructed in the XML architecture to serve as a unified geometric object area, and is used for storing vertexes and triangular patches of the structural circuit integrated model; introducing a vertex deduplication algorithm based on a hash table, and performing deduplication on vertexes of the unified geometric object area to obtain an index database; when the triangular patches are generated, reading a file header based on the XML architecture, obtaining material IDs corresponding to the triangular patches, and binding each triangular patch in the index database with the corresponding material ID; and writing the metadata area, the resource area and the index database of the structural circuit integrated model and the index database bound with the material ID into a disk, generating an H-AMF file, and realizing digital expression of the structural circuit integrated model.

The present application relates to nuclear power engineering drawing semi-structured and search method, device and electronic equipment, including: the drawing to be processed is detected, and the symbol, identification and position information of all objects in the drawing are obtained; the figure text association is carried out; the identification of the figure text association is identified, and the text content of the identification is obtained; the type of the object is determined; the text content is compiled to generate the function position code of the object; the data table is formed based on the text content, the object type, the position information, the function position code and the drawing number of the drawing to be processed; the link relationship of the data table and the drawing, the link relationship of the data entry and the object in the drawing page are constructed, and the semi-structured of the drawing to be processed is completed. The present application greatly reduces the labor cost of engineering drawing data extraction; greatly improves the efficiency and quality of engineering drawing data extraction; also can realize the second search and second determination of the drawing, and effectively improves the efficiency and accuracy of the use of engineering drawing in the business field.

The embodiment of the invention provides a data table evaluation method and device, computer equipment and a storage medium, and relates to the technical field of data management. The method comprises the following steps: acquiring a first data table, a vector database and at least one standard field information of an enterprise; effective metadata in the first data table is subjected to HTML (HyperText Markup Language) tagging processing, a reference text field is obtained, and the reference text field comprises a table name, a field name and classification; converting the reference text field into a first vector; and based on the first vector, the vector database and the at least one standard field information, determining an evaluation result of the first data table, the evaluation result being used for indicating whether the first data table meets the evaluation requirements of the enterprise. By adopting the technical scheme provided by the embodiment of the invention, the evaluation accuracy of the data table can be improved.

The invention discloses a compressed XML database type, system and method supporting efficient query, and relates to the technical field of XML databases, and the database type adopts a hierarchical compression storage architecture and a reverse search query architecture; in the storage stage, XML non-keyword information is coded through Huffman coding, recoverable ordered arrangement is carried out on the coded information through a tree structure, then transcoding is carried out, the information is stored in a disk in a binary form, and compared with a traditional XML database storage mode, at least 50% of storage space is saved. In the query stage, a reverse search mode is adopted, a corresponding Huffman code initial position is positioned through a keyword, then under the optimization effect of a reverse index, reverse search is performed to recover a complete XML structure, and the query complexity is reduced from traditional O (n) to O (1). According to the method, the storage efficiency and the query performance of the XML data are greatly improved, and the method is suitable for large-scale XML data management and application scenes.

The application discloses a cigarette defect detection method and device, the cigarette defect detection method comprising: receiving a cigarette image collected by an industrial camera; identifying defects in the cigarette image based on a defect detection model to obtain a prediction box containing defect type and position information; converting information of the prediction box into an XML file and storing; digitally analyzing the XML file to obtain a defect value corresponding to the cigarette image; and evaluating the defect level of the cigarette image according to the defect value. The application uses an XML file to store the defect identification result, which not only retains important information of the original cigarette image, but also greatly reduces the storage space occupied by each cigarette image, effectively reducing the storage cost to a certain extent, so that the defect identification result of each cigarette image can be recorded, laying a foundation for full-sample monitoring of cigarette production. Meanwhile, the defect detection model is used for defect identification, which can improve the detection accuracy, adaptability and universality.

To generate logic from a schema, such as a database schema.SOLUTION: A method includes accessing a schema that specifies relationships among datasets (DSs), computations on the DSs, or transformations of the DSs, selecting a DS from among the DSs, and identifying, from the schema, other DSs that are related to the selected DS. Attributes of the DSs are identified, and logical data representing the identified attributes and relationships among the attributes is generated. The logical data is provided to a development environment, which provides access to portions of the logical data representing the identified attributes. A specification that specifies at least one of the identified attributes in performing an operation is received from the development environment. Based on the specification and the relationships among the identified attributes represented by the logical data, a computer program is generated to perform the operation by accessing, from storage, at least one DS having the at least one of the attributes specified in the specification.SELECTED DRAWING: Figure 1

The application provides a kind of unstructured data access method and device, scheme includes: receiving JSON format interface parameter data;Based on the JSON Schema mode definition of pre-configuration, data is format checked, and after checking, it is converted into JsonObject general data carrier;Based on the JsonPath path expression of pre-definition, field in carrier is addressed and read-write operation;The carrier after processing is converted into the document data of database interaction format, simultaneously, JsonPath expression is converted into the field key expression form required by database query command;Based on the field key expression form, database operation command is constructed and executed, and document data is stored or queried in database.

An example may involve receiving a request to generate a user interface component, wherein the request indicates data usable to populate the user interface component; generating a prompt for a natural language model based on the request and the data; receiving, from the natural language model, a representation of the user interface component based on the prompt; and providing the representation of the user interface component for display.

The invention provides a lossless extraction method and device for a chart vector diagram in a PDF and a computer readable medium, and relates to the technical field of computers, in response to a chart vector diagram extraction request, obtaining target PDF data according to the chart vector diagram extraction request, and converting the target PDF data into target markup language data; acquiring position information of one or more target chart vector diagrams according to preset chart vector diagram labels in the target markup language data, confirming corresponding target candidate frame ranges according to the position information of the target chart vector diagrams, and performing statistics on the target candidate frame ranges to generate a candidate frame statistical table; calling and acquiring a target candidate frame range in the candidate frame statistical table, and selecting and performing lossless extraction on to-be-extracted chart vector diagram data according to the chart type of the target chart vector diagram to obtain final chart vector diagram data; and converting the markup language data corresponding to the final chart vector diagram data into a target chart vector diagram.

The invention discloses a data processing method, device and system and a storage medium, and belongs to the field of storage. The method is applied to the storage system, the storage system comprises multiple layers of storage areas, each layer of storage area comprises at least one data set, the data set is used for storing multiple data records, and each data record at least comprises a key. The method comprises the steps that a first data record set is obtained, the first data record set comprises data records stored in a first data set in an ith layer of storage area and data records stored in at least one second data set in an (i + 1) th layer of storage area, and i is an integer larger than or equal to 0. And merging a plurality of data records including the same key in the first data record set into one data record to obtain a second data record set. And updating the data records stored in the at least one second data set in the (i + 1) th layer of storage area into the data records included in the second data record set. According to the invention, storage resources can be saved.

The invention discloses a track fusion method based on open source data and AIS data, and relates to the technical field of Internet open source data processing and situation fusion. An acquired AIS original message is analyzed to generate a track of a ship target, open source text data acquired through the Internet also includes track information of the ship target, entity extraction is performed by adopting a deep neural network algorithm, the open source text data is processed and recognized, and the track information of the ship target is acquired. Information such as an IMO number, an MMSI number and a ship name of a ship in the data and information such as occurrence time and occurrence place of a related event are identified, ship information and event information are integrated to form a trace point of the ship, and the mark, time and space information of the ship correspond to records in an AIS original message, so that the information of the ship is obtained. And inserting the ship trace point obtained from the open source text data into the track of the ship in the AIS, and further smoothing the track through a Kalman filtering algorithm to form track information after ship target fusion, thereby realizing track fusion based on the open source data and the AIS data.

This invention provides an XML-based method for extracting tables from scientific and technological documents, belonging to the field of PDF file information extraction. The method includes converting the PDF to DOCX, decompressing the DOCX to obtain an XML file, filtering out interfering characters using text font size nodes and keywords in the XML tree structure, retaining table title keywords, obtaining table headers and splitting columns through cell attribute nodes, correcting the columns of other table rows (excluding the header) based on the header columns, inserting data from the previous row into table rows with missing columns according to rules, restoring the table row structure, and finally extracting and storing the table column data using an ontology model. This method is not constrained by the table border type of scientific and technological documents and accurately extracts related table data through a semantic model, restoring the logical relationships of the tables, improving the accuracy and automation of table extraction, and is applicable to scientific and technological documents in different fields.

This invention proposes a lossless extraction method, device, and computer-readable medium for chart vector graphics in PDF, relating to the field of computer technology. Responding to a chart vector graphics extraction request, the method involves: acquiring target PDF data based on the request and converting the target PDF data into target markup language (TRAIL) data; obtaining the position information of one or more target chart vector graphics based on preset chart vector graphics tags in the TRIL data, confirming the corresponding target candidate box range based on the position information of the target chart vector graphics, and generating a candidate box statistics table; calling and retrieving the target candidate box range from the candidate box statistics table, selecting the chart vector graphics data to be extracted based on the chart type of the target chart vector graphics, and performing lossless extraction to obtain the final chart vector graphics data; and converting the TRIL data corresponding to the final chart vector graphics data into the target chart vector graphics.

A computer implemented method (300) and an apparatus (102) for handling data in an industrial automation infrastructure (100) are disclosed. The method (300) comprises receiving, by a processing unit (202), a stream of binary data from a file; storing the binary data in a data structure; converting the stored binary data to an encoded character sequence; determining whether an identifier (508) associated with a target attribute exists within the character sequence; and when the identifier (508) exists within the encoded character sequence, performing: recording a start position (508-1) and an end position (508-N) of the identifier (508), determining preconfigured positions (520) relative to at least the start position (508-1) and the end position (508-N) that accommodate data elements being indicative of length of attribute values by referencing a schema (516, 600), and extracting attribute values from the character sequence based on the preconfigured positions and length of attribute values.

This invention discloses a distributed storage method and apparatus for meteorological and oceanographic data. The method includes: acquiring a meteorological and oceanographic dataset; the meteorological and oceanographic dataset includes a subset of forecast data; the subset of forecast data includes a forecast model, start time, forecast lead time, barosphere, spatial latitude and longitude partitioning values, and meteorological forecast data; the meteorological forecast data includes radar data, satellite inversion data, and meteorological and oceanographic raster data; each meteorological forecast data has a corresponding forecast model, start time, forecast lead time, barosphere, and spatial latitude and longitude partitioning values; extracting a set of KEY field information based on the meteorological and oceanographic dataset; and classifying and storing the meteorological and oceanographic dataset based on the set of KEY field information to obtain a meteorological physical database and a meteorological logical database.

Embodiments of the present application disclose a message conversion method and device, electronic equipment and a storage medium. The method comprises: obtaining a structure document of an initial message, and parsing the structure document to obtain structured configuration data, wherein the structured configuration data comprises at least one node absolute path; for each node absolute path in the at least one node absolute path, determining a node relative path corresponding to each absolute node on the node absolute path according to the structured configuration data, and determining target configuration data according to each node relative path; constructing tree structure data based on the target configuration data, and determining a target message based on the tree structure data to convert the initial message into the target message. The technical solution of the embodiments of the present application improves the message conversion efficiency and compatibility.

This invention provides a method and apparatus for inserting semi-structured data into a database, comprising: acquiring an insertion request; parsing the semi-structured data to be processed; determining whether the semi-structured data to be processed contains a predefined identifier field; if it contains a predefined identifier field, using the field value as a target identifier; if it does not contain a predefined identifier field, generating a target identifier; and performing an insertion operation of the target identifier and the semi-structured data to be processed based on pre-set index rules. The solution of this invention has dual logic of identifier fields and the generation of unique identifiers, ensuring that each piece of semi-structured data to be processed corresponds to a unique target identifier. Combined with pre-set index rules, it can prevent the writing of duplicate identifiers from the insertion stage. Performing the insertion operation based on preset index rules avoids the cumbersome process of additional full data traversal verification required when inserting semi-structured data.

The embodiment of the invention provides a data processing method, device and equipment. The method comprises the following steps: generating to-be-written data; determining a structure type of the to-be-written data, wherein the structure type is a structured type or an unstructured type; under the condition that the structure type is an unstructured type, performing multi-modal feature analysis on the to-be-written data, and determining feature information of the to-be-written data; the feature information is used for indicating the content attribute, the security level and the use scene of the to-be-written data; and storing the to-be-written data in a target node according to the feature information, the target node being a computing node in a data cluster managed by the server. The data processing mode provided by the invention is suitable for read-write operation of the unstructured data.

Aspects of the subject disclosure may include, for example, a technique for processing a decision network that includes obtaining an index encoding a mapping from potential values of an input parameter to decision parameters of the network's predicates, wherein the mapping associates potential values of the input parameter with decision parameters affected by those potential values; evaluating decision parameters affected by specified values of the input parameter, including identifying each decision parameter to which the index maps at least one specified values of the input parameter, and setting the values of those decision parameters in accordance with the input parameter's specified values; and analyzing the decision network, including evaluating the predicates of one or more of the decision nodes based on the values of the predicates' decision parameters, and determining, based on the values of the evaluated predicates and a topology of the decision network, that a particular terminal node encodes the network's output. Other embodiments are disclosed.

Embodiments of the present application provide a data conversion method and device, and the present application relates to the field of cloud technology. The data conversion method in the embodiments of the present application comprises: obtaining to-be-converted data, the to-be-converted data being data in JSON format; based on a conversion category of the to-be-converted data, searching a data conversion rule library for a data conversion rule matched with the conversion category as a target data conversion rule for data conversion of the to-be-converted data; checking a field value in the to-be-converted data according to the target verification rule; if the field value in the to-be-converted data passes the check, performing conversion processing on the field value in the to-be-converted data based on the target conversion rule to generate converted data in JSON format. The technical scheme of the embodiments of the present application can effectively improve the conversion efficiency of data and reduce the workload of developers.

A computer-implemented technique for decomposing semi-structured data is provided. In this technique, metadata for a predetermined number of records can be collected from semi-structured data that includes several records. A structured format is generated based on the metadata and the plurality of records is decomposed with the structured format.

The invention provides an automatic reconstruction method for an aviation three-dimensional CAD (computer aided design) model based on XML (extensible markup language), which belongs to the technical field of computer aided design and comprises the following steps: extracting geometrical characteristics, modeling history and parameter constraint information of a model by using a CATIA CAA interface, and storing the geometrical characteristics, the modeling history and the parameter constraint information in an XML format; an XML file is analyzed through a Python script, a JS modeling script suitable for a domestic CAD platform is automatically generated, and automatic and accurate reconstruction of a model in domestic CAD is achieved; according to the method, the parameterization characteristics and the modeling sequence of the original model are reserved, the data migration efficiency is remarkably improved, the problem that the models among the heterogeneous CAD systems are difficult to reuse is solved, and the method has good universality and engineering application value.