100 results about "Formatted text" patented technology

Certain aspects of the present disclosure provide techniques for preprocessing data prior to generating vector representations. The data is preprocessed to generate one or more natural language texts describing entities and relationships included in the data. Vector representations of the natural language format texts are stored in a vector database. The embeddings may be retrieved to augment a prompt for a generative model. The embeddings may be selected by performing a search of the vector database using an embedding for the prompt to determine relevant or similar embeddings. The augmented prompt can be input into a large language model. Although the model may be unaware of the data from which the embeddings of the embedding database were generated, the augmented prompt may enable the model to use the data to improve breadth and depth of responses. The preprocessing of the data improves response results of the model.

The invention relates to an engineering document index consistency proofreading method and system based on a multi-modal large model, and the method comprises the steps: Q1. OCR detection and recognition: carrying out the optical character recognition and format analysis of a source document, converting an uploaded PDF document into a processable text message in a Markdown format, and carrying out the format discrimination of a table, a formula and a plain text; and Q2, table and formula processing: adopting a hierarchical processing strategy, intelligently selecting an optimal processing mode according to the complexity of the table, and converting table information into a descriptive long text through a language large model and cue words. According to the method, accurate, reliable and efficient document index checking service can be provided for a user, the quality and efficiency of professional document processing are remarkably improved, the efficiency and quality of knowledge graph construction are remarkably improved, a knowledge verification system capable of being evolved continuously is established, and the method is suitable for popularization and application. And a reliable technical support is provided for knowledge management and professional decision-making in a complex field.

The invention provides a vertical field data construction method based on a large model, which belongs to the technical field of data processing and artificial intelligence, and comprises the following steps: converting a vertical field source document into an intermediate format text, and segmenting the intermediate format text into a plurality of text blocks; inputting the text blocks into a pre-trained generative language model, guiding the pre-trained generative language model according to pre-designed cue words to generate a plurality of candidate questions according to the content of the text blocks, and performing preliminary screening and fine screening on each candidate question to obtain a question set; pre-defining a mode of a knowledge graph according to field characteristics of the vertical field, processing all text blocks based on an information extraction model, and constructing a field knowledge graph; and performing local context retrieval on each final question in the question set based on the text block of the question source, performing global knowledge retrieval based on the domain knowledge graph, and generating a final answer and a final thinking chain. The method is suitable for different vertical fields, the data quality can be effectively improved, and the problem generation accuracy is guaranteed.

The invention discloses an alloy field-oriented text and table double-track collaborative data extraction method and system. The method comprises the following steps of: obtaining an alloy related document DOI; taking the DOI as an index to obtain a full-text literature and storing the full-text literature in a JSON format; performing document screening based on an LDA topic distribution model and establishing a target topic document library; preprocessing the unstructured table, and obtaining structured table data by using a large language model; performing automatic text segment division on the JSON format text through rule matching analysis; and sorting the text segments by using a large language model to generate semi-structured text data, fusing and normalizing the text and table data, and outputting a component-process-organization-performance-strengthening mechanism alloy data chain. According to the method, through a double-track collaborative data extraction mechanism, the problems of multi-source heterogeneity, format fragments, information islands, semantic layering and the like of material subject data are effectively solved, structured treatment of literature data in the alloy field is achieved, and a unified technical path is provided for constructing a high-quality alloy database.

The invention discloses a plant protection field literature information batch structured extraction method based on an AI large model, and belongs to the technical field of computer data processing and artificial intelligence application. According to the method, a PDF format literature is converted into a Markdown format text through a PDF literature self-adaptive preprocessing module and a Markdown conversion module; then, a double-strategy self-adaptive AI extraction module is adopted, different modes are adopted for processing according to a text length threshold value, and JSON data are extracted in combination with a JSON format data restoration strategy; then, the CPU intensive conversion task and the I / O intensive AI analysis task are processed in parallel through a two-stage parallel scheduling module, and a structured database file is generated through aggregation according to a predefined mapping rule through a multi-dimensional aggregation export module. According to the method, end-to-end automation from literature acquisition to structured data output is realized, and the problem of low extraction efficiency of manual literature reading information is solved.

The invention relates to the technical field of passenger service, and discloses a construction method and device of a dynamic extension word bank, equipment and a medium. The method comprises the following steps: acquiring multi-source heterogeneous original knowledge data related to civil aviation passenger service, and converting the multi-source heterogeneous original knowledge data into a standard format text; processing the unstructured data in the standard format text based on the adaptive word segmentation model and the stop word list to obtain a plurality of candidate words; performing multi-dimensional corpus feature quantitative analysis on each candidate word to determine candidate keywords; and performing similarity calculation based on the entity word segmentation and the candidate keywords, determining effective candidate keywords, and adding the effective candidate keywords into the dynamic expansion word bank. By means of the method and device, the technical problems that in the prior art, a word bank used by a civil aviation service system is usually based on a static vocabulary or depends on manual compiling and updating, the static word bank cannot be updated in real time, the characteristics of the civil aviation field cannot be flexibly handled, and the maintenance cost is high due to manual updating and maintenance are solved.

The invention discloses an intelligent cue word generation evaluation method and system based on multi-component collaboration, and relates to the technical field of artificial intelligence and natural language processing. The method comprises the following steps: acquiring a multi-format text, converting the multi-format text into an input-output pair, and generating a plurality of candidate cue words; after determining a communication mode according to the text form, standardizing cue words according to the structured template of each large language model; calling each platform to generate answer analysis and unify formats, and extracting target contents by utilizing regularization; and carrying out multi-mode evaluation on the candidate cue words, selecting an optimal cue word, and outputting a structured evaluation result and an optimization suggestion based on multiple indexes. According to the method, full-process automatic processing of prompt word generation, optimization and evaluation is achieved, various model platforms, data sets, algorithms and evaluation indexes are supported, and good universality and expansibility are achieved.

The invention discloses a vertical field objective test data set construction method, system and device and a medium, and relates to the technical field of natural language processing, the construction method comprises the following steps: obtaining text data, and carrying out knowledge extraction and arrangement on the text data to obtain formatted text information; taking the formatted text information as input, and performing question generation, correct answer generation, interference option generation and analysis generation through a pre-established large model to obtain test data; performing quality detection on the obtained test data, and screening out the test data meeting the quality detection requirements; and performing self-inspection on the test data meeting the quality detection requirements, and retaining the test data passing the self-inspection to obtain a vertical field objective test data set. According to the method, the vertical domain test questions can be automatically generated on a large scale, the dependency degree of testers on professional domain knowledge is reduced, meanwhile, the labor cost is saved, the problem difficulty is controllable, and the professionality and accuracy of test question generation are improved.

A computing services environment may include a database system storing a plurality of database records for client organizations accessing computing services including a conversational chat assistant accessible via various communication channels. The computing services environment may also include a communication interface configured to receive an input message from a client machine via a communication channel, a generative language model interface providing access to one or more generative language models, and an orchestration and planning service. The orchestration and planning service may be configured to analyze the input message to determine a novel text passage via a generative language model, to determine novel text formatting information based on designated text formatting configuration information specifying one or more parameters for formatting text generated for transmission via the communication channel, and to transmit the novel text passage and the novel text formatting information to the client machine via the communication interface.

The invention provides an industrial internet network intrusion detection method based on a pre-trained large language model, and the method comprises the steps: carrying out the data preprocessing of original network flow data through protocol self-adaptive flow aggregation and session segmentation, feature screening and feature coding; constructing a stream format text data set used for training a generation model GPT-2 and a packet level classification text-label data set used for training a classification model DistilBERT; then fine tuning training is carried out on the generation model GPT-2 and the classification model DistilBERT; then calling a trained generation model GPT-2 to generate a traffic sequence, obtaining a prediction data packet, calling a trained classification model DistilBERT, performing anomaly judgment on sequence data in the prediction data packet one by one, and outputting a classification result of anomaly judgment; the active intrusion detection of first generation and then discrimination is realized. According to the method, prediction can be made before real attack traffic arrives, and network attacks are prevented.

The present invention belongs to the technical field of text processing, and relates to an intelligent extraction method and system for an input text containing a mathematical formula. The method comprises: 1) performing format determination, conversion and preprocessing on an input text; 2) performing angle correction on the preprocessed image-formatted text; 3) performing formula detection; 4) performing layout analysis; 5) for inline formulas, according to a formula detection box, determining whether a corrected OCR detection box contains an inline formula, and splitting the OCR detection box containing the inline formula, so as to obtain a text-only OCR detection box; 6) performing formula recognition to obtain a formula recognition result; 7) performing text recognition to obtain a text recognition result; and 8) on the basis of a layout analysis box and a layout category thereof, performing same-line detection box determination and merging on the formula recognition result and the text recognition result, so as to obtain an extraction result for the input text. The present invention can effectively improve the extraction efficiency and accuracy for input texts containing mathematical formulas.

The invention relates to the technical field of aeronautical text data processing, and discloses an aeronautical text content cleaning and labeling method, system, device and medium wherein the method comprises: noise filtering: identifying and removing noise in an aeronautical text in combination with static cleaning and a general large model; format standardization: converting the aviation text after noise removal into a standardized format text; duplicate removal and error correction: detecting duplicate contents based on a hash algorithm, and correcting spelling errors and grammar errors based on a general large model to obtain an aviation text subjected to duplicate removal and error correction; entity identification: key entities are extracted based on the general large model, and the extracted key entities are labeled; active learning: screening high-value samples in the marked key entities based on an uncertainty query strategy; and dynamic optimization: performing verification and iterative optimization on the marking result in combination with the aviation knowledge base. According to the method, the automation level, the labeling accuracy and the system self-adaptive capability of aviation text processing can be remarkably improved.

The invention discloses a zero-sample-oriented cross-domain recommendation method and system, and belongs to the field of recommendation systems. The method comprises the following steps: pre-training a residual quantization variational auto-encoder by using a multi-field article data set, wherein the residual quantization variational auto-encoder is used for encoding an article into a multi-level concept identifier (ID); during pre-training, formatted text description needs to be carried out on an article, optimization is carried out based on a curriculum learning method, and entropy penalty loss is introduced; training an autoregressive large language model by using the cross-domain recommendation data set, so that the autoregressive large language model predicts the multi-level concept identifier ID of the next to-be-interacted article according to the multi-level concept identifier ID of the historical interaction article of the user; and sorting the to-be-recommended article list by adopting the trained model. According to the method, articles are encoded into multi-level concept identifiers (ID), article marks are unified, course learning is adopted to reduce the initial training complexity and relieve the codebook collapse problem, the codebook use frequency is dynamically monitored, the knowledge migration efficiency is improved, and the method can be applied to article recommendation tasks in different fields.

The invention discloses a style and style aligned DOCX format document translation system, which comprises the following steps of: analyzing a DOCX format document through a document analysis tool to obtain an XML format text; each Paragraph object style label in the XML format text is simplified; the simplified XML format text is sent into a large language model to obtain a translation XML format text; carrying out XML label integrity verification on each XML format text of the translation; if the verification result meets the integrity requirement, original style tags are restored according to the corresponding sequence of the text in the XML format of the translation; and generating the reduced text in the XML format of the translation into a document in the DOCX format of the translation through a document analysis tool. According to the method, the problem of balance between format fidelity and translation quality in professional document translation is effectively solved, the pattern alignment accuracy and translation efficiency are remarkably improved, and the method is particularly suitable for professional document translation scenes, such as legal contracts and technical manuals, in which formats need to be strictly reserved.

A message detection method, device and equipment based on a large model and a readable storage medium relate to the field of automobile network and data analysis, and comprise the following steps: performing format conversion on analyzed SOME / IP message information corresponding to each target SOME / IP message to obtain a structured format text comprising message contents, timestamps, service IDs and method IDs corresponding to the target SOME / IP messages; based on the SOME / IP protocol matrix table, the relationship between the ECUs, the corresponding relationship between the client and the server and all services between each pair of client and server, analyzing the structured format text to generate a target relationship table; determining target message change data according to the target relation table; and inputting the target message change data and a preset exception check rule into a preset large language model for message detection so as to output a target detection result. According to the invention, the dependence on expert experience can be reduced, and the detection efficiency can be effectively improved.

Image pages are generated from a document. Assembling the image pages generates a collage. Two-dimensional text and bounding boxes are extracted from the image pages. A structure verbalizer spatially formats the two-dimensional text in one-dimension with spatial information to generate spatial-formatted text. The spatial-formatted text is concatenated to generate a text extraction. A multimodal embedding model is applied to the collage and the text extraction to generate a target artifact vector. The target artifact vector is compared against a set of preexisting artifact vectors to identify a corresponding artifact vector associated with a corresponding document having a corresponding metadata tag. A distance is determined between the corresponding artifact vector and the target artifact vector. Responsive to the distance being within a threshold distance additional steps are performed, including performing both applying the corresponding metadata tag to the document to generate a modified document and outputting the modified document.

The embodiment of the invention relates to a contract information extraction method and device, equipment and a medium. The method comprises the steps of obtaining a to-be-processed clause text in a to-be-processed contract text; responding to the to-be-processed clause text as a target clause text, obtaining a prompt word template corresponding to the to-be-processed clause text, and setting a corresponding prompt identifier for the prompt word template corresponding to the to-be-processed clause text to obtain a target prompt word template; and performing information extraction on the to-be-processed term text based on the target cue word template through a preset large model to obtain target format text information so as to determine contract information corresponding to the to-be-processed contract text. Therefore, the key information of each clause text can be quickly and accurately extracted through the large model, the accurate information of the whole contract text can be obtained, the prompt identifier is added for the specific clause to improve the recognition accuracy, and the contract information recognition efficiency and effect are further improved; therefore, the contract-related business processing efficiency is improved, and the business error risk is reduced.

The invention relates to the technical field of automatic testing, in particular to a webpage element positioning method and device, equipment, a storage medium and a program product, and the method comprises the steps that in response to a selection operation of a target element in a front-end page, a first tree structure corresponding to the target element in a webpage is obtained; according to a current node corresponding to the target element in the first tree structure and an associated node of a link where the current node is located, performing node cutting on the first tree structure to obtain a cut target format text; selecting a target node from the associated nodes according to the semantic information of the associated nodes; according to the semantic information of the target node, the target format text and the element positioning description information, cue words are obtained; and generating a positioning code according to the cue word and the target model. According to the design method for conducting UI element positioning based on the large model, the finally obtained positioning code has semantics and is high in readability, and the professional requirement for research and development workers does not need to be too high.

A system for adaptive text rendering includes an editor, a context analyzer, an adaptive text generator, and a feedback and learning engine. The editor renders a text box on a digital canvas and receives user input via the text box. The context analyzer extracts contextual information from digital content associated with the digital canvas. The adaptive text generator uses the user input, text box dimensions, and contextual information to direct an artificial intelligence (AI) model to generate text. It then formats the generated text to fit within the text box based on one or more fit criteria. The feedback and learning engine captures and evaluates user feedback related to the formatted text and generates tuning adjustments to refine the adaptive text generator or the AI model.

In some implementations, a device may identify a form field, of a form, that is to be filled with text. The device may determine a formatting for the text that is to be used for filling the form field. The formatting for the text may be determined based on content that is indicative of the formatting that is to be used for filling the form field, or information relating to at least one previous filling of the form field that is associated with a successful submission of the form. The device may modify the text in accordance with the formatting that is determined to obtain formatted text. The device may cause filling of the form field with the formatted text.

Systems and methods for device naming-indicator generation are disclosed. Friendly names for accessory devices, such as smart-home devices, may be utilized to generate formatted text data that includes capitalization and / or punctuation for the friendly names. The formatted text data may be utilized to generate tag data indicating attributes of the friendly name. The tag data and / or contextual data indicating historical usage of the accessory device may be utilized to generate naming indicator(s) for the accessory device. The naming indicator(s) may be utilized, for example, during target inference and / or for communicating with a user about the accessory device.

This application discloses a data query method, device, and storage medium, relating to the field of Internet technology. This method addresses the existing problem of being unable to perform text queries based on query statements. The method includes obtaining a query statement and structured text data; parsing the query statement into an abstract syntax tree; attaching the structured text data as a parameter to the abstract syntax tree; compiling the abstract syntax tree to generate a high-order function; executing the high-order function to generate a row-level data access handle; using the row-level data access handle to access the structured text data, obtain query results, and return tabular data of the query results. This method enables efficient querying of formatted text data using commonly used query statements.

The application discloses a ship structure strength calculation formula fine-grained knowledge extraction method, which faces a ship structure strength calculation scene and comprises the following steps: generating a specified format text file by analyzing a ship structure strength calculation guide PDF; constructing a ship structure strength calculation formula fine-grained extraction large model; obtaining text-level full-quantity triple by adopting a general ontology model guided text-level full-quantity triple extraction method and a semantic-based triple edge relationship supplementing and correcting method; perfecting the ontology model by adopting an ontology model generation and attribute expansion method based on rule filtering and hierarchical clustering; formulating attribute value text description by adopting a triple snowflake type expansion method based on attribute semantic depth mining; and specifying attribute value reference description by adopting a triple supplementing method based on reference association expansion, so as to obtain formula-level full-quantity triple, thereby constructing a calculation formula graph and realizing the structured management of the calculation formula.

A system and method for automatically generating a report based on cockpit voice recognition is provided. The system includes a data acquisition module, a speech engine, and a report generator. The data acquisition module is configured to acquire cockpit voice data and vehicle parameter data. The speech engine is configured to convert the cockpit voice data into formatted text data. The speech engine includes: a pre-processing module configured to convert the cockpit voice data into formatted voice data; an ASR module configured to convert the formatted voice data into text data using automatic speech recognition; and a post-processing module configured to reorganize the text data into formatted text data using text processing. The report generator is configured to generate a vehicle standard operating procedure (SOP) compliance report based on the formatted text data and a report template. The vehicle SOP compliance report indicates how the investigated behavior derived from the cockpit voice data complies with the standard operating procedures defined by the report template.

Electronic system and method for assisting a civil aircraft operator in decision-making with the development of aeronautical indicator(s), civil aircraft and associated computer program. The present invention relates to an electronic system (15) for assisting a civil aircraft operator in decision-making (10), comprising: - an electronic device (18) for displaying information; - an electronic device (20) for developing aeronautical indicators, comprising: + a module (30) for acquiring at least one aeronautical information message comprising a header and a useful part, the useful part including several data fields, including a free-format text field, called the free field;+ a module (32) for calculating several aeronautical indicators for each aeronautical information message via the application of an artificial intelligence algorithm to said message, the artificial intelligence algorithm receiving the free field as input and delivering the indicators as output, these being distinct from the data contained in said message; + a module (34) for displaying a rendering of the aeronautical indicators. Figure for the abbreviation: Figure 1;

The invention belongs to the field of natural language processing and large language models, and particularly discloses an intelligent question and answer method and system for a document in the field of water taking permission, and the method comprises the steps: analyzing a water taking permission document through layout recognition, and obtaining a json format text; performing knowledge extraction on the text, namely identifying a paragraph main entity through a paragraph main entity dual-channel detection and entity identification iterative training method, complementing the content of an indication pronoun in the text through long-distance anaphora resolution based on an elastic sliding window, and guiding a large language model to extract a triple by using knowledge extraction Prompt, data cleaning is carried out on the data; constructing a vector database index based on the cleaned triples, and when a user question is received, retrieving candidate triples related to the user question by using the vector database index; and inputting the retrieved candidate triples into the large language model to generate answers to the user questions. According to the method, the knowledge can be efficiently and accurately extracted and retrieved from the water taking permission document.

The embodiment of the invention discloses a machine-readable building engineering standard digital processing method and a machine-readable building engineering standard digital processing device. A specific embodiment of the method comprises the steps of generating text sequence information according to a first preset processing mode; generating layout information and bibliography and reference relation information according to preset layout annotation information, the layout level identification model and preset bibliography annotation information; text information is generated according to preset text labeling information and a paragraph extraction model; according to a pre-trained formula identification model, generating special format text information; determining the text sequence information, the layout information, the bibliography and reference relation information, the text information and the special format text information as construction standard control information; and generating control parameter information according to the construction standard control information, and controlling the construction robot to perform construction processing according to the control parameter information. According to the embodiment, the accuracy of obtaining the building engineering standard information is improved, the construction time consumption is shortened, and the waste of equipment resources is reduced.

The present disclosure provides a presentation document parsing method and system, wherein the presentation document parsing method comprises: obtaining recognition results of all pages of a presentation document; wherein the recognition results comprise intermediate state JSON data and Markdown format text content of all pages; obtaining a structured information extraction outline of the presentation document according to a user intention and the recognition results; determining key pages of the presentation document corresponding to the structured information extraction outline; inputting original page images, text recognition results and chart structure codes of the obtained key pages into a visual large model to output a presentation document structured parsing result conforming to the user intention. The present disclosure generates a structured information extraction outline of the presentation document according to the user intention, so that the user does not need to know the structure and content of the presentation document in advance, and the parsing path of the presentation document can be automatically planned, greatly reducing the use threshold.

The invention provides a Word document generation method and system, electronic equipment and a storage medium. The method comprises the following steps: displaying a Web page containing a text editing area; obtaining input content and format requirements in the text editing area; judging whether the input content contains a structure identifier or not; if not, calling a semantic recognition model to perform structured analysis on the input content; if yes, performing hierarchical structure analysis on the input content according to a regularization rule and / or calling a semantic recognition model to perform structured analysis on the input content; and adjusting the input content according to the analysis result and the format requirement, and generating a Word document. Word document generation does not depend on a template, a fixed template does not need to be designed in advance, a text of a fixed structure does not need to be provided according to the template, a user can obtain the standard Word document by submitting the text of the free format through the Web page, operation is convenient and fast, universality is high, and flexibility is high.

This application relates to a method, apparatus, and storage medium for displaying text to be displayed. The method includes: upon acquiring the text to be displayed, acquiring the global offset of the target text within the text to be displayed that requires formatting adjustment; determining the global character range of each slice in the slicing results of the text to be displayed; determining, based on the comparison result between the global character range and the global offset, a target slice containing the character to be adjusted, and determining the local offset of the character to be adjusted within its respective target slice; and during the streaming display of the target slice of the text to be displayed, adjusting the format of the character to be adjusted corresponding to the local offset within the target slice before displaying it. This application solves the technical problem that formatted text cannot be displayed when displaying long text output by LLM.