450 results about "Paragraph" patented technology

Systems and methods for semantic temporal segmentation based on topic recognition are disclosed. Text classification and segmentation may be used to index media content for subsequent searches. The method may include using a text classification model to semantically analyze sentences in a text file (such as a transcript) to determine topics with which sentences are associated. The output of the text classification model may be provided to a text segmentation model to enable more-accurate identification of text segments (e.g., paragraphs) within the text file. In some examples, the output of a text segmentation model is provided to a text classification model to enable the text classification model to perform more-accurate classification based on text segments rather than (or in addition to) performing classification on single sentences. The classification of the text segments may be used to assign labels to the text segments to enable subsequent searches based on the labels.

The invention provides a method for creating a structured document based on a deep learning model, and relates to the technical field of data processing, and the method comprises the steps: obtaining basic text information collected by a user, obtaining text data, fragmenting the text data, obtaining text fragmented data, and creating a structured document according to the text fragmented data. Analyzing the structures of the sentences in the paragraphs, determining keywords in the sentences, analyzing the logical relationship between the sentences in the paragraphs, associating the sentence keywords in the paragraphs, analyzing the association degree between the paragraphs in the chapters, identifying core keywords of the chapters, obtaining a chapter keyword data set, and analyzing the field involved in the text. The method comprises the steps of extracting field information of a text to obtain a text keyword data set, constructing a structured document model based on a graph neural network model, and inputting the text keyword data set and text data into the structured document model to obtain a structured document. According to the method, the structured document can be constructed through the deep learning model.

The invention provides a long document semantic coherence processing method and device, a storage medium and computer device.The method comprises the steps that block coding is conducted on a long document through a sliding window converter, and the sliding window converter comprises a window dividing unit, a position coding unit and a converter coding unit; the window division unit adjusts block positions according to sentences and paragraph boundaries and sets overlapping areas between adjacent blocks; the encoded document content is stored and managed through a memory management module, the memory management module comprises a chapter-level memory pool and a gating memory unit, the chapter-level memory pool stores the document encoding content, and the gating memory unit maintains entities and event timelines; performing deep semantic understanding on the document content through a semantic processing module, and calculating a multi-dimensional attention weight capture dependency relationship between paragraphs; providing a retrieval result through a retrieval module; and generating semantically coherent output content through a content generation module. According to the scheme, the semantic coherence of long document processing is improved.

The invention provides a semantic enhancement adaptive partitioning method and system for natural resource large model questions and answers, and aims to solve the problems of difficulty in term boundary recognition, damage to semantic integrity and the like. According to the method, three core technologies including theme perception coarse-grained paragraph division, self-adaptive sliding window theme hierarchy division and embedded perception context self-adaptive text segmentation are fused. The system firstly analyzes a natural resource long text structure, identifies titles and theme levels and aligns associated contents; paragraphs are extracted according to a theme perception strategy and are subdivided into sentence sets according to grammar rules; an improved sliding window mechanism is adopted to divide sentences into window sentence block groups. The method is characterized in that a dynamic aggregation threshold mechanism is introduced, the semantic association degree between adjacent sentence blocks is calculated through an embedded perception context semantic segmentation technology, whether the sentence blocks are combined or not is judged by combining a similarity distribution change trend and a dynamic adjustment threshold, self-adaptive delimitation of semantic boundaries is achieved, and text blocks which are clear in structure and coherent in semantics are generated.

The invention discloses an event association analysis method and system based on a large language model and a pre-training model, and the method comprises the steps: carrying out the sentence segmentation and segmentation processing of each event, inputting the processed events into a large language model, and generating the sentence representation of an event pair; aggregating global expressions of paragraphs and sentence levels of the event by the large language model; retrieving a semantic paragraph related to the current event; splitting the event reasoning task into a plurality of steps, wherein each step corresponds to a specific reasoning task in event association analysis; generating a possible reasoning path, and performing marginalization processing on the reasoning path; the multiple reasoning paths are evaluated at the same time, the most reasonable reasoning result is obtained and serves as input, and a final classification result is generated through weighted fusion; and according to a classification result, determining a specific relationship type between the event pairs, and outputting a determination result of a causal, sequential or hyponymy relationship between the events. According to the method, the complex relationship between events can be more accurately captured.

The invention belongs to the technical field of text processing, and particularly relates to a method and system for retrieving DOCX document content based on keywords, which comprises the following steps: analyzing an Office Open XML structure of a DOCX document, combining with multi-dimensional features such as style names, and utilizing a title classification score model to accurately distinguish a title and a text, so that a semantic hierarchical structure of the document is effectively reserved; and secondly, a multi-level semantic extension mechanism is introduced, and a Sension-BERT, a HowNet knowledge base and a Word2Vec model are fused, so that intelligent extension of synonyms and synonyms of keywords is realized, and the recall rate and semantic understanding ability of retrieval are remarkably improved. And in addition, a BM25 model is combined with paragraph length normalization and structure position weight to calculate a correlation score, so that retrieval results are sorted more accurately and reasonably. The construction of the reverse index is combined with the position coding and compression optimization strategy, and the retrieval efficiency and the storage performance are both considered.

The invention relates to the technical field of natural language processing, in particular to an AI reading management and control method and system based on artificial intelligence, and the method comprises the following steps: carrying out text word segmentation processing on an input original text, segmenting the text into independent sentences, recognizing basic word units in each sentence, analyzing semantic adjacency relationships and syntactic structure features among vocabularies, and carrying out word segmentation processing on the word units; screening and extracting potential phrases representing paragraph meanings, and establishing a candidate semantic unit set; according to the method, through word segmentation, syntactic structure recognition and semantic adjacency analysis of the original text, potential phrases capable of representing paragraph significance are extracted, the candidate semantic unit set is constructed, and modeling of the semantic structure in the text is achieved. And associating the set with the reading fixation duration and the playback action of the user sentence by sentence to obtain a reading behavior response of each semantic fragment, and executing semantic weighting and reading behavior cross analysis according to the reading behavior response. Through the linkage mode, the semantic focus actually focused by the user at present can be recognized.

The invention discloses a hierarchical retrieval enhancement generation method and system. The method comprises the following steps: determining question information to be retrieved according to a hierarchical retrieval enhancement generation task; performing document-level retrieval on an external knowledge base based on the problem information to obtain a most relevant document set; slicing the most relevant document set according to paragraphs and performing paragraph-level retrieval to obtain a most relevant paragraph set; slicing the most relevant paragraph set according to sentences and performing sentence-level retrieval to obtain a most relevant sentence set; performing fragment generation operation on each sentence in the most relevant sentence set by utilizing a sliding window, and then performing fragment-level retrieval to obtain a most relevant fragment set; and carrying out information integration on the most relevant fragment set and knowledge in the model by utilizing a large language model, and carrying out context information supplement by utilizing factual knowledge obtained based on an external knowledge graph to obtain a context answer text of question information. According to the scheme, the accuracy and the reliability of a retrieval result can be remarkably improved.

The invention relates to the technical field of cross-language information processing, and provides a cross-language text fusion intelligent alignment method and system.The cross-language text fusion intelligent alignment method comprises the steps that a hierarchical alignment model is constructed through preprocessing and label recognition of multi-coding-type texts, deep semantic feature extraction and labeling of a multi-language pre-training model and text semantic and format information analysis; transform is taken as a core, cross-language semantic association is enhanced through a multi-head attention mechanism, character-level and paragraph-level format collaboration is realized through label weight allocation and condition constraint, the format alignment accuracy is obviously improved in a multi-language mixed typesetting scene, and the multi-language mixed typesetting efficiency is improved. The analysis capability of the model on the structured information can be extended to layout relation processing of texts, images and tables, so that the comprehensive alignment efficiency in a multi-modal fusion scene is remarkably improved. The accuracy of text and label fusion is guaranteed, the problem of confusion of messy codes and labels is avoided, and high-precision cross-language text alignment from semantics to formats, from single mode to multiple modes and from semantics to formats is achieved.

Methods and systems for retrieving relevant information in response to an input question. The method includes obtaining text content related to the input question and partitioning the content into one or more paragraphs based on predefined rules. The method further involves extracting one or more evidence spans that are relevant to the input question by inputting the text content and the question into a trained language model. A semantic search is then performed on both the paragraphs and the extracted evidence spans, ranking the candidate passages based on their relevance to the input question. Each candidate passage may comprise either a paragraph or an evidence span that addresses the question. The disclosed methods and systems improve the quality and relevance of retrieved information by combining heuristic-based content partitioning with machine learning-based evidence extraction.

The invention provides a method for training a small parameter model for automatic analysis of electric power contract terms and a computing device, and the method comprises the steps: obtaining contract data, carrying out the preprocessing of the contract data, and converting the contract data into a standardized paragraph list; analyzing the standardized paragraph list through a large language model, and outputting a labeled structured data set as training data; fine tuning training is conducted on the small-parameter model through the training data, the fine-tuned small-parameter model is obtained, and the fine-tuned small-parameter model outputs clause classification labels and extracted clause key fields in the reasoning stage according to the input unlabeled contract paragraph text. According to the technical scheme of the invention, the clause extraction precision and efficiency can be improved, the calculation cost can be greatly reduced, the field adaptability can be enhanced, the clause change and the newly added contract type of the power industry can be quickly adapted, and an innovative solution can be provided for intelligent contract management.

The invention discloses an industry instruction fine tuning data set automatic generation method and system, and belongs to the technical field of artificial intelligence, S10: selecting industry standard document data, and analyzing the data into a plurality of paragraph texts as data sources according to chapters; s20, generating different types of questions for the paragraph text by using a large language model; s30, scoring the generated questions according to a scoring rule, and filtering the questions with scores lower than a preset threshold value; s40, the solvability and difficulty of the question are adjusted through a large language model, and the quality of the question is optimized, and S50, a first version answer of the question is generated by using the large language model, and the answer is optimized by using a hierarchical region optimization search algorithm; s60, constructing a sample pool through global and local selection methods, calculating a sample compression ratio and screening a data set; the method has the beneficial effects that through a highly automatic process and an optimization algorithm, the data set generation efficiency and quality are improved, the labeling cost is reduced, and support is provided for rapid deployment of a large language model in industry application.

The invention relates to the technical field of large language models, in particular to an official document content generation and compliance verification system based on a large language model. Comprising an official document extraction module used for analyzing a model essay by using a large language model, extracting a writing style and a fixed sentence pattern, constructing a knowledge graph, storing format specifications, common expressions and logic structures, mining and constructing a core intention library, and storing typical intention prototype vectors; the content generation module is used for generating paragraphs through context sensing, expanding and writing keywords, analyzing behavior data, predicting writing intention vectors and performing condition control; the compliance verification module is used for warning improper words in real time, retrieving policy verification logic and verification formats, and performing pre-verification based on intention vectors; and the writing auxiliary module is used for providing real-time modification suggestions and forward-looking prompts based on writing intention vectors. According to the technical scheme, document writing efficiency and quality can be improved.

The invention provides a text processing method and system based on semantic density, which are applied to the technical field of text processing, and the method comprises the following steps: obtaining a target text, and carrying out feature extraction on the target text to obtain a plurality of text features of each paragraph of the target text; calculating a semantic density score of each paragraph according to the text feature of each paragraph; classifying each paragraph according to the semantic density score of each paragraph to obtain a paragraph category of each paragraph; if the paragraph category of the paragraph is high in density, performing block processing on the paragraph according to the text type of the target text and the semantic density score of the paragraph to obtain a plurality of text blocks corresponding to the paragraph; and if the paragraph category of the paragraph is low density, processing the paragraph according to the text type of the target text, the paragraph and the semantic density score thereof, and each other paragraph and the semantic density score thereof, so that information splitting and context loss can be avoided, redundant calculation and resource waste are reduced, and the field adaptability is improved.

The invention relates to the field of text data analysis, in particular to a data recommendation method based on a large model, which comprises the following steps: acquiring a retrieval keyword of a target problem text, and acquiring a retrieval text set of the target problem text based on the retrieval keyword and related keywords; performing retrieval association evaluation on the retrieval text set to determine a retrieval state of the retrieval text set, and determining a text processing strategy according to the retrieval state; when a paragraph analysis mode is adopted, determining a phrase extraction method corresponding to each text paragraph according to the reference extraction coefficient of each text paragraph; when the combined analysis mode is adopted, key analysis texts and key keywords are obtained, and matched text phrases are determined according to the matching keyword proportion and the matching validity; according to the invention, the effectiveness of the obtained recommended text content is improved.

The invention relates to a text generation method and device based on structured semantic association retrieval, and the method comprises the steps: carrying out the vector coding of a question text through a vector coding model after the question text inputted by a user is received, and obtaining a user question coding vector corresponding to the question text; searching a plurality of target sub-node coding vectors corresponding to the user question coding vector from a vector database; determining a father node corresponding to each target child node coding vector in the plurality of target child node coding vectors based on the association relationship to obtain a plurality of father nodes, and screening the text paragraphs corresponding to each father node in the plurality of father nodes to obtain the screened text paragraphs; and fusing the question text and the paragraph corresponding to the screened text paragraph based on the cue word project to obtain an input vector of the large language model, and inputting the input vector into the large language model to obtain a question answer, so that the retrieval accuracy can be improved.

The invention belongs to the technical field of construction technical scheme compilation, and particularly relates to an intelligent construction special scheme paragraph intelligent compilation method based on an information network, which comprises the following steps of: 1, dividing content processing into two categories according to different input contents: project general content and project characteristic content; step 2, vector embedding; step 3, index construction; step 4, index storage; step 5, carrying out similarity retrieval; and 6, outputting a retrieval result. According to the intelligent construction special scheme paragraph intelligent compilation method based on the information network, through automatic word segmentation, vector embedding and index construction, the construction special scheme compilation efficiency is greatly improved, and the time needed by manual compilation is shortened; historical schemes and standard requirements are utilized to ensure that the generated scheme content conforms to industrial standards and specifications, and the standardization degree of the schemes is improved; by distinguishing the project general content and the project characteristic content, a customized scheme can be generated according to the characteristics and requirements of a specific project.

The invention provides a system comparison system and method based on a multi-granularity large language model, and the method comprises the steps: carrying out the multi-granularity splitting of a to-be-compared document, and obtaining the to-be-compared contents of multiple granularities; respectively extracting a text vector corresponding to the to-be-compared content of each granularity; based on a vector distance between the text vector of each granularity and a system text vector in a preset knowledge base, determining a similarity score between the system texts; performing semantic difference analysis on the to-be-compared document and the system text vectors in the preset knowledge base by using a preset large model to obtain a difference analysis result between the system texts; and determining a system comparison result based on the similarity score and the difference analysis result between the system texts. According to the method, multi-granularity splitting processing is carried out on the to-be-compared document, comprehensive comparison and semantic difference analysis of the system text are carried out from three levels of sentences, paragraphs and full texts, details and global semantic features of the system text are comprehensively captured, and a more accurate comparison result is generated.

The invention discloses a document information comparison method and device, equipment and a medium. In the method, firstly, after a word document set with a plurality of to-be-compared word documents is obtained, text contents of paragraphs in the plurality of to-be-compared word documents are extracted; carrying out literal difference detection on text contents of paragraphs in a standard word document in the word document to be compared and text contents of paragraphs in comparison word documents except the standard word document by using a character comparison algorithm, and identifying a difference part in each paragraph; and finally, performing semantic difference detection on the text content corresponding to the difference part by using a semantic difference model, and identifying the difference type of the difference part. In the process, due to the fact that the model is used for conducting semantic difference detection on the difference part, the difference type of the specific difference part is detected, the semantic difference of the text can be accurately recognized, the whole difference detection process is efficient, and large-scale documents can be rapidly processed.

The invention discloses a local knowledge base RAG method based on Bayesian reasoning, and belongs to the technical field of information retrieval. The method comprises the steps of obtaining all knowledge data of a local knowledge base, performing paragraph classification and coding on the knowledge data to obtain a paragraph set, and calculating the semantic probability of each paragraph in the paragraph set; obtaining a target professional vocabulary set of the target question, and calculating the semantic probability of each vocabulary in the target professional vocabulary set in each paragraph and the semantic probability of all vocabulary in the target professional vocabulary set in each paragraph in the paragraph set Calculating the conditional probability of each vocabulary in the target professional vocabulary set in each paragraph based on the occurrence frequency; and calculating the conditional probability of the target professional vocabulary set in each paragraph by adopting a Bayesian formula, sorting to obtain a conditional probability sorting result, selecting the paragraphs of which the total word number does not exceed a preset threshold value as a target paragraph set, and generating target retrieval information based on the target paragraph set and the target question. According to the method, the accuracy and reliability of question retrieval are improved.

The invention discloses an intelligent RAG enhanced large model auxiliary writing system, and relates to the technical field of artificial intelligence auxiliary writing. In order to overcome the defects of an existing writing auxiliary tool, the adopted scheme comprises an interface and user layer for receiving a writing demand and redisplaying a generation result; the service layer is used for deploying three types of intelligent agents including writing, interactive writing enhancement and checking; the platform layer is used for carrying out industry adaptation analysis, mixed retrieval and RAG enhancement on writing demands and then sending the writing demands to the large model layer; three intelligent agents of writing, interactive writing enhancement and checking are called in sequence based on the output of the large model layer, and then a text is redisplayed to the interface and user layer after fact accuracy verification is completed through a tense knowledge graph; the data layer is used for providing a multi-type data storage system; the large model layer outputs a paragraph-level text and a reference index based on a writing large model, the paragraph-level text is returned to the platform layer, and reference index information is fed back to the interpretability and security layer for tracing; and the infrastructure layer is used for providing bottom layer resource support for each layer.

The invention relates to the technical field of cross-culture translation, in particular to a cross-culture translation method and system based on context perception fusion, and the method comprises the steps: recognizing a text data set and an image data set, carrying out the paragraph division of the text data set, obtaining a text paragraph set, recognizing an image description text of image data, carrying out the word segmentation of the image description text, and obtaining a text paragraph set; calculating the matching similarity of the text paragraphs and the image description phrases, judging whether the matching similarity is greater than a standard matching similarity or not, if so, recording the matching similarity as a target matching similarity to obtain a target matching similarity set, and obtaining a plurality of associated text paragraph sets according to the target matching similarity set; and translating the associated text paragraph set to obtain an associated translated text, extracting an unassociated text paragraph set from the text paragraph set, and translating the unassociated text paragraphs to obtain an unassociated translated text. According to the invention, the accuracy and efficiency of cross-culture translation can be improved.

Provided in the embodiments of the present disclosure are a document processing method and apparatus, a content generation method and apparatus, and an electronic device. The document processing method comprises: in response to the reception of a first instruction for a target document in a preset format, analyzing a non-text object partition and a text object partition in the target document, so as to obtain at least one non-text object partition and at least one text object partition; respectively performing paragraph splicing on non-character objects and character objects in the non-text object partition and the text object partition on the basis of semantic information and geometric information of the objects, so as to obtain a plurality of paragraphs; and generating semantic tags of the paragraphs on the basis of position information and semantic information of the paragraphs. By means of the embodiments of the present disclosure, document objects in the target document can be accurately identified, and the layout of the document objects can be accurately restored.

The invention discloses a course intelligent question-answering method and system based on retrieval enhancement generation and a large model, and the method comprises the steps: processing a to-be-answered artificial intelligence course knowledge field question, obtaining a question sequence, and obtaining a text representation vector of the question; obtaining a similarity score based on the text representation vector and the paragraph representation vector, and calculating a BM25 score between the question and the paragraph; based on the similarity score and the BM25 score, calculating to obtain a mixed score corresponding to each paragraph, and sorting the paragraphs from high to low according to the mixed scores to obtain a topN document; paragraphs in the topN document are reordered, and a topK document is obtained; and taking the topK document as the context of the question, taking the topK document and the question as input by the large language model, and outputting the answer of the question. According to the method, intelligent question answering of artificial intelligence course knowledge in the education field is realized, the learning efficiency of students is improved, and the teaching burden of teaching assistance of teachers is reduced.

The invention belongs to the field of large models, and particularly relates to a document question and answer method and system based on multi-modal primitives, a terminal and a medium, and the method comprises the steps: extracting title primitives, paragraph primitives, image primitives and table primitives of a plurality of documents uploaded by a user; constructing a correlation mapping table of the paragraph primitives, the image primitives and the table primitives; converting the primitives of the text modality into text vectors for storage; converting the user question into a query vector; screening out a target title primitive and a target paragraph primitive according to the query vector, and obtaining a related target image primitive and a target table primitive; and constructing the target title primitive, the target paragraph primitive, the target image primitive, the target table primitive and the user question into a cue word, inputting the cue word into a multi-modal large language model for processing, and outputting a question result. Question answering is carried out based on the multi-modal information, and the accuracy and interpretability of the question result output by the model are improved.

Disclosed is a video summarisation method that includes converting an input video file to a video transcript using speech recognition, segmenting the video transcript into a plurality of paragraphs using a transformer based model, performing relevance ranking of the plurality of paragraphs by assigning a paragraph importance score to each paragraph, based on relevance of each paragraph to an overall content of the video transcript, creating a plurality of candidate summaries from the plurality of paragraphs, each candidate summary comprising two or more paragraphs, performing coherence reranking of the plurality of candidate summaries based on a combination of coherence and relevance of each candidate summary, and selecting a summary based on the coherence reranking.

The embodiment of the invention provides a retrieval method and device based on document space layout, electronic equipment and a medium, and belongs to the technical field of financial science and technology and artificial intelligence. The method comprises the steps that polar coordinate coding is conducted on word segmentation candidate titles and word segmentation candidate paragraphs, Gaussian space layout position calibration is conducted on title word position information and paragraph word position information after polar coordinate coding, calibration word position information is obtained, and retrieval is conducted according to the calibration word position information and retrieval indication information. According to the embodiment of the invention, the initial spatial positions of the paragraph blocks and the title blocks in the document can be preliminarily determined by performing polar coordinate coding on the word segmentation candidate paragraphs and the word segmentation candidate titles, and Gaussian spatial layout position calibration is performed on the paragraph word position information and the title word position information; spatial layout information of words and words can be fused while semantic features of the document are extracted, retrieval is carried out according to position information of calibration words and retrieval indication information, and the accuracy of document retrieval is improved.

The invention relates to the technical field of document intelligent review, in particular to a multi-mode digital publishing intelligent review system and method based on a large model, and the system comprises a sample construction module, a multi-mode analysis module, a structure recognition module, a term verification module and an annotation output module. According to the method, a model configuration data set is generated by analyzing a subject-object combination relationship in a text and jointly screening part-of-speech, sentence pattern and semantic structure, the accuracy of semantic recognition and structure judgment is improved, and local features of semantic offset and image-text expression separation are recognized by combining a comparison relationship between an image action path and a text behavior object; a paragraph logic structure and a theme connection mode are analyzed based on semantic offset information, the problems of content dislocation and theme jump between paragraphs are effectively revealed, matching change tracks and part-of-speech continuation fluctuation in term cross-paragraph contexts are recognized, label overlapping and coverage redundancy conditions are analyzed from sentence group hierarchy, and the term cross-paragraph contexts are obtained. And forming a label combination suggestion and constructing an annotation structure record.

The invention discloses a complex hierarchy table and large language model collaborative question and answer method. The method comprises the steps of S1, obtaining a user question, a complex hierarchy table and an unstructured text paragraph associated with the complex hierarchy table; s2, preprocessing the complex hierarchical table to obtain a multi-modal input sequence; s3, performing joint coding on the sequence by adopting a LayoutLM model, and obtaining a two-dimensional table with a reduced structure through a graph convolutional network; s4, based on the user question, performing fine-grained evidence retrieval on the two-dimensional table and the unstructured text paragraph to obtain a mixed fine-grained evidence set; and S5, taking the user question, the mixed fine-grained evidence set and the specific task prompt template as input of a large language model, and outputting to obtain a final answer of the user question. According to the method, the problems that a complex hierarchical table structure is difficult to understand and the numerical reasoning performance of the mixed context of the table and the text is insufficient are solved, and the high-precision and generalizable table question and answer ability is achieved.

The invention is applicable to the technical field of file archiving, and provides an intelligent electronic file sorting and archiving method and system based on an AI large model, and the method comprises the following steps: receiving electronic text information, and carrying out title recognition and paragraph recognition on the electronic text information to obtain title information and paragraph texts; all paragraph texts are input into an AI large model for extraction description, and an extraction statement of each paragraph text is obtained; performing semantic matching on the title information and the extracted statement with classification tags, and determining classification tags corresponding to the title information and each paragraph text; the weight value of each classification label is calculated according to the title information weight and the paragraph text weight, classification label arrangement information is obtained, and N labels are arranged in the classification label arrangement information in sequence; and inputting the classification label arrangement information into the classification path identification model to obtain an archiving category and a storage path. According to the method, all contents of the paragraph text do not need to be processed, so that the consumption of computing resources is greatly reduced.