Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

661 results about "Text matching" patented technology

Multimodal data heterogeneous transformer-based asset recognition method, system, and device

This invention discloses a multimodal data heterogeneous Transformer-based asset recognition method, system, and device, the method including: collecting various-modal information of an asset, including text information and image information; building an ALBERT model, a ViT model, and a CLIP model; by the ALBERT model, extracting a text information feature; by the ViT model, extracting an image information feature; by the CLIP model, extracting image-text matching information feature; by different channels, applying asset type recognition to information in different modalities; outputting classification information from the different channels; by the CLIP model, generating asset void information; and discriminatively fusing the classification information from the different channels with the matching degree between the image information and the text information obtained by the CLIP model, and outputting final asset class information. This invention realizes comprehensive discrimination by drawing from multiple modalities to improve the accuracy of asset recognition.
Owner:JINAN UNIVERSITY

Gradual semantic aggregation and structured cognitive enhancement-based image-text matching method

The invention discloses an image-text matching method based on progressive semantic aggregation and structured cognitive enhancement, and relates to the technical field of image-text matching, comprising the following steps: S1, multi-modal feature extraction; s2, introducing a hierarchical self-enhanced semantic perception pooling mechanism; s3, introducing a structured knowledge auxiliary learning module; s4, introducing a cross-modal semantic alignment mechanism; s5, introducing a multi-scale semantic association matrix; according to the method, through multi-level semantic enhancement and fine-grained level self-adaptive feature aggregation, a significant word field is enhanced, and consistency and unification between local representation and global representation are achieved; extracting structured semantics between entities from an image title corpus, mining high-order relationships in long and short sentences, and expanding concepts missed in text global embedding by utilizing the priori knowledge; a cross-modal relation interaction mechanism is introduced, semantic embedding is enhanced, and more consistent semantic understanding is promoted; a multi-scale semantic association matrix is introduced, and a more comprehensive and accurate similarity measurement matrix is generated.
Owner:CHONGQING NORMAL UNIVERSITY

Intelligent question answering system based on large language model

The invention discloses an intelligent question answering system based on a large language model, and relates to the technical field of natural language processing and information retrieval. According to the system, a retrieval enhancement generation (RAG) and knowledge reordering (Re-rankig) technology is utilized, and the knowledge processing capacity for mechanical manufacturing process design, industrial standards and related specifications is improved. The system is mainly composed of four parts, wherein the first part is used for constructing a knowledge base covering machine manufacturing knowledge; in the second part, a LoRA + fine-tuned Qwen2-7B model is used for guaranteeing high-precision requirements such as process parameter calculation; in the third part, the RAG technology is integrated to effectively retrieve and process domain knowledge; and 4, introducing a reordering model to optimize answers. The system is remarkably superior to a traditional large language model LLM in BLEU-4 and ROUGE text matching evaluation indexes, the reliability and speciality of application of the system in the mechanical manufacturing field are verified, an effective scheme is provided for intelligent question answering in the field, and the prospect is wide.
Owner:NANJING UNIV OF AERONAUTICS & ASTRONAUTICS

Intelligent diagnosis and treatment method and system based on knowledge graph and large language model

The invention relates to an intelligent diagnosis and treatment method and system based on a knowledge graph and a large language model. The method comprises the steps of constructing a basic framework of a medical field knowledge graph based on collected medical field data, constructing multi-level knowledge graph expression, obtaining symptom description of a patient through multiple rounds of dialogues, diagnosing the patient and generating a diagnosis report, rechecking and adjusting the diagnosis report by a professional physician, and finally outputting the diagnosis report. The intelligent diagnosis and treatment system comprises a knowledge graph construction module, an intelligent question and answer system, an entity matching module, an intelligent diagnosis module, a doctor review and diagnosis report adjustment module, a diagnosis report output module and the like. And main diseases are screened, so that the coverage is wider. And three levels of knowledge maps are constructed, so that text matching with finer granularity can be carried out. The non-standardized expression of the patient is firstly converted into standardized medical terms through a large model, and drug allergy and the like are taken into consideration, so that the diagnosis accuracy is improved, and the diagnosis and treatment risk is reduced.
Owner:CHENGDU LINGSHU YICHEN HEALTH TECHNOLOGY CO LTD

Multi-modal entity and relation extraction method and system based on cross-modal alignment and fusion

The invention discloses a multi-modal entity and relation extraction method and system based on cross-modal alignment and fusion, and the method comprises the steps: carrying out the processing and coding of an input text and an image, and obtaining a plurality of types of image and text features; performing feature alignment on the fine-grained and coarse-grained text features and the pixel-level image representation by taking the semantic representation of the image as an anchor point, and mapping the image and the text features to the same semantic space; performing multi-granularity feature fusion through text-guided dynamic gating aggregation, visual prefix cross-modal fusion and cross-modal image-text matching, modeling association between noun phrases and image objects in a text while increasing feature complementarity, and obtaining multi-granularity multi-modal feature representation; fusing multi-granularity multi-modal features through entity guidance attention gating, and gathering visual information related to a text entity to obtain final multi-modal fusion representation; according to the multi-modal fusion representation, task prediction of multi-modal named entity recognition and multi-modal relation extraction is carried out.
Owner:YANBIAN UNIV

Vision-language pre-training general framework for realizing multi-granularity cross-modal alignment

The invention discloses a vision-language pre-training general framework for realizing multi-granularity cross-modal alignment. Image-text pairs are used as training data to be respectively input into an image encoder and a text encoder to obtain coarse-granularity global features and fine-granularity local features; multi-granularity cross-modal alignment is carried out on the obtained global features and local features, and the aligned global features of the image and the text are used for cross-modal pre-training; the multi-granularity cross-modal alignment is composed of a text perception image block selection module and a cross-granularity alignment module; a text perception image block selection module in the step S3 selects the image block with the highest correlation through the correlation between text words or phrases and the image blocks to form a corresponding region or target object, so that more sufficient fine-grained alignment is realized; the cross-modal pre-training target comprises the multi-granularity cross-modal pre-training target constructed in the step S6 and universal image-text comparison loss, image-text matching loss and mask text modeling loss.
Owner:HEFEI UNIV OF TECH +1

Power transmission line insulator defect detection method and system based on machine vision

The invention discloses a power transmission line insulator defect detection method and system based on machine vision, and relates to the technical field of machine vision, and the method comprises the steps: combining a pre-trained DenseCLIP model with a YOLOv11 network, extracting an insulator image feature through a visual encoder of the DenseCLIP model, converting the insulator image feature into a visual embedded vector, and carrying out the detection of the defect of the insulator through the YOLOv11 network; constructing a text prompt according to a defect category, converting the text prompt into a language embedded vector, calculating the similarity between the vision embedded vector and the language embedded vector, generating a pixel-text matching score graph to guide YOLOv11 network learning, and constructing an insulator defect detection model; and constructing a mixed training set by using an artificial sample and a real sample, carrying out training by adopting a transfer learning staged training strategy, importing a to-be-detected insulator image into an insulator defect detection model, obtaining an enhanced feature graph and a pixel-text score graph after visual and language embedding vector fusion, and generating a final prediction result. According to the invention, the detection precision and real-time performance of insulator defects are improved, and the intelligent operation and maintenance efficiency of a power grid is remarkably improved.
Owner:GUANGDONG UNIV OF TECH

Speech recognition model optimization method and device based on reinforcement learning, equipment and medium

The invention discloses a voice recognition model optimization method and device based on reinforcement learning, equipment and a medium, and relates to the technical field of voice recognition, and the method comprises the steps: determining a current optimization strategy of a current time step; randomly selecting one piece of enhanced speech training data and corresponding real text training data from the data training set as target training data, and inputting the target training data into the current automatic speech recognition model for prediction to obtain a plurality of prediction texts output by the current automatic speech recognition model; determining a text matching degree and a semantic similarity between each prediction text and real text training data to obtain a group of target reward values, updating an optimization strategy at the current moment based on the target reward values, and skipping to execute the step of determining the current optimization strategy of the current time step to obtain a target reward value; and outputting the corresponding optimized automatic speech recognition model until the current automatic speech recognition model meets the optimization ending condition. And reasonable optimization of the automatic speech recognition model is realized.
Owner:SHANDONG INSPUR SCI RES INST CO LTD

Voice conversation method and device, equipment, storage medium and product

The invention discloses a voice dialogue method and device, equipment, a storage medium and a product, and relates to the technical field of cloud computing, and the voice dialogue method comprises the steps: obtaining voice information input by a user and historical dialogue information of the user, and carrying out the text recognition of the voice information, and obtaining initial text information; based on the historical dialogue information and the initial text information, performing context matching through a preset text matching model to obtain matched target text information; generating prompt words corresponding to the target text information; and inputting the cue word into a preset large language model, and receiving and outputting answer voice information output by the large language model. According to the method, context matching is carried out on the historical dialogue information and the initial text information to obtain the target text information of the complete context obtained by contacting multiple sessions, so that a large language model can more accurately distinguish the actual intention of a user, and the voice dialogue effect is improved.
Owner:SHENZHEN MAXTOP DIGITAL TECH CO LTD

Picture auditing method and device, electronic equipment and storage medium

PendingCN119274190Aimprove accuracyreduce manslaughterInstrumentsText detectionEngineering
The invention provides a picture auditing method and device, electronic equipment and a storage medium, and relates to the field of artificial intelligence. The picture auditing method comprises the following steps: performing character detection and recognition on a picture to be audited by utilizing a character recognition module, and extracting text information in the picture to be audited; the text information is matched with a preset text label, a text matching result is determined, and the text matching result is used for indicating whether the text information is successfully matched with the preset text label or not; the to-be-audited picture is matched with a preset image label, an image matching result is determined, and the image matching result is used for indicating whether the to-be-audited picture is successfully matched with the preset image label or not; and according to the text matching result and the image matching result, determining whether the to-be-audited picture is compliant or not. According to the embodiment of the invention, the accuracy of picture auditing can be effectively improved, and the situation of mistaken killing and missing killing can be reduced.
Owner:TENCENT TECHNOLOGY (SHENZHEN) CO LTD

Multi-modal geographic positioning method based on knowledge graph

The invention provides a multi-modal geographic positioning method based on a knowledge graph. The method comprises the following steps: graph construction: constructing a geographic knowledge graph; image positioning: calculating the entity similarity between the to-be-queried image and each sub-image so as to select an image positioning candidate sub-image, calculating the image matching similarity between the to-be-queried image and each image positioning candidate sub-image so as to determine an image positioning target sub-image, and taking the latitude and longitude coordinates of each image positioning target sub-image as a positioning result; text positioning: converting a to-be-queried text into map representation, calculating the map similarity between a text query sub-graph and each sub-graph to select a text positioning candidate sub-graph, and calculating the semantic similarity between the to-be-queried text and each text positioning candidate sub-graph, and calculating a text matching similarity corresponding to each text positioning candidate sub-graph based on the graph similarity and the semantic similarity corresponding to each text positioning candidate sub-graph so as to determine a text positioning target sub-graph, and taking latitude and longitude coordinates of each text positioning target sub-graph as a positioning result.
Owner:INST OF COMPUTING TECH CHINESE ACAD OF SCI

Transformer substation defect identification method and system based on multi-mode open set associative reasoning

The invention discloses a transformer substation defect identification method and system based on multi-mode open set associative reasoning. The method comprises the steps that transformer substation oily equipment defect images are collected and marked; constructing a multi-modal data set of image-text matching; yOLOv8 is adopted to extract visual features, and a CLIP model is adopted to process text features; designing a re-parameterized visual language path aggregation network for feature fusion; training the model based on a contrast loss function; the applications are deployed after the performance is evaluated through multiple indexes; through the multi-modal feature fusion and the re-parameterization path aggregation network, the problem that the oil leakage defect is difficult to mark due to liquid flowability and form variability is effectively solved, the false detection condition under multi-device cross interference is remarkably improved, and by combining multi-scale feature learning and an attention mechanism, the oil leakage detection accuracy is improved. And the identification accuracy of the oil stain defect is greatly improved.
Owner:NARI TECH CO LTD +1

Method and system for building compilable customized module

Disclosed in the present invention are a method and system for building a compilable customized module. The method comprises: customizing a module on the basis of a process, defining a modeling language syntax, and generating a modeling language file corresponding to the customized module; establishing a text matching rule and a grammar matching rule, using flex to generate a source program file of a lexical analyzer from the text matching rule, and using bison to generate a source program file of a syntax analyzer from the grammar matching rule; performing lexical analysis and generating an abstract syntax tree; traversing the syntax tree, generating a translation C++ file for a modeling language on the basis of a syntax-based translation configuration file, and compiling the translation C++ file; and importing the built customized module and an executable binary file into a simulation platform, and performing flow simulations for different scenarios including process simulation, optimization and parameter tuning. The present invention can realize the rapid commissioning of a process and can seamlessly switch a customized model to various application scenarios.
Owner:SUPCON TECH CO LTD

Similar law judgment document matching method based on prompt engineering and judgment prediction

The invention discloses a similar law judgment document matching method based on prompt engineering and judgment prediction, belongs to the field of text matching, aims to solve the problem of inaccurate case matching in case retrieval of law case handling personnel, and is characterized by comprising the following steps: S10, acquiring a training data set; wherein the data set comprises case serial numbers, judgment texts and legal elements; s20, training a class case matching model by using the training data set, wherein the class case matching model comprises a plurality of Bert models sharing weights; wherein the class case matching model is trained by adopting a joint learning method; and S30, performing similarity prediction on the judgment text contents of a plurality of input judgment documents by using the trained class case matching model.
Owner:DALIAN UNIV OF TECH

Myocardial infarction complication identification method based on heart multi-mode image

PendingCN120071059AMedical automated diagnosisNeural learning methodsHeart diseaseMyocardial infarction complications
The invention belongs to the technical field of medical image recognition, and particularly discloses a myocardial infarction complication recognition method based on a heart multi-modal image, and the method comprises the steps: firstly obtaining multi-modal medical image data and corresponding diagnosis report text data; constructing a myocardial infarction complication identification model; dividing the whole network into a feature extraction stage and an image text feature matching stage by adopting a separation training thought, carrying out back propagation on the network in the feature extraction stage through L1 loss, and optimizing weight parameters of the feature extraction network; performing back propagation on the network in the image text matching stage by comparing loss, and optimizing parameters of an image and a text encoder; and finally, collecting a medical image of the heart of a patient, inputting the medical image into the trained myocardial infarction complication identification model, and generating a character diagnosis report of heart disease complication. Therefore, early automatic diagnosis of myocardial infarction complications is realized.
Owner:GENERAL HOSPITAL OF SOUTHERN THEATRE COMMAND OF PLA

A multi-channel short text matching method integrating external knowledge and syntactic structure

The invention discloses a multi-channel short text matching method integrating external knowledge and syntactic structure, belonging to the field of deep learning technology, the method is based on the acquired text to be matched, based on the semantic relationship of the semantic original in the external knowledge base HowNet, the semantic similarity between words is calculated by topological distance, and the semantic original similarity matrix between words is determined; the attention weight of each word in the text is obtained by calculating the attention score of the text; according to the semantic original similarity matrix between words and the attention weight of each word in the text, the semantic feature vector of the text pair integrating the semantic original information is generated; by parsing the dependency relationship of the text and constructing a dependency structure graph, the nodes, edges and features of the graph are encoded as vector representations, and the syntactic structure feature vector of the text is obtained; the semantic feature vector and the syntactic structure feature vector are fused to obtain a comprehensive feature vector and input it into a classifier, and the text matching result is output. The method can achieve accurate matching of short texts.
Owner:CHANGAN UNIV

Training method and apparatus for image-text matching model, device and storage medium

The present disclosure provides a training method and apparatus for an image-text matching model, a device and a storage medium. The method includes: acquiring a positive sample and a negative sample; where the positive sample includes text and an image, the text in the positive sample is used to describe content of the image in the positive sample; the negative sample includes text and an image, the text in the negative sample describes content that is inconsistent with content of the image in the negative sample; training the image-text matching model by using the acquired positive sample and the acquired negative sample based on a manner of contrastive learning; where the image-text matching model is used to predict, for an input image and input text, whether the input text is used to describe content of the input image.
Owner:BEIJING BOE TECH DEV CO LTD +1

Graph representation learning and retrieval method and system based on code warehouse graph structure

The invention discloses a graph representation learning and retrieval method and system based on a code warehouse graph structure, and the method comprises the steps: constructing a heterogeneous code graph related to an original code, a newly-added code or a modified code, carrying out the graph representation learning of the heterogeneous code graph, generating a node embedding vector, carrying out the self-supervised learning through an intelligent model, and carrying out the retrieval of the node embedding vector. And outputting the updated node embedding vector. The method comprises the steps of detecting whether newly-added codes or modified codes exist or not in real time, only performing heterogeneous graph modification on the newly-added or modified codes, receiving a query request from a user, generating corresponding code snippets according to the request, checking whether the target code snippets meet grammar specifications or not, and outputting a query result. According to the method, the structural relationship between codes is fully modeled, a structure perception retrieval mode is realized, the method is superior to a traditional text matching scheme, and the code semantic understanding capability is improved; and large-scale code graph online query is supported, and the applicability is wider.
Owner:PEKING UNIV +1

Dance costume prop retrieval method based on causal reasoning and cross-modal matching

The invention discloses a dance costume prop retrieval method based on causal reasoning and cross-modal matching, which comprises the following steps: in a matching task, calculating the similarity between costume prop images and text matching features by a model, and judging whether the costume prop images and the text matching features are matched or not according to a threshold value; and when the task is retrieved, calculating the similarity between the query text and the image feature set, and sorting to obtain a result. The process of calculating the features and the similarity comprises the following steps: firstly, extracting multi-scale visual features of an image by using Faster R-CNN, and extracting multi-granularity semantic features of a text by using BERT; then, the two types of features are subjected to self-attention and gating fusion respectively and then are input into a Transform layer, and final visual and text representation is obtained; then, multi-head attention calculation is carried out on the two representations, self-attention calculation is carried out after sequences are combined, and matching features are obtained; and finally, dividing the dot product of the two feature vectors by the modular length product to calculate the similarity. According to the invention, the matching accuracy of dance costume props can be improved.
Owner:CHANGSHU INSTITUTE OF TECHNOLOGY

Intelligent matching system based on big data and matching method thereof

The invention relates to the technical field of policy matching, and discloses an intelligent matching system based on big data and a matching method thereof. The system comprises a data acquisition module, a condition analysis module and an intelligent matching module. The data acquisition module is used for acquiring policy text, enterprise qualification and historical case data and constructing a matching basic database; the condition analysis module is used for extracting policy dominant conditions through a regular expression, mining implicit conditions from failure cases by utilizing an association rule algorithm, and calculating condition weights by adopting a random forest algorithm; the intelligent matching module integrates dominant and implicit conditions to calculate a matching coefficient, dynamically divides risk levels and generates an early warning, the matching comprehensiveness and the early warning sensitivity are improved by mining the implicit conditions, dynamically adjusting the matching model and reinforcing a learning optimization mechanism, upgrading from text matching to risk prejudgment is achieved, and the method has the advantages of being high in practicability and easy to popularize. The enterprise declaration success rate can be improved, and the repeated declaration rate is reduced.
Owner:BEIJING LIANYUE TECHNOLOGY CO LTD

Alarm information processing method and device and storage medium

The invention discloses an alarm information processing method and device and a storage medium, and relates to the technical field of data processing.The method comprises the steps that an unstructured alarm text is converted into an accurate keyword set through word segmentation processing, the limitation of traditional simple text matching is broken through, alarm core semantic information is effectively extracted, and redundant interference is reduced; according to the method, the mapping relation between the keyword and the alarm identifier is established and stored in the database, so that an efficient retrieval structure based on the inverted index is realized, the overhead of full-text scanning is avoided, and the retrieval speed of massive alarm data is remarkably improved; according to the method, the target alarm text can be more accurately positioned based on semantic relevance of the keywords, and the rigidity of traditional fixed condition retrieval is overcome, so that the practicability of alarm information retrieval is synchronously optimized in the aspects of efficiency and precision. The technical problem of low retrieval efficiency in a traditional retrieval mode is solved, and the technical effect of improving the retrieval speed of mass alarm data is achieved.
Owner:ZHENGZHOU YUNHAI INFORMATION TECH CO LTD

Semantic enhancement text matching method and system based on achievement transformation large model

The invention belongs to the technical field of large model application, and particularly relates to a semantic enhancement text matching method and system based on an achievement transformation large model. The semantic enhancement text matching method based on the achievement transformation large model comprises five parts, namely preparation of data and knowledge, enhancement of multi-level semantic knowledge, learning of semantic difference, intelligent weight distribution of semantic information and prediction of text similarity. The preparation of data and knowledge is used for constructing basic data and knowledge resources required by a text matching task through text preprocessing and knowledge base construction. And enhancement of multi-level semantic knowledge: enriching semantic representation of the text by injecting external knowledge at three levels of word level, phrase level and sentence pattern level. And semantic difference learning: learning fine-grained semantic features by comparing a learning framework, and improving the recognition capability of the model on the text semantic difference. Intelligent weight distribution of semantic information is carried out, and dynamic balance and optimal combination of different source information are realized by evaluating feature importance and dynamically adjusting weights. And predicting the text similarity, calculating the similarity score of the text pair by integrating the feature information of each stage, and providing an interpretable prediction result. The complete text matching method and system are designed based on the achievement transformation large model, the problems of insufficient knowledge utilization, incomplete semantic representation, fixed feature weight and the like of the large model in a text matching task are effectively solved, the text matching effect of the large model in an actual application scene is improved, and the text matching efficiency is improved. And a feasible solution is provided for achievement transformation of a large model technology.
Owner:GUANGZHOU UNIVERSITY +1

Harbor ship unloading area analysis method based on AIS data

The invention relates to an AIS data-based port ship unloading area analysis method. The method comprises the following steps of: 1, preprocessing AIS data, namely carrying out cleaning screening and attribute supplementation on original data; 2, ship staying leg extraction: defining a staying leg by using motion features and expressing the staying leg by using a Geohash grid; step 3, port destination text matching: matching irregular destinations with standard port names; 4, bidirectional destination retrieval: analyzing an accurate destination by using a ship trajectory, and determining an affiliation port of a staying grid; and 5, grid integration and route cutting: integrating grids belonging to different ports to obtain an unloading area, and cutting a whole-cycle route of the ship. According to the method, the problems of complex unloading mode and complex route staggering around a port are solved, and reliable technical support is provided for trade monitoring of bulk energy goods such as crude oil by utilizing the complementary advantages of time-space information and destination texts in AIS data.
Owner:NANJING UNIV

Multi-modal data processing method and device based on large model, equipment and medium

The invention discloses a multi-modal data processing method and device based on a large model, equipment and a medium. Performing feature extraction and association mining on the multi-modal data to obtain multi-modal data features; according to a retrieval sequence of the multiple text matching methods, sequentially using the multiple text matching methods to perform text retrieval on the multi-modal data features in a pre-constructed knowledge base to obtain a first associated sub-graph; based on the semantic similarity between the multi-modal data features and multi-modal data in a pre-constructed knowledge base and a similarity threshold value, determining a second associated sub-graph; performing standard evaluation on the plurality of candidate results in the first associated sub-graph and the second associated sub-graph, and sorting the plurality of candidate results according to an evaluation result to obtain a retrieval result; inputting the retrieval result and the multi-modal data features into a large model for information analysis to obtain prompt information and a target tool; and sending the prompt information and the target tool to the terminal, and receiving a data processing result of the terminal.
Owner:QINGDAO HISENSE TRANS TECH

Method for constructing vectorized knowledge base based on dynamic fields for RAG system

The invention discloses a method for constructing a vectorized knowledge base based on dynamic fields for an RAG system, and the method comprises the following steps: 1, designing knowledge base fields: designing vectorized knowledge base fields, including static fields and dynamic fields, according to needs; step 2, document analysis and vectorization: analyzing the knowledge documents in various formats to generate a plurality of text blocks, and performing vectorization representation on each text block; step 3, constructing an index: constructing an index for each text block; and 4, determining a recall mechanism, improving the recall accuracy and coverage rate by adopting a text matching or vector similarity mode, and serving the constructed knowledge base for the RAG system. According to the method, a user can be helped to design the knowledge base based on dynamic fields, the construction process of the knowledge base is simplified, the construction efficiency of the vectorized knowledge base is improved, and later data updating and maintenance are more intelligent.
Owner:SHANGHAI ELECTRIC POWER DESIGN INST

Language large model construction method oriented to vertical field of forestry

The invention discloses a large language model construction method for the vertical field of forestry, which belongs to the technical field of computer models and comprises the following steps: constructing a first data set of forestry management business questions and answers; constructing a multi-modal coding model; freezing a text stream of the multi-modal coding model, forming the multi-modal coding model and an LLM decoder into a large language model, and training the large language model for generating a target text related to user input during intelligent question answering; constructing an RAG knowledge base; and performing intelligent question and answer based on RAG retrieval. Aiming at the characteristic that forestry management business images and texts have high matching, innovative technologies of multi-modal joint coding, weighted similarity calculation and LLM training by a frozen text encoder are introduced, and the quality and application value of forestry intelligent question answering and document generation are greatly improved. And the problem that the existing image and text matching precision is insufficient is solved, and more flexible and accurate retrieval and intelligent question answering can be realized in combination with an RAG technology.
Owner:JIANGXI WOODPECKER TECH CO LTD

Children book content semantic analysis method and system based on image recognition

The invention discloses a child book content semantic analysis method and system based on image recognition, and the system comprises an image-text processing module, an image recognition module, an image-text matching module, a semantic analysis module, and an intelligent question answering module, and aims to provide accurate and safe book content semantic analysis service meeting the cognitive level for children. According to the method, a multi-modal large model image recognition technology is fused, training is performed based on massive children book image-text data, illustrations, scenes and roles in books can be accurately recognized, and semantic information of book contents can be deeply understood in combination with a natural language processing technology. Through a multi-link image recognition mode, semantic analysis is carried out by fully combining images and texts, accurate guidance is provided for children, points of interest and understanding ability of the children in the reading process are accurately grasped, answering and content optimization are dynamically carried out according to feedback of the children, and intelligent and accurate semantic analysis of the children's book content is truly achieved.
Owner:CITIC UNITED CLOUD TECH CO LTD

Image-text matching analysis method, system and device based on large model and multiple modes

The invention relates to an image-text matching analysis method, system and device based on a large model and multiple modes. The method comprises the following steps: 1) image-text analysis: analyzing an image into an image and a text; 2) carrying out fine-tuning multi-mode large model processing on the image, and carrying out word segmentation processing on the characters; 3) constructing a bisection weight; 4) performing Few-shot cue word construction on the processed characters and the constructed equally-divided weight; 5) performing training / fine tuning on the characters and the pictures in the step 4 and the step 2 to serve as a large oracle model; according to the multi-mode image-text matching method, cooperative work of the multi-mode large model and the large language model is introduced, the advantages of the multi-mode large model and the large language model in semantic comprehension, context analysis and dynamic adjustment capacity are comprehensively utilized, a comprehensive image-text matching framework is established, and the accuracy, flexibility and interpretability of matching can be improved.
Owner:DEQING ALPHA INNOVATION RES INST

Weak supervision instance segmentation method and device, computer equipment and storage medium

The invention relates to a weak supervision instance segmentation method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a training image set and an image-level label, and generating a pseudo-semantic segmentation label; extracting multi-scale image features in the training image set, performing prompt contrast learning based on the multi-scale image features and the pseudo-semantic segmentation labels, and generating contrast loss and a pixel-text matching graph; splicing the multi-scale image features and the pixel-text matching graph to generate enhanced features, and training an instance positioning module based on the enhanced features to generate an instance center point and positioning loss; generating a semantic mapping graph through the segmentation network, and calculating self-matching loss of prompt guidance; calculating the total loss of the model based on the obtained loss, and performing iterative training on the instance segmentation model based on the total loss of the model to generate a target instance segmentation model; and generating a target instance segmentation result based on the to-be-processed image and the category text through the target instance segmentation model. According to the invention, the precision of instance segmentation is improved.
Owner:ZHEJIANG UNIV

Name disambiguation method based on multi-relation deep retrieval text matching

The invention provides a name disambiguation method based on multi-relation deep retrieval text matching, and relates to the technical field of name disambiguation. Cross-modal data related to enterprises and persons are obtained, data alignment and data fusion are conducted on the cross-modal data through an entity alignment algorithm, and a person-enterprise structured data set is formed; a multi-relation deep retrieval model is established based on a pre-training language model to generate a semantic vector, semantic similarity is calculated according to the semantic vector, a character embedding vector is generated according to co-occurrence frequency and time-space correlation characteristics calculated by a human-enterprise structured data set, and a disambiguation recognition model is established based on an adversarial neural network. Performing name disambiguation according to the character embedding vector, establishing a structure updating model through a graph attention network, updating a graph structure in real time, calculating semantic similarity and the confidence coefficient of the co-occurrence frequency according to the Bayesian theorem, and updating the weight of the semantic similarity and the weight of the co-occurrence frequency according to the confidence coefficient.
Owner:ANHUI CNBI SOFTWARE TECH CO LTD