Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

82 results about "Dynamic text" patented technology

Dynamic text is text on a map layout that changes based on the current properties of the project, map frame, map, and so on. When you insert a piece of dynamic text, it automatically displays the current value of its respective property. When that property is updated, the dynamic text automatically updates.

Text block dynamic segmentation method and system based on RAG

The invention provides an RAG-based text block dynamic segmentation method and system, and the method comprises the steps: analyzing a multi-level directory structure of a document, and segmenting a text block by taking a directory node as a reference; for a document without a directory or with an incomplete directory, a mixed segmentation mode of rules and semantics is switched; performing potential Dirichlet distribution topic modeling on the continuous text stream, and calculating a topic distribution vector of each text segment in real time; performing adaptive segmentation on the detected topic boundary, inserting a hard segmentation mark at a topic mutation point, and performing soft segmentation on a gradual change topic area; the initial window size of the sliding window is set according to the document type, and the semantic density in the window is monitored in real time; and layering, blocking and recombining. The system comprises a segmentation mode module, a mark confirmation module and a block recombination module. According to the method, the RAG retrieval accuracy is improved, the memory occupation is reduced, and meanwhile, the streaming throughput is supported.
Owner:北京三维天地科技股份有限公司

Target multi-modal model system and construction method, video processing model training method, and video processing method

Embodiments of the present invention provide a target multi-modal model system and construction method, a video processing model training method, and a video processing method. The video processing model training method comprises: inputting a video sample and each initial text sample into a video processing model, wherein the initial text sample is a text for performing category description on video content of the video sample; using the video processing model to perform feature extraction on the video sample to obtain a temporal motion feature and a fused image feature; using the video processing model to perform feature extraction on the initial text sample to obtain a dynamic text feature and a fused text feature; and training the video processing model on the basis of the temporal motion feature and the dynamic text feature, and the fused image feature and the fused text feature.
Owner:CLOUD INTELLIGENCE ASSETS HOLDING (SINGAPORE) PTE LTD

RAG multi-mode document analysis method and device and medium

The embodiment of the invention discloses an RAG multi-mode document analysis method and device and a medium, relates to the technical field of natural language processing and is used for solving the problem that an existing analysis mode is low in processing capacity. The method comprises the following steps: identifying a document type of a multi-modal document, and distributing an analysis task of the multi-modal document to a corresponding Kubernetes task node based on the document type; based on the analysis requirement of the multi-modal document, the analysis process of each Kubernetes task node is adjusted, the analysis task of the multi-modal document is executed based on the updated analysis process, and analysis data is obtained; preprocessing the parsed data, and performing semantic slicing on the processed parsed data in a dynamic text windowing mode based on a preset large language model to obtain semantic blocks corresponding to the processed parsed data; and converting the semantic blocks into embedded vectors, and storing the embedded vectors in a preset vector database so as to carry out rapid retrieval based on the preset vector database.
Owner:SHANDONG INSPUR SCI RES INST CO LTD

Personalized and dynamic text to speech voice cloning using incompletely trained text to speech models

Systems and methods are provided for machine learning models configured as zero-shot personalized text-to-speech models which comprise a feature extractor, a speaker encoder, and a text-to-speech module. The feature extractor is configured to extract acoustic features and prosodic features from new target reference speech associated with the new target speaker. The speaker encoder is configured to generate a speaker embedding corresponding to the new target speaker based on the acoustic features extracted from the new target reference speech. The text-to-speech module is configured to generate the personalized voice corresponding for the new target speaker based on the speaker embedding and the prosodic features extracted from the new target reference speech without applying the text-to-speech module on new labeled training data associated with the new target speaker.
Owner:MICROSOFT TECHNOLOGY LICENSING LLC

Multi-stage serial knowledge graph construction method and system based on cue words

The invention discloses a cue word-based multi-stage serial knowledge graph construction method, which comprises the following steps of: pre-configuration: initializing various parameters and resources required by a system through large language model configuration, cue word set configuration and example set configuration; knowledge extraction: realizing accurate knowledge extraction through dynamic text partitioning and a multi-stage serial extraction chain, extracting entities, attributes and relationships from a text, and generating a preliminary knowledge triple; and graph construction: organizing the knowledge triples into a structured knowledge graph through database configuration, knowledge storage and construction and interface and query, and realizing query and visual display. According to the method, the sizes of text blocks are dynamically calculated according to a dynamic text blocking strategy, the situation of too many or too few blocks is reduced, the knowledge extraction efficiency and accuracy are improved, a knowledge extraction task is decomposed into an entity recognition stage, an attribute completion stage and a relation extraction stage through a multi-stage serial extraction chain, and the accuracy and consistency of extraction results are ensured.
Owner:SOUTH CHINA UNIV OF TECH

Hydrofracture question-answering system and method based on cross-language retrieval enhanced generation

The invention discloses a system and a method for enhancing generation of hydrofracture questions and answers based on cross-language retrieval. The knowledge question-answering system aiming at the hydraulic fracturing field is developed by utilizing a cross-language retrieval enhancement generation technology, so that a user can obtain an optimal answer and a corresponding source from a multi-language knowledge base by matching only by using Chinese retrieval; the complex process that a user needs to find useful information from numerous and jumbled papers and reports is changed. According to the system, a dynamic text vector library framework is adopted, multi-language and complex document formats are supported, the analysis function of tables and formulas is integrated, documents in the hydraulic fracturing field are stored in the same knowledge base, the knowledge base is continuously updated along with document expansion in the professional field, a user can inquire related problems in the field in the system, and the user experience is improved. By integrating and updating a large amount of document information, required professional knowledge in the field can be efficiently obtained.
Owner:ZHEJIANG UNIV

Training method and device of open vocabulary detection model and electronic equipment

The embodiment of the invention provides a training method and device of an open vocabulary detection model and electronic equipment. The open vocabulary detection model training method comprises the steps of obtaining sample data; the sample data comprises a sample image, a sample text and label information, inputting the sample data into a neural network model to be trained, and extracting text feature information of the sample text and image feature information of the sample image; generating dynamic text prompt information according to the image feature information; performing fusion processing on the text feature information and the dynamic text prompt information to obtain semantic fusion feature information; predicting an object in the sample image according to the semantic fusion feature information; and performing iterative training on the neural network model according to the object prediction result, the label information, the text feature information, the dynamic text prompt information and the semantic fusion feature information to obtain an open vocabulary detection model. According to the invention, the context sensing, semantic generalization and fine-grained understanding capabilities of the open vocabulary detection model can be improved.
Owner:HANGZHOU WEIMING XINKE TECH CO LTD +1

Multi-degraded image restoration method based on semantic guidance

The invention discloses a multi-degraded image restoration method based on semantic guidance. According to the method, for degraded images captured by a vehicle-mounted camera under severe weather conditions, potential features of the images are extracted by adopting a trunk network based on Transform, multi-modal semantic information is extracted in combination with a CLIP visual language model, and semantic guidance is provided for different degradation types through a dynamic text prompt generation mechanism. A self-adaptive feature fusion module is designed, channel attention and space attention mechanisms are combined to realize effective integration of multi-modal features, and a degradation feature extraction and fusion module is introduced to enhance the generalization ability of the model. According to the semantic guidance multi-type image recovery network SGIRN provided by the invention, the global modeling capability of the Transform and the cross-modal representation capability of the CLIP visual language model are combined, so that high-quality recovery of various weather degradation types is realized.
Owner:SHENYANG INST OF COMPUTING TECH CO LTD THE CHINESE ACAD OF SCI

Short video stitching and distributing method, program and storage medium

The invention relates to a short video stitching and distributing method, a program and a storage medium, the method comprises the following steps: constructing an intelligent layered material library, the intelligent layered material library comprises a six-dimensional material management system, which is respectively a frame picture material, a background video material, a dynamic patch material, a background music material, a transition special effect material and a dynamic text material; the dynamic combination duplicate removal engine realizes intelligent combination and duplicate removal of each material in the intelligent layered material library to generate a short video, and the dynamic combination duplicate removal engine adopts a double-factor duplicate removal mechanism of a frequency control module and a content detection module to ensure that the generated short video is unique in intelligent combination and content presentation of each material; and the intelligent distribution control system completes high-efficiency compliance distribution of short video contents through a multi-stage state circulation and real-time decision-making mechanism. Through large-scale commercial verification, the generation amount of a single template per hour is increased from 80 to 380, and the material reuse rate is reduced from 47% to 8.5%; the image content similarity is reduced from 61.2% to 4.3%, and the audio repetition rate is reduced from 58.7% to 2.1%.
Owner:WUHAN YOUYOU TECHNOLOGY CO LTD

Thermal runaway prediction method and device of power battery, storage medium and electronic equipment

The invention provides a thermal runaway prediction method and device for a power battery, a storage medium and electronic equipment, and the method comprises the steps: obtaining a time sequence of a plurality of operation parameters of the power battery in a first historical time period, and determining a plurality of embedded matrixes according to the time sequence, the plurality of operation parameters comprising the voltage and temperature of the power battery; obtaining a static text matrix and a dynamic text matrix of the power battery; and obtaining a fusion data matrix according to the embedded matrix, the static text matrix and the dynamic text matrix, and inputting the fusion data matrix into a target large language model to predict an operation parameter prediction value of the power battery in a future time period so as to predict the thermal runaway state of the power battery. The operation parameter prediction value comprises a voltage prediction value and a temperature prediction value of the power battery. According to the method, through the steps of time sequence data reprogramming, cross-modal information fusion, large language model intelligent analysis and the like, all-around perception and risk assessment of the running state of the battery are achieved.
Owner:WEICHAI POWER CO LTD

Dynamic text response method and system for education question-answering robot

The invention provides a dynamic text response method and system of an educational question-answering robot, and relates to the technical field of data processing, and the method comprises the steps that the educational question-answering robot locates an initial guide node after matching a target guide skeleton according to real-time questions of students; performing question and answer scene feature filling on the static guidance language, and outputting an initial dynamic guidance language; iteratively executing cross-level dynamic matching and binding guide language filling until the real-time education goal reaches a closed loop; real-time weak nodes are positioned according to the learning situation portrait and the text response tracking data; loading to an education target association topology, and limiting, shrinking and positioning an education association topology node; and executing random guidance on the students until the associated education objectives reach a closed loop. The technical problem that in a traditional education question and answer robot, guide words are usually generated based on a fixed template, guide content cannot be dynamically adjusted according to specific learning conditions or answer feedback of students, and the learning effect is not ideal is solved.
Owner:QTONE EDUCATION GROUP (GUANGDONG) CO LTD

Multi-modal medical text extraction method based on adaptive domain knowledge fusion

The invention relates to a multi-modal medical text extraction method based on adaptive domain knowledge fusion, and belongs to the technical field of medical text mining, and the method comprises the following steps: text entity-relationship mining: extracting related entities and semantic relationships from clinical structured and unstructured texts, and generating a preliminary text disease network; professional knowledge linking: performing knowledge alignment by using an external medical knowledge graph; and extracting a medical spatio-temporal event chain: extracting an event sequence with a time sequence and a spatial position from a clinical text, constructing a dynamic text disease knowledge network, and extracting a multi-modal medical text.
Owner:THE SECOND AFFILIATED HOSPITAL ARMY MEDICAL UNIV

Allergic rhinitis diagnosis and allergen tracing system based on dynamic text guidance

The invention discloses an allergic rhinitis diagnosis and allergen traceability system based on dynamic text guidance, and belongs to the technical field of intelligent medical treatment. According to the method, the problems that in the prior art, patient description is inaccurate, allergens are various in variety and have differences, and allergens are difficult to accurately determine and trace are solved, the initial situational sub-graph is verified by introducing periodic features of the patient, the causal contribution degree of the initial situational sub-graph is evaluated through a Bayesian reasoning algorithm, and a mode association result library is formed; according to the method, the crossing of the allergic rhinitis diagnosis and tracing from the traditional static judgment to the dynamic accurate inference is realized, and a personalized and scientific allergen avoidance scheme is provided for patients; through similarity retrieval, environmental information injection verification and calculation of a mode goodness-of-fit score, under the condition that patient symptoms are in multi-factor mixing, a mixed causal graph is created through a graph fusion technology, and the mode goodness-of-fit score is calculated again, so that the accuracy of obtaining an allergen traceability result in practical application is ensured.
Owner:THE THIRD MEDICAL CENT OF THE CHINESE PEOPLES LIBERATION ARMY GENERAL HOSPITAL

Dynamic text tokenization for index-based searching of annotated data assets using keyword-based text searching

Devices, systems, and methods for tokenizing search attributes and terms of a search query for an index-based search. A method may include receiving, by a search service of a provider network, a first search query to search a first searchable document set, the first search query including a first search term in a first language; applying a first tokenization rule to identify the first search term in the first search query; determining that the first search term is in the first language; applying a second tokenization rule to tokenize the first search term based on the first search term being in the first language; causing a launch of a search instance by a managed compute service of the provider network, the search instance to execute a search function for a keyword-based text search using the tokenized first search term.
Owner:AMAZON TECH INC

Cultural interaction device and method based on artificial intelligence and multi-modal perception

The invention discloses a culture interaction device and method based on artificial intelligence and multi-modal perception, the culture interaction device based on artificial intelligence and multi-modal perception comprises a multi-modal perception unit, a generative content processing unit, an entity display unit and a control system, the multi-modal perception unit is connected with the generative content processing unit, and the entity display unit is connected with the control system. The generative content processing unit is connected with the entity display unit through the control system; the multi-modal sensing unit is used for acquiring visual data of a user and sending the visual data to the generative content processing unit; the generative content processing unit is used for processing the visual data, generating dynamic character image information of the user and sending the dynamic character image information to the entity display unit through the control system; and the entity display unit is used for displaying the dynamic character image information to the user. According to the method and the device, capturing, processing and analyzing of user input features and generation and dynamic display of personalized digital contents can be realized, and immersive culture interaction experience is provided for users.
Owner:SOUTHERN UNIVERSITY OF SCIENCE AND TECHNOLOGY

Industrial dynamic tracking semantic analysis method and system based on artificial intelligence

The invention provides an industry dynamic tracking semantic analysis method and system based on artificial intelligence, and the method comprises the steps: firstly obtaining an industry dynamic text set which comprises a plurality of industry dynamic text units with source identifiers, constructing source semantic hierarchical mapping based on the source identifiers, and obtaining a hierarchical association relationship between the text units; calling the pre-training model to generate a semantic association rule set, constructing a dynamic semantic tracking link according to the semantic association rule set, extracting a core semantic development direction to determine a core tracking dimension, and integrating semantic information according to the dimension to generate an industry dynamic analysis report, so that the industry dynamic semantics can be efficiently and accurately tracked; and a comprehensive and accurate industry analysis result is provided for a user.
Owner:BEIXI INTELLIGENT FUTURE (SHANGHAI) ARTIFICIAL INTELLIGENCE TECHNOLOGY CO LTD

Wild animal posture estimation method based on dynamic prompt and semi-supervised multi-mode learning

The invention discloses a wild animal attitude estimation method based on dynamic prompt and semi-supervised multi-mode learning, and relates to the technical field of computer vision and animal behavior analysis, and the method comprises the steps: firstly carrying out the motion state analysis of an input video, so as to extract parameters such as the centroid velocity, the acceleration and the steering angle of a moving material; the method comprises the following steps: generating a dynamic text prompt containing directional semantics, constructing a cross-modal fusion time sequence convolutional network, fusing visual features and text prompt features through a space alignment mask mechanism and a cross attention module, capturing time sequence dependence by using a bidirectional ConvGRU network to realize attitude estimation, and finally, based on a semi-supervised multi-task learning framework, carrying out dynamic text prompt processing on the basis of the semi-supervised multi-task learning framework. A joint loss function including supervision loss, time difference loss, attitude PCA reconstruction loss, multi-modal alignment loss and motion consistency loss is adopted, a loss weight is adaptively adjusted in combination with prediction uncertainty to optimize network parameters, a smooth 2D / 3D animal skeleton is finally output, real-time reasoning can be realized on edge equipment, and the real-time reasoning efficiency is improved. Ecological analysis software is compatible, high precision is still kept under the condition of low-label data, and the performance is remarkably improved especially in an animal sharp turning scene.
Owner:BEIJING FORESTRY UNIVERSITY

Dynamic text message processing implementing endpoint communication channel selection

The present disclosure relates generally to providing a concierge service to handle a wide variety of topics and user intents via a text messaging interface. The concierge service can be part of a connection management system that can dynamically manage and facilitate natural language conversations between a user making a request or providing an instruction and one or more endpoints for the purposes of fulfilling the request or instruction.
Owner:LIVEPERSON INC

A Chromosomal Abnormality Detection Method Based on a Multimodal Large Model

This invention relates to the field of chromosome abnormality recognition technology, specifically to a chromosome abnormality detection method based on a multimodal large model. The method includes: constructing an image dataset and a text dataset; fusing image features and text features to obtain multimodal fusion features; assigning anomaly scores to image blocks belonging to band regions based on anomaly scoring rules formulated from the multimodal fusion features, and comprehensively processing the anomaly scores of all image blocks corresponding to the chromosome to determine whether the chromosome image is abnormal; and decoding and generating natural language text that meets the requirements of chromosome abnormality detection based on the multimodal fusion feature representation and anomaly scoring rules. This invention achieves accurate chromosome abnormality detection and band location positioning through multimodal fusion and dynamic text generation mechanisms, generating interpretable natural language text descriptions, and improving the practicality and interpretability of the detection results.
Owner:笑纳科技(苏州)有限公司

Method and system for enhancing reasoning stability of large model in text scene

The invention discloses a large model reasoning stability enhancement method and system in a text scene, and relates to the technical field of knowledge enhancement deep learning, and the method comprises the steps: building a structured text knowledge graph based on a text knowledge base, and generating a dynamic text knowledge embedding matrix through a graph attention network; inserting a text knowledge gating cross attention module into a decoder selection layer of the pre-trained large model, taking the text knowledge gating cross attention module as an external knowledge source, obtaining a knowledge enhanced hidden state after gating fusion, and constructing a transformation model; semantic equivalent perturbation is carried out on an input text to obtain a perturbation sample, the perturbation sample is input into the transformation model in parallel to obtain extraction probability distribution, and divergence and gradient direction consistency loss between two distributions are calculated; and combining cross entropy loss and gradient direction consistency loss to train and transform the model, and updating parameters to convergence to obtain a final large model. According to the method, the fact consistency of output can be improved, common optimization of knowledge guidance and stability constraint is realized, and the result is accurate and reliable.
Owner:DIGITAL HEALTH CHINA TECHNOLOGIES CO LTD

A lesion segmentation method of adaptive dynamic text prompt

The present application relates to the technical field of image processing, and more particularly to a lesion segmentation method of adaptive dynamic text prompt, comprising: acquiring a lesion image, and generating an adaptive dynamic text prompt based on the lesion image; constructing a multi-modal enhanced fusion text prompt adapter based on a FiLM global channel recalibration module, a spatial cross-attention interaction module and a zero initialization gate nonlinear integration module cascade; the global channel recalibration module of FiLM utilizes a feature linear modulation mechanism to perform channel-level weighting on a visual feature map; before the feature enters spatial interaction, according to the text semantic enhancement band response, the activation of the background noise channel is inhibited. The present application overcomes the problems of insufficient utilization of multi-modal prompts, dependence on artificial static labeling of text prompts, serious dependence on prior information in the reasoning stage and difficulty in multi-modal feature fusion in a low-contrast environment in the existing medical image segmentation technology.
Owner:CHANGZHOU UNIV

A text guidance and cross-attention auxiliary clothes-changing pedestrian re-identification method and system

PendingCN122369057APattern recognitionData set
This invention discloses a text-guided and cross-attention-assisted method and system for re-identifying pedestrians changing clothes. The method includes: preparing a dataset for re-identifying pedestrians changing clothes; resampling using a diversity identity sampler, prioritizing samples with high diversity to construct training batches; building a cross-modal feature representation model based on contrastive language-image pre-training, generating dynamic text prompts, and performing visual and text cross-modal alignment training; extracting key biometric features using pedestrian parsing technology, and constructing a residual-cross-attention mechanism to fuse RGB global features with biometric features; performing a second-stage optimization training by combining identity loss, triplet loss, and adaptive salient feature loss; inputting a query image into the trained model to obtain a list of pedestrian targets ranked by similarity. This invention significantly improves the recognition accuracy and model generalization performance in solving problems such as uneven data distribution and lack of semantic information in clothing-changing scenarios.
Owner:NANJING UNIV OF AERONAUTICS & ASTRONAUTICS

Character dynamic effect data generation method and device, storage medium and electronic equipment

The invention relates to a text dynamic effect data generation method and device, a storage medium and electronic equipment. The method comprises the following steps: constructing a copywriting library and a background library; according to the character dynamic effect template, randomly selecting target character content from a copywriting library, and generating a corresponding black-matrix character dynamic effect video; fusing the black-matrix character dynamic effect video with a target background image randomly selected from a background library by adopting a grid division strategy to generate a target dynamic effect video; and extracting a first frame of mask image of the target dynamic effect video to obtain a character mask marking image of the target dynamic effect video, and generating a training data set of the target character content in combination with the copywriting index information of the target dynamic effect video. The technical problems that a large amount of diversified dynamic character and background pairing data in a real scene is lacked, and high-quality character dynamic effect training samples are difficult to automatically generate are solved.
Owner:BEIJING QIYI CENTURY SCI & TECH CO LTD

Dynamic text response method and system for educational question-answering robot

The present invention provides a dynamic text response method and system for an educational question-and-answer robot, which relates to the field of data processing technology. The method includes: after the educational question-and-answer robot matches the target guidance skeleton according to the student's real-time question, it locates the starting guidance node; fills the static guidance language with question-and-answer scenario features and outputs the starting dynamic guidance language; iteratively performs cross-level dynamic matching and binding guidance language filling until the real-time educational goal is closed; locates real-time weak nodes based on the learning situation portrait and text response tracking data; loads it into the educational goal association topology, and narrows and locates the educational association topology nodes; and performs random guidance on students until the associated educational goal is closed. The present invention solves the technical problem that in traditional educational question-and-answer robots, the guidance language is usually generated based on a fixed template, and the guidance content cannot be dynamically adjusted according to the student's specific learning situation or answer feedback, resulting in unsatisfactory learning results.
Owner:QTONE EDUCATION GROUP (GUANGDONG) CO LTD

Industrial marine meteorological disaster forecasting method based on deep learning

The invention relates to the technical field of deep learning and ocean engineering meteorology, and particularly provides an industrial ocean meteorological disaster forecasting method based on deep learning. The method comprises the following steps: acquiring real-time meteorological data, and constructing a dynamic text prompt set; performing semantic-physical feature joint modeling on the dynamic text prompt set by using a multi-modal fusion expert module to obtain a real disaster sample; constructing a dream learning framework, entering a dream stage, and obtaining a virtual disaster dream sample; in an awakening stage, high-dimensional potential representation is obtained through a real disaster sample and a virtual disaster dream sample; according to the high-dimensional potential representation, a forecast result is output, a full-closed-loop industrial meteorological disaster prediction system is formed, and the method improves the accuracy, robustness and interpretability of meteorological disaster prediction in a marine industrial scene.
Owner:SHANDONG UNIV

User interfaces for indicating time

User interfaces for indicating time, displaying a user interface based on a day of the week, user interfaces that include a dynamic text string, user interfaces that include a customizable border complication, user interfaces that include a watch hand that changes color at predetermined times, and user interfaces that include a simulated lighting visual effect.
Owner:APPLE INC

User interfaces for indicating time

User interfaces for indicating time, displaying a user interface based on a day of the week, user interfaces that include a dynamic text string, user interfaces that include a customizable border complication, user interfaces that include a watch hand that changes color at predetermined times, and user interfaces that include a simulated lighting visual effect.
Owner:APPLE INC

Multimodal intention recognition method, system, device, and medium under uncertain modality

The present invention belongs to the field of data processing technology, and specifically relates to a multimodal intent recognition method, system, device, and medium under uncertain modalities. In order to cope with the uncertainty problem of text modality quality, text features and common features are subjected to multi-head attention processing to generate dynamic text weights and act on the text modality, thereby dynamically optimizing the text modality; in order to solve the problem of insufficient information caused by uncertain modalities, the cosine similarity function is used to calculate the similarity between the modality with the highest priority and the corresponding modality in a full-modal sample data set, and similar full-modal samples are obtained and spliced ​​to form target enhancement features, and then the target enhancement features and fusion features are deeply associated and learned through cross-attention processing, thereby enhancing the quality of the fusion features.
Owner:YANTAI UNIV

Asymmetric cross-modal momentum contrast learning-based galaxy form classification method and system

The invention discloses a galaxy form classification method and system based on asymmetric cross-modal momentum contrast learning, and belongs to the technical field of image classification, and the system comprises a visual encoder which extracts image features and stores the image features in a momentum feature queue; the text encoder is used for encoding the text information to generate initial text embedding; the dynamic text embedding projection layer is used for embedding the initial text into the projection to obtain a text projection; the classification head is used for predicting a galaxy form classification result; the processing module is used for calculating total loss including visual contrast loss and cross-modal alignment loss, and updating model parameters of the visual encoder, the dynamic text embedding projection layer and the classification head by using the total loss; for the total loss, the weighted weight of the visual contrast loss is greater than the cross-modal alignment loss. According to the method, asymmetric interaction of images and texts can be realized, text information is effectively fused in a visual dominant galaxy classification task, the classification precision is improved, and resource consumption is optimized.
Owner:JIANGSU HONGXIN SYST INTEGRATION

Open vocabulary object detection method, apparatus, device, and storage medium

PendingCN122313006AVisual perceptionVocabulary Object
This invention provides an open-vocabulary object detection method, apparatus, device, and storage medium, relating to the field of object detection technology. The method includes: extracting a visual feature map of the image to be detected based on an image encoder in an open-vocabulary object detection model; obtaining an alignment matrix of the visual feature map based on the positive dynamic text embedding, negative dynamic text embedding, and visual features of each target region in the visual feature map for each text category; wherein the positive and negative dynamic text embeddings for each text category are trained based on the visual features and easily confused visual features of that text category; and inputting the alignment matrix between each target region and each text category into a detection head to obtain the detection result. This invention improves the ability to distinguish easily confused categories during the detection process by establishing discriminative constraints for easily confused categories, thereby effectively improving the accuracy of open-vocabulary object detection results.
Owner:HARBIN INSTITUTE OF TECHNOLOGY (SHENZHEN) (INSTITUTE OF SCIENCE AND TECHNOLOGY INNOVATION HARBIN INSTITUTE OF TECHNOLOGY SHENZHEN)