123 results about "Semantic relationship" patented technology

The invention discloses a lightweight multi-modal content recognition system based on a double-track migration framework, and relates to the technical field of content recognition, and the system comprises a data collection module which is used for synchronously collecting multi-source content of a text and an image and carrying out standardization processing and tensor construction to form a fusion tensor X; and the model construction module is used for inputting the fusion tensor into a dual-track migration structure constructed based on a Transform backbone network, and the dual-track migration structure realizes task semantic alignment and structure migration under parameter freezing through Prompt Learning embedding and Adapter-Tuning insertion, and outputs an intermediate representation of modal alignment. According to the method, the training and deployment cost of multi-modal content recognition is remarkably reduced, the semantic expression ability in the modal is enhanced, the cross-modal alignment precision and fusion depth are effectively improved, and the perception and recognition ability of the model to the complex semantic relationship is enhanced.

The invention belongs to the technical field of cross-modal information retrieval, and provides a quick cross-modal retrieval method and system fused with fine-grained semanteme, multi-modal data to be retrieved are obtained, a trained cross-modal retrieval model is used for learning to obtain continuous Hash features, discrete Hash codes are generated according to the continuous Hash features, and the multi-modal data to be retrieved are retrieved according to the discrete Hash codes. Performing cross-modal matching and retrieval based on Hamming distance matching, and feeding back a retrieval result; in the training process of the cross-modal retrieval model, the fusion loss function, the cross-modal alignment loss function and the shared subspace loss function are subjected to weighted fusion, and corresponding parameters are optimized with the purpose of minimizing the total loss function after weighted fusion until training requirements are met. According to the method, the cross-modal semantic consistency among different modal features is optimized, so that the generated Hash code can reflect semantic association among multi-modal data more accurately, and the precision and efficiency of cross-modal retrieval are improved.

The invention discloses a camouflage target detection method and system based on a double-domain fusion enhanced network, and relates to the technical field of target detection. Through the nonlinear double-domain fusion module, in combination with nonlinear mapping of a spatial domain and a frequency domain, key difference characteristics of a frequency domain amplitude spectrum and a phase spectrum are captured, the problem that the detection performance is reduced in a scene of low contrast and the like depending on an RGB spatial domain is solved, and the target discrimination degree is improved; based on a lightweight scale perception modulation converter and a double-feature fusion module, multi-scale features are extracted, aligned and fused, a semantic relation is integrated by means of cross attention, and the problems of detail loss and boundary fuzziness caused by scale diversity are solved; the context feature enhancement module integrates cross attention and edge auxiliary injection, accumulates multi-layer feature integration, gives consideration to a global boundary and a local structure, effectively reduces false detection, missing detection and edge roughness, and further enhances robustness through multi-layer auxiliary supervision.

The invention provides an enterprise knowledge graph automatic construction and intelligent retrieval method, which comprises the following steps: collecting multi-source data from a heterogeneous enterprise information system, and carrying out data cleaning and standardization processing; on the basis of a comprehensive scoring mechanism of field similarity and behavior semantic vectors, entities from different systems are merged, and a standard entity set with a unique identifier is generated; based on the standard entity set, in combination with a scoring mechanism of a task-type relationship and a collaborative relationship, extracting a semantic relationship from a behavior record, and constructing an enterprise knowledge graph structure; extracting representative semantic paths from the knowledge graph structure, screening high-quality paths through a path scoring model, and organizing the high-quality paths into a structured path index set; and receiving a natural language query statement, encoding the natural language query statement into a semantic vector, matching the semantic vector with the path index set, executing query in the atlas in combination with an authority control mechanism, and returning a result.

The invention relates to the technical field of text processing, in particular to a text classification method and system based on semantic analysis, and the method comprises the following steps: segmenting semantic units, constructing a direction change sequence, positioning mutation nodes, generating a consistency section, forming a convergence section, and outputting a classification result. According to the method, a continuous change sequence is formed by constructing a semantic embedding vector and calculating a direction difference, a semantic mutation point can be anchored and divided into sections by combining mutation intensity identification and local jump tracking, and a semantic closed structure and a convergence section are extracted by means of context direction consistency judgment and generic label comparison; precise recognition of a semantic relation chain is realized, semantic jump and conflict starting points can be dynamically sensed, the semantic boundary recognition capability is improved, and the understanding and classification capability of a model on semantic attribution in a complex context is enhanced on the premise of not depending on a fixed dictionary and shallow statistics. The problems that a traditional model is slow in response to an abrupt change structure and weak in semantic convergence recognition are effectively solved.

The invention discloses an anti-fact multi-mode dialogue emotion causal reasoning method based on a double-branch hypergraph. The method comprises the following steps: respectively extracting sentence level feature vectors of three modes of text, voice and vision from input multi-mode dialogue data; carrying out modeling on a high-order relationship in the modals and between the modals by utilizing a hypergraph structure, and constructing a dialogue hypergraph containing multi-modal nodes and emotion nodes; introducing a hypergraph attention network on the hypergraph, learning contribution weight of each modal node to a target emotion node, and selecting a candidate reason node set; the candidate reason nodes are intervened, an anti-fact branch is constructed, a fact situation and final node feature representation under the anti-fact situation are calculated, and a causal effect vector is obtained; and designing a joint optimization objective function, and carrying out joint training on emotion recognition loss and causal consistency loss to realize synchronous prediction of emotion categories and emotion reasons. According to the method, a high-order semantic relationship can be effectively modeled in a multi-modal dialogue scene, and a key reason for emotion formation is reasoned.

The invention provides a culture symbol transmission method and system based on Sichuan opera facial makeup art patterns, and particularly relates to the field of facial makeup digital transmission. According to the method, multi-level modeling of a Sichuan opera facial makeup pattern from a pixel layer to a component layer and then to a semantic layer is realized by fusing image processing, component contour matching, image structure modeling and an image neural network technology; according to the method, main and auxiliary color block areas and component information in an image can be automatically identified and structurally stored, a spatial semantic relationship of the image is learned through a graph neural network, a structural semantic code is output, and finally a multi-dimensional cultural label index capable of being filed, clustered and classified is generated; the method effectively overcomes the limitation that in the prior art, only stays at an image level and is lack of structure and semantic modeling, enables the Sichuan opera facial makeup pattern to have machine-readable component structure representation and semantic feature representation, and lays a foundation for cultural inheritance and intelligent propagation.

The invention relates to a multi-modal knowledge representation learning method fusing a multi-attention mechanism and semantic enhancement, and aims to solve the problems that the multi-modal knowledge representation method is insufficient in multi-modal feature fusion and difficult to effectively model complex semantic relationships such as symmetric and anti-symmetric. The method comprises the following steps: firstly, extracting entity image and text features from a multi-modal knowledge graph by using a CLIP model, and extracting entity audio features by using a VGGish model; then image features are enhanced through spatial attention, and image-text feature fusion is realized through cross attention; designing a cross-modal fusion module to dynamically fuse the image-text features and the audio features to obtain multi-modal fusion features; and finally, extracting structured features based on a ComplEx model, integrating the structured features with multi-modal fusion features through an adaptive dual-channel scoring function, and optimizing a training process by adopting a contrast learning loss function. The multi-modal features can be fully fused, the semantic expression ability is enhanced, and the performance of downstream tasks such as intelligent question and answer is improved.

The invention relates to the technical field of artificial intelligence, and provides a multi-modal data fusion modeling method based on multi-task learning, and the method comprises the steps: respectively extracting image and text features in data, and carrying out the fusion to obtain first fusion data; respectively inputting the fused data into a Transform decoder and at least two weight layers for processing, and fusing the fused data with the original fused data again through a self-adaptive gating layer to obtain second fused data; and finally, based on the second fusion data, respectively calculating classification loss and comparison loss, and when the two types of loss meet preset conditions, outputting a final multi-modal data fusion model. The invention further discloses a system. According to the method and the system, the semantic alignment between the image and the text is enhanced, the cross-modal semantic problem is effectively solved, and the understanding ability of the model to the complex semantic relationship is improved. The model is enhanced to have adaptability in different task scenes, the generalization ability and robustness of the model are remarkably improved, and data noise is reduced.

The invention discloses a sparse view angle three-dimensional Gaussian language field construction method and system for humanoid robot grabbing. Comprising the following steps: acquiring an RGB image set and a user natural language instruction under a sparse view angle; reconstructing a three-dimensional point cloud through stereo matching, and initializing a three-dimensional Gaussian primitive field after noise elimination and measurement alignment; two-dimensional semantic features are embedded into the field, joint optimization is carried out through a double-path semantic supervision module, and a three-dimensional Gaussian language field with consistent semantics is constructed; wherein the dual-path semantic supervision comprises an object perception path and a global context path, the local semantic consistency and the overall semantic relationship are constrained respectively, and the final semantic representation of each Gaussian primitive is obtained through weighted fusion; and finally, candidate grabbing postures are generated, semantic reordering is carried out in combination with a language instruction, and the optimal grabbing posture is screened through geometric-semantic joint scoring. Under the sparse image set input condition, the execution accuracy of a robot grabbing task under a complex instruction can be improved.

The invention discloses an intelligent dialogue memory management method and system based on the logistics field, and relates to the technical field of logistics intelligent dialogues, and the method comprises the steps: obtaining the historical logistics dialogue data of a target user, generating a user feature-interactive entity-semantic relationship initial knowledge graph based on a QWen2.5-32B large language model, and constructing a Milvus semantic vector library; monitoring a logistics dialogue flow in real time, and starting an adaptive memory management mechanism by taking three rounds of dialogue as a judgment threshold value; extracting user dialogue content attribute features and behavior preferences, and dynamically updating a user feature-interactive entity-semantic relationship initial knowledge graph and a Milvus semantic vector library; and triggering a logistics dialogue according to a target user, starting a multi-modal memory recall mechanism, cooperatively retrieving a user feature-interactive entity-semantic relationship knowledge graph and a Milvus semantic vector library, generating a personalized intelligent dialogue response, and realizing a logistics intelligent dialogue memory management closed loop. The beneficial effect of the invention is that the intelligent and personalized capabilities of the system are enhanced.

This invention discloses a lightweight multimodal content recognition system based on a dual-track transfer framework, belonging to the field of content recognition technology. It includes a data acquisition module for simultaneously acquiring multi-source content from text and images, performing standardization processing and tensor construction to form a fused tensor X; and a model construction module for inputting the fused tensor into a dual-track transfer structure built on a Transformer backbone network. This dual-track transfer structure achieves task semantic alignment and structural transfer under parameter freezing through Prompt Learning embedding and Adapter-Tuning insertion, outputting an intermediate representation of modality alignment. This invention significantly reduces the training and deployment costs of multimodal content recognition, enhances the semantic expression capabilities within modalities, effectively improves cross-modal alignment accuracy and fusion depth, and enhances the model's ability to perceive and recognize complex semantic relationships.

The invention discloses a method and a system for searching by combining user input information, and particularly relates to the technical field of language paragraption.The method comprises the following steps: firstly, converting a plurality of user inputs into an input semantic structure containing keywords, semantic relationships and context tags; comparing the structures, extracting interactive focuses and generating a semantic consensus score; constructing a fusion path diagram according to the consensus score, the keyword weight and the semantic adaptation degree; semantic fusion and conflict mediation are executed according to the path diagram, and a unified expression structure is generated; and finally, sorting the keywords according to semantic levels and task orientation, and constructing a comprehensive search expression to finish query output. According to the method, through semantic analysis and cross comparison input by multiple users, the consensus content is extracted, the fusion path diagram is constructed, and it is ensured that the fusion sequence is reasonable and semantic consistency is achieved; and finally, a comprehensive semantic expression structure is generated and a search expression is constructed, so that accurate alignment of query contents and user intentions is realized, and representativeness and accuracy of search results are improved.

The invention provides an intelligent processing method and system from structured data to a text based on a natural language, and relates to the technical field of data processing.The method comprises the steps that the text chunk analysis process is optimized and adjusted according to analysis optimization parameters, sentences are segmented into non-overlapping phrases with syntactic function labels, and the text after chunk analysis is obtained; performing syntactic and semantic structure analysis on the text subjected to block analysis, and establishing a semantic association relationship between internal structures of sentences through component analysis, dependency analysis and semantic dependency graph analysis to obtain structured semantic information; performing multi-sentence logic association analysis on the structured semantic information on a chapter level to obtain semantic information of the whole chapter; and based on the semantic information of the overall chapter, obtaining a target natural language text by utilizing a pre-trained large language model. According to the method, the accuracy and fluency of conversion from the structured data to the text are improved.

Techniques for performing contextualized intent and slot extraction using a large language model (LLM) are disclosed. The LLM is generally pre-trained on an arbitrary corpus of language training data. A prompt is provided to the LLM. This prompt includes a limited number of prompt phrases. The prompt phrases share a semantic relationship with one another. A spoken utterance is recorded and then converted to text, resulting in generation of a transcription. The transcription is provided to the LLM. The LLM extracts, from the transcription, an extracted intent and an extracted slot. The extracted intent is determined to be related to a prompt-described intent that was included in the prompt. The prompt is supplemented by adding the extracted intent and the extracted slot to the prompt, resulting in the extracted intent being identified as sharing the semantic relationship with the other prompt phrases in the prompt.

The invention discloses a knowledge graph semantic path guided remote sensing data recommendation method, which introduces a representation learning mechanism based on knowledge graph semantic path guidance, and realizes modeling of structured semantic association between nodes by constructing Meta-Path with task semantics. According to the method, a path-oriented random walk strategy is designed by combining a Meta-Path2Vec embedding method, semantic representation in an embedding space is obtained, and a sorting recommendation result is calculated and generated based on path similarity. According to the method, computable modeling of the semantic chain in the knowledge graph structure is realized, and path controllability, semantic interpretability and structural universality in the recommendation process are enhanced. According to the method, a complex semantic relationship among a natural disaster type, a remote sensing application task and remote sensing data can be modeled through meta-path constraint in a multi-source heterogeneous environment. In combination with a Meta-Path2Vec embedding method and a random walk strategy, high-order semantic association modeling between tasks and remote sensing resources is realized, and potential relationships between entities are effectively captured.

The invention relates to the technical field of multi-modal dense subtitle generation, in particular to a multi-modal dense subtitle generation method and system based on heterogeneous graph convolution enhancement, and the method comprises the steps: obtaining image data, and extracting visual features and position codes of a region of interest in an image through a multi-modal encoder; constructing a multi-modal heterogeneous graph based on the regional features and the position codes, and defining three edge types of a spatial relationship, a semantic relationship and cross-modal interaction; performing multiple rounds of message passing and feature aggregation by adopting a hierarchical heterogeneous graph convolutional network to generate enhanced features; the enhanced features are input into a context relation modeling module, and representation containing rich context information is generated; the system comprises a data acquisition module, a heterogeneous graph construction module, a feature aggregation module and a subtitle generation module. According to the method, the multi-mode heterogeneous interaction capability can be improved, the accuracy of generated subtitle semantics is high, and the method can be applied to scenes such as automatic driving, virtual assistants and intelligent media analysis.

The invention discloses a four-dimensional index-based scientific and technical literature knowledge automatic labeling method and system, and relates to the technical field of natural language processing, and the method comprises the following steps: constructing a four-dimensional index architecture; the method comprises the following steps: receiving multi-format scientific and technical literatures through a data input layer, and performing index analysis to obtain a document layer semantic structure, a paragraph layer semantic relation graph, a sentence semantic relation network and an entity semantic relation; performing semantic density calculation and knowledge enrichment region recognition to obtain a knowledge enrichment region recognition result; and establishing a cross-hierarchy semantic association model, performing entity boundary prediction and multi-level labeling on a knowledge enrichment area recognition result to obtain a structured semantic labeling result, and performing visual output and display. The technical problem that in the prior art, scientific and technical literature knowledge recognition is not accurate, hierarchical labeling is insufficient, and consequently the literature processing efficiency and accuracy are low is solved, and the technical effects that precise recognition and multi-hierarchical labeling of scientific and technical literature knowledge are achieved, and the literature processing efficiency and accuracy are improved are achieved.

Disclosed is a system and method for enhancing value for advertisers by helping them select and initiate the purchase of an advertisement associated with advertising (ad) words or phrases that have strong semantic relationships to a given context, but which are not necessarily the most popular ad words or phrases with the highest costs. Advertisements associated with ad words or phrases that have strong semantic relationships to a given context, and yet are still cost effective in that their calculated value exceeds the costs of purchasing the ad keywords, are bid for and bought. The system and method may be adapted to automatically purchase advertisements associated with ad words or phrases when they fall within a desired price range based on their calculated value. As the prices of advertisements associated with these words or phrases fluctuate over time based on their popularity, the automated bidding and buying of advertisements may be used to purchase advertisements associated with words or phrases at a price desirable to a given ad purchaser. By automatically purchasing such advertisements, the return on investment for an advertiser may be improved.

The application discloses a kind of multi-modal video sequence segmentation methods based on multi-scale codec, steps include: extracting image features and text features;Obtain the joint feature representation containing image and language semantic information simultaneously;Extract multi-scale fusion feature sequence under different spatial resolutions;Obtain the feature representation sequence after space-time modeling;Obtain the cross-scale fusion feature representation in unified semantic space;Get foreground features;Visualize segmentation mask to generate semantic segmentation map.The multi-modal video sequence segmentation method can realize the deep interaction of image and language semantics, establish the context and semantic relationship, introduce cross-modal information interaction through multi-modal collaborative mechanism, enhance the robustness and stability of the model in complex dynamic scene, effectively improve the segmentation effect and generalization ability of image sequence segmentation model in segmentation task.

A system and method for an artificially intelligent (AI) platform comprising a Knowledge Net (KN) Platform for automatically creating accessible, transparent knowledge stores which reflect a complete spectrum of semantic relationships between concepts. The system comprises a knowledge discovery engine that creates a machine representation of the semantic knowledge graph of a concept, namely a Concept Semantic Knowledge Graph (CSKG), for explainable, human-analogous understanding of natural language data for the full range of automated natural language understanding tasks—wherein the knowledge discovery engine, in discovering the related concepts, uses a variety of linguistic learning methods.

The application discloses a mineral prediction method based on spatial intelligence, relates to the technical field of spatial intelligence, and comprises the following steps: identifying the spatial relationship of a spatial entity element list, constructing a topological connected graph, extracting connected structure attributes, executing semantic relationship reasoning, and generating an object-level spatial intelligent scene graph; based on the object-level spatial intelligent scene graph, constructing an anisotropic cost field, calculating a minimum cost channel distance, mapping into a channel weight expression, and generating a channel weight layer; performing spatial alignment and multi-modal spatial correlation feature fusion on a multi-source mineralization evidence data set, the object-level spatial intelligent scene graph and the channel weight layer, executing mineral inference through a mineral prediction machine learning model, and generating a mineral prediction layer. The application realizes dynamic modeling of a mineralization path, and improves prediction accuracy and engineering practicability.

The invention discloses a text data structured processing method and system based on semantic recognition, and relates to the technical field of natural language process.The method comprises the steps that a predetermined recognition extraction strategy is introduced to process standardized original text data, text key information is obtained and analyzed to obtain a first relation triple, and a second relation triple is obtained; after a triple set is established, a target entity matrix is obtained through graph embedding analysis, after clustering processing, initial semantic features are adjusted in combination with a domain dictionary library so as to obtain target semantic features, and finally, a structured result of the target semantic features serves as a structured result of original text data. According to the method, the technical problems of incomplete key information extraction and inaccurate structured processing result caused by difficulty in accurately capturing deep semantic association of the text due to ineffective use of a semantic recognition technology in a traditional text data processing method are solved, semantic-level structured processing of the text data is achieved, and the text data processing efficiency is improved. And the key information extraction integrity and the structured result accuracy are improved.

The invention relates to the technical field of semantic comprehension, in particular to a water and soil conservation monitoring summary report auxiliary auditing system based on semantic comprehension, which comprises a graph and certificate judgment module, a phrase screening module, an entry matching module, a data error correction module and a revision and collection module. According to the method, the corresponding relation between image components and text main words is extracted, semantic deviation between image-text contents is recognized, a verb group and a time word are combined to carry out sequential relation comparison, the logical sequence in a statement is analyzed to be associated with actions, construction terms are extracted and are associated with standard terms, and phrase contents lacking the orientation of the terms are screened; the method comprises the following steps of: analyzing a connection state of a data segment and text description on a field level by combining a semantic relationship between a field subject and a monitoring unit, constructing a multi-dimensional check chain covering image-text association, a time sequence, term classification and data matching, and promoting collaborative application of semantic processing in structure association, rule matching and logic recognition; and the depth and breadth of text review are expanded.

This invention relates to the field of privacy protection technology, specifically to a method and system for protecting user privacy data based on data security. The method utilizes phrase association analysis of candidate keywords in text to identify core sentences in the user's text; it then filters keyword groups based on their sensitivity within the core sentences to determine noisy placement, and divides text units by combining different semantic relationships between keyword groups; finally, it analyzes the confidentiality requirements of text units based on lexical sensitivity, and clusters text units based on their similarity, adding noise based on the confidentiality requirements of different clusters to obtain noisy text data. This invention combines fine-grained semantic association analysis between words in the text, adaptively adding noise to sensitive parts with varying risks while preserving basic semantics, thus protecting user privacy while ensuring the accuracy and reliability of subsequent data analysis.

The invention provides an application function recommendation method and device, and the method comprises the steps: constructing a text feature vector and a behavior feature vector based on application program data, and carrying out the fusion to obtain a fusion feature vector; determining an embedded vector corresponding to the fusion feature vector by using a heterogeneous graph information network model; inputting the embedded vector into a prediction model, and outputting prediction scores of the multiple functions; according to the prediction scores, the multiple functions are arranged in a descending order, and a preset number of functions with the highest ranking are selected as recommendation results to be output. By constructing a heterogeneous graph information network model with multiple types of nodes and multiple semantic edges and combining multivariate path mining, the accuracy, interpretability and generalization ability of processing a complex semantic relationship between a user and a function are improved. By capturing the attention weights of different relationships and enhancing the distinction degree of feature expression, the method can accurately model user requirements under the condition of data sparsity, and improves the robustness and performance of a recommendation system, thereby meeting the multi-dimensional requirements of different users.

This invention relates to the field of distributed computing technology, specifically to a method and system for multi-agent collaborative task allocation based on a large model. It addresses the technical problem of existing methods struggling to effectively assess and mitigate the risk of blocking propagation along task chains in scenarios with complex task dependencies and semantic relationships, leading to low system resource utilization. The method includes: constructing a semantic blocking risk index based on the semantic blocking strength and semantic transmission capability of all preceding tasks of the task to be allocated, the exclusivity index of candidate agents, and the real-time load data of candidate agents; adjusting the observation window length of the time-series prediction model based on the semantic blocking risk index; predicting the future load and blocking trend of each agent using the adjusted time-series prediction model; and allocating tasks based on the prediction results and the semantic blocking risk index. This invention is applicable to multi-agent collaborative task allocation scenarios.

This invention discloses a method and system for semantic analysis of underwater swimming movements based on multi-agent collaboration, belonging to the field of multi-agent collaboration technology. The method includes identifying and analyzing underwater swimming video datasets based on a visual semantic generation agent model, recognizing natural language commands based on a planning agent, coordinating and interacting with a large language model and multiple agents, outputting kinematic parameter analysis results and personalized training suggestions, and generating an interpretable evaluation report. In this invention, the visual semantic generation agent model is trained using non-uniform soft-label knowledge distillation, so that the output visual semantic vector retains fine-grained inter-class semantic relationships while eliminating redundant information. Furthermore, the multi-agent collaborative architecture, through task orchestration of the planning agent, links structured data retrieval, professional knowledge enhancement, and credibility verification agents, realizing full-process automation from raw video to kinematic parameter analysis and personalized training suggestion generation.

The invention discloses a data-driven agile project management system, and relates to the technical field of intelligent project management, and the system comprises a project management center which is in communication connection with the following modules: a semantic graph construction module which dynamically captures the semantic association of a project task and an activity based on a knowledge graph and natural language processing technology, and stores the semantic association of the project task and the activity; and constructing a project semantic knowledge graph and continuously updating a semantic network between tasks. Through semantic graph construction and a dynamic mapping mechanism, semantic association between a traditional waterfall model and an agile model task can be automatically identified, limitation of traditional static keyword matching is overcome, context changes in task description are captured in real time based on a knowledge graph and a natural language processing technology, and the task description efficiency is improved. The semantic network is updated through incremental learning, the accuracy and adaptability of the mapping relation are ensured, project management can be seamlessly connected in a mixed development mode, and task traceability and management coherence are improved.

This application discloses a multimodal large-scale jailbreak risk detection and defense method and an electronic device. The method acquires original malicious commands and constructs visual and textual embedding features based on these commands, fusing them to form multimodal embedding fusion features. Based on these features, it generates a model response to the original malicious commands and a jailbreak attack evaluation type for the model response. If a successful jailbreak attack is determined, the corresponding attack text and image are used as the jailbreak attack scheme, and then the model jailbreak risk detection and defense processing is performed based on this scheme. This application, through multimodal embedding fusion features, deeply explores the semantic relationships and feature essence of test commands, extracts the attack text and image combinations of successful jailbreak scenarios, and constructs a jailbreak attack scheme that combines concealment and attack. Based on this, targeted model jailbreak risk detection and defense processing is carried out, improving the model's ability to identify and defend against jailbreak attacks in multimodal scenarios.