303results about "Multimedia data clustering/classification" patented technology

The invention relates to the technical field of information retrieval and generation, in particular to a retrieval enhancement method based on multi-modal data fusion and modal perception. According to the method, firstly, a dual-channel architecture is adopted to perform feature extraction and coding on a text and an image respectively, and mutually independent embedded representation spaces are constructed, so that high-quality collaboration and matching of cross-modal representation are realized; and a pseudo-pairing generation mechanism is introduced to effectively mine and reconstruct the existing non-paired data in the knowledge base. And designing a query modal perception and dynamic weighting mechanism for accurately controlling the fusion proportion of the image-text bimodal information in the retrieval stage so as to match the modal demand difference of different query contents. And further executing aggregation retrieval and reordering of the cross-modal information by using dynamic weighted fusion retrieval to generate a candidate set of multi-modal responses. According to the method, accurate matching and dynamic weight adjustment of the image-text content are realized, and the accuracy and expression integrity of the generated content are improved.

The invention discloses a personalized teaching content generation method and system based on a digital portrait, and the method comprises the steps: collecting the multi-dimensional learning data of a student in real time, analyzing the data keyword of the student, constructing the portrait of the student, generating a teaching target based on the portrait of the student, and enabling the teaching target to comprise a teaching theme, exercises related to the theme, and knowledge points related to the theme. Converting the teaching target into an executable instruction of a large language model through a structured prompt engineering technology; the large language model generates initial teaching content according to the teaching instruction; verifying and optimizing the generated initial teaching content to ensure that the generated teaching content conforms to a teaching target and can adapt to the current cognitive level and learning requirements of students; and outputting the verified and optimized teaching content to the students and automatically adapting to the content presentation form according to the learning style preference of the students.

The invention discloses a conference content intelligent generation processing method and system based on a multi-modal large model, and the method comprises the steps: collecting the original data of a conference, and completing the standardization preprocessing; inputting a multi-modal large model, extracting multi-modal features and carrying out semantic alignment; executing cross-modal hash coding, generating binary codes and establishing an index database; performing hash retrieval on the related fragments, and constructing a conference content directed graph; based on a conference content directed graph structure, searching an optimized path by adopting a Monte Carlo tree; and generating structured conference content, and outputting a summary, an abstract and an action item. According to the method, efficient extraction, accurate retrieval and structured intelligent generation of the conference content are realized by fusing a multi-modal large model, cross-modal Hash coding and Monte Carlo tree search.

The invention discloses a cross-platform content generation and distribution method based on a multi-modal AI, and belongs to the technical field of cross-platform content generation and distribution, and the method comprises the steps: carrying out the content analysis and feature extraction of an original material based on a multi-modal AI model, and generating a structured content label and a semantic vector. By combining a vector matching degree formula of target portrait features and platform features, an adaptation strategy is dynamically generated, it is ensured that content not only conforms to platform rules, but also can accurately reach a target group, the conversion rate and user viscosity are finally improved, visual, text and semantic vectors are aligned through a Transform multi-modal fusion model, a cross-modal joint representation vector is generated, and the user experience is improved. And in combination with an adversarial generative network, differentiated variants of the same theme are generated in batches, and a content diversity score mechanism ensures that generated contents are balanced between creativity and compliance.

The invention provides an image video retrieval method based on a domain fine-tuning large language model, which comprises the following steps: performing fine-tuning on a pre-training model to obtain a fine-tuning pre-training model for intention classification and keyword extraction; performing dynamic iteration screening on an optimal prompt template through Monte Carlo tree search in combination with a hidden Markov model (HMM); performing noise filtering on the keyword list, and predicting category labels of the filtered keywords through a conditional random field model to obtain a keyword enhancement set; combining with the user intention to generate a query condition, and obtaining a candidate resource set; and according to the similarity between the user query text and the candidate resource set, and in combination with the optimal prompt template, obtaining the resource path with the highest matching score between the user query and the candidate resource, and obtaining the retrieved image or video, so that the identification deviation possibly occurring when a general model processes proper nouns and terminologies can be effectively solved, and the user experience is improved. And the retrieval accuracy and response speed are improved, so that the retrieval accuracy and professional adaptability are improved.

The invention discloses a digital media content element accurate screening method based on artificial intelligence image recognition, and relates to the technical field of digital media, and the method comprises the steps: building a distributed capture network to form a dynamic content pool, and building a metadata index database; calling a multi-modal image perception engine to generate a double-layer characteristic spectrum containing dominant and recessive elements; constructing a distributed recognition model cluster based on federated learning; converting the user demand into a screening parameter set and generating a decision tree; screening and secondarily verifying an output result through a double-path matching mechanism; and constructing a reinforcement learning reward function based on user behaviors, and driving the model and the decision tree to co-evolve. Multi-source heterogeneous content full-dimension analysis is achieved, the recognition comprehensiveness and depth are improved, knowledge barriers and privacy risks are solved, the screening accuracy and flexibility are improved, the system is endowed with the continuous optimization capacity, and the method is suitable for efficient and accurate digital media content screening scenes.

The application discloses a kind of PDF intelligent retrieval and generation method and system based on RAG, by obtaining the document data of input, using the classification model established in advance to parse document data, extract text content and image content to form first data set;Using deep learning model to the image content in first data set carries out feature extraction, while the text content in first data set applies natural language processing technology to carry out semantic analysis, obtains multimodal feature set;According to multimodal feature set, application information integration algorithm is uniformly encoded and is handled to generate second data set, if detecting the integrity of fusion feature vector in second data set is lower than preset threshold value, then supplementary context semantic analysis fills in missing information;Using preset index construction mechanism to the clustering processing of fusion feature vector in second data set, generates the retrieval index library containing classification index structure.The application improves the accuracy and comprehensiveness of document retrieval.

The invention discloses a dynamic multimedia data hash retrieval method and system based on extensible increment, and relates to the technical field of multimedia data retrieval. The method comprises the following steps: acquiring dynamic multimedia data to be retrieved; a pre-trained Hash retrieval large model is constructed, the Hash retrieval large model is trained by taking a bit extensible Hash center as global supervision information and taking tag cosine similarity as local supervision information, and specifically, generalization feature representation of new multimedia data maintaining new and old class discrimination is obtained through forward propagation; constructing a linear mapping relationship between the generalization feature representation and the hash code, and introducing an auxiliary variable to continuously update the hash function without playback; and utilizing the trained Hash retrieval large model to generate a query Hash code for the dynamic multimedia data to be retrieved, and utilizing the query Hash code to retrieve. According to the method, low memory occupation, high updating efficiency and non-forgetting retrieval of the dynamic multimedia data stream in the open environment are realized.

A system includes a computing platform having processing hardware, and a memory storing software code and a machine learning (ML) model-based feature classifier. When executed, the software code receives media content including a first media component corresponding to a first media mode and a second media component corresponding to a second media mode, encodes the first media component using a first encoder to generate multiple first embedding vectors, and encodes the second media component using a second encoder to generate multiple second embedding vectors. The software code further combines the first embedding vectors and the second embedding vectors to provide an input data structure for a neural network mixer, process, using the neural network mixer, the input data structure to provide feature data corresponding to a feature of the media content, and predict, using the ML model-based feature classifier and the feature data, a classification of the feature.

The present invention relates to a method and apparatus to quantify unusual aerial phenomena via a network of citizen operated personal devices. A feature of the present invention is an app that runs on popular personal devices. A further feature of the invention is a server system in communication with said app. A further feature of the invention is a means by which a user having spotted a potential event can quickly engage the app to begin a data collection mode. A further feature of the invention is a data collection mode that simultaneously records data including but not limited to video, audio, 9-axis IMU data, GPS coordinates, and time. A further feature of the invention is a method of tagging the collected data with a cryptographic signature 800 to ensure integrity. A further feature of the invention is real-time app communication to a centralized server. A further feature of the invention is an alert to other users indicating something of interest is happening nearby. A further feature of the invention is the a means by which the server can distinguish interesting events from non-interesting events. A further feature of the inventions is a means by which the server can analyze the data to obtain scientifically useful quantitative information.

Provided are an animal monitoring system, an animal monitoring server, an animal monitoring method, and an animal monitoring program capable of watching a behavior of a monitored animal with high accuracy while reducing power consumption. An animal monitoring system 1 that monitors a monitored animal 6 includes: a motion detection unit 502 attached to the monitored animal 6 and configured to detect motion information of the monitored animal 6; a behavior estimation unit 212 configured to estimate a behavior of the monitored animal 6 on the basis of the motion information detected by the motion detection unit 502; and a power saving control unit 508 configured to start control of reducing power consumption in the motion detection unit 502 when the behavior estimation unit 212 estimates that the behavior of the monitored animal 6 is in an inactive state.

The application relates to the field of information technology, in particular to an intelligent processing method and system for new media data. The method comprises the following steps: reading historical new media material data; analyzing the historical new media material data, calculating an availability estimation value of the historical new media material, and sorting and screening the historical new media resource according to the availability estimation value; storing the analyzed historical new media material data in a classified manner, and establishing a multidimensional index of the historical new media material data; receiving a new media content generation demand, converting the new media content generation demand into a new media material matching condition; screening and sorting new media materials based on the new media material matching condition, and selecting the first N new media material data as basic new media material data; and fusing a theme content based on the basic new media material, generating multi-form new media content, and realizing intelligent processing of high-efficiency, high-quality and self-optimizable new media content.

The invention discloses a picture book interaction method and a picture book content artificial intelligence generation method and system.The picture book content artificial intelligence generation system comprises picture book application software and a server system, and the picture book application software comprises a picture book interaction AI engine front end SDK and a role playing mode program; the server system comprises a picture book interaction AI engine back-end service system, an interactive picture book media resource library system and a picture book content artificial intelligence generation system, and artificial intelligence processing of voice recognition is changed from a current television voice assistant to the picture book interaction AI engine front-end SDK and the picture book interaction AI engine back-end service system. Therefore, the picture book application software is independent of the artificial intelligence processing capability of the intelligent voice assistant of the existing television (set top box), not only is the adaptive work of the intelligent voice assistant of a television manufacturer reduced, but also the service data security protection of the picture book application software is realized.

The invention provides an intelligent civil aircraft separated emergency flight data storage system facing flight accident scenes. The system collects multi-source data such as flight parameters, audios, videos and data links, carries out preprocessing and semantic modeling on the multi-source data, and extracts multi-modal feature information; multi-modal features are fused based on a modal perception attention mechanism and a diagnosis feedback readjustment mechanism, and intelligent diagnosis of a current flight state is realized in combination with a semantic consistency discrimination mechanism. When the flight state is judged to be abnormal or accident, starting a satellite communication channel to carry out remote emergency transmission on key data; and when the flight state is judged to be normal, performing modular hierarchical storage on the acquired data according to the data type. And the separated emergency data transmission subsystem is immediately started under the extreme conditions of power failure of the main system and the like. According to the method, strategies such as multi-mode intelligent diagnosis and multi-level storage are fused, and the flight data acquisition efficiency and the flight safety guarantee capability are remarkably improved.

The invention is suitable for the technical field of online education, and provides an intelligent learning platform based on a multi-modal knowledge graph and a large language model, and the platform comprises a multi-modal knowledge graph construction and intelligent analysis engine which is used for carrying out the entity relation extraction, fusion and alignment of multi-modal education data, constructing a structured knowledge graph, and carrying out the fusion and alignment of the structured knowledge graph; fusing a large language model to provide an analysis service; the account management and authority control system is used for realizing user identity authentication and authority isolation based on an RBAC model and a JWT token; the student function module supports multi-mode homework submission, intelligent question answering, wrong question analysis and personalized question generation; the teacher function module is used for homework correction, teaching analysis and teacher-student interaction; and the parent function module is used for learning progress monitoring, report acquisition and home-school interaction. According to the platform, through multi-modal data fusion and knowledge graph driving, a large language model is introduced to realize accurate learning condition analysis, personalized learning path planning and family-school cooperation, and the intelligent level of the education process is improved.

The invention discloses a data development management processing method and device and electronic equipment. The data development management processing method comprises the following steps: taking a dimension correlation model structure definition as a center of communication construction management; a project space is used as a basic unit of management, data architecture is flattened and labeled, and data component content is abstracted into resources and managed in a unified mode; access limitation setting is performed on resources in a project space, preprocessing and post-calculation are performed on data, and multiple forms of data services which are safe, compliant, credible and controllable are provided. According to the technical scheme, a low-threshold, low-cost and easy-to-expand capability is provided for data development management processing, meanwhile, an effective method for simply sharing data is provided, and the data construction efficiency, the data circulation efficiency and the data use efficiency are improved.

The invention discloses a resume generation and optimization method and system based on multi-round natural language interaction, and belongs to the technical field of natural language processing, the resume generation and optimization method and system based on multi-round natural language interaction comprises the following specific steps: 1, receiving initial information input by a user through a natural language mode; the natural language mode comprises voice input, text input and multi-mode input containing images, and if the input is voice input, the input is converted into initial text information through an automatic voice recognition engine. Through multiple rounds of natural language interaction, the resume input and updating threshold is remarkably reduced, and a user can generate a first version of resume and match posts in real time only by short voices. The system actively excavates the potential advantages of the user, guides and complements key information, supports multi-modal input and dynamic updating, and effectively improves the resume quality, the matching efficiency and the user experience.

The invention provides a teaching courseware generation method and device, and relates to the technical fields of document processing, artificial intelligence, natural language processing and the like. According to the specific scheme, the method comprises the following steps: generating a teaching design general outline comprising a unit identifier of a teaching material unit and module information of at least two teaching modules based on teaching elements of the teaching material unit input by a user; based on the teaching design general outline, teaching resources are generated for each teaching module; based on the teaching design general outline, detecting whether the learning targets of the teaching resources among different teaching modules are consistent and / or whether the teaching resources of each teaching module deviate from respective teaching targets; in response to detection that the learning targets of the teaching resources are inconsistent and / or deviate from the teaching targets, regenerating the teaching resources of the corresponding teaching modules, and continuing detection until the learning targets of the teaching resources of different teaching modules are consistent and the teaching resources of all the teaching modules do not deviate from the respective teaching targets; and generating a target teaching courseware of the teaching material unit based on the teaching resources.

The invention provides a multimedia data processing method and device based on data classification and electronic equipment, and relates to the technical field of knowledge maps, and the method comprises the steps: carrying out the recommendation classification of multimedia data based on a knowledge map; performing multi-dimensional quantitative evaluation on the classification result, and determining a privacy fraudulent feeling numerical value of the classification result; recent interaction behaviors and long-term preferences of the user are fused, a comfort tolerance baseline is updated by introducing an attenuation factor, and a dynamic comfort threshold is adjusted in real time according to the comfort tolerance baseline; and only outputting a classification result of which the privacy fraudulent feeling value is lower than the comfort threshold, and forming a self-adaptive output closed loop capable of adapting to the psychological state drift of the user. By quantifying a privacy fraudulent feeling value and combining with a dynamically adjusted comfort threshold, the system effectively filters sensitive contents which may cause discomfort of the user, and ensures that the privacy boundary and psychological comfort of the user are fully respected while the recommendation result meets the interest demand.

The invention discloses a multi-modal corpus deduplication method and device based on an AI large model, equipment and a medium, and relates to the technical field of data processing, the method comprises the following steps: performing type identification on each pre-training corpus contained in an obtained pre-training corpus set, and determining a modal type; preprocessing each pre-training corpus based on the modal type to obtain a current pre-training corpus set; performing feature extraction on each current pre-training corpus based on the modal type to obtain a semantic vector and a structure vector corresponding to each current pre-training corpus; based on the semantic vector and the structure vector, determining a structure sensing semantic fingerprint corresponding to each current pre-training corpus; determining a redundant cluster based on the structure perception semantic fingerprint; determining redundant corpora from the redundant clusters; and deleting redundant corpora in the current pre-training corpus set to obtain a de-duplicated pre-training corpus set, and the multi-modal corpus de-duplication accuracy and efficiency based on the AI large model can be improved.

The invention discloses an image-text retrieval method and system based on multi-modal fusion and depth spectral clustering, and relates to the technical field of data retrieval, and the method comprises the steps: extracting image training features and text training features; splicing to obtain a global feature; constructing local image training features and local text training features; performing alignment processing on the local image training features and the local text training features; fusing the global features and the aligned local image training features and local text training features; performing clustering analysis on the normalized features after fusion feature normalization to obtain a plurality of clustering centers; and extracting query features of the query sample, and determining a retrieval result according to the cosine similarity. According to the method, the characterization capability and robustness of the model to the multi-modal data are improved through data-level fusion and local structure constraint, self-supervised alignment of the image and text modal data is realized through the positive sample pair and the negative sample pair, the clustering center of the image-text data is learned through depth spectral clustering, and new sample data are quickly adapted.

The invention relates to the technical field of computer data processing, and discloses an intelligent exhibition hall multi-modal interaction digital human system and an implementation method, which are used for solving the problem that multi-source data of multi-modal interaction lacks a unified clock and verifiable timestamp alignment mechanism in a traditional method. According to the method, a unified time domain and a time version are established on an edge side, terminal access is restrained, and an acquisition time mark, an access time mark and a serial number are written in data or a state; performing gating shunting according to a time version, performing de-duplication and out-of-order rearrangement based on a serial number and double time marks, and generating and solidifying a session window evidence index; locking a transaction window boundary according to the evidence index, generating a participation source list and a transaction number, establishing a fragment reference relationship and generating an alignment voucher; in the linkage stage, phase division issuing is carried out according to a preparation phase, an execution phase and a confirmation phase, backward reading verification is carried out according to an action sequence number, a transaction log is archived, and alignment, rechecking and playback of an interaction link are achieved.

The invention discloses a personalized English learning recommendation method and system based on a knowledge graph, and relates to the technical field of intelligent education and natural language processing, and the method comprises the steps: collecting the multi-source learning data of a learner; extracting four types of knowledge entities including vocabularies, grammar, topics and skills by adopting a sequence labeling model, and calculating entity association degree through an attention mechanism to construct a hierarchical knowledge graph; constructing a user ability model based on the learner data, and positioning weak knowledge entities and associated entities thereof to form a target knowledge entity set; screening matched contents from a resource library, analyzing and determining a knowledge logic sequence in combination with a knowledge graph path, and generating a personalized recommendation result; and evaluating the effect according to the learned data and dynamically updating the user capability model and the knowledge graph. According to the invention, structured organization of English knowledge, accurate description of learner ability and coherent recommendation of learning content are realized, and the personalized level and effect of English learning are effectively improved.

The invention discloses an automatic story generation method and system based on a user portrait, and relates to the technical field of user portrait modeling, and the method comprises the steps: collecting real-time interaction data of a user, generating a candidate keyword set in combination with an RAKE algorithm, calculating a VAD emotion vector, and generating an emotion resonance keyword set based on the candidate keyword set and the VAD emotion vector; generating an event sequence based on an emotional resonance keyword set, constructing an event graph by using the event sequence, optimizing the event sequence through a LaMDA model, calculating an edge weight, updating the event graph, obtaining probability distribution of the updated event graph through VGAE coding, and generating a plot skeleton in combination with a VAD emotional vector; and calculating a gating weight vector, calculating a dynamic user portrait vector based on the gating weight vector, calculating a loss function and updating an encoder-decoder model, and obtaining an optimized story text. The emotional consistency, the plot continuity and the individuation level of the generated stories are improved.

The invention belongs to the field of multimedia information intelligent processing and classification application, and relates to a multimedia field label information labeling method based on TSK rule migration. The method comprises three parts of label set division, rule migration modeling and multi-label classification. The method comprises the following steps: firstly, constructing a label separation mechanism according to label distribution characteristics of multimedia multi-label data, and dividing a label set into conventional labels and scarce labels; secondly, constructing a rule migration TSK fuzzy system; the system comprises a combined If-part and a migration-driven ten-part, not only can realize feature-label reasoning modeling, but also can establish relevance between conventional labels and scarce labels according to label distribution differences. And finally, on the basis of RT-TSK-FS, a multi-label classification method based on rule migration is provided, and sharing and supplement of knowledge among different labels are realized through a rule migration mechanism, so that the prediction performance of multimedia label information is improved.

The invention discloses a film series intelligent dynamic collection method and device based on a multi-modal large model, and belongs to the technical field of film and television content processing, and the method comprises the steps: extracting text information, audio information and visual information of film and television content; the text information is analyzed to extract key semantic features, the audio information is analyzed to extract emotion tone features, the visual information is analyzed to extract image hue, shot language and scene composition visual style features, and multi-modal feature vectors are obtained; inputting the multi-modal feature vectors into a pre-trained artificial intelligence large model, and inferring an association relationship between the film and television contents and a series episode category to which the film and television contents belong in combination with a preset series episode structure rule base; and carrying out dynamic mapping comparison on the inferred incidence relation between the film and television contents and the category of the series with the stock series, and storing a comparison result. The collection efficiency and accuracy are improved, the labor cost is reduced, and the experience of watching the series of the user is improved.

The invention relates to a method and system for generating a three-dimensional object based on semantic analysis. More specifically, the present invention relates to a method and system for automatically generating a three-dimensional object corresponding to the meaning of a sentence by analyzing the sentence in a text form.

Systems and methods for creating a fan profile are disclosed. One embodiment includes querying a user media database. A set of locations are generated based on location metadata associated with media included in the user media database. The embodiment may further include iterating through the user media database to create one or more clusters comprised of similar locations at which one or more media files stored in the user media database have been captured by the user, and filtering out any clusters of the clusters that are less than a threshold size. For each remaining cluster, a time range may be created. All remaining clusters and associated locations within the time range may be categorized and aggregated. All categorized and aggregated remaining clusters may be analyzed to infer demographic traits associated with the user, and a final fan profile associated with the user may be exported based on the analyzing.

The invention relates to an interactive training data generation method and device, vehicle-mounted equipment, a storage medium and a program product. According to the method, historical dialogue data of a vehicle user is obtained, a virtual user model corresponding to the vehicle user is constructed according to the historical dialogue data, and then the virtual user model is utilized to initiate multiple rounds of dialogues to an initial intelligent interaction model. And after each round of conversation, adjusting an emotion label of the virtual user model according to reply information of the initial intelligent interaction model, generating conversation content of the next round based on the adjusted emotion label, and then generating interactive training data according to conversation data and corresponding emotion labels in the multi-round conversation process, the interactive training data is used for training the initial intelligent interactive model. According to the method, massive interactive training data with emotion tags can be generated, and thus the intelligence of an intelligent interaction model trained through the interactive training data can be improved.

The embodiment of the invention discloses a multimedia data processing method and device, equipment and a storage medium, and is applied to the artificial intelligence technology, and the method comprises the steps: predicting a first public object feature of an object interested in sample multimedia data according to an original media feature through an initial media recommendation model, predicting a second public object feature of an object which is not interested in the sample multimedia data according to the original media feature; according to the first public object feature, the second public object feature and the original media feature, generating an interaction feature, and according to the original object feature and the interaction feature of the sample object, predicting the prediction interestingness of the sample object; and performing iterative training on the initial media recommendation model according to the labeled interestingness and the predicted interestingness of the sample object, and the original object feature, the first public object feature and the second public object feature of the sample object. According to the method, the training accuracy of the media recommendation model can be improved, so that the recommendation accuracy for the multimedia data is improved.