865results about "Metadata video data retrieval" patented technology

The invention relates to the technical field of disaster monitoring, in particular to a mine inrush water image processing system based on AI recognition, which comprises a fissure water flow tracking module, a voiceprint disturbance extraction module, an acceleration mapping module, a form analysis module and a risk output module. According to the method, a fine voiceprint perturbation sequence is formed by extracting crack water flow boundary pixel coordinates from continuous frame images, accurately capturing water flow micro-displacement and generating a deviation track in combination with an inter-frame difference between a gray gradient amplitude and a direction angle, and synchronously extracting a liquid level perturbation wave crest position and a voiceprint primary and secondary frequency amplitude difference; the acceleration is calculated according to the pixel displacement speed and mapped to a time axis, so that the boundary movement trend and morphological change are synchronously analyzed in the time dimension, risk frame locking is completed by multi-source signals in the same time period under double-threshold judgment, and three-dimensional presentation of the water flow dynamic state is achieved. The accuracy and reliability of sudden water gushing risk judgment in a complex mine environment are remarkably improved, and the probability of missing detection and misjudgment is reduced.

The invention discloses a universal scene retrieval analysis method and system based on multi-modal feature fusion, the method comprises a video analysis step and an application service step, and the application service step comprises the steps of receiving a user input request, describing a multi-dimensional standardized video tag based on a video summary, and obtaining a multi-dimensional standardized video tag; the steps of cross-modal video retrieval, dynamic knowledge enhancement question answering and interactive enhancement analysis can be executed, efficient video preprocessing is achieved by constructing an offline feature library, the retrieval precision is improved by adopting a cross-modal feature fusion technology, the analysis authority is enhanced in combination with a dynamic knowledge base, and the interactive enhancement analysis is supported to achieve abnormal early warning. The method has the advantages that the offline video processing efficiency is improved, cross-modal feature fusion retrieval is realized, and the authority of an analysis result is enhanced.

The invention relates to the technical field of video content understanding, in particular to a video content understanding method of a multi-mode advertisement inventory intelligent matching system, and the method comprises the steps: taking a video frame of an original advertisement video as a benchmark, and in a shot where a timestamp of each video frame is located, obtaining a video frame of the original advertisement video; analyzing the similarity of adjacent video frames and the dispersion degree of feature matrix elements corresponding to the voice segments, and determining a first search window; meanwhile, analyzing the occurrence frequency and duration of the text fragment in unit time, and confirming a second search window; based on the search window, analyzing the feature similarity degree of the voice segment and the text segment with the video frame in the corresponding window, and obtaining an optimal alignment path; and aligning the timestamps of the voice segments and the text segments with the timestamps of the video frames, and realizing multi-level semantic analysis through a multi-modal fusion technology and a deep learning model. The invention aims to synchronize the sequential relationship among the audio, the text and the video frame and prompt the accuracy of video content understanding.

The invention relates to the field of movies, and discloses a movie recommendation method and system based on big data, and the method comprises the steps: collecting the multi-dimensional behavior data of a user to form an initial behavior data set; based on the initial behavior data set, calculating preference scores of the user for different categories of short play contents to obtain a user interest distribution matrix; dividing user groups and labeling user category labels; forming an initial short episode content feature set; constructing a content semantic feature matrix; if the content semantic feature matrix and the user category label have the matching requirement, judging the integrating degree of the short episode content and the user preference, and outputting a preliminarily recommended short episode content list; and on the basis of the preliminarily recommended episode content list, sorting and optimizing the recommended episode content by using click rate and complete playing rate indexes in the historical distribution data to obtain a final recommended episode content list. The method has the following effect that the accuracy and the individuation degree of short play recommendation are improved.

The invention belongs to the technical field of medical treatment and public health, and particularly relates to a hypertension group knowledge recommendation method and system based on a large language model multi-agent. Hypertension knowledge question-answer pair information, medical short video information and user personal health portraits are finely extracted by constructing a hypertension knowledge question-answer pair library; according to the method, video content quality evaluation and matching degree evaluation are performed, and user platform interaction data adjustment is combined, so that personalized and accurate knowledge recommendation services can be provided for hypertension users, the requirements of hypertension patients for health knowledge are met, the accuracy and effectiveness of knowledge recommendation are improved, and self-health management of the patients is facilitated.

The invention relates to a weak supervision online video moment positioning method and system based on memory perception, and belongs to the technical field of artificial intelligence, and the method comprises the steps: carrying out the multi-modal feature fusion of a given video and a text query thereof, and obtaining the unified frame level representation at each stage; inputting the fused features into an offline module and an online module in an integral and frame-by-frame manner by using an offline guide online model architecture; in the off-line module, generating a Gaussian mask to reconstruct query of a covered part of words, and obtaining a proposal of an action starting moment; in the on-line module, the long-term historical memory in the window is used for enhancing the score, the attention weight of the score in the window is dynamically generated, and the score of the current frame is calculated in a weighted mode; taking the proposal obtained by the offline module as a pseudo tag, and providing supervision information for the score sequence of the online module; and high-performance weak supervision on-line moment positioning can be completed only by independently deducing the on-line module. The expansion capability and the application value of the model are remarkably improved.

The invention relates to the technical field of data processing, and discloses a film and television supervision analysis method and system based on data mining and a storage medium. The method comprises the following steps: collecting film and television data in a panoramic manner, and carrying out cross-media transcoding processing to obtain a multi-dimensional data set; performing video semantic extraction, audio emotion recognition and text tendency analysis on the data set to generate a time sequence mark list; scoring the key elements according to social influence, propagation depth and audience acceptability to form a dynamic score table; analyzing the propagation path based on the score table, and constructing an influence map; sorting supervision points according to the atlas, and making an intelligent supervision scheme; the execution deviation is analyzed through effect feedback, and the supervision parameters are optimized. According to the method, multi-dimensional content feature extraction, social influence evaluation and propagation path analysis of the film and television works are realized in a mass multimedia content environment, a precise and differentiated intelligent supervision scheme is formed, and the method has self-optimization and adjustment capabilities at the same time.

The invention discloses a film and television content label processing method and terminal based on a large model, and belongs to the technical field of artificial intelligence and film and television content processing, and the method comprises the steps: pre-integrating to form a multi-source film and television data set; performing fine adjustment on a plurality of specified artificial intelligence large models in advance; pre-deploying a task automatic triggering module and constructing a cue word engineering library; acquiring input film and television content, automatically selecting unrechecked content for the film and television content to trigger a task, calling corresponding cue words through the cue word engineering library, and respectively outputting error correction results of film and television content labels through each specified artificial intelligence large model after fine tuning; judging and correcting the error correction result of the output film and television content label by adopting a preset artificial intelligence model, and mapping the correction result to a label library; and after AI auditing is carried out, the identification recheck authentication state of the film and television content is updated. The accuracy, efficiency and diversity of film and television label processing are improved, and the requirements of scenes such as film and television content management, recommendation and analysis are met.

The invention discloses an information retrieval method, a related device, equipment and a storage medium, and is applied to the field of artificial intelligence. The method comprises the steps of obtaining to-be-queried information and target instruction information; based on the to-be-queried information and the target instruction information, obtaining a first token sequence through a feature extraction layer included in an information retrieval model; performing bidirectional attention coding on the first token sequence through a backbone network included in the information retrieval model to obtain a second token sequence; pooling the second token sequence to obtain a target embedded vector; and according to the target embedding vector, obtaining a preset number of candidate information with the highest similarity as a retrieval result. According to the method, the complexity and the calculation cost of model deployment are reduced, and the target embedding vector generated by the model is more discriminative, so that the accuracy of a retrieval result is improved.

The invention relates to the field of information retrieval, and discloses a cross-modal retrieval method for a text video. In the training stage, candidate pairs are input in batches, and the similarity of the candidate pairs is calculated respectively; then, taking the similarity of each candidate pair as a classification probability, and based on Dirichlet distribution, carrying out modeling on the uncertainty of the classification probability; and taking the probability of the classification probability of the candidate pair corresponding to the sample pair as 100% and the probability of other candidate pairs as 0%, constructing a label, and completing the training of the model. In the retrieval stage, the candidate pairs are input in batches, uncertainty modeling is aligned while the similarity is obtained, the similarity is corrected through uncertainty, and matching is conducted through the corrected similarity. Therefore, the method provided by the invention can quantify the uncertainty in the text video pair so as to carry out more reliable similarity evaluation, effectively improve the retrieval performance, and solve the reliability problem caused by only dependence on similarity in the prior art.

The invention discloses a media information processing method and device, a storage medium and electronic equipment. The method comprises the steps that to-be-retrieved source media information and prompt template information are obtained, the source media information and the prompt template information are input into a target retrieval model, target media information is determined, and the target retrieval model represents a model obtained through joint training according to at least two retrieval task types; a loss function used in the training process of the target retrieval model is a joint loss function, the joint loss function is determined by at least two sub-loss functions in one-to-one correspondence with the at least two retrieval task types, and the target media information represents media information obtained by retrieving the source media information according to the prompt template information. According to the method and the device, the technical problem of relatively low retrieval efficiency of the media information caused by single multi-modal retrieval task is solved.

The embodiment of the invention provides an information processing method and device, and the method comprises the steps: collecting hotspot information related to a target service, and obtaining a hotspot video related to the hotspot information and a reference video copywriting of the hotspot video through a target interface; inputting the hotspot information, the hotspot video and the reference video copywriting into a large language model for processing to obtain a target video copywriting matched with a target platform content publishing style; generating a target audio and a target image corresponding to the target video copywriting, and constructing a target video according to the target audio and the target image; and publishing the target video on the target platform according to a preset publishing strategy.

A method for performing a video content search is provided. The method includes receiving a search query associated with one or more videos and obtaining subtitle files associated with the one or more videos. The subtitle files each include a corresponding transcript having a plurality of transcript passages and video timestamps for the plurality of transcript passages. The method further includes determining query responses based on contents of video segments of the one or more videos. The video segments correspond to passage time ranges relating to the search query. The passage time ranges each include video timestamps associated with a transcript passage in the subtitle files that correlates with the search query. The method further includes causing a query result to be presented to a user on a graphical user interface, the query result being based on the determined query responses.

The invention provides a video content on-demand method and system applied to school education, and relates to the technical field of school education streaming media. Fusing student personalized attributes and behavior features to generate student feature representations, and predicting applicability scores in combination with video tags; grouping the student feature data and calculating a group feature vector; fusing the personalized prediction score and the group feature vector to generate a recommendation list; and time sequence preference representation is constructed based on student watching history, and a recommendation strategy is dynamically optimized through an adaptive scoring and calibration mechanism. According to the invention, accurate personalized recommendation of the education video is realized, and the learning effect and the resource utilization rate are improved.

The invention discloses an AI-based creative thinking auxiliary generation method and system, relates to the technical field of artificial intelligence, and solves the problem of inaccurate sound and picture matching in a traditional method by performing timestamp alignment and feature extraction on audio data and a visual image frame and calculating the correlation between the audio data and the visual image frame by using a cross-modal attention mechanism. According to the method, user eye movement track data is introduced, real attention points of a user are mapped into a visual sequence, and an optimized weight matrix is generated by constructing attention masks and fusing model attention weights, so that a generation result is more in line with perception key points of the user. And meanwhile, a feedback mechanism is established based on the synchronization error score, and when the sound and the picture are detected to be asynchronous, the visual frame timestamp can be dynamically adjusted, so that the self-adaptive correction of the content is realized. On the whole, the method has remarkable advantages in the aspects of improving modal alignment precision, enhancing user perception consistency and optimizing generation result naturalness.

The invention relates to the technical field of data processing and information, and discloses a video file film and television information matching method and device, a terminal and a storage medium, and the method comprises the steps: obtaining the video file information of a video file in a target storage device; inputting the video file information into a pre-trained target recognition model to obtain a recognition result output by the target recognition model; when the recognition result is that a target keyword is recognized, querying from a plurality of media resource libraries according to the target keyword to obtain a fused media resource data set; metadata corresponding to the video file is extracted from the target storage device; and sequentially performing weight matching calculation on each piece of fusion media resource data in the fusion media resource data set and the metadata to obtain optimal fusion media resource data corresponding to the video file, and storing the optimal fusion media resource data in a database. And the matching accuracy of the film and television information of the video file is improved.

The invention belongs to the technical field of automatic driving test, and particularly relates to a multi-mode accident scene library construction method for end-to-end automatic driving test. The method specifically comprises the following steps: step 1, extracting text information based on an MCAT-BiLSTM-CRF algorithm; 2, designing the ontology architecture, and storing accident scene information by using a knowledge graph to obtain an accident text knowledge base; step 3, derivative expansion is carried out on the accident scene element combination based on a HyCon-Sg-Net algorithm; step 4, performing enhanced fine tuning on the Open-Sora model to realize modal conversion from the text knowledge base to the accident video database so as to construct a multi-modal accident scene library; the method can be used for testing the performance of an end-to-end automatic driving automobile in an extreme accident scene, and the adaptive capacity of an automatic driving algorithm to the extreme accident scene is remarkably improved.

The invention discloses a video retrieval method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring each current to-be-retrieved video; dividing each video to be retrieved into a plurality of split videos; for each target retrieval dimension, obtaining a text description of the target retrieval dimension of each split video; wherein the target retrieval dimension comprises a subtitle dimension, an audio dimension and a visual dimension; the text description of the visual dimension of one split video is combined by the context text description of the split video and the text description of each key frame of the split video; combining the text description of the target retrieval dimension of each split video with a retrieval problem, and performing retrieval enhancement to obtain a plurality of retrieval videos based on the target retrieval dimension; weighting the text description of each target retrieval dimension of each retrieval video to obtain a plurality of sub-mirror multi-dimensional text descriptions; and combining the multi-dimensional text description of each sub-mirror with a retrieval problem to carry out retrieval enhancement to obtain a final retrieval result.

Disclosed herein are systems and method for curating and distributing digital content; including: digital video, music, pictures, etc. For example, presented herein are systems and methods for providing a digital content program, including a plurality of continuously provided digital assets streamed over a digital content platform. Example embodiments include: (a) curating a plurality of assets; (b) selecting a subset of assets from the plurality of assets, based on similarity metrics between assets; and (c) ordering the subset of assets into a digital content program.

The invention provides a video recommendation method and system based on an AB experiment, electronic equipment and a storage medium. The method comprises the steps of obtaining user data and experiment data of a user based on an online recommendation request sent by the user; loading each pre-trained shunting model, pre-estimating an estimated value of each experimental index according to the user data and the experimental data through the shunting model, and determining a user preference score of a user for a video recommendation result of the shunting model according to the estimated value of each experimental index; for each shunting model, counting the selection times of the video recommendation result of the shunting model, and determining the model winning rate of the shunting model according to the selection times; calculating a video recommendation score of each shunting model according to the user preference score and the model winning rate of each shunting model, and determining a shunting proportion among the shunting models according to the video recommendation score of each shunting model; and displaying the current video recommendation result of each shunting model to the user according to the shunting proportion.

The invention provides a public opinion video tag aggregation method and system based on artificial intelligence, and relates to the technical field of artificial intelligence. Comprising the following steps: acquiring pictures and text information in a short video, and performing semantic alignment; different large language models are adopted to generate preliminary labels for the pictures and the text information after semantic alignment; clustering the pictures and the texts after semantic alignment to obtain clusters; calculating the labeling probability of each primary label type in the current cluster by each large language model, and selecting the primary label with the highest probability sum as a clustering label of the current cluster; calculating the reliability weight of each large language model in the current cluster based on the clustering label of the current cluster; and based on the reliability weight, calculating the weighted support degree of all the large language models to different preliminary label types of each piece of data in the current cluster, calculating the weighted label of the current data, and further determining a final label. According to the method, the condition of few labels or no labels can be effectively processed, and the manual workload is greatly reduced.

A video tag processing method, comprising: determining a video to be processed, and performing extraction on the video to obtain modality information of at least two modalities (202); on the basis of the modality information of the at least two modalities, respectively performing cross retrieval according to feature dimensions of the at least two modalities, so as to obtain respective multi-modal retrieval tags of the modality information of the at least two modalities (204); on the basis of the respective multi-modal retrieval tags of the modality information of the at least two modalities, determining at least one candidate tag for the video (206); and on the basis of the at least one candidate tag and the modality information of the at least two modalities, performing tag prediction, so as to obtain a video tag for the video (208).

The invention discloses a video file management system and method based on a block chain network. The method comprises the following steps: a submitter sends an identity authentication request message to a block chain network, then the block chain network performs identity authentication on the submitter according to the identity authentication request message, and after the authentication is passed, the submitter sends the identity authentication request message to the submitter; sending a proposal constructed based on the identification information of the video file in the request message, the encrypted video fingerprint data and the fingerprint generation timestamp to other participants; then other participants can call the smart contract to carry out standardization processing on the proposal to obtain a standardization processing result, and the standardization processing result is fed back to the submitting party; and finally, the submitting party decrypts each standardized processing result to obtain corresponding standardized video fingerprint data, and records the standardized video fingerprint data to the target block chain when the data meets a pre-standard. According to the method and the device, the technical problem of relatively poor security and privacy of video file storage in a related centralized storage technology is solved.

Self-learning systems process data in real-time and output the processed data to client applications in an effective manner. They comprise a capture platform that captures data and generates a stream of text, a text decoding server that extracts individual words from the stream of text, an entity extractor that identifies entities, a trending engine that outputs trending results, and a live queue broker that filters the trending results. The self-learning systems provide more efficient realization of Boxfish technologies, and provide or work in conjunction with real-time processing, storage, indexing, and delivery of segmented video. Furthermore, the self-learning systems efficiently perform entity relationing by creating entity network graphs, and are operable to identify advertisements from the data.

This disclosure describes automatically identifying videos of a video dataset having high-content similarity and grouping the videos together with one or more possible tags for the videos for consideration by annotators. A labeling service of a service provider network receives (i) a dataset of unlabeled videos and (ii) category tags for labeling the videos from a client device associated with a user. A video data modeling engine analyzes the dataset to provide a set of ranked unlabeled videos from the dataset, wherein the set of ranked unlabeled videos are ranked according to confidence scores with respect to at least one tag. Based at least in part on the confidence scores, at least some of the unlabeled videos are labeled with the at least one tag to provide a dataset of labeled videos. The dataset of labeled videos is provided to a database and / or the user.

The invention discloses a short video retrieval method combining a pre-training model and a dependency syntax tree, and relates to the technical field of multi-modal learning, computer vision and information retrieval cross fusion. The semantic information in the context is captured by using the advantages of BERT, and the deep semantic representation is generated, so that the meaning of the sentence is more accurately understood. A clear syntactic relationship is provided by using the dependency syntactic tree, and the structured dependency relationship between words can be revealed. The relationship can help the model to better understand logic semantics in sentences and extract text features related to tasks in a more targeted manner. Through the dependency syntax tree and the model, primary and secondary information in the long sentences can be distinguished more accurately, the problems of word order ambiguity and the like are eliminated, feature extraction is more stable, and robustness is improved. The dependency syntactic tree is explicit grammatical representation, and structured information of the dependency syntactic tree can assist in analyzing BERT implicit features, so that more explainable basis is provided for the decision process of the model, and the interpretability is remarkably enhanced.

The invention discloses a grid-coding-based video space-time retrieval method and device, the grid-coding-based video space-time retrieval method is executed in computing equipment, and the method comprises the following steps: mapping a coordinate range of a target area into a corresponding grid code set according to a Beidou grid standard on the basis of the coordinate range of the target area; acquiring video data with an intersection or inclusion relationship between the spatial range of the shot object and the target area; combining the shooting time of any video data with each grid code to generate a space-time grid code set; constructing an index structure corresponding to the video data by taking each time-space grid code in the time-space grid code set as a main key; and in response to query information of the user, analyzing a space-time range in the query of the user, mapping the space-time range into a corresponding space-time grid code range, executing retrieval based on the range, and returning hit video data. According to the method, rapid positioning and accurate retrieval of large-scale aerial photography video data are realized.

According to the automatic driving questioning and answering method and device based on the knowledge graph and the multi-modal large model, the method comprises the steps of firstly responding to a questioning and answering triggering condition, obtaining a real-time road condition video and a traffic safety problem, further obtaining a multi-modal query problem, then performing analysis processing on the multi-modal query problem based on a visual language model, and finally obtaining an automatic driving questioning and answering result. According to the method, visual semantic keywords are obtained, a candidate entity node set related to the visual semantic keywords is determined from a plurality of entity nodes of the knowledge graph, core knowledge nodes related to query questions are accurately focused, the invalid search range is reduced, question retrieval and answer generation efficiency is improved, and user experience is improved. The comprehensive similarity between each candidate node in the candidate entity node set and the visual semantic keyword is calculated, the key entity node set is obtained, the accuracy of question retrieval and answer generation is improved, and finally the question and answer result corresponding to the traffic safety question is generated according to each key entity node in the key entity node set.

The invention discloses a short video promotion optimization method and system, relates to the technical field of short video promotion optimization, and aims to solve the problems that when an existing short video promotion method is used for processing subtle elements which may cause different understandings of users in video contents, intentions of merchants are difficult to accurately convey, target users are difficult to reach, and the user experience is poor. The method comprises the steps of obtaining content data of a to-be-promoted video, performing multi-meaning element identification on the content data to determine at least one potential multi-meaning element in the to-be-promoted video, and determining a situational misreading risk assessment result of the potential multi-meaning element according to the potential multi-meaning element and a preset user situation model, and according to the situational misreading risk assessment result, a pre-stored promotion constraint condition and a content modification cost model, making a decision between the content modification strategy and the audience screening strategy to generate an initial intervention strategy, executing the initial intervention strategy, and optimizing the promotion process based on user feedback data obtained after execution.