633results about "Metadata video data retrieval" patented technology

The invention relates to the technical field of disaster monitoring, in particular to a mine inrush water image processing system based on AI recognition, which comprises a fissure water flow tracking module, a voiceprint disturbance extraction module, an acceleration mapping module, a form analysis module and a risk output module. According to the method, a fine voiceprint perturbation sequence is formed by extracting crack water flow boundary pixel coordinates from continuous frame images, accurately capturing water flow micro-displacement and generating a deviation track in combination with an inter-frame difference between a gray gradient amplitude and a direction angle, and synchronously extracting a liquid level perturbation wave crest position and a voiceprint primary and secondary frequency amplitude difference; the acceleration is calculated according to the pixel displacement speed and mapped to a time axis, so that the boundary movement trend and morphological change are synchronously analyzed in the time dimension, risk frame locking is completed by multi-source signals in the same time period under double-threshold judgment, and three-dimensional presentation of the water flow dynamic state is achieved. The accuracy and reliability of sudden water gushing risk judgment in a complex mine environment are remarkably improved, and the probability of missing detection and misjudgment is reduced.

The invention discloses a universal scene retrieval analysis method and system based on multi-modal feature fusion, the method comprises a video analysis step and an application service step, and the application service step comprises the steps of receiving a user input request, describing a multi-dimensional standardized video tag based on a video summary, and obtaining a multi-dimensional standardized video tag; the steps of cross-modal video retrieval, dynamic knowledge enhancement question answering and interactive enhancement analysis can be executed, efficient video preprocessing is achieved by constructing an offline feature library, the retrieval precision is improved by adopting a cross-modal feature fusion technology, the analysis authority is enhanced in combination with a dynamic knowledge base, and the interactive enhancement analysis is supported to achieve abnormal early warning. The method has the advantages that the offline video processing efficiency is improved, cross-modal feature fusion retrieval is realized, and the authority of an analysis result is enhanced.

The invention relates to the field of movies, and discloses a movie recommendation method and system based on big data, and the method comprises the steps: collecting the multi-dimensional behavior data of a user to form an initial behavior data set; based on the initial behavior data set, calculating preference scores of the user for different categories of short play contents to obtain a user interest distribution matrix; dividing user groups and labeling user category labels; forming an initial short episode content feature set; constructing a content semantic feature matrix; if the content semantic feature matrix and the user category label have the matching requirement, judging the integrating degree of the short episode content and the user preference, and outputting a preliminarily recommended short episode content list; and on the basis of the preliminarily recommended episode content list, sorting and optimizing the recommended episode content by using click rate and complete playing rate indexes in the historical distribution data to obtain a final recommended episode content list. The method has the following effect that the accuracy and the individuation degree of short play recommendation are improved.

The invention belongs to the technical field of medical treatment and public health, and particularly relates to a hypertension group knowledge recommendation method and system based on a large language model multi-agent. Hypertension knowledge question-answer pair information, medical short video information and user personal health portraits are finely extracted by constructing a hypertension knowledge question-answer pair library; according to the method, video content quality evaluation and matching degree evaluation are performed, and user platform interaction data adjustment is combined, so that personalized and accurate knowledge recommendation services can be provided for hypertension users, the requirements of hypertension patients for health knowledge are met, the accuracy and effectiveness of knowledge recommendation are improved, and self-health management of the patients is facilitated.

The invention relates to a weak supervision online video moment positioning method and system based on memory perception, and belongs to the technical field of artificial intelligence, and the method comprises the steps: carrying out the multi-modal feature fusion of a given video and a text query thereof, and obtaining the unified frame level representation at each stage; inputting the fused features into an offline module and an online module in an integral and frame-by-frame manner by using an offline guide online model architecture; in the off-line module, generating a Gaussian mask to reconstruct query of a covered part of words, and obtaining a proposal of an action starting moment; in the on-line module, the long-term historical memory in the window is used for enhancing the score, the attention weight of the score in the window is dynamically generated, and the score of the current frame is calculated in a weighted mode; taking the proposal obtained by the offline module as a pseudo tag, and providing supervision information for the score sequence of the online module; and high-performance weak supervision on-line moment positioning can be completed only by independently deducing the on-line module. The expansion capability and the application value of the model are remarkably improved.

The invention relates to the technical field of data processing, and discloses a film and television supervision analysis method and system based on data mining and a storage medium. The method comprises the following steps: collecting film and television data in a panoramic manner, and carrying out cross-media transcoding processing to obtain a multi-dimensional data set; performing video semantic extraction, audio emotion recognition and text tendency analysis on the data set to generate a time sequence mark list; scoring the key elements according to social influence, propagation depth and audience acceptability to form a dynamic score table; analyzing the propagation path based on the score table, and constructing an influence map; sorting supervision points according to the atlas, and making an intelligent supervision scheme; the execution deviation is analyzed through effect feedback, and the supervision parameters are optimized. According to the method, multi-dimensional content feature extraction, social influence evaluation and propagation path analysis of the film and television works are realized in a mass multimedia content environment, a precise and differentiated intelligent supervision scheme is formed, and the method has self-optimization and adjustment capabilities at the same time.

The invention discloses a film and television content label processing method and terminal based on a large model, and belongs to the technical field of artificial intelligence and film and television content processing, and the method comprises the steps: pre-integrating to form a multi-source film and television data set; performing fine adjustment on a plurality of specified artificial intelligence large models in advance; pre-deploying a task automatic triggering module and constructing a cue word engineering library; acquiring input film and television content, automatically selecting unrechecked content for the film and television content to trigger a task, calling corresponding cue words through the cue word engineering library, and respectively outputting error correction results of film and television content labels through each specified artificial intelligence large model after fine tuning; judging and correcting the error correction result of the output film and television content label by adopting a preset artificial intelligence model, and mapping the correction result to a label library; and after AI auditing is carried out, the identification recheck authentication state of the film and television content is updated. The accuracy, efficiency and diversity of film and television label processing are improved, and the requirements of scenes such as film and television content management, recommendation and analysis are met.

The invention discloses an information retrieval method, a related device, equipment and a storage medium, and is applied to the field of artificial intelligence. The method comprises the steps of obtaining to-be-queried information and target instruction information; based on the to-be-queried information and the target instruction information, obtaining a first token sequence through a feature extraction layer included in an information retrieval model; performing bidirectional attention coding on the first token sequence through a backbone network included in the information retrieval model to obtain a second token sequence; pooling the second token sequence to obtain a target embedded vector; and according to the target embedding vector, obtaining a preset number of candidate information with the highest similarity as a retrieval result. According to the method, the complexity and the calculation cost of model deployment are reduced, and the target embedding vector generated by the model is more discriminative, so that the accuracy of a retrieval result is improved.

The embodiment of the invention provides an information processing method and device, and the method comprises the steps: collecting hotspot information related to a target service, and obtaining a hotspot video related to the hotspot information and a reference video copywriting of the hotspot video through a target interface; inputting the hotspot information, the hotspot video and the reference video copywriting into a large language model for processing to obtain a target video copywriting matched with a target platform content publishing style; generating a target audio and a target image corresponding to the target video copywriting, and constructing a target video according to the target audio and the target image; and publishing the target video on the target platform according to a preset publishing strategy.

The invention discloses an AI-based creative thinking auxiliary generation method and system, relates to the technical field of artificial intelligence, and solves the problem of inaccurate sound and picture matching in a traditional method by performing timestamp alignment and feature extraction on audio data and a visual image frame and calculating the correlation between the audio data and the visual image frame by using a cross-modal attention mechanism. According to the method, user eye movement track data is introduced, real attention points of a user are mapped into a visual sequence, and an optimized weight matrix is generated by constructing attention masks and fusing model attention weights, so that a generation result is more in line with perception key points of the user. And meanwhile, a feedback mechanism is established based on the synchronization error score, and when the sound and the picture are detected to be asynchronous, the visual frame timestamp can be dynamically adjusted, so that the self-adaptive correction of the content is realized. On the whole, the method has remarkable advantages in the aspects of improving modal alignment precision, enhancing user perception consistency and optimizing generation result naturalness.

The invention belongs to the technical field of automatic driving test, and particularly relates to a multi-mode accident scene library construction method for end-to-end automatic driving test. The method specifically comprises the following steps: step 1, extracting text information based on an MCAT-BiLSTM-CRF algorithm; 2, designing the ontology architecture, and storing accident scene information by using a knowledge graph to obtain an accident text knowledge base; step 3, derivative expansion is carried out on the accident scene element combination based on a HyCon-Sg-Net algorithm; step 4, performing enhanced fine tuning on the Open-Sora model to realize modal conversion from the text knowledge base to the accident video database so as to construct a multi-modal accident scene library; the method can be used for testing the performance of an end-to-end automatic driving automobile in an extreme accident scene, and the adaptive capacity of an automatic driving algorithm to the extreme accident scene is remarkably improved.

The invention provides a public opinion video tag aggregation method and system based on artificial intelligence, and relates to the technical field of artificial intelligence. Comprising the following steps: acquiring pictures and text information in a short video, and performing semantic alignment; different large language models are adopted to generate preliminary labels for the pictures and the text information after semantic alignment; clustering the pictures and the texts after semantic alignment to obtain clusters; calculating the labeling probability of each primary label type in the current cluster by each large language model, and selecting the primary label with the highest probability sum as a clustering label of the current cluster; calculating the reliability weight of each large language model in the current cluster based on the clustering label of the current cluster; and based on the reliability weight, calculating the weighted support degree of all the large language models to different preliminary label types of each piece of data in the current cluster, calculating the weighted label of the current data, and further determining a final label. According to the method, the condition of few labels or no labels can be effectively processed, and the manual workload is greatly reduced.

A video tag processing method, comprising: determining a video to be processed, and performing extraction on the video to obtain modality information of at least two modalities (202); on the basis of the modality information of the at least two modalities, respectively performing cross retrieval according to feature dimensions of the at least two modalities, so as to obtain respective multi-modal retrieval tags of the modality information of the at least two modalities (204); on the basis of the respective multi-modal retrieval tags of the modality information of the at least two modalities, determining at least one candidate tag for the video (206); and on the basis of the at least one candidate tag and the modality information of the at least two modalities, performing tag prediction, so as to obtain a video tag for the video (208).

The invention discloses a grid-coding-based video space-time retrieval method and device, the grid-coding-based video space-time retrieval method is executed in computing equipment, and the method comprises the following steps: mapping a coordinate range of a target area into a corresponding grid code set according to a Beidou grid standard on the basis of the coordinate range of the target area; acquiring video data with an intersection or inclusion relationship between the spatial range of the shot object and the target area; combining the shooting time of any video data with each grid code to generate a space-time grid code set; constructing an index structure corresponding to the video data by taking each time-space grid code in the time-space grid code set as a main key; and in response to query information of the user, analyzing a space-time range in the query of the user, mapping the space-time range into a corresponding space-time grid code range, executing retrieval based on the range, and returning hit video data. According to the method, rapid positioning and accurate retrieval of large-scale aerial photography video data are realized.

According to the automatic driving questioning and answering method and device based on the knowledge graph and the multi-modal large model, the method comprises the steps of firstly responding to a questioning and answering triggering condition, obtaining a real-time road condition video and a traffic safety problem, further obtaining a multi-modal query problem, then performing analysis processing on the multi-modal query problem based on a visual language model, and finally obtaining an automatic driving questioning and answering result. According to the method, visual semantic keywords are obtained, a candidate entity node set related to the visual semantic keywords is determined from a plurality of entity nodes of the knowledge graph, core knowledge nodes related to query questions are accurately focused, the invalid search range is reduced, question retrieval and answer generation efficiency is improved, and user experience is improved. The comprehensive similarity between each candidate node in the candidate entity node set and the visual semantic keyword is calculated, the key entity node set is obtained, the accuracy of question retrieval and answer generation is improved, and finally the question and answer result corresponding to the traffic safety question is generated according to each key entity node in the key entity node set.

The invention discloses a short video promotion optimization method and system, relates to the technical field of short video promotion optimization, and aims to solve the problems that when an existing short video promotion method is used for processing subtle elements which may cause different understandings of users in video contents, intentions of merchants are difficult to accurately convey, target users are difficult to reach, and the user experience is poor. The method comprises the steps of obtaining content data of a to-be-promoted video, performing multi-meaning element identification on the content data to determine at least one potential multi-meaning element in the to-be-promoted video, and determining a situational misreading risk assessment result of the potential multi-meaning element according to the potential multi-meaning element and a preset user situation model, and according to the situational misreading risk assessment result, a pre-stored promotion constraint condition and a content modification cost model, making a decision between the content modification strategy and the audience screening strategy to generate an initial intervention strategy, executing the initial intervention strategy, and optimizing the promotion process based on user feedback data obtained after execution.

Systems and methods are provided herein for personalizing images that correspond to a media asset identifier by using user profile information. As an example, the television series “Community” has several actors, such as Joel McHale, Chevy Chase, and Ken Jeong. Poster art developed by an editor o50533238_1f “Community” may include an image that portrays each of Joel McHale, Chevy Chase, and Ken Jeong. In order to personalize the image, control circuitry may determine which actor(s) the user prefers, and crop out only those actors in the poster art to create a personalized image. As an example, if the user prefers Joel McHale, control circuitry may crop out the portrayal of Joel McHale and use only that portion of the image to display next to other text describing “Community.”

In some embodiments, a method of processing image data may include receiving environmental data and associated capture time data from a sensor of a mobile computing device, the capture time data reflecting capture time of the environmental data; processing the environmental data to generate metadata; time stamping the metadata using the capture time data; receiving video data and video time data at a processor; correlating the metadata to the video data using the capture time data and the video time data; receiving a search query; and / or identifying a frame within the video data by performing a search of the metadata using the search criterion.

A system includes a computing platform having processing hardware, and a memory storing software code and a machine learning (ML) model-based feature classifier. When executed, the software code receives media content including a first media component corresponding to a first media mode and a second media component corresponding to a second media mode, encodes the first media component using a first encoder to generate multiple first embedding vectors, and encodes the second media component using a second encoder to generate multiple second embedding vectors. The software code further combines the first embedding vectors and the second embedding vectors to provide an input data structure for a neural network mixer, process, using the neural network mixer, the input data structure to provide feature data corresponding to a feature of the media content, and predict, using the ML model-based feature classifier and the feature data, a classification of the feature.

The embodiment of the invention provides a model training method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the steps of obtaining a plurality of training sample groups; each training sample group comprises a plurality of sample data; the sample data comprises target sample data, positive sample data and negative sample data; the sample data comprises a sample video or a video tag; each video tag has a corresponding tag description text; the sample data of the at least one training sample group comprises a sample video and a video tag; and carrying out at least one training operation on the initial video feature extraction model and the initial text feature extraction model through the training data set until a training ending condition is met, and obtaining a trained video feature extraction model and a trained text feature extraction model. The video features extracted by the trained video feature extraction model and the text features extracted by the trained text feature extraction model are aligned to the same feature space, so that the retrieval efficiency is improved during retrieval.

Disclosed herein are methods, systems, and computer-readable media for prompting a machine learning model to generate answer data based on a recording. Some embodiments involve preprocessing a prompt corresponding to a query for a first system by receiving the prompt and a timestamp corresponding to a time position of the query in a recording, acquiring a text transcript based on the recording, and selecting, based on the timestamp and the text transcript, a first data domain from the text transcript. Some embodiments involve transmitting at least one of the prompt, the text transcript, and the first data domain to a second system, the second system including a machine learning model. Some embodiments involve generating answer data corresponding to the prompt by querying the machine learning model with the prompt, receiving answer data from the machine learning model, and transmitting the answer data to the first system.

The invention provides an interaction processing method, device and equipment and a computer program product. The method comprises the following steps: segmenting an obtained video to be processed to obtain a plurality of video clips; obtaining a video metadata text of the to-be-processed video, and performing structured information extraction on each video clip to obtain video understanding structured information of each video clip; generating a current fragment understanding text and a historical fragment understanding text by using the video metadata text and the video understanding structured information of each video fragment; based on the existing interaction information of each video clip, determining an interaction emotion tendency and reference interaction information corresponding to each video clip; and generating target interaction information corresponding to each video clip by using the current clip understanding text, the historical clip understanding text, and the interaction emotion tendency and reference interaction information corresponding to each video clip. According to the invention, high-quality target interaction information with high matching degree with the video content can be generated.

An information processing apparatus comprises a controller configured to: output, based on a travel history of at least one vehicle, position information of the vehicle for generating a map on which a plurality of icons indicating positions of the vehicle at a plurality of time points are arranged, output time information corresponding to at least one icon among the plurality of icons, and transmit a video request requesting transmission of a video captured at a corresponding time point, to a vehicle or an onboard apparatus corresponding to a first icon specified by a first user operation.

The present disclosure generally relates to an integrated approach for retrieving biomedical information from clinical video presentations. In particular, the present disclosure is directed to video retrieval systems and methods of text-video retrieval from clinical video presentations.

The invention relates to the field of monitoring, in particular to a monitoring video retrieval method and device, equipment and a program product. The method comprises the following steps: receiving a surveillance video retrieval request, wherein the surveillance video retrieval request comprises a first time range and a type range; performing preliminary retrieval according to the first time range, and determining an index ID range of a first monitoring video event index set associated with the first time range; and performing second retrieval in the monitoring video event corresponding to the index ID range, determining a second monitoring video event index set associated with the type range in batches, and sequentially returning the second monitoring video event index set to the retrieval terminal according to the batches. The index ID range obtained through preliminary retrieval only occupies a small cache, and the second monitoring video event index set generated in batches is written into the cache, so that the monitoring video event and video loss illusion caused by insufficient cache can be effectively avoided on the premise of not increasing the cache, and the user experience is improved.

One embodiment of the present invention sets forth a technique for determining a time-varying deformation associated with a scene. The technique includes matching a query time to a time interval associated with the scene and generating, via execution of a machine learning model, (i) a first set of attributes associated with a set of canonical coordinates in the scene at a starting time of the time interval and (ii) a second set of attributes associated with the set of canonical coordinates at an ending time of the time interval. The technique also includes computing a third set of attributes associated with the set of canonical coordinates at the query time based on a spline interpolation associated with the first and second sets of attributes. The technique further includes generating a representation of the scene at the query time based on the third set of attributes.

Embodiments of the present disclosure provide a video processing method and apparatus, and a device. The method comprises: acquiring a reference image and text information; determining a target action video corresponding to the text information; if first human body key point information of the reference image fails to match second human body key point information of a target image frame in the target action video, performing inpainting processing on the reference image to obtain a target reference image; and generating a target video on the basis of the target reference image, the text information, and the target action video. In this way, an electronic device can automatically determine the target action video on the basis of the text information, and a user does not need to manually upload a guiding action video; and additionally, automatic inpainting processing for the reference image can be achieved, avoiding limitations on image integrity, enriching interaction forms of the user, improving the flexibility of user interaction, and meeting diversified requirements of the user.

The invention discloses a three-dimensional modeling method and system for a chemical industrial park, and the method comprises the steps: carrying a multi-lens inclined camera and a monitoring sensor through employing an unmanned plane, and obtaining the multi-view image data, positioning and attitude determination data of the chemical industrial park, and the real-time operation data of chemical equipment; performing data processing on the multi-view image data to generate sparse point clouds, and generating three-dimensional point clouds through dense matching and depth estimation; performing block processing on the three-dimensional point cloud, and performing splicing by adopting a point cloud registration algorithm to obtain global point cloud data; performing surface reconstruction on the global point cloud data to generate a three-dimensional grid model, and completing model texture mapping through a texture mapping algorithm; a three-dimensional visual management platform is constructed, a three-dimensional grid model and real-time monitoring data are integrated, and park global visualization, risk visualization and intelligent management of chemical engineering devices, equipment and facilities are achieved.

The invention discloses a timing sequence statement positioning model training method and device based on proposal selection and anchor point distribution, and relates to the technical field of timing sequence statement positioning. The method comprises the following steps: initializing a fixed number of learnable queries according to a first static anchor point set; on the basis of the first learnable query, according to the unpruned long video and the natural language description text, performing proposal generation through a time sequence statement positioning model; based on the proposal selection module, redundant proposal filtering is carried out by using a non-maximum suppression algorithm; based on an anchor point distribution module, according to the untrimmed long video, the first static anchor point set and the candidate proposal set, performing bipartite graph matching by using a Hungary algorithm; and carrying out loss function weighting calculation according to the unpruned long video and the optimal proposal set, and carrying out parameter optimization on the time sequence statement positioning model to obtain an optimized time sequence statement positioning model. The method is an efficient and accurate time sequence statement positioning model training method combining proposal selection and anchor point distribution.

The invention discloses a short video personalized recommendation system and method based on deep reinforcement learning. Firstly, user features, video features and feedback features are converted into embedded sequence representations, then states are modeled through a self-adaptive attention mechanism, potential relationships among embedded sequences are captured, and personalized recommendation state representations reflecting user preferences are generated. An optimal recommendation strategy of deep reinforcement learning is used, a Q network comprising an LSTM layer and an MLP layer is constructed, the LSTM layer is responsible for capturing time sequence information of user behaviors, and the MLP layer combines the information with a current environment and outputs a Q value of each action. By continuously optimizing the Q network, an optimal recommendation strategy is learned, so that personalized short video recommendation is provided. Compared with a traditional short video recommendation method, the method has the advantages that the adaptive capacity is higher, the preference change of the user is quickly responded, the recommendation strategy is dynamically adjusted, and higher recommendation accuracy and diversity are achieved, so that long-term return is maximized, and the satisfaction degree of the user is improved.