313results about "Video data clustering/classification" patented technology

The present disclosure relates to the technical field of target detection, and particularly relates to a model deployment method, an end-side device and a storage medium, which are used for solving the problem in the related art of the accuracy of a deployed model being low. The method comprises: performing target detection on a video frame image input into a first model, and acquiring a first target detection result and a first confidence; if the first confidence is greater than or equal to a first confidence threshold value, recording the video frame image and the first target detection result as samples in a training set; if the first confidence is less than the first confidence threshold value, performing target detection on the video frame image on the basis of a second model, and recording the video frame image and an acquired second target detection result as samples in the training set; and training the first model on the basis of the training set, and replacing the current first model with a trained first model for subsequent target detection. In this way, the accuracy and model generalization capability of a first model are improved.

A data processing system implements receiving a call requesting a generative model to generate a semantic tree for a source content; constructing a first prompt including the source content and instructions to the model to analyze a semantic structure of the source content and to generate a semantic outline and content chunks of the source content, the semantic outline including one or more topics each connected with one or more of the content chunks, to compute one summary for each of the content chunks, to apply indices to reference each topic node of the semantic tree to one of the topics, and to apply indices to reference each leaf node of the semantic tree to one of the content chunks and the respective summary; providing the first prompt to the model and receiving the semantic tree of the source content; and storing the semantic tree in a database.

The invention relates to the technical field of security and protection visualization, in particular to a security and protection visualization display method and system and a storage medium. The method comprises the following steps: acquiring a data stream acquired by security and protection equipment, and compressing the data stream into a unified acquisition frame set; identifying a road boundary point set in the unified collection frame set, and detecting a dynamic target and a static target at each moment in the unified collection frame set; performing semantic annotation according to the motion state change condition of the dynamic target at each moment in the unified collection frame set to obtain a road boundary semantic distribution map; according to the road boundary semantic distribution map, identifying and predicting the motion state change condition of the static target; and predicting an overlapping time point of the motion state of the target based on the motion state change condition of the static target and the motion state change condition of the dynamic target, and transmitting the overlapping time point to communication equipment of the static target through the Internet of Things so as to execute a voice prompt task. According to the invention, the display precision and response efficiency of security and protection visualization can be improved.

A data processing system implements receiving a call requesting a generative model to generate a semantic tree for a source content; constructing a first prompt including the source content and instructions to the model to analyze a semantic structure of the source content and to generate a semantic outline and content chunks of the source content, the semantic outline including one or more topics each connected with one or more of the content chunks, to compute one summary for each of the content chunks, to apply indices to reference each topic node of the semantic tree to one of the topics, and to apply indices to reference each leaf node of the semantic tree to one of the content chunks and the respective summary; providing the first prompt to the model and receiving the semantic tree of the source content; and storing the semantic tree in a database.

The invention relates to the technical field of data processing, and discloses a film and television supervision analysis method and system based on data mining and a storage medium. The method comprises the following steps: collecting film and television data in a panoramic manner, and carrying out cross-media transcoding processing to obtain a multi-dimensional data set; performing video semantic extraction, audio emotion recognition and text tendency analysis on the data set to generate a time sequence mark list; scoring the key elements according to social influence, propagation depth and audience acceptability to form a dynamic score table; analyzing the propagation path based on the score table, and constructing an influence map; sorting supervision points according to the atlas, and making an intelligent supervision scheme; the execution deviation is analyzed through effect feedback, and the supervision parameters are optimized. According to the method, multi-dimensional content feature extraction, social influence evaluation and propagation path analysis of the film and television works are realized in a mass multimedia content environment, a precise and differentiated intelligent supervision scheme is formed, and the method has self-optimization and adjustment capabilities at the same time.

Methods, systems, and apparatuses, including computer programs encoded on computer storage media, for receiving a query relating to a data item that includes multiple data item samples and processing the query and the data item to generate a response to the query. In particular, the described techniques include adaptively selecting a subset of the data item samples using a selection neural network conditioned on features of the data item samples and the query. Then processing the subset and query using a downstream task neural network to generate a response to the query. By adaptively selecting the subset of data item samples according to the query, the described techniques generate responses to queries that are more accurate and require less computation resources than would be the case using other techniques.

The invention belongs to the technical field of automatic driving test, and particularly relates to a multi-mode accident scene library construction method for end-to-end automatic driving test. The method specifically comprises the following steps: step 1, extracting text information based on an MCAT-BiLSTM-CRF algorithm; 2, designing the ontology architecture, and storing accident scene information by using a knowledge graph to obtain an accident text knowledge base; step 3, derivative expansion is carried out on the accident scene element combination based on a HyCon-Sg-Net algorithm; step 4, performing enhanced fine tuning on the Open-Sora model to realize modal conversion from the text knowledge base to the accident video database so as to construct a multi-modal accident scene library; the method can be used for testing the performance of an end-to-end automatic driving automobile in an extreme accident scene, and the adaptive capacity of an automatic driving algorithm to the extreme accident scene is remarkably improved.

The present application provides a video segmentation method, a server, a storage medium, and a program product. In the method of the present application, video data to be segmented is segmented into multiple data segments, unimodal features of the data segments, including text features of a text modality and visual features of a visual modality, are respectively extracted by means of a video topics segmentation model, and then the text features and visual features of the data segments are fused, so that the fusion of multimodal information can be performed at the intermediate representation level, the relationship and interaction between different modalities can be better captured, and higher-quality multimodal fusion features of the data segments are obtained. Furthermore, on the basis of the multimodal fusion features of the data segments, whether the data segments are topic boundaries is predicted, so that the topic boundaries of the video data can be accurately predicted, improving the accuracy of topic boundary recognition, thereby improving the accuracy and quality of video topics segmentation results.

The invention provides a public opinion video tag aggregation method and system based on artificial intelligence, and relates to the technical field of artificial intelligence. Comprising the following steps: acquiring pictures and text information in a short video, and performing semantic alignment; different large language models are adopted to generate preliminary labels for the pictures and the text information after semantic alignment; clustering the pictures and the texts after semantic alignment to obtain clusters; calculating the labeling probability of each primary label type in the current cluster by each large language model, and selecting the primary label with the highest probability sum as a clustering label of the current cluster; calculating the reliability weight of each large language model in the current cluster based on the clustering label of the current cluster; and based on the reliability weight, calculating the weighted support degree of all the large language models to different preliminary label types of each piece of data in the current cluster, calculating the weighted label of the current data, and further determining a final label. According to the method, the condition of few labels or no labels can be effectively processed, and the manual workload is greatly reduced.

A video tag processing method, comprising: determining a video to be processed, and performing extraction on the video to obtain modality information of at least two modalities (202); on the basis of the modality information of the at least two modalities, respectively performing cross retrieval according to feature dimensions of the at least two modalities, so as to obtain respective multi-modal retrieval tags of the modality information of the at least two modalities (204); on the basis of the respective multi-modal retrieval tags of the modality information of the at least two modalities, determining at least one candidate tag for the video (206); and on the basis of the at least one candidate tag and the modality information of the at least two modalities, performing tag prediction, so as to obtain a video tag for the video (208).

The invention relates to the technical field of video content analysis, and discloses a video scene content label determination method based on a knowledge graph. The method comprises the following steps: inputting a multi-modal feature sequence formed by visual, audio and text features extracted from a video stream into a knowledge graph inference engine comprising an entity relationship network and a semantic association rule base; performing node mapping through an engine to generate an initial scene entity set; performing hierarchical reasoning on the set based on a semantic association rule base to obtain a scene semantic topological structure; screening core entities according to entity weight distribution in the topological structure, and generating a candidate tag set; performing time sequence consistency verification on the candidate tag set, and correcting tag time sequence offset in combination with timestamp information; performing cross-modal disambiguation on the corrected label set by using an entity relationship network to eliminate semantic conflicts; and according to a disambiguation result, constructing a scene label knowledge sub-graph containing entity attributes and relation path constraints.

A system includes a computing platform having processing hardware, and a memory storing software code and a machine learning (ML) model-based feature classifier. When executed, the software code receives media content including a first media component corresponding to a first media mode and a second media component corresponding to a second media mode, encodes the first media component using a first encoder to generate multiple first embedding vectors, and encodes the second media component using a second encoder to generate multiple second embedding vectors. The software code further combines the first embedding vectors and the second embedding vectors to provide an input data structure for a neural network mixer, process, using the neural network mixer, the input data structure to provide feature data corresponding to a feature of the media content, and predict, using the ML model-based feature classifier and the feature data, a classification of the feature.

A data processing system implements receiving a call requesting a generative model to generate a semantic tree for a source content; constructing a first prompt including the source content and instructions to the model to analyze a semantic structure of the source content and to generate a semantic outline and content chunks of the source content, the semantic outline including one or more topics each connected with one or more of the content chunks, to compute one summary for each of the content chunks, to apply indices to reference each topic node of the semantic tree to one of the topics, and to apply indices to reference each leaf node of the semantic tree to one of the content chunks and the respective summary; providing the first prompt to the model and receiving the semantic tree of the source content; and storing the semantic tree in a database.

The invention discloses a timing sequence statement positioning model training method and device based on proposal selection and anchor point distribution, and relates to the technical field of timing sequence statement positioning. The method comprises the following steps: initializing a fixed number of learnable queries according to a first static anchor point set; on the basis of the first learnable query, according to the unpruned long video and the natural language description text, performing proposal generation through a time sequence statement positioning model; based on the proposal selection module, redundant proposal filtering is carried out by using a non-maximum suppression algorithm; based on an anchor point distribution module, according to the untrimmed long video, the first static anchor point set and the candidate proposal set, performing bipartite graph matching by using a Hungary algorithm; and carrying out loss function weighting calculation according to the unpruned long video and the optimal proposal set, and carrying out parameter optimization on the time sequence statement positioning model to obtain an optimized time sequence statement positioning model. The method is an efficient and accurate time sequence statement positioning model training method combining proposal selection and anchor point distribution.

The present document discloses a method to generate a database for synchronization of a text, a video and / or audio media, namely a computer-implemented method to generate a database for synchronization of a text, a video and / or audio media with a video stream to be generated using said database, comprising the steps of: receiving an input text; splitting the input text into text segments; clustering the text segments into at least one text group; labelling each text segment with a sequential timestamp; and, storing each text segment with the label on a data record wherein the text group comprises a time interval label which corresponds to the duration of at least a portion of a video and / or audio media. It is further disclosed a method for retrieving information from said database, a system, and a computer program thereof.

The invention discloses a film and television content intelligent recommendation method and system based on artificial intelligence. The method comprises the steps of multi-source data collection, data optimization, group film and television content recommendation, personalized recommendation optimization sorting and film and television content intelligent recommendation. The invention relates to the technical field of data processing, in particular to an intelligent film and television content recommendation method and system based on artificial intelligence. According to the scheme, group recommendation and personalized optimization are innovatively combined, and dynamic balance between group representativeness and individual difference is achieved; a density correction mechanism, weighted distance calculation, a representative point merging mechanism and a fuzzy weighted distribution strategy are innovatively proposed to improve the clustering algorithm, so that the clustering accuracy is improved, and the group recommendation precision and the user satisfaction are improved; and an adaptive reference individual selection strategy and an iteration-based global exploration weight improvement optimization algorithm are adopted, so that the accuracy of a model output result is remarkably improved, and the stability of personalized film and television content recommendation optimization sorting is realized.

A method and system for classifying a video file within an environment in which the file is located and when a file is classified as sensitive, controlling transmission of the file outside the environment. Classifying the video comprises analysing, using at least one machine learning model, the video to recognise any individuals in the video; obtaining a transcript of any speech in the video and generating, using the analysis, obtained transcript and a database of individuals linked to the environment, a labelled transcript which identifies each individual linked to the environment that is in the video. Information about each identified individual may be obtained from a connected database. A first generative AI model generates a text-based summary of the video by using the labelled transcript and information about identified individuals as prompts. A second generative AI model then determines a sensitivity classification of the video using the generated text-based summary.

A method of generating a set of predictions associated with position-based sporting events using an axial transformer neural network, the method including: receiving an input tuple, including a set of tensors representing game context, team strength, player strength, live team features, live player features, game events, and a super feature; inputting the input tuple into an axial transformer neural network by inputting each tensor from the set of tensors within a corresponding initial embedding layer; concatenating the initial embedding layers to form a single tensor; applying self-attention to the single tensor through axial transformer layers of the axial transformer neural network; mapping output embeddings from the axial transformer layers to target layers; and generating a set of target metric predictions for each racer, team, and overall for the position-based sporting events, based on the output embeddings from the target layers.

In various examples, generating and using interaction graphs for video information retrieval systems and applications is described herein. Systems and methods are disclosed that process videos generated using one or more image sensors in order to generate a graph that represents at least interactions between entities depicted by the videos. For instance, nodes of the graph may be associated with the entities—such as people and / or other objects—as well as attributes associated with the entities. Additionally, edges of the graph may be associated with interactions between the entities, times that the interactions occurred, and / or indications of which videos depict the interactions. Systems and methods are then further disclosed that use the graph to perform information retrieval associated with the videos. For instance, the graph may be used to identify relevant information associated with a query, where the information may then be used to generate a response.

The invention belongs to the technical field of intelligent traffic, and relates to a vehicle image-text retrieval method and system based on multistage semantic graph alignment and attribute enhancement. According to the method, fusion visual embedding is obtained according to the vehicle video, and fusion text embedding is obtained according to the text data; obtaining an updated visual attribute embedding vector and an updated text attribute embedding vector based on fusion visual embedding and fusion text embedding; performing semantic matching on the updated visual attribute embedding vector and the updated text attribute embedding vector, and aligning vehicle cross-modal semantic attributes from three different levels by utilizing multi-granularity semantics to obtain final visual attribute embedding and final text attribute embedding; and according to the final visual attribute embedding and the final text attribute embedding, obtaining the similarity between the vehicle video and the artificial text description, wherein the vehicle video with the highest similarity is a vehicle video retrieval result. According to the method, the target vehicle can be quickly and accurately positioned, and higher robustness and higher retrieval success rate are shown.

The invention discloses a lens-level professional video image marking and quick retrieval method, and aims to solve the problems of disordered management and low retrieval efficiency and accuracy caused by inconsistent marking terms and overhigh dimensionality of film and television materials in the prior art, multi-level marking is carried out on lens-level professional video images, and each level focuses on different directions, so that the retrieval efficiency and the retrieval accuracy are improved. The used keywords are selected from a pre-constructed keyword library, so that the unification of the keywords is realized, and the keywords in the keyword library are associated with synonym groups to support fuzzy retrieval. And during retrieval, the required video image can be obtained by inputting the user-defined keyword. According to the method, fuzzy retrieval is supported, synonyms do not need to be tried one by one, the retrieval difficulty is low, the retrieval efficiency is high, noise keywords are not introduced in the marking method, main keywords are not omitted, and the retrieval accuracy is high. AI tools are used for auxiliary marking, and the marking efficiency is high.

The invention relates to the field of safety training, in particular to a safety talent training resource sharing system and method based on cloud computing, and the method comprises the steps: determining a test question obtaining mode according to the number of historical training times and effective duration corresponding to a generation request; the test question obtaining mode is that when test questions are obtained based on the calibrated training test questions, the calibrated training test questions are selected based on the label support proportion determination basis processing evaluation value or the label subordinate learning degree; according to a position change reference value corresponding to a first-class standard selection box in a test question video corresponding to the calibrated training test question, determining that the video feature difference degree is a second-class standard selection box influence quantity difference value or a first-class standard selection box surrounding area difference value; according to the video feature difference degree corresponding to the to-be-selected test questions and the calibrated training test questions, primary training is selected; when the test question obtaining mode is test question obtaining based on the rotation number, the current training test question is selected according to the rotation number corresponding to the test question to be selected; the selection precision of the training test questions is improved.

The disclosure provides a video author identification method, a video resource recommendation method and device, and relates to the technical field of computers, in particular to the technical field of video technology and intelligent recommendation. The video author identification method comprises: determining first image features of a plurality of video works of a video author according to classification results of cover image of the plurality of video works; determining first video works of a target type from the plurality of video works according to text features and the first image features of the plurality of video works; wherein the text features are obtained according to label information of the video works; determining first work features of the video author according to a proportion of the first video works in the plurality of video works; determining second work features of the video author according to a proportion of second video works containing target attributes in the plurality of video works; and determining an author type label of the video author according to the first work features and the second work features. The disclosure is helpful for mining of video authors and video resources.

A method includes obtaining a first unstructured video stream that provides pixel values for a plurality of pixels and corresponds to a portion of a second unstructured video stream being displayed on a second electronic device different from the first electronic device. Obtaining the first unstructured video stream includes obtaining pass-through image data including the portion of a second unstructured video stream. The method includes generating respective pixel characterization vectors for a first portion of the plurality of pixels. Generating each of the respective pixel characterization vectors includes determining a respective instance label value. The method includes identifying a first object within the first portion of the plurality of pixels associated with a particular instance label value. The method includes generating respective semantic label values corresponding to pixels associated with the first object. The respective semantic label values are added to pixel characterization vectors associated with the first object.

The present disclosure relates to a target video generation method and device, computer equipment and a storage medium. The method comprises: determining a to-be-processed video segment in a plurality of video segments, and determining a matching adjustment duration of each to-be-processed video segment; in response to the to-be-processed video segment being a to-be-expanded video segment, determining a clustering number according to the adjustment duration, clustering according to feature information of the to-be-expanded video segment, time information of each frame of video image in the to-be-expanded video segment and the clustering number, and generating a target segment; in response to the to-be-processed video segment being a to-be-cut video segment, determining a first video style of the to-be-cut video segment, determining a target segment according to a relationship between the adjustment duration matching the to-be-cut video segment, the first video style and a second video style of a clip template; and generating a target video according to at least the generated target segment and the clip template. The method can generate a video adapted to the clip template.

The present application relates to the fields of sensor technology and Internet of Things, and provides a fitness video and training equipment cooperative control method and system, which comprises: collecting a cloud-based fitness teaching video stream to extract a sequence of key points of a skeleton of a demonstration action, an action intensity level and a standard motion trajectory, collecting real-time physiological data and real-time motion posture of a user; identifying a target action of each stage in the fitness teaching video stream, generating a target control parameter of a training equipment corresponding to the target action to construct a video-parameter mapping library of the training equipment; determining a physical fitness adaptation coefficient of the user, calculating an action completion degree and a motion risk coefficient of the user, and generating a safety control threshold of the training equipment; and generating a cooperative control instruction of the training equipment to execute cooperative control of the training equipment. The present application can improve the adaptability and safety of the training equipment cooperative control.

The invention discloses a short video recommendation method and system based on graph contrast learning. The short video recommendation method comprises the following steps: inputting a short video into a video recommendation model to generate preference of a user for the short video; the training process of the video recommendation model is as follows: constructing a user-creator interaction graph and a user-short video interaction graph; the user-creator interaction graph is divided into sub-graphs according to creator content field clustering, local multi-layer information propagation is carried out in each sub-graph, and user embedding and creator embedding under a creator view are obtained through aggregation; performing multi-layer information spreading on the user-short video interaction diagram, and aggregating to obtain user embedding and short video embedding under a short video view; taking the inner product of the dynamically fused creator embedding and short video embedding and the user embedding as the preference score of the user for the short video, arranging the preference score in a descending order, and taking the first K recommendation results as the preference of the user for the short video; according to the recommendation method, accurate and robust short video recommendation is realized.

The present disclosure relates to a method for managing a large number of videos, an electronic device and a computer program product, which are applied to a monitoring system, including detecting an event and outputting a large number of videos according to the event; labeling a plurality of tags for the large number of videos, the tags including a plurality of attributes of at least one object in the large number of videos; receiving video search information; comparing the relevance of the video search information and the tags of the large number of videos, and outputting a plurality of recommended videos in a terminal device according to the relevance, the terminal device including a display screen; detecting the resolution of the display screen, and adaptively playing the recommended videos in a user interface on the display screen according to the resolution. The method of the present disclosure uses attribute tag indexing technology closest to human search logic, effectively manages a large number of videos, automatically searches for relevant video results according to input information, greatly simplifies the search process and improves user experience.

The invention belongs to the field of artificial intelligence networks, and particularly relates to a method for performing visual questions and answers by using an attention mechanism from words to regions. Comprising the following steps: (1) extracting features; and (2) generating candidate answers: firstly positioning related image areas and keywords in questions by adopting a collaborative attention mechanism, then obtaining fine-grained image features and question features, and finally fusing the two features to generate the candidate answers. According to the method, in order to generate candidate answers with higher quality, two question and answer stages are cascaded, a traditional single-stage visual model is expanded into a double-stage model, semantic information contained in the answers is fully mined, and accurate prediction of the final answers is promoted. According to the invention, image areas and keywords related to questions can be extracted and utilized, so that more accurate candidate answers can be generated.

The invention provides a petroleum scene operator violation action detection method based on posture recognition, and relates to the technical field of action recognition, and the method comprises the steps: obtaining a joint point position sequence, hazard source dynamic data and an operation process state identifier of an operator; performing time sequence evolution analysis based on the joint point position sequence to obtain a subsequent attitude evolution trend; establishing an operator prediction motion envelope and a hazard source prediction motion envelope according to a subsequent attitude evolution trend and hazard source dynamic data, and calculating a minimum distance evolution curve of the operator prediction motion envelope and the hazard source prediction motion envelope in a future time window to obtain a bidirectional interaction risk value representing a relative motion collision probability; and according to the bidirectional interaction risk value and the operation process state identifier, generating a violation judgment result containing a violation emergency degree and an early warning opportunity through a context-dependent risk weight rule. According to the invention, the prospective risk assessment of the dynamic hazard source can be realized, and the accuracy of violation detection and the timeliness of early warning are improved.