633results about "Video data clustering/classification" patented technology

The present disclosure relates to the technical field of target detection, and particularly relates to a model deployment method, an end-side device and a storage medium, which are used for solving the problem in the related art of the accuracy of a deployed model being low. The method comprises: performing target detection on a video frame image input into a first model, and acquiring a first target detection result and a first confidence; if the first confidence is greater than or equal to a first confidence threshold value, recording the video frame image and the first target detection result as samples in a training set; if the first confidence is less than the first confidence threshold value, performing target detection on the video frame image on the basis of a second model, and recording the video frame image and an acquired second target detection result as samples in the training set; and training the first model on the basis of the training set, and replacing the current first model with a trained first model for subsequent target detection. In this way, the accuracy and model generalization capability of a first model are improved.

A data processing system implements receiving a call requesting a generative model to generate a semantic tree for a source content; constructing a first prompt including the source content and instructions to the model to analyze a semantic structure of the source content and to generate a semantic outline and content chunks of the source content, the semantic outline including one or more topics each connected with one or more of the content chunks, to compute one summary for each of the content chunks, to apply indices to reference each topic node of the semantic tree to one of the topics, and to apply indices to reference each leaf node of the semantic tree to one of the content chunks and the respective summary; providing the first prompt to the model and receiving the semantic tree of the source content; and storing the semantic tree in a database.

A method and system for classifying a video file within an environment in which the file is located and when a file is classified as sensitive, controlling transmission of the file outside the environment. Classifying the video comprises analysing, using at least one machine learning model, the video to recognise any individuals in the video; obtaining a transcript of any speech in the video and generating, using the analysis, obtained transcript and a database of individuals linked to the environment, a labelled transcript which identifies each individual linked to the environment that is in the video. Information about each identified individual may be obtained from a connected database. A first generative AI model generates a text-based summary of the video by using the labelled transcript and information about identified individuals as prompts. A second generative AI model then determines a sensitivity classification of the video using the generated text-based summary.

Systems, methods, and computer-readable media are disclosed for systems and methods multimodal indexing of video using machine learning. An example method may include deceiving, by a video encoder of an audio-video transformer neural network comprising one or more computer processors coupled to memory, a first frame and a second frame associated with a first segment of a video. The example method may also include receiving, by an audio encoder of the audio-video transformer neural network, an audio spectrogram comprising first audio data associated with the first segment of the video. generating, by the video encoder, a first video embedding. The example method may also include generating, by the audio encoder, a first audio embedding. The example method may also include determining a fusion of the first video embedding and the first audio embedding using a multimodal bottleneck token. The example method may also include determining an output including the first video embedding and the first audio embedding. The example method may also include determining a classification of the first portion of the video based on the output.

Systems and methods for automating the management of a warehouse are disclosed. The method includes monitoring. in real-time, a plurality of items in the warehouse, a plurality of tasks associated with the warehouse, and / or incoming vehicles to the warehouse. The method also includes receiving, via a plurality of sensors, image data and / or video data associated with the plurality of items, the plurality of tasks, and / or the incoming vehicles. The method further includes inputting the image data and / or the video data into a machine learning model to automate the management of the warehouse, wherein the machine learning model has been trained based on training data to generate predictions for the inputted data, and wherein the machine learning model utilizes the predictions for the inputted data for retraining to generate predictions for newly inputted data and improve accuracy of error handling.

The invention discloses a video abstract generation method and system based on feature expression enhancement, and relates to the technical field of computer vision, and the method comprises the steps: obtaining an original video, and extracting a video frame image sequence; a video frame image sequence is input into an OCVF video understanding model, initial features of each input video frame image are extracted through a feature extraction module, key spatio-temporal features of the initial features are extracted through an attention module, and then high-level feature representation is generated through feature mapping and jump connection. The high-level feature representation outputs the importance score of each video frame through a classifier, key frames are determined, and a final video abstract result is generated; wherein the attention module comprises a channel attention layer, an average pooling layer, a position coding layer, a self-attention layer and a space-time attention layer which are arranged in sequence. According to the method, more comprehensive modeling and efficient understanding of the video content can be realized, and more robust and more accurate video abstract generation is realized.

The invention discloses a multi-modal video question-answering method and device based on large language model optimization, and the method employs a CoVGT basic framework, optimizes video feature extraction and text input, and adds a video question-answering model output optimizer. The method comprises the steps of video feature extraction, text feature extraction, feature fusion, preliminary answer prediction, related sample generation, Prompt text generation and final answer generation. According to the method, the answer optimizer is designed by using the learning characteristics in GPT-3 analogy, related sample generation, Prompt text generation and final answer generation, and compared with a basic model, the model accuracy is greatly improved.

The invention relates to the technical field of security and protection visualization, in particular to a security and protection visualization display method and system and a storage medium. The method comprises the following steps: acquiring a data stream acquired by security and protection equipment, and compressing the data stream into a unified acquisition frame set; identifying a road boundary point set in the unified collection frame set, and detecting a dynamic target and a static target at each moment in the unified collection frame set; performing semantic annotation according to the motion state change condition of the dynamic target at each moment in the unified collection frame set to obtain a road boundary semantic distribution map; according to the road boundary semantic distribution map, identifying and predicting the motion state change condition of the static target; and predicting an overlapping time point of the motion state of the target based on the motion state change condition of the static target and the motion state change condition of the dynamic target, and transmitting the overlapping time point to communication equipment of the static target through the Internet of Things so as to execute a voice prompt task. According to the invention, the display precision and response efficiency of security and protection visualization can be improved.

A data processing system implements receiving a call requesting a generative model to generate a semantic tree for a source content; constructing a first prompt including the source content and instructions to the model to analyze a semantic structure of the source content and to generate a semantic outline and content chunks of the source content, the semantic outline including one or more topics each connected with one or more of the content chunks, to compute one summary for each of the content chunks, to apply indices to reference each topic node of the semantic tree to one of the topics, and to apply indices to reference each leaf node of the semantic tree to one of the content chunks and the respective summary; providing the first prompt to the model and receiving the semantic tree of the source content; and storing the semantic tree in a database.

The invention provides a video generation method and device, and relates to the technical field of computers, in particular to the technical fields of large models, deep learning, image processing, voice recognition and the like. According to the specific implementation scheme, semantic partitioning and position marking are conducted on a to-be-processed material, and a semantic text block set comprising at least one semantic text block is obtained; based on the key event information, classifying semantic text blocks in the semantic text block set to obtain a key text block set and a non-key text block set; generating a key image sequence based on the key text block set, the to-be-processed material and a video material library; generating a non-key image sequence based on the non-key text block set and a video material library; the target video is generated based on the key image sequence and the non-key image sequence, and the video generation effect is improved.

The invention relates to the technical field of data processing, and discloses a film and television supervision analysis method and system based on data mining and a storage medium. The method comprises the following steps: collecting film and television data in a panoramic manner, and carrying out cross-media transcoding processing to obtain a multi-dimensional data set; performing video semantic extraction, audio emotion recognition and text tendency analysis on the data set to generate a time sequence mark list; scoring the key elements according to social influence, propagation depth and audience acceptability to form a dynamic score table; analyzing the propagation path based on the score table, and constructing an influence map; sorting supervision points according to the atlas, and making an intelligent supervision scheme; the execution deviation is analyzed through effect feedback, and the supervision parameters are optimized. According to the method, multi-dimensional content feature extraction, social influence evaluation and propagation path analysis of the film and television works are realized in a mass multimedia content environment, a precise and differentiated intelligent supervision scheme is formed, and the method has self-optimization and adjustment capabilities at the same time.

Methods, systems, and apparatuses, including computer programs encoded on computer storage media, for receiving a query relating to a data item that includes multiple data item samples and processing the query and the data item to generate a response to the query. In particular, the described techniques include adaptively selecting a subset of the data item samples using a selection neural network conditioned on features of the data item samples and the query. Then processing the subset and query using a downstream task neural network to generate a response to the query. By adaptively selecting the subset of data item samples according to the query, the described techniques generate responses to queries that are more accurate and require less computation resources than would be the case using other techniques.

Systems and methods for tracking inventory items in an area of real space are disclosed. The method includes receiving a signal generated in dependence on sensors. The signal indicates a change to a portion of an image of an area of real space. The method includes, in response to receiving the signal, implementing a trained location detection model to determine, based on inputs, whether an inventory item identified in the portion of the image has changed a position in the area of real space. The method includes implementing a trained item classification model to determine a classification of the inventory item. The method includes updating an inventory database with inventory item data determined in dependence on the classification of the inventory item to provide an updated map of the area of real space as a result of the received signal indicating the change to the portion of the image.

Embodiments are disclosed for performing content authentication. A method of content authentication may include dividing a query video into a plurality of chunks. A feature vector may be generated, using a fingerprinting model, for each chunk from the plurality of chunks. Similar video chunks are identified from a trusted chunk database based on the feature vectors using a multi-chunk search policy. One or more original videos corresponding to the query video are then returned.

The invention discloses a long video positioning method, system and device and a storage medium, which are one-to-one corresponding schemes, in the scheme, a step-by-step positioning mode is adopted, firstly, a highlight video frame is identified through video and text guidance, so that a video clip aligned with a query text is obtained, and the problem that in an existing method, a clip is directly extracted from a long video, so that the video clip cannot be directly extracted is avoided. The problem of lack of overall perception of a long video on the text is solved; further positioning on the basis of obtaining the video clip aligned with the query text to obtain an accurate moment; in the step-by-step positioning mode, each step of positioning further strengthens the relation between the video and the query text, so that a more accurate positioning result can be obtained.

The invention discloses a media information processing method and device, a storage medium and electronic equipment. The method comprises the steps that to-be-retrieved source media information and prompt template information are obtained, the source media information and the prompt template information are input into a target retrieval model, target media information is determined, and the target retrieval model represents a model obtained through joint training according to at least two retrieval task types; a loss function used in the training process of the target retrieval model is a joint loss function, the joint loss function is determined by at least two sub-loss functions in one-to-one correspondence with the at least two retrieval task types, and the target media information represents media information obtained by retrieving the source media information according to the prompt template information. According to the method and the device, the technical problem of relatively low retrieval efficiency of the media information caused by single multi-modal retrieval task is solved.

The invention discloses a multi-granularity video retrieval method and device based on a multi-modal large model, computer equipment and a readable storage medium, and the method comprises the steps: firstly obtaining video query information input by a user, carrying out the intention recognition to obtain a query field, rewriting the query information and the field to obtain a video query vector, and carrying out the retrieval of the video query vector; and retrieving in a preset retrieval video knowledge base according to the vector and the field, and finally obtaining the target retrieval video content, so as to improve the efficiency and accuracy of video retrieval and adapt to multi-field retrieval requirements.

The invention relates to a label generation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a video identifier and description data of a target video; obtaining classification reference data of the target video according to the video identifier; generating a candidate label set based on the description data and the classification reference data; and inputting the description data and the candidate label set into a preset classification model to obtain a classification label of the target video. According to the embodiment of the invention, the classification label of the target video can be automatically generated, furthermore, the classification label closer to the description data of the target video is selected from the expanded richer candidate label set, the label generation accuracy is improved, and the classification label generation efficiency is improved due to the fact that the classification label is generated based on the same standard. Errors caused by individual differences in manual labeling are avoided, and the consistency of the labels is improved.

The invention discloses an intelligent conference recording and recording method and system. According to the invention, cloud service equipment receives multi-modal information sent by edge equipment; processing the multi-modal information, and generating conference record information in combination with context sensing error correction, content segmentation, a dynamic segmentation strategy and a preset storage strategy; carrying out encryption processing on the conference record information based on the participant role information, and generating target format conference information; processing the target format conference information to generate initial conference scene mode information including a conference outline, a PPT and a mind map; processing the target format conference information and the initial conference scene mode information to generate conference information data after priority ranking and strategy processing; performing semantic analysis, feature fusion and cross-modal association processing on the conference information data to generate target conference scene mode information; and determining a target application program based on the target conference scene mode information, and sending the target format instruction information to the target application program.

Disclosed embodiments include a computational memory system. The computational memory system includes at least one computational memory chip including one or more processor subunits and one or more memory banks formed on a common substrate. The at least one computational memory chip is configured to store one or more portions of an embedding table in the one or more memory banks, the embedding table including one or more feature vectors. The one or more processor subunits are configured to receive a sparse vector indicator from a host external to the at least one computational memory chip and, based on the received sparse vector indicator and the one or more portions of the embedding table, generate one or more vector sums.

A method includes receiving a series of video segments and providing the series of video segments as input to a first machine learning model to produce text data. The text data is provided as input to a second machine learning model to produce categorized text data that includes a classification indication. The classification indication is added to metadata of the video segment, and the categorized text data is provided as input to a third machine learning model to produce a semantic vector. The method also includes causing the video segment and the metadata that includes the classification indication to be stored at a location of a database based on the semantic vector, the database being configured to be searched based on a search query associated with the semantic vector.

Method and apparatus for video processing. A plurality of videos are accessed. An index is generated based on one or more key frames and one or more video features extracted from the plurality of video. A user input is processed to identify user intentions. The index is filtered to identify a plurality of candidate key frames based on the user intentions. A recommended video is generated based on the plurality of candidate key frames, further comprising selecting one or more additional frames based on processing the plurality of candidate key frames using one or more clustering operations, and aggregating the plurality of candidate key frames and the one or more additional frames.

The present disclosure provides methods and apparatuses for image processing and image classification. In one embodiment, the method for image processing comprises: receiving an image; obtaining a first classification result for the image based on a classification model; processing the image for classification based on a preset process, and providing a processing result into a re-ranking model to obtain a second classification result for the image; and determining a target classification result for the image, based on the first classification result and the second classification result.

The invention relates to the technical field of Internet video content distribution and distribution, in particular to a micro and short play content distribution and distribution system. The system comprises a content management module, a user portrait module, a copyright protection module, a commercial realization module and a data analysis module, the content management module realizes efficient uploading, auditing, storage and intelligent distribution of a movie, and content retrieval and popularization efficiency is improved through accurate classification and tagging management; the user portrait module utilizes data preprocessing, feature extraction and clustering analysis to construct an accurate user model, and user experience is improved based on a personalized recommendation algorithm; the copyright protection module comprehensively guarantees content security through request verification, URL encryption, anti-theft link and a dynamic access control mechanism; the commercial realization module adopts various strategies to improve the platform profitability; and the data analysis module optimizes a content recommendation and distribution strategy through big data collection and mining, and improves the overall operation effect. According to the invention, high-quality experience is provided for the user while the rights and interests of the content creator are ensured.

The invention belongs to the technical field of automatic driving test, and particularly relates to a multi-mode accident scene library construction method for end-to-end automatic driving test. The method specifically comprises the following steps: step 1, extracting text information based on an MCAT-BiLSTM-CRF algorithm; 2, designing the ontology architecture, and storing accident scene information by using a knowledge graph to obtain an accident text knowledge base; step 3, derivative expansion is carried out on the accident scene element combination based on a HyCon-Sg-Net algorithm; step 4, performing enhanced fine tuning on the Open-Sora model to realize modal conversion from the text knowledge base to the accident video database so as to construct a multi-modal accident scene library; the method can be used for testing the performance of an end-to-end automatic driving automobile in an extreme accident scene, and the adaptive capacity of an automatic driving algorithm to the extreme accident scene is remarkably improved.

The present application provides a video segmentation method, a server, a storage medium, and a program product. In the method of the present application, video data to be segmented is segmented into multiple data segments, unimodal features of the data segments, including text features of a text modality and visual features of a visual modality, are respectively extracted by means of a video topics segmentation model, and then the text features and visual features of the data segments are fused, so that the fusion of multimodal information can be performed at the intermediate representation level, the relationship and interaction between different modalities can be better captured, and higher-quality multimodal fusion features of the data segments are obtained. Furthermore, on the basis of the multimodal fusion features of the data segments, whether the data segments are topic boundaries is predicted, so that the topic boundaries of the video data can be accurately predicted, improving the accuracy of topic boundary recognition, thereby improving the accuracy and quality of video topics segmentation results.

Embodiments of the invention provide a retrieval result determination method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining an input image; determining an open set target of the input image, and extracting a category label of the open set target; generating a content description text for the category label, and obtaining text features of the content description text; acquiring position information of the open set target; generating an unstructured visual coding feature and an attribute feature based on the category label and the position information; an open set target index is constructed through the text features, the unstructured visual coding features and the attribute features; when the retrieval request is obtained, the retrieval result corresponding to the retrieval request is determined based on the open set target index, and therefore the situation that when a new target type appears every time, a model needs to be retrained is avoided, and retrieval for the new target type can be achieved.

The invention discloses a video retrieval method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring each current to-be-retrieved video; dividing each video to be retrieved into a plurality of split videos; for each target retrieval dimension, obtaining a text description of the target retrieval dimension of each split video; wherein the target retrieval dimension comprises a subtitle dimension, an audio dimension and a visual dimension; the text description of the visual dimension of one split video is combined by the context text description of the split video and the text description of each key frame of the split video; combining the text description of the target retrieval dimension of each split video with a retrieval problem, and performing retrieval enhancement to obtain a plurality of retrieval videos based on the target retrieval dimension; weighting the text description of each target retrieval dimension of each retrieval video to obtain a plurality of sub-mirror multi-dimensional text descriptions; and combining the multi-dimensional text description of each sub-mirror with a retrieval problem to carry out retrieval enhancement to obtain a final retrieval result.

The invention provides a public opinion video tag aggregation method and system based on artificial intelligence, and relates to the technical field of artificial intelligence. Comprising the following steps: acquiring pictures and text information in a short video, and performing semantic alignment; different large language models are adopted to generate preliminary labels for the pictures and the text information after semantic alignment; clustering the pictures and the texts after semantic alignment to obtain clusters; calculating the labeling probability of each primary label type in the current cluster by each large language model, and selecting the primary label with the highest probability sum as a clustering label of the current cluster; calculating the reliability weight of each large language model in the current cluster based on the clustering label of the current cluster; and based on the reliability weight, calculating the weighted support degree of all the large language models to different preliminary label types of each piece of data in the current cluster, calculating the weighted label of the current data, and further determining a final label. According to the method, the condition of few labels or no labels can be effectively processed, and the manual workload is greatly reduced.

A video tag processing method, comprising: determining a video to be processed, and performing extraction on the video to obtain modality information of at least two modalities (202); on the basis of the modality information of the at least two modalities, respectively performing cross retrieval according to feature dimensions of the at least two modalities, so as to obtain respective multi-modal retrieval tags of the modality information of the at least two modalities (204); on the basis of the respective multi-modal retrieval tags of the modality information of the at least two modalities, determining at least one candidate tag for the video (206); and on the basis of the at least one candidate tag and the modality information of the at least two modalities, performing tag prediction, so as to obtain a video tag for the video (208).