864results about "Video data clustering/classification" patented technology

The present disclosure relates to the technical field of target detection, and particularly relates to a model deployment method, an end-side device and a storage medium, which are used for solving the problem in the related art of the accuracy of a deployed model being low. The method comprises: performing target detection on a video frame image input into a first model, and acquiring a first target detection result and a first confidence; if the first confidence is greater than or equal to a first confidence threshold value, recording the video frame image and the first target detection result as samples in a training set; if the first confidence is less than the first confidence threshold value, performing target detection on the video frame image on the basis of a second model, and recording the video frame image and an acquired second target detection result as samples in the training set; and training the first model on the basis of the training set, and replacing the current first model with a trained first model for subsequent target detection. In this way, the accuracy and model generalization capability of a first model are improved.

A data processing system implements receiving a call requesting a generative model to generate a semantic tree for a source content; constructing a first prompt including the source content and instructions to the model to analyze a semantic structure of the source content and to generate a semantic outline and content chunks of the source content, the semantic outline including one or more topics each connected with one or more of the content chunks, to compute one summary for each of the content chunks, to apply indices to reference each topic node of the semantic tree to one of the topics, and to apply indices to reference each leaf node of the semantic tree to one of the content chunks and the respective summary; providing the first prompt to the model and receiving the semantic tree of the source content; and storing the semantic tree in a database.

A method and system for classifying a video file within an environment in which the file is located and when a file is classified as sensitive, controlling transmission of the file outside the environment. Classifying the video comprises analysing, using at least one machine learning model, the video to recognise any individuals in the video; obtaining a transcript of any speech in the video and generating, using the analysis, obtained transcript and a database of individuals linked to the environment, a labelled transcript which identifies each individual linked to the environment that is in the video. Information about each identified individual may be obtained from a connected database. A first generative AI model generates a text-based summary of the video by using the labelled transcript and information about identified individuals as prompts. A second generative AI model then determines a sensitivity classification of the video using the generated text-based summary.

The invention discloses a digital intelligent teaching recording and broadcasting system and method, and relates to the technical field of data encryption, and the method comprises the steps: collecting teaching videos and audios of a teacher through a high-definition camera and an audio device, carrying out the preprocessing, obtaining the course content, recognizing the key information of the course content through a natural language processing NLP and computer vision technology, and carrying out the recording and broadcasting. The method comprises the following steps: segmenting a video into single knowledge point fragments, performing classified storage, forming a learning resource library, collecting basic information and learning habit data of students, constructing a user portrait, matching learning resources, extracting the learning habit data in the user portrait, performing feature extraction, and obtaining a comprehensive feature vector; constructing a learning state evaluation model by using a convolutional neural network to comprehensively analyze the comprehensive feature vector to obtain a learning state comprehensive score, formulating a playing strategy according to the learning state comprehensive score and matched learning resources, regularly testing knowledge points of students, evaluating the mastering degree of the students, and updating a user portrait according to a result.

Systems, methods, and computer-readable media are disclosed for systems and methods multimodal indexing of video using machine learning. An example method may include deceiving, by a video encoder of an audio-video transformer neural network comprising one or more computer processors coupled to memory, a first frame and a second frame associated with a first segment of a video. The example method may also include receiving, by an audio encoder of the audio-video transformer neural network, an audio spectrogram comprising first audio data associated with the first segment of the video. generating, by the video encoder, a first video embedding. The example method may also include generating, by the audio encoder, a first audio embedding. The example method may also include determining a fusion of the first video embedding and the first audio embedding using a multimodal bottleneck token. The example method may also include determining an output including the first video embedding and the first audio embedding. The example method may also include determining a classification of the first portion of the video based on the output.

Systems and methods for automating the management of a warehouse are disclosed. The method includes monitoring. in real-time, a plurality of items in the warehouse, a plurality of tasks associated with the warehouse, and / or incoming vehicles to the warehouse. The method also includes receiving, via a plurality of sensors, image data and / or video data associated with the plurality of items, the plurality of tasks, and / or the incoming vehicles. The method further includes inputting the image data and / or the video data into a machine learning model to automate the management of the warehouse, wherein the machine learning model has been trained based on training data to generate predictions for the inputted data, and wherein the machine learning model utilizes the predictions for the inputted data for retraining to generate predictions for newly inputted data and improve accuracy of error handling.

The invention discloses a video abstract generation method and system based on feature expression enhancement, and relates to the technical field of computer vision, and the method comprises the steps: obtaining an original video, and extracting a video frame image sequence; a video frame image sequence is input into an OCVF video understanding model, initial features of each input video frame image are extracted through a feature extraction module, key spatio-temporal features of the initial features are extracted through an attention module, and then high-level feature representation is generated through feature mapping and jump connection. The high-level feature representation outputs the importance score of each video frame through a classifier, key frames are determined, and a final video abstract result is generated; wherein the attention module comprises a channel attention layer, an average pooling layer, a position coding layer, a self-attention layer and a space-time attention layer which are arranged in sequence. According to the method, more comprehensive modeling and efficient understanding of the video content can be realized, and more robust and more accurate video abstract generation is realized.

The invention discloses a multi-modal video question-answering method and device based on large language model optimization, and the method employs a CoVGT basic framework, optimizes video feature extraction and text input, and adds a video question-answering model output optimizer. The method comprises the steps of video feature extraction, text feature extraction, feature fusion, preliminary answer prediction, related sample generation, Prompt text generation and final answer generation. According to the method, the answer optimizer is designed by using the learning characteristics in GPT-3 analogy, related sample generation, Prompt text generation and final answer generation, and compared with a basic model, the model accuracy is greatly improved.

The invention relates to the technical field of text video retrieval, in particular to a video retrieval method and system based on large language model semantic driving, and the method comprises the steps: obtaining a query text and a candidate video set; constructing a text video retrieval model; inputting the candidate video set into a text video retrieval model, and obtaining video features of each candidate video through a video feature extractor; after the query text is embedded into a preset prompt statement, inputting the query text and the video features of all the candidate videos into a large language model, and outputting the similarity between the query text and each candidate video; and outputting a video retrieval result according to the similarity between the query text and each candidate video. According to the method, the video features containing dynamic changes and spatial details are constructed, the cross-modal semantic relationship is obtained by using the powerful semantic reasoning ability of the large language model, the cognitive behavior of human retrieval is met, the interpretability of the model is enhanced, and the precision and efficiency of cross-modal text video retrieval are improved.

The invention relates to the technical field of security and protection visualization, in particular to a security and protection visualization display method and system and a storage medium. The method comprises the following steps: acquiring a data stream acquired by security and protection equipment, and compressing the data stream into a unified acquisition frame set; identifying a road boundary point set in the unified collection frame set, and detecting a dynamic target and a static target at each moment in the unified collection frame set; performing semantic annotation according to the motion state change condition of the dynamic target at each moment in the unified collection frame set to obtain a road boundary semantic distribution map; according to the road boundary semantic distribution map, identifying and predicting the motion state change condition of the static target; and predicting an overlapping time point of the motion state of the target based on the motion state change condition of the static target and the motion state change condition of the dynamic target, and transmitting the overlapping time point to communication equipment of the static target through the Internet of Things so as to execute a voice prompt task. According to the invention, the display precision and response efficiency of security and protection visualization can be improved.

A data processing system implements receiving a call requesting a generative model to generate a semantic tree for a source content; constructing a first prompt including the source content and instructions to the model to analyze a semantic structure of the source content and to generate a semantic outline and content chunks of the source content, the semantic outline including one or more topics each connected with one or more of the content chunks, to compute one summary for each of the content chunks, to apply indices to reference each topic node of the semantic tree to one of the topics, and to apply indices to reference each leaf node of the semantic tree to one of the content chunks and the respective summary; providing the first prompt to the model and receiving the semantic tree of the source content; and storing the semantic tree in a database.

The invention provides a video generation method and device, and relates to the technical field of computers, in particular to the technical fields of large models, deep learning, image processing, voice recognition and the like. According to the specific implementation scheme, semantic partitioning and position marking are conducted on a to-be-processed material, and a semantic text block set comprising at least one semantic text block is obtained; based on the key event information, classifying semantic text blocks in the semantic text block set to obtain a key text block set and a non-key text block set; generating a key image sequence based on the key text block set, the to-be-processed material and a video material library; generating a non-key image sequence based on the non-key text block set and a video material library; the target video is generated based on the key image sequence and the non-key image sequence, and the video generation effect is improved.

The invention relates to the technical field of data processing, and discloses a film and television supervision analysis method and system based on data mining and a storage medium. The method comprises the following steps: collecting film and television data in a panoramic manner, and carrying out cross-media transcoding processing to obtain a multi-dimensional data set; performing video semantic extraction, audio emotion recognition and text tendency analysis on the data set to generate a time sequence mark list; scoring the key elements according to social influence, propagation depth and audience acceptability to form a dynamic score table; analyzing the propagation path based on the score table, and constructing an influence map; sorting supervision points according to the atlas, and making an intelligent supervision scheme; the execution deviation is analyzed through effect feedback, and the supervision parameters are optimized. According to the method, multi-dimensional content feature extraction, social influence evaluation and propagation path analysis of the film and television works are realized in a mass multimedia content environment, a precise and differentiated intelligent supervision scheme is formed, and the method has self-optimization and adjustment capabilities at the same time.

Methods, systems, and apparatuses, including computer programs encoded on computer storage media, for receiving a query relating to a data item that includes multiple data item samples and processing the query and the data item to generate a response to the query. In particular, the described techniques include adaptively selecting a subset of the data item samples using a selection neural network conditioned on features of the data item samples and the query. Then processing the subset and query using a downstream task neural network to generate a response to the query. By adaptively selecting the subset of data item samples according to the query, the described techniques generate responses to queries that are more accurate and require less computation resources than would be the case using other techniques.

Systems and methods for tracking inventory items in an area of real space are disclosed. The method includes receiving a signal generated in dependence on sensors. The signal indicates a change to a portion of an image of an area of real space. The method includes, in response to receiving the signal, implementing a trained location detection model to determine, based on inputs, whether an inventory item identified in the portion of the image has changed a position in the area of real space. The method includes implementing a trained item classification model to determine a classification of the inventory item. The method includes updating an inventory database with inventory item data determined in dependence on the classification of the inventory item to provide an updated map of the area of real space as a result of the received signal indicating the change to the portion of the image.

Embodiments are disclosed for performing content authentication. A method of content authentication may include dividing a query video into a plurality of chunks. A feature vector may be generated, using a fingerprinting model, for each chunk from the plurality of chunks. Similar video chunks are identified from a trusted chunk database based on the feature vectors using a multi-chunk search policy. One or more original videos corresponding to the query video are then returned.

The invention discloses a long video positioning method, system and device and a storage medium, which are one-to-one corresponding schemes, in the scheme, a step-by-step positioning mode is adopted, firstly, a highlight video frame is identified through video and text guidance, so that a video clip aligned with a query text is obtained, and the problem that in an existing method, a clip is directly extracted from a long video, so that the video clip cannot be directly extracted is avoided. The problem of lack of overall perception of a long video on the text is solved; further positioning on the basis of obtaining the video clip aligned with the query text to obtain an accurate moment; in the step-by-step positioning mode, each step of positioning further strengthens the relation between the video and the query text, so that a more accurate positioning result can be obtained.

The invention discloses a media information processing method and device, a storage medium and electronic equipment. The method comprises the steps that to-be-retrieved source media information and prompt template information are obtained, the source media information and the prompt template information are input into a target retrieval model, target media information is determined, and the target retrieval model represents a model obtained through joint training according to at least two retrieval task types; a loss function used in the training process of the target retrieval model is a joint loss function, the joint loss function is determined by at least two sub-loss functions in one-to-one correspondence with the at least two retrieval task types, and the target media information represents media information obtained by retrieving the source media information according to the prompt template information. According to the method and the device, the technical problem of relatively low retrieval efficiency of the media information caused by single multi-modal retrieval task is solved.

The invention belongs to the technical field of airport boundary security and protection, and discloses an airport boundary detection method and system based on multiple modes. According to the method, the environment around the enclosure is analyzed in real time through the video personnel detection module, and whether a person approaches the enclosure or not is observed; detecting whether the prompt word and the content of the image are illegal or not, and if yes, not processing; analyzing the content in the image through the image understanding capability, the text understanding capability and the analysis capability of the multi-modal visual large model, and returning an analysis result; judging whether the input text content has the intention of damaging and invading the enclosure or not; if there is no intention of invading and damaging the enclosure, the abnormal alarm module is directly used. According to the method, a large model and traditional deep learning are combined, so that the accuracy and the overall efficiency of a security system can be remarkably improved. The ever-increasing safety requirement of the civil aviation industry is met, and new powerful power is injected for continuous and healthy development of the unmanned aerial vehicle.

The invention discloses a multi-granularity video retrieval method and device based on a multi-modal large model, computer equipment and a readable storage medium, and the method comprises the steps: firstly obtaining video query information input by a user, carrying out the intention recognition to obtain a query field, rewriting the query information and the field to obtain a video query vector, and carrying out the retrieval of the video query vector; and retrieving in a preset retrieval video knowledge base according to the vector and the field, and finally obtaining the target retrieval video content, so as to improve the efficiency and accuracy of video retrieval and adapt to multi-field retrieval requirements.

The invention relates to a label generation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a video identifier and description data of a target video; obtaining classification reference data of the target video according to the video identifier; generating a candidate label set based on the description data and the classification reference data; and inputting the description data and the candidate label set into a preset classification model to obtain a classification label of the target video. According to the embodiment of the invention, the classification label of the target video can be automatically generated, furthermore, the classification label closer to the description data of the target video is selected from the expanded richer candidate label set, the label generation accuracy is improved, and the classification label generation efficiency is improved due to the fact that the classification label is generated based on the same standard. Errors caused by individual differences in manual labeling are avoided, and the consistency of the labels is improved.

The invention discloses an intelligent conference recording and recording method and system. According to the invention, cloud service equipment receives multi-modal information sent by edge equipment; processing the multi-modal information, and generating conference record information in combination with context sensing error correction, content segmentation, a dynamic segmentation strategy and a preset storage strategy; carrying out encryption processing on the conference record information based on the participant role information, and generating target format conference information; processing the target format conference information to generate initial conference scene mode information including a conference outline, a PPT and a mind map; processing the target format conference information and the initial conference scene mode information to generate conference information data after priority ranking and strategy processing; performing semantic analysis, feature fusion and cross-modal association processing on the conference information data to generate target conference scene mode information; and determining a target application program based on the target conference scene mode information, and sending the target format instruction information to the target application program.

Disclosed embodiments include a computational memory system. The computational memory system includes at least one computational memory chip including one or more processor subunits and one or more memory banks formed on a common substrate. The at least one computational memory chip is configured to store one or more portions of an embedding table in the one or more memory banks, the embedding table including one or more feature vectors. The one or more processor subunits are configured to receive a sparse vector indicator from a host external to the at least one computational memory chip and, based on the received sparse vector indicator and the one or more portions of the embedding table, generate one or more vector sums.

The present invention discloses a multimedia video stream management system and method based on artificial intelligence, and relates to the technical field of video stream label processing. The present invention combs the reference element information based on which the AI ​​module divides the video label types of any video stream, and extracts a number of neighboring labels based on data analysis for the phenomenon of neighboring judgment when the AI ​​module performs differential classification. The present invention monitors whether there is data deviation in the classification result output by the AI ​​module within the range of video labels that are mutually neighboring labels, thereby providing a judgment basis for whether the AI ​​module needs to be optimized. The present invention can realize adaptive monitoring of the performance of the AI ​​module in the field of multimedia video stream label processing, and effectively improves the role of AI in the discoverability of multimedia video content and user experience.

A method includes receiving a series of video segments and providing the series of video segments as input to a first machine learning model to produce text data. The text data is provided as input to a second machine learning model to produce categorized text data that includes a classification indication. The classification indication is added to metadata of the video segment, and the categorized text data is provided as input to a third machine learning model to produce a semantic vector. The method also includes causing the video segment and the metadata that includes the classification indication to be stored at a location of a database based on the semantic vector, the database being configured to be searched based on a search query associated with the semantic vector.

Method and apparatus for video processing. A plurality of videos are accessed. An index is generated based on one or more key frames and one or more video features extracted from the plurality of video. A user input is processed to identify user intentions. The index is filtered to identify a plurality of candidate key frames based on the user intentions. A recommended video is generated based on the plurality of candidate key frames, further comprising selecting one or more additional frames based on processing the plurality of candidate key frames using one or more clustering operations, and aggregating the plurality of candidate key frames and the one or more additional frames.

The present disclosure provides methods and apparatuses for image processing and image classification. In one embodiment, the method for image processing comprises: receiving an image; obtaining a first classification result for the image based on a classification model; processing the image for classification based on a preset process, and providing a processing result into a re-ranking model to obtain a second classification result for the image; and determining a target classification result for the image, based on the first classification result and the second classification result.

The invention relates to the technical field of Internet video content distribution and distribution, in particular to a micro and short play content distribution and distribution system. The system comprises a content management module, a user portrait module, a copyright protection module, a commercial realization module and a data analysis module, the content management module realizes efficient uploading, auditing, storage and intelligent distribution of a movie, and content retrieval and popularization efficiency is improved through accurate classification and tagging management; the user portrait module utilizes data preprocessing, feature extraction and clustering analysis to construct an accurate user model, and user experience is improved based on a personalized recommendation algorithm; the copyright protection module comprehensively guarantees content security through request verification, URL encryption, anti-theft link and a dynamic access control mechanism; the commercial realization module adopts various strategies to improve the platform profitability; and the data analysis module optimizes a content recommendation and distribution strategy through big data collection and mining, and improves the overall operation effect. According to the invention, high-quality experience is provided for the user while the rights and interests of the content creator are ensured.

Video classification accuracy can be improved by utilizing multiple features. Classification based on a combination of an image classification model, an audio classification model, and a textual description classification model may improve classification. The image classification result is based on an image feature of the image frame. The audio classification result is based on an audio feature of the audio. The textual description classification result is based on a text feature of the textual description information. A target classification result of the target video is determined based on to the image classification result, the audio classification result, and the textual classification result.

The invention belongs to the technical field of automatic driving test, and particularly relates to a multi-mode accident scene library construction method for end-to-end automatic driving test. The method specifically comprises the following steps: step 1, extracting text information based on an MCAT-BiLSTM-CRF algorithm; 2, designing the ontology architecture, and storing accident scene information by using a knowledge graph to obtain an accident text knowledge base; step 3, derivative expansion is carried out on the accident scene element combination based on a HyCon-Sg-Net algorithm; step 4, performing enhanced fine tuning on the Open-Sora model to realize modal conversion from the text knowledge base to the accident video database so as to construct a multi-modal accident scene library; the method can be used for testing the performance of an end-to-end automatic driving automobile in an extreme accident scene, and the adaptive capacity of an automatic driving algorithm to the extreme accident scene is remarkably improved.