36 results about "Video annotation" patented technology

The invention provides an education video content intelligent analysis and labeling system based on deep learning, and belongs to the technical field of video analysis, and the system comprises a data processing module which is used for extracting a plurality of scene features based on preprocessed education video data through a pre-trained deep learning model; the feature screening module inputs the extracted scene features into a feature fusion network, and performs feature screening and fusion based on a preset condition to obtain fused features; the content reasoning module is used for constructing a final graph structure of the video content through a graph neural network based on the fusion features, performing semantic reasoning in combination with a preset teaching logic rule, and generating semantic description of the video content; the annotation and optimization module is used for generating a video annotation through a reinforcement learning model based on a knowledge graph and semantic description to obtain a final annotation result; and the storage and retrieval module is used for associatively storing the final labeling result and the preprocessed education video data, and constructing a retrieval index. And the accuracy and the intelligent level of the labeling result are improved.

The invention relates to a video generation method, which comprises the steps of collecting video associated information input by a user based on a first operation of the user, determining video copywriting information based on a second operation of the user, performing preset title processing on the video copywriting information to generate at least one video title, and performing multi-dimensional classification on the video copywriting information, generating video annotation information, determining a target video title based on a third operation of the user, and generating a target video according to the target video title, the video annotation information and the video material selected by the user. By collecting the real intention of the user for the video, performing multi-dimensional classification labeling on the video text content and associating multiple video feature information, the content of the generated video is close to the real intention of the user, and the picture is vivid and deeply impressed by people, so that the high quality of video generation is ensured.

Systems and methods for annotating and tracking objects in a video are described herein. The methods include operating at least one processor to: receive, from at least one image device proximal to the manufacturing device, a sequence of frames of a video showing the plurality of parts within the manufacturing device; receive at least one annotated frame having labelling of a subset of parts of the plurality of parts in a plurality of frames of the video, the annotated frame being video annotation data; apply the video annotation data as input to a propagation algorithm to annotate an additional subset of parts of the plurality of parts within the frames of the video, the additional annotated frames being additional video annotation data; apply a segmentation model to the additional video annotation data to generate image segmentation masks of each of the parts, the image segmentation masks being trained segmentation model output data; and apply an object detection model to the trained segmentation model output tracking data to get a fine-tuned object detection model to detect and track the parts.

Multiple video files that are captured by calibrated imaging devices may be annotated based on a single annotation of an image frame of one of the video files. An operator may enter an annotation to an image frame via a user interface, and the annotation may be replicated from the image frame to other image frames that were captured at the same time and are included in other video files. Annotations may be updated by the operator and / or tracked in subsequent image frames. Predicted locations of the annotations in subsequent image frames within each of the video files may be determined, e.g., by a tracker, and a confidence level associated with any of the annotations may be calculated. Where the confidence level falls below a predetermined threshold, the operator may be prompted to delete or update the annotation, or the annotation may be deleted.

The embodiment of the invention provides a small sample scene-oriented image and video marking system, which comprises a client and a server, and is characterized in that the client is used for requesting an image file, marking a target marking detection frame and a key point in a picture or marking an event in a video, and marking the target marking detection frame and the key point in the image or marking an event in the video after the marking is completed; uploading the labeled image file to a server side in a set labeling format; the server side is used for managing an image data set, a video data set and annotation data, monitoring and controlling annotation quality of the client side, calculating user scores and conducting data distribution according to rules, and annotation quality detection, a data distribution strategy, an annotation sampling inspection strategy and a data augmentation algorithm are included. The method is advantaged in that improvement of data labeling quality is facilitated, a neural network training overfitting problem caused by too little image data is solved, an effect of obtaining large-batch diversified image data from small-batch image data is realized, and related technical research of subsequent neural network target detection and event detection is supported.

The embodiment of the invention provides a surgical robot and a video labeling method thereof, and the surgical robot comprises a patient surgical platform, a doctor main console, an electronic equipment cart and a processor, and the patient surgical platform comprises a surgical instrument for executing a surgery and an endoscope for obtaining a surgical environment. The doctor main console is connected with the patient operation platform through a network, and the doctor main console is used for controlling the operation instruments and the endoscope to move. The surgical robot and the video labeling method thereof obtain a target object of a labeling operation when receiving the labeling operation on a surgical environment image output by an endoscope from a patient surgical platform, a doctor main console or an electronic equipment cart, and perform video labeling on the surgical environment image when determining that the target object is a surgical instrument. The state of the target object is marked in the surgical environment image, so that the state of the surgical instrument can be conveniently and timely acquired in the surgery, the real-time feedback of the surgical state is realized, and the surgical operation accuracy and the safety of the remote surgery can be improved.

The invention discloses a video labeling method, system and device and a storage medium. The method comprises the following steps: labeling a start key frame and an end key frame in a video; marking the target articles in the start key frame and the end key frame; forming a sequence frame by all video frames between the start key frame and the end key frame, and acquiring positioning information of a target object in the sequence frame by using a video labeling device; and associating the positioning information of the same target article in the start key frame, the sequence frame and the end key frame to obtain the movement track and the size change of the target article, and further converting the movement track and the size change into a labeling result of the target article. According to the method, the precision and efficiency of video labeling can be improved, the cost is reduced, and the method can flexibly adapt to the dynamic and continuous characteristics of video data.

The invention relates to a video description method and system based on annotation quality and reward deviation, and a medium. The method comprises the following steps: extracting video features of a target video; coding the video features by using a hierarchical visual network to obtain multi-level video coding features; decoding the multi-level video coding features through a video description generation model to obtain a video description result; the step of training the video description generation model comprises the substeps of comprehensively evaluating the video annotation quality through a comprehensive scoring module, encouraging the video description generation model to learn towards high-quality annotations according to a comprehensive evaluation score, and inhibiting noise interference introduced by low-quality annotations; and calculating a deviation reward of each video description based on a reinforcement learning algorithm through a self-review module, estimating an expected reward income of each prediction description according to the deviation reward, and guiding the video description generation model to carry out optimization through the expected reward income. Compared with the prior art, the accuracy and semantic richness of video description are improved.

The invention provides a video labeling method, device and equipment and a computer program product, and relates to the technical field of video processing. The video labeling method comprises the following steps: acquiring a target video; labeling the target video based on a preliminary labeling strategy to obtain video labeling data, the preliminary labeling strategy comprising a labeling tool and / or configuration information for performing video labeling; performing quality evaluation on the video annotation data to determine a quality evaluation result; analyzing problems existing in the video annotation data based on the quality evaluation result, and generating an optimized annotation strategy based on the problems existing in the video annotation data, the optimized annotation strategy comprising an optimized annotation tool and / or configuration information; and marking the target video based on the optimized marking strategy, and updating the video marking data.

This application provides a video annotation method, electronic device, and storage medium. The method includes: acquiring the original video and user-annotated information for the first frame of the original video; generating an initial contour mask based on the SAM3 model and the first frame and its annotation information; determining whether the number of frames in the original video exceeds a preset frame count threshold; if so, dividing the original video into multiple sub-videos; performing temporal propagation annotation on each sub-video sequentially based on the initial contour mask to obtain the initial contour mask for each frame in the original video; and generating the annotation result of the original video based on the original video and the initial contour masks for each frame in the original video. This application generates the initial contour mask based on the SAM3 model, requiring only minimal human interaction to automatically locate the target and identify its state, significantly reducing the workload of manual frame-by-frame annotation and lowering labor costs.

The embodiment of the invention relates to the technical field of video annotation, and discloses a front-end video playing page picture frame method based on a Vue framework. According to the video playing page diversified picture frame technology based on interaction of the Vue frame, the U I frame and the picture frame plug-in, the picture frame is performed on the video to be played in the U I frame, so that accurate picture frame in the video page is realized. Besides, through a method of acquiring historical echo data of the camera, an area needing a picture frame can be selected more conveniently and quickly, selection and searching time is reduced, correctness and availability of the historical echo data to the current label are further determined through correctness judgment of the echo data, picture frame time and resource occupation are saved, and the method is suitable for popularization and application. And the picture frame can work more quickly and accurately.

A customized video stream is provided. A second content is searched for that includes the plurality of entities, and a plurality of user data associated with the social media account. A hierarchy of additional content is generated based, at least in part, on the second content that is associated with the plurality of entities, and the user data associated with the social media account, wherein the hierarchy of additional content contains a plurality of second content. A customized video stream is generated based, at least in part, on the video stream and the hierarchy of additional content. The customized video stream is populated on a display screen of the user device, wherein the video stream is displayed on the display screen of the user device, and the hierarchy of additional content is displayed in a portion of the display screen and displayed concurrently with the video stream.

The invention discloses a video annotation method and device, a storage medium, electronic equipment and a program product, and the method comprises the steps: responding to an annotation instruction triggered by a target time axis corresponding to a target element contained in a plurality of videos, which is received through a visual interaction interface, in a process of synchronously playing the videos of the same scene, which are synchronously collected from a plurality of different visual angles; based on first frame information and second frame information corresponding to a target time axis, video clips corresponding to the first frame information and the second frame information are determined in multiple videos, and the labeling instruction comprises attribute information of a target element; according to the method, the attribute information is marked for the target element of each frame of image in the video clip to obtain the marked video clip, and the time axis is utilized to realize simultaneous marking of the videos synchronously acquired at multiple different visual angles, so that the efficiency, accuracy and consistency of marking the videos at multiple visual angles can be improved.

The invention discloses a video annotation method, system and device based on continuous insertion and a medium, and relates to the technical field of data processing, and the method comprises the steps: obtaining a target video and a preset annotation object set, obtaining a time axis, and obtaining a current frame, a historical frame and a rear frame; generating a first recognition result, if the first recognition result is matched with the preset labeling object set, generating a second recognition result, and obtaining a first difference degree; if the first difference degree exceeds a first preset threshold value, inserting initial annotation content into the current frame of the ith time point, performing target identification on the frame behind the ith time point, generating a third identification result, and obtaining a second difference degree according to the third identification result and the first identification result; and if the second difference degree exceeds a second preset threshold value, updating the initial annotation content according to a third identification result, and continuously performing frame-by-frame identification on the target video end point and updating the annotation. The method has the advantages of accurate judgment, self-adaptive adjustment and light-weight operation.

DEVICES, SYSTEMS, AND METHODS FOR GAMING VIDEO ANNOTATION The invention relates to devices, systems, and methods for gamifying the video annotation process to maintain engagement in the annotation process and increase the quality and quantity of annotated dataset. Methods for gamifying the annotation of gestures in a video include determining gamified feedback based on input and presenting the gamified feedback to an operator to keep the operator engaged in the annotation process. Methods for gamifying the annotation of self-perceived gestures in a video by a subject are capable of being performed by the subject without requiring an operator, in which case the method captures an aspect of the subject's experience, and the gamification maintains or increases the subject's engagement in the annotation process.Annotated datasets are used to train artificial intelligence and machine learning systems for the automated detection and characterization of subjects' gestures and self-perception gestures. Figure for the abstract: none.

This application relates to a video annotation method, apparatus, device, medium, and product. The method includes: acquiring media information of a video to be annotated, the media information including image data and descriptive text of the video; using a video classification model to determine the confidence level of each category label mapped from the image data to a category label set, and identifying category labels with confidence levels exceeding a preset confidence threshold as target category labels; using category labels with confidence levels below the confidence threshold as pending category labels, and verifying whether the pending category labels are target category labels based on the descriptive text; and annotating the video to be annotated using the target category labels. By using the descriptive text of the video to be annotated to verify and confirm the low-confidence category labels identified in the video classification model, the accuracy of video annotation can be improved, overcoming the long-tail effect caused by insufficient training samples for category labels, and also improving the utilization efficiency of video information resources.

A video annotation method, comprising: by means of key frame matching, automatically determining a key frame comprising a target object from a video to be annotated, and segmenting said video by using the key frame as a video segmentation point; and separately performing target tracking processing on two image sequences located before and after the key frame in the video. Thus, without changing the working principle of the target tracking algorithm, the video annotation method does not need to separately determine whether the first frame in the video to be annotated comprises the target object, thereby achieving the automation of video annotation and effectively improving the efficiency and accuracy of video annotation.

1. The name of the design product: video annotation graphical user interface of electronic equipment. 2. The use of the design product: for an electronic device. 3. The design points of the design product: in the graphical user interface. 4. The picture or photo that best indicates the design points: design 1 front view. 5. Design 1 is designated as the basic design. 6. The use of the graphical user interface: for video annotation and quality inspection; in the middle of the interface in the design 1 to design 5 front view, the mark area can be added in the coating area and marked; in the design 6 front view, the mark can be edited in the click pop-up box. 7. Other circumstances that need to be explained: the coating content is the content picture.

This invention discloses a method and system for annotating highway event videos based on structured multi-turn question answering. The method includes quality assessment and video enhancement of the surveillance video stream, determination of the event observation time and segmentation and cropping of the surveillance video stream, identification of scene attributes for each surveillance video segment, construction of a question-answering mapping model to obtain annotation questions and generate JSON-formatted annotation templates, pre-annotation on the annotation templates using a pre-trained video understanding model, human-computer collaborative annotation to generate annotation text based on the pre-annotation results and quality assessment results, storage of the annotation text in a preset file directory, and writing the basic metadata and annotation text storage paths of the same surveillance video segment into a database table. The database table is used for association retrieval of surveillance video segments and annotation text. This method not only improves the efficiency and accuracy of highway event video annotation but also has good interpretability and can be directly applied to highway event video annotation systems.

This application provides a fall detection method, electronic device, and program product based on human posture image prediction. The method includes: acquiring video frames; converting the video frames into human posture images using a preset posture estimation algorithm; predicting the predicted posture image for the next moment based on historical human posture images using a constraint-based generative adversarial network (CGN); using the human posture image corresponding to the video frame of the target person captured at the next moment as the real posture image; determining the individual posture features of the target person based on the real posture image using a preset individual posture recognition strategy; determining the error features between the predicted posture image and the real posture image based on the loss function of the CGN; and determining the detection result representing whether the target person has fallen based on the individual posture features and the error features. This improves upon the problems of weak generalization ability, unstable detection results, and cumbersome video annotation in traditional fall detection methods.

The invention discloses a commodity acquisition video data tracking method, equipment and a medium, relates to the technical field of video data tracking, and solves the problem that the authenticity of traffic data is difficult to guarantee due to various data types of the traffic data and limitation of a video platform after the current acquisition video is put. The method specifically comprises the following steps: putting a business order video on a video platform, and carrying out content annotation on the business order video to obtain video annotation data of the business order video; performing data monitoring on the business order video to obtain video traffic data of the business order video; analyzing the delivery effect of the business order video according to the video traffic data, and performing cooperation intention re-evaluation on a business order partner and a video platform; according to the method, intelligent analysis of the propaganda effect and benefit corresponding to the commodity video to be picked up is realized on the basis of video data tracking, and the business order video investment and the benefit of the target commodity are comprehensively analyzed and the propaganda strategy of the business order video is adjusted according to the analysis result.

The embodiment of the invention provides a video labeling method and device, electronic equipment, a storage medium and a product, and relates to the technical field of video processing, and the method comprises the steps: obtaining a to-be-labeled video, and carrying out the recognition of the to-be-labeled video, and obtaining a recognition result; generating an attribute description text of each object in the to-be-annotated video and a scene description text of the to-be-annotated video by combining the to-be-annotated video and the corresponding identification result and utilizing the multi-modal large model and the cue word to obtain a to-be-corrected description text; obtaining a correction result obtained by correcting the to-be-corrected description text, and taking the correction result as a final labeling result of the to-be-labeled video; and obtaining a new cue word obtained by correcting the cue word, and adjusting parameters of the multi-modal large model by using a difference between a final labeling result of the to-be-labeled video and the to-be-corrected description text to obtain a new multi-modal large model. In this way, video annotation can be quickly and accurately realized.

The embodiment of the invention discloses a method and device for video annotation, electronic equipment and a storage medium, and the method comprises the steps: extracting a video frame set from a target video; acquiring correction information of each key point in each video frame of the video frame set; based on the correction information and the annotation information of each key point in each video frame of the video frame set, determining whether the annotation information marked by the annotation model is credible; filtering the target video in response to the fact that the labeling information labeled by the labeling model is credible; determining whether the filtering effect meets the requirement or not based on the filtered labeling information of each key point in each video frame of the filtered target video and the labeling information of each key point in each video frame of the target video; and storing the filtered target video in response to the condition that the filtering effect meets the requirement. According to the embodiment of the invention, the video with the high-precision key point position of the target object can be obtained, and the recognition precision of object behavior recognition for the target video can be improved.

The embodiment of the invention provides a method, apparatus, device, medium and product of video annotation, and the method includes: determining a sub-segment to be annotated in a video to be annotated to obtain a target sub-segment; obtaining a first frame annotation result corresponding to a first frame of the target sub-segment; generating an end frame annotation result corresponding to an end frame of the target sub-segment based on the first frame annotation result; generating an annotation result of an intermediate frame of the target sub-segment based on the first frame annotation result and the end frame annotation result, to obtain an annotation result of the target sub-segment to be annotated; and generating a target annotation result of the video to be annotated based on the annotation result of the target sub-segment.

The present application relates to a video generation method, which collects video-related information input by the user based on a first operation of the user, determines video text information based on a second operation of the user, wherein the video text information is converted from the video-related information, performs preset title processing on the video text information to generate at least one video title, performs multi-dimensional classification on the video text information to generate video annotation information, determines a target video title based on a third operation of the user, and generates a target video based on the target video title and video annotation information, as well as the video material selected by the user. By collecting the user's true intention for the video, and multi-dimensionally classifying and annotating the video text content, and associating multiple video feature information, the generated video has content close to the user's true intention, and the picture is vivid and impressive, thus ensuring the high quality of video generation.

The present invention discloses a data processing method and device for video annotation, which comprises: obtaining video information to be processed; the video information to be processed comprises video comment information and video emotion information; performing feature analysis processing on the video comment information in the video information to be processed by using a target text processing model to obtain target comment emotion information; performing feature fusion processing on the target comment emotion information and the video emotion information in the video information to be processed to obtain target video annotation information.

The present disclosure provides a sample data generation method and device, a model training method and device, an electronic device and a storage medium, relates to the technical field of artificial intelligence, in particular to the technical field of video annotation, the technical field of multi-modal model and the technical field of large model. The specific implementation scheme is: obtaining video data, the video data comprising a plurality of video frames and a text description corresponding to each video frame; extracting a scene description and at least one basic event from the text description; based on the scene description, aggregating the basic events associated with semantics to generate at least one trigger event and a monitoring instruction of the trigger event; determining a time interval set of the trigger event occurrence according to at least one time interval of each basic event occurrence in the video data; for each monitoring instruction, based on the time interval set of the trigger event corresponding to the monitoring instruction, state labeling is performed on the video data to obtain sample data.

The application provides a video labeling method, comprising the following steps: extracting a plurality of video frames from an unlabeled video, displaying at least one video frame in the plurality of video frames to a user through a display interface, obtaining a user labeling result of the user labeling the at least one video frame in the display interface, and obtaining a labeling result of other video frames in the plurality of video frames according to the user labeling result. The method makes the user only need to label a key frame once or even only confirm whether the key frame is labeled correctly in the video labeling process, so that the complete labeling result of the video can be obtained, the labeling operation of the user is greatly reduced, and the video labeling efficiency and the use experience of the user are improved.

This application relates to the field of autonomous driving technology, specifically to a video annotation method, intelligent device, and computer-readable storage medium, aiming to solve the technical problem of how to accurately annotate video streams with timestamps. To this end, this application acquires multiple video frames of the video stream to be annotated. Based on each video frame and its timestamp, it acquires the timestamp-based visual coding features of each video. Based on the timestamp-based visual coding features of multiple video frames, it acquires the annotation results of a preset annotation task for the video stream to be annotated. The annotation results include the timestamp of the preset annotation task, enabling accurate acquisition of the temporal information of the video stream to be annotated, effectively improving the accuracy of the temporal information of the visual coding features, and achieving the fusion of the visual coding features and temporal information of the video stream to be annotated. This effectively enhances the video understanding capability of the video stream and obtains more real-time and accurate annotation results for the video stream to be annotated.

This invention relates to the field of image recognition technology, specifically to a short video annotation method and system, comprising the following steps: acquiring frame feature regions, constructing multi-scale mapping, analyzing dynamic levels, matching semantic tags, and generating a temporal annotation sequence. In this invention, by fusing directional difference and color statistics in short video frames, a joint measure of texture and color changes is achieved, improving the accuracy of regional dynamic feature expression. Edge direction histograms and color moment features extracted by multi-scale windows are standardized and screened using cosine similarity to ensure consistency of regional features across multiple scales and reduce interference. The normalized product of centroid displacement and brightness changes constructs a dynamic intensity distribution, enhancing the discriminativeness of motion level division. Semantic matching combines motion parameters and semantic vectors for joint calculation, improving the semantic association and accuracy of tag generation. Temporal reorganization uses trajectory aggregation to achieve temporal continuity of tags, ensuring the uniformity and stability of multi-frame annotation in both spatial and temporal dimensions.