A video data structured processing method and system for intelligent video generation
Through preprocessing and object detection of video sources, combined with grid division and trajectory tracking, the video data characteristics are stored using adjacency tables, which solves the complexity and diversity of video data structured processing and realizes efficient video data storage and calculation Efficiency, suitable for smart video generation.
Patent Information
- Application Number
- CN202411107811.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-13
AI Technical Summary
The three-dimensional characteristics and diversity dynamics of video data make its structured processing complex, and requires high flexibility, generalization capabilities, real-time and interactiveness. It is difficult for the existing technology to effectively capture the spatial and temporal characteristics of videos, while taking into account computing efficiency and storage costs. and network adaptability.
By acquiring the video source for preprocessing and segmentation, performing the first and second object detection, obtaining the target attribute characteristics and trajectory characteristics, and storing these characteristics using an adjacency table to realize the structured processing of video data. The specific steps include grid division, target center point determination, attribute feature parameter recognition, trajectory tracking and traceability judgment, and finally, using a linked list to store the feature set.
It improves the structured storage of video data, provides data guarantee for smart video generation, improves computing efficiency and storage costs, and meets network adaptability needs.
Smart Images

Figure CN118656516B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to image recognition technology, and in particular to a video data structured processing method and system for intelligent video generation. Background Art
[0002] The development of image recognition technology has brought revolutionary changes to the generation and processing of video data. By utilizing the automatic feature extraction and powerful pattern recognition capabilities of deep neural networks, new video generation models such as generative adversarial networks and variational autoencoders can create high-quality, realistic video content while optimizing video coding efficiency. By learning the spatiotemporal correlations between video frames, these models can not only generate continuous and smooth video sequences, but also realize intelligent video editing, style transfer, and scene synthesis.
[0003] However, the structured processing of video data still faces many challenges. First, the three-dimensional characteristics of video data make its structured processing more complicated than that of images. Second, the diversity and dynamics of video content require the processing method to have a high degree of flexibility and generalization. Furthermore, the real-time and interactive requirements also put forward higher standards for the response speed of video processing and the user interaction experience. Therefore, developing a video data structured processing method that can effectively capture the spatiotemporal characteristics of video while taking into account computational efficiency, storage cost and network adaptability is an important direction of current research. Summary of the invention
[0004] The present invention provides a video data structured processing method for intelligent video generation, comprising:
[0005] S110, obtaining a video source that needs to be structured, and preprocessing and segmenting the video source;
[0006] S120, performing a first target detection on the image sequence to obtain a target attribute feature set;
[0007] S130, performing a second target detection on the image sequence to obtain a target trajectory feature set;
[0008] S140 , storing the target's attribute feature set and trajectory feature set through an adjacency table to achieve structured processing of video data.
[0009] The video data structured processing method for intelligent video generation as described above is characterized in that the specific method of the first target detection includes the following sub-steps:
[0010] Grid the sequence images and divide the target range and boundary of the grid for the same target;
[0011] The target center point is determined based on the target boundary, the attribute feature parameters of the target are determined using image recognition technology, and a target attribute feature set is established.
[0012] The video data structured processing method for intelligent video generation as described above is characterized in that the second target detection method specifically includes the following sub-steps:
[0013] Track the target according to the position change of the center point in the image sequence and determine the trackability of the target;
[0014] If the target's trackability is true, the target's trajectory characteristic parameters are recorded. If the target's trackability is false, the target is considered lost and is not recorded.
[0015] The video data structured processing method for intelligent video generation as described above is characterized in that the adjacency list, as a commonly used data structure, is suitable for storing the attribute feature set and trajectory feature set of the target. The attribute feature set of the target is used as the header node, and the trajectory feature set of the target is used as the edge node. After the header node is connected with a linked list, this storage method structuredly represents the relationship between the attributes and running trajectories of each target. When regenerating the intelligent video, it is only necessary to modify the value of the header node or edge node of a certain target.
[0016] The present invention also provides a video data structured processing system for intelligent video generation, including: a video source collection and decoding module, a first target detection module, a second target detection module, and a structured processing module.
[0017] Video source collection and decoding module: obtains the video source that needs to be structured, and pre-processes and segments the video source;
[0018] First target detection module: performs the first target detection on the image sequence to obtain a set of target attribute features;
[0019] Second target detection module: performs a second target detection on the image sequence to obtain a target trajectory feature set;
[0020] Structured processing module: The target's attribute feature set and trajectory feature set are stored in an adjacency table to achieve structured processing of video data.
[0021] The video data structured processing system for intelligent video generation as described above is characterized in that the first target detection module specifically includes:
[0022] Division submodule: divide the sequence images into grids, and divide the target range and boundary of the grid under the same target;
[0023] Attribute feature set establishment submodule: determine the target center point based on the target boundary, identify the target attribute feature parameters, and establish the target attribute feature set.
[0024] The video data structured processing system for intelligent video generation as described above is characterized in that the second target detection module specifically includes:
[0025] Trackability judgment submodule: tracks the target trajectory according to the position change of the center point in the image sequence and judges the trackability of the target;
[0026] Trajectory characteristic parameter establishment submodule: If the target's trackability is true, the target's trajectory characteristic parameters are recorded; if the target's trackability is false, the target is considered lost and is not recorded.
[0027] A video data structured processing system for intelligent video generation as described above is characterized in that the adjacency list, as a commonly used data structure, is suitable for storing the attribute feature set and trajectory feature set of a target. The attribute feature set of the target is used as a header node, and the trajectory feature set of the target is used as an edge node. After the header node is connected with a linked list, this storage method structuredly represents the relationship between the attributes and running trajectories of each target. When regenerating an intelligent video, it is only necessary to modify the value of the header node or edge node of a certain target.
[0028] The beneficial effects achieved by the present invention are as follows: the structuring of video data storage is improved through image recognition technology, providing data guarantee for intelligent video generation. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.
[0030] Figure 1 It is a flow chart of a video data structured processing method for intelligent video generation provided in Example 1 of the present application;
[0031] Figure 2 It is a schematic diagram of a video data structured processing system for intelligent video generation provided in Embodiment 2 of the present invention. DETAILED DESCRIPTION
[0032] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention. Embodiment 1
[0033] like Figure 1 As shown, Embodiment 1 of the present application provides a method for structuring video data for intelligent video generation.
[0034] Step S110: obtaining a video source that needs to be structured, and preprocessing and segmenting the video source;
[0035] Get the video source that needs to be structured. There are many sources of video sources, including major video platforms, video material libraries, user-uploaded videos, and live broadcast sources. The collected videos include virtual human commentary videos, live broadcast editing videos, and user-recorded videos. Preprocess the collected video sources, including format conversion, resolution adjustment, and denoising. Specifically, the formula is used: Preprocess the video source, including Represents the preprocessed video source, Represents the original video, Represents a preprocessing operation.
[0036] The preprocessed video source is divided into multiple segments according to the timeline to form a picture sequence, and the start and end time of each segment are identified. Specifically, the formula is used: Cut the preprocessed video source, represents the set of segments after segmentation, , Respectively represent The start and end time of each segment, Indicates the subscript of the fragment, with a value of 1-n.
[0037] Step S120: performing a first target detection on the image sequence to obtain a target attribute feature set;
[0038] Perform the first target detection on the image sequence. The main purpose of the first target detection is to lock the target of interest from a single frame of the image and extract the attribute features of the target through image recognition technology. The attribute features of the target include target type, target scene, target action, target emotion, target text and voice, target background music and sound effects, target style and target quality assessment. The first target detection specifically includes the following sub-steps:
[0039] Step S121: gridding the image sequence, and dividing the target range and boundary of the same target into grids;
[0040] The sequence images are divided into N×N grids. The first grid is used as the initial target, and its four edges are used as the initial target boundaries. The credibility of its neighboring grids as the same target is calculated. If the credibility exceeds the credibility threshold, the neighboring grids are divided into the target range, and the target boundary is expanded accordingly. Until the credibility of all neighboring grids in the target range as the same target does not exceed the credibility threshold, the range and boundary division of the target is completed, and the next target is divided until all grid divisions are completed.
[0041] The credibility of the same target is determined by the difference in image pixel values and structural similarity. The smaller the difference in image pixel values, the higher the credibility of the same target, which reflects the average pixel difference between the two images as a whole; the higher the structural similarity of the images, the higher the credibility of the same target. When the two images are highly similar in brightness, contrast and structure, the structural similarity value is close to 1, and when they are not similar, the structural similarity value is close to 0. Specifically, the formula is used: Calculate the credibility of the same target, where Indicates the credibility of the same target, The influencing factor representing the difference in image pixel values, represents the impact factor of structural similarity, and Respectively represent the local parameters of the two images, and Represents the height and width of the image in pixels, and Represents the two images at pixel coordinates The pixel value on and Respectively represent images and The local brightness average, and Representing images and The local variance of , a higher variance means the image has a higher contrast, Representing images and The local covariance of the two images shows the same brightness change trend at the same position. A higher value of indicates similar structural information. and Represents a constant to avoid zero denominator.
[0042] Specifically, the formula is used: represents a cyclic process, where Indicates calling the same target credibility calculation formula, Represents the current target range image, Indicates the location of the current initial target range image, The value of is 1-N×N, represents the neighboring grid image, represents the neighboring grid direction, where Indicates the north, Indicates the south, Indicates the West, Indicates the east, and the values of the four directions are all less than the number of grid cells N. Represents the credible threshold, and uses the ternary operator to determine whether the credibility of the same target is greater than the threshold. If it is greater than the threshold, the current target range image is merged with the adjacent grid image, otherwise the original target range is maintained. The cycle continues until the adjacent grid image Until all the merge conditions are not met, the target range at this time is the final range of the target, and the boundary at this time is the final boundary of the target. ++, continue to divide the next target until all targets are divided.
[0043] Step S122: determining the target center point according to the target boundary, identifying the attribute characteristic parameters of the target, and establishing a target attribute characteristic set;
[0044] 1. Determine the target center point based on the target boundary;
[0045] The target center point is determined according to the target boundary. Specifically, the formula is used: , calculate the coordinates of the target center point, where represents the center point coordinates of the i-th target, , , , Respectively represent the west boundary, east boundary, north boundary and south boundary of the i-th target.
[0046] 2. Identify the attribute characteristic parameters of the target;
[0047] Image recognition technology is used to determine the attribute characteristic parameters of the target, which include target type, target scene, target action, target emotion, target text and voice, target background music and sound effects, target style and target quality assessment.
[0048] The target types are divided into objects, humans and animals. Specifically, the formula is used: Identify faces in videos and analyze their expressions, including Indicates The faces detected in the target, Represents the set of expressions corresponding to the face, Represents the current target image, represents the face recognition algorithm, Represents the expression analysis algorithm; the formula used is: Identify animals in videos, where Indicates The type of animal in the clip, Represents the animal recognition algorithm; using the formula: Identify objects in a video, where Indicates objects in the fragment, Represents an object detection algorithm.
[0049] The target scenes include indoor, outdoor, day and night, using the formula: Perform target scene recognition, where Indicates The category of the scene, Represents the scene detection algorithm.
[0050] The target action represents the actions and behaviors of the characters in the video, including walking, running, jumping, etc., using the formula: Perform target action recognition, where Indicates The set of actions in a fragment, Represents the action detection algorithm.
[0051] The target emotion represents the emotional expressions in the video, including excitement, calmness, tension, etc., using the formula: Target emotion recognition is performed, where Indicates The emotional characteristics of the fragments, Represents a sentiment analysis algorithm.
[0052] The target text and speech recognition means recognizing the speech in the video, converting it into text, and extracting keywords using the formula: Perform target text and grammar recognition, where Indicates The text content of the fragment, represents the set of keywords extracted from the text, represents the text recognition algorithm, Represents the keyword extraction algorithm.
[0053] The target background music and sound effects represent the background music type and sound effect characteristics in the identified video. Specifically, the formula is used: Extract target background music and sound effects, Indicates The background music and sound effects characteristics of each clip, Represents the background music and sound effect extraction algorithm.
[0054] The target style represents the overall style of the detected video, including movie style, documentary style, and news report style. Specifically, the formula is used: Target style extraction is performed, where Indicates The style category of the clips, Represents a style recognition algorithm.
[0055] Target quality assessment refers to assessing the quality of the video, including clarity, stability, and color. Specifically, the formula is used: Conduct target quality assessment, where Indicates The quality score of each segment, Represents the target quality assessment algorithm.
[0056] 3. Establish a target attribute feature set;
[0057] After the recognition is completed, the target attribute feature set is established. Specifically, the formula is used: represents the attribute feature set of target i, where represents the attribute feature set of target i, represents the center point of target i, represents the target type of target i, represents the target scene of target i, represents the target action of target i, represents the target sentiment of target i, represents the target text and speech of target i, Indicates the target background music and sound effects of target i, represents the target style of target i, represents the target quality evaluation of target i.
[0058] Step S130: performing a second target detection on the image sequence to obtain a target trajectory feature set;
[0059] The second target detection is performed on the image sequence. The main purpose of the second target detection is to track the moving trajectory of the target from the image sequence and determine the trackability of the target. The method of the second target detection specifically includes the following sub-steps:
[0060] Step S131: tracking the target according to the position change of the center point in the picture sequence, and determining the traceability of the target;
[0061] In each frame, the first target detection method is used to find the target and determine its new center point coordinates. The movement trajectory between adjacent frames is represented by the displacement vector, displacement distance and displacement direction angle (relative to the east direction) of the center point. Specifically, the formula is used: , calculate the displacement vector, displacement distance and displacement direction angle of the center point, where represents the displacement vector from the center point of segment i to i+1, represents the displacement distance from the center point of the i to i+1 fragment, represents the displacement direction angle from the i to the i+1 segment center point, Indicates the horizontal displacement of the target between the two segments, Represents the vertical displacement of the target between the two segments. The coordinates of the center point of the target in segment i are , the coordinates of the center point of the target in the i+1 segment are , the value of i is 1-n.
[0062] Traceability of this target It is determined by the displacement vector, displacement distance and displacement direction angle of the center point of the target in each frame. If the displacement vector, displacement distance and displacement direction angle of the center point of the target do not change, the target is considered to be a stationary target. If all three change, the target is considered to move in a curve. If the displacement position and displacement distance change but the displacement direction angle does not change, the target is considered to move in a straight line. The above three cases are considered to be legal target movement and trackability is true. Other cases are considered to be illegal and trackability is false.
[0063] Step S132: If the target's trackability is true, the target's trajectory characteristic parameters are recorded; if the target's trackability is false, the target is considered lost and is not recorded;
[0064] If the target's trackability is true, the target's trajectory characteristic parameters are recorded. Specifically, the formula is used: Represents the trajectory feature set of the target, where represents the trajectory feature set of target i, represents the displacement vector from the center point of segment i to i+1, represents the displacement distance from the center point of the i to i+1 fragment, It represents the displacement direction angle from the i to the i+1 segment center point, i is 1-n. Indicates the traceability of target i. If the traceability of a target is false, the target is considered lost and will not be recorded.
[0065] Step S140: storing the target's attribute feature set and trajectory feature set through an adjacency table to achieve structured processing of video data;
[0066] As a commonly used data structure, adjacency list is suitable for storing the target's attribute feature set and trajectory feature set. The target's attribute feature set is used as the header node, and the target's trajectory feature set is used as the edge node. After connecting the header node with a linked list, this storage method structuredly represents the relationship between the attributes and running trajectories of each target. When regenerating a smart video, you only need to modify the value of a target's header node or edge node. Embodiment 2
[0067] like Figure 2 As shown, Embodiment 2 of the present application provides a video data structuring processing system for intelligent video generation.
[0068] Video source collection and decoding module 21: obtains the video source that needs to be structured, and pre-processes and segments the video source;
[0069] Get the video source that needs to be structured. There are many sources of video sources, including major video platforms, video material libraries, user-uploaded videos, and live broadcast sources. The collected videos include virtual human commentary videos, live broadcast editing videos, and user-recorded videos. Preprocess the collected video sources, including format conversion, resolution adjustment, and denoising. Specifically, the formula is used: Preprocess the video source, including Represents the preprocessed video source, Represents the original video, Represents a preprocessing operation.
[0070] The preprocessed video source is divided into multiple segments according to the timeline to form a picture sequence, and the start and end time of each segment are identified. Specifically, the formula is used: Cut the preprocessed video source, represents the set of segments after segmentation, , They represent the start and end time of the ith segment respectively. i represents the subscript of the segment, which takes a value of 1-n.
[0071] First target detection module 22: performs first target detection on the image sequence to obtain a target attribute feature set;
[0072] The first target detection is performed on the image sequence. The main purpose of the first target detection is to lock the target of interest from a single frame image and extract the target's attribute features through image recognition technology. The target's attribute features include target type, target scene, target action, target emotion, target text and voice, target background music and sound effects, target style and target quality assessment.
[0073] The first target detection module 22 specifically includes:
[0074] Division submodule: divide the sequence images into grids, and divide the target range and boundary of the grid under the same target;
[0075] The sequence images are divided into N×N grids. The first grid is used as the initial target, and its four edges are used as the initial target boundaries. The credibility of its neighboring grids as the same target is calculated. If the credibility exceeds the credibility threshold, the neighboring grids are divided into the target range, and the target boundary is expanded accordingly. Until the credibility of all neighboring grids in the target range as the same target does not exceed the credibility threshold, the range and boundary division of the target is completed, and the next target is divided until all grid divisions are completed.
[0076] The credibility of the same target is determined by the difference in image pixel values and structural similarity. The smaller the difference in image pixel values, the higher the credibility of the same target, which reflects the average pixel difference between the two images as a whole; the higher the structural similarity of the images, the higher the credibility of the same target. When the two images are highly similar in brightness, contrast and structure, the structural similarity value is close to 1, and when they are not similar, the structural similarity value is close to 0. Specifically, the formula is used: Calculate the credibility of the same target, where Indicates the credibility of the same target, The influencing factor representing the difference in image pixel values, represents the impact factor of structural similarity, and Respectively represent the local parameters of the two images, and Represents the height and width of the image in pixels, and Represents the two images at pixel coordinates The pixel value on and Respectively represent images and The local brightness average, and Representing images and The local variance of , a higher variance means the image has a higher contrast, Representing images and The local covariance of the two images shows the same brightness change trend at the same position. A higher value of indicates similar structural information. and Represents a constant to avoid zero denominator.
[0077] Specifically, the formula is used: represents a cyclic process, where Indicates calling the same target credibility calculation formula, Represents the current target range image, Indicates the location of the current initial target range image, The value of is 1-N×N, represents the neighboring grid image, represents the neighboring grid direction, where Indicates the north, Indicates the south, Indicates the West, Indicates the east, and the values of the four directions are all less than the number of grid cells N. Represents the credible threshold, and uses the ternary operator to determine whether the credibility of the same target is greater than the threshold. If it is greater than the threshold, the current target range image is merged with the adjacent grid image, otherwise the original target range is maintained. The cycle continues until the adjacent grid image Until all the merge conditions are not met, the target range at this time is the final range of the target, and the boundary at this time is the final boundary of the target. ++, continue to divide the next target until all targets are divided.
[0078] Attribute feature set establishment submodule: determine the target center point according to the target boundary, identify the target attribute feature parameters, and establish the target attribute feature set;
[0079] 1. Determine the target center point based on the target boundary;
[0080] The target center point is determined according to the target boundary. Specifically, the formula is used: , calculate the coordinates of the target center point, where represents the center point coordinates of the i-th target, , , , Respectively represent the west boundary, east boundary, north boundary and south boundary of the i-th target.
[0081] 2. Identify the attribute characteristic parameters of the target;
[0082] Image recognition technology is used to determine the attribute characteristic parameters of the target, which include target type, target scene, target action, target emotion, target text and voice, target background music and sound effects, target style and target quality assessment.
[0083] The target types are divided into objects, humans and animals. Specifically, the formula is used: Identify faces in videos and analyze their expressions, including Indicates The faces detected in the target, Represents the set of expressions corresponding to the face, Represents the current target image, represents the face recognition algorithm, Represents the expression analysis algorithm; the formula used is: Identify animals in videos, where Indicates The type of animal in the clip, Represents the animal recognition algorithm; using the formula: Identify objects in a video, where Indicates objects in the fragment, Represents an object detection algorithm.
[0084] The target scenes include indoor, outdoor, day and night, using the formula: Perform target scene recognition, where Indicates The category of the scene, Represents the scene detection algorithm.
[0085] The target action represents the actions and behaviors of the characters in the video, including walking, running, jumping, etc., using the formula: Perform target action recognition, where Indicates The set of actions in a fragment, Represents the action detection algorithm.
[0086] The target emotion represents the emotional expressions in the video, including excitement, calmness, tension, etc., using the formula: Target emotion recognition is performed, where Indicates The emotional characteristics of the fragments, Represents a sentiment analysis algorithm.
[0087] The target text and speech recognition means recognizing the speech in the video, converting it into text, and extracting keywords using the formula: Perform target text and grammar recognition, where Indicates The text content of the fragment, represents the set of keywords extracted from the text, represents the text recognition algorithm, Represents the keyword extraction algorithm.
[0088] The target background music and sound effects represent the background music type and sound effect characteristics in the identified video. Specifically, the formula is used: Extract target background music and sound effects, Indicates The background music and sound effects characteristics of each clip, Represents the background music and sound effect extraction algorithm.
[0089] The target style represents the overall style of the detected video, including movie style, documentary style, and news report style. Specifically, the formula is used: Target style extraction is performed, where Indicates The style category of the clips, Represents a style recognition algorithm.
[0090] Target quality assessment refers to assessing the quality of the video, including clarity, stability, and color. Specifically, the formula is used: Conduct target quality assessment, where Indicates The quality score of each segment, Represents the target quality assessment algorithm.
[0091] 3. Establish a target attribute feature set;
[0092] After the recognition is completed, the target attribute feature set is established. Specifically, the formula is used: represents the attribute feature set of target i, where represents the attribute feature set of target i, represents the center point of target i, represents the target type of target i, represents the target scene of target i, represents the target action of target i, represents the target sentiment of target i, represents the target text and speech of target i, Indicates the target background music and sound effects of target i, represents the target style of target i, represents the target quality evaluation of target i.
[0093] Second target detection module 23: performs a second target detection on the image sequence to obtain a target trajectory feature set;
[0094] A second target detection is performed on the image sequence. The main purpose of the second target detection is to track the moving trajectory of the target from the image sequence and determine the trackability of the target.
[0095] The second target detection module 23 specifically includes:
[0096] Trackability judgment submodule: tracks the target trajectory according to the position change of the center point in the image sequence and judges the trackability of the target;
[0097] In each frame, the first target detection method is used to find the target and determine its new center point coordinates. The movement trajectory between adjacent frames is represented by the displacement vector, displacement distance and displacement direction angle (relative to the east direction) of the center point. Specifically, the formula is used: , calculate the displacement vector, displacement distance and displacement direction angle of the center point, where represents the displacement vector from the center point of segment i to i+1, represents the displacement distance from the center point of the i to i+1 fragment, represents the displacement direction angle from the i to the i+1 segment center point, Indicates the horizontal displacement of the target between the two segments, Represents the vertical displacement of the target between the two segments. The coordinates of the center point of the target in segment i are , the coordinates of the center point of the target in the i+1 segment are , the value of i is 1-n.
[0098] Traceability of this target It is determined by the displacement vector, displacement distance and displacement direction angle of the center point of the target in each frame. If the displacement vector, displacement distance and displacement direction angle of the center point of the target do not change, the target is considered to be a stationary target. If all three change, the target is considered to move in a curve. If the displacement position and displacement distance change but the displacement direction angle does not change, the target is considered to move in a straight line. The above three cases are considered to be legal target movement and trackability is true. Other cases are considered to be illegal and trackability is false.
[0099] Trajectory characteristic parameter establishment module: If the target's trackability is true, the target's trajectory characteristic parameters are recorded; if the target's trackability is false, the target is considered lost and is not recorded;
[0100] If the target's trackability is true, the target's trajectory characteristic parameters are recorded. Specifically, the formula is used: Represents the trajectory feature set of the target, where represents the trajectory feature set of target i, represents the displacement vector from the center point of segment i to i+1, represents the displacement distance from the center point of the i to i+1 fragment, It represents the displacement direction angle from the i to the i+1 segment center point, i is 1-n. Indicates the traceability of target i. If the traceability of a target is false, the target is considered lost and will not be recorded.
[0101] Structural processing module 24: storing the attribute feature set and trajectory feature set of the target through an adjacency table to achieve structural processing of video data;
[0102] As a commonly used data structure, adjacency list is suitable for storing the target's attribute feature set and trajectory feature set. The target's attribute feature set is used as the header node, and the target's trajectory feature set is used as the edge node. After connecting the header node with a linked list, this storage method structuredly represents the relationship between the attributes and running trajectories of each target. When regenerating a smart video, you only need to modify the value of a target's header node or edge node.
[0103] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.
Claims
1. A video data structured processing method for intelligent video generation, characterized in that: include: S110, obtaining a video source that needs to be structured, and preprocessing and segmenting the video source; S120, performing the first target detection on the image sequence to obtain a target attribute feature set, specifically including the following sub-steps: Grid the sequence images, and divide the target range and boundary for the same target; divide the sequence images into N×N grids, with the first grid as the initial target and its four edges as the initial target boundary. Calculate the credibility of its neighboring grids as the same target. If the credibility exceeds the credibility threshold, the neighboring grids are divided into the target range, and the target boundary is expanded accordingly. Until the credibility of all neighboring grids in the target range as the same target does not exceed the credibility threshold, the target range and boundary division is completed, and the next target is divided until all grid divisions are completed. The calculation formula for the same target credibility is: ,in Indicates the credibility of the same target, The influencing factor representing the difference in image pixel values, represents the impact factor of structural similarity, and Respectively represent the local parameters of the two images, and Represents the height and width of the image in pixels, and Represents the two images at pixel coordinates The pixel value on and Respectively represent images and The local brightness average, and Representing images and The local variance of , a higher variance means the image has a higher contrast, Representing images and The local covariance of the two images shows the same brightness change trend at the same position. A higher value of indicates similar structural information. and Indicates a constant to avoid a denominator of zero; Determine the target center point based on the target boundary, use image recognition technology to determine the target attribute feature parameters, and establish a target attribute feature set; S130, performing a second target detection on the image sequence to obtain a target trajectory feature set; S140 , storing the target's attribute feature set and trajectory feature set through an adjacency table to achieve structured processing of video data.
2. The video data structured processing method for intelligent video generation according to claim 1, characterized in that: The second target detection method specifically includes the following sub-steps: Track the target according to the position change of the center point in the image sequence and determine the trackability of the target; If the target's trackability is true, the target's trajectory characteristic parameters are recorded. If the target's trackability is false, the target is considered lost and is not recorded.
3. The video data structured processing method for intelligent video generation according to claim 1, characterized in that: As a commonly used data structure, adjacency list is suitable for storing the target's attribute feature set and trajectory feature set. The target's attribute feature set is used as the header node, and the target's trajectory feature set is used as the edge node. After connecting the header node with a linked list, this storage method structuredly represents the relationship between the attributes and running trajectories of each target. When regenerating a smart video, you only need to modify the value of a target's header node or edge node.
4. A video data structured processing system for intelligent video generation, characterized in that: include: Video source collection and decoding module: obtains the video source that needs to be structured, and pre-processes and segments the video source; First target detection module: Perform the first target detection on the image sequence to obtain the target attribute feature set, including: Grid the sequence images, and divide the target range and boundary for the same target; divide the sequence images into N×N grids, with the first grid as the initial target and its four edges as the initial target boundary. Calculate the credibility of its neighboring grids as the same target. If the credibility exceeds the credibility threshold, the neighboring grids are divided into the target range, and the target boundary is expanded accordingly. Until the credibility of all neighboring grids in the target range as the same target does not exceed the credibility threshold, the target range and boundary division is completed, and the next target is divided until all grid divisions are completed. The calculation formula for the same target credibility is: ,in Indicates the credibility of the same target, The influencing factor representing the difference in image pixel values, The impact factor representing the structural similarity, and Respectively represent the local parameters of the two images, and Represents the height and width of the image in pixels, and Represents the two images at pixel coordinates The pixel value on and Respectively represent images and The local brightness average, and Representing images and The local variance of , a higher variance means the image has a higher contrast, Representing images and The local covariance of the two images shows the same brightness change trend at the same position. A higher value of indicates similar structural information. and Represents a constant to avoid zero denominator; Determine the target center point based on the target boundary, use image recognition technology to determine the target attribute feature parameters, and establish a target attribute feature set; Second target detection module: performs a second target detection on the image sequence to obtain a target trajectory feature set; Structured processing module: The target's attribute feature set and trajectory feature set are stored in an adjacency table to achieve structured processing of video data.
5. A video data structured processing system for intelligent video generation as claimed in claim 4, characterized in that: The second target detection module specifically includes: Trackability judgment submodule: tracks the target trajectory according to the position change of the center point in the image sequence and judges the trackability of the target; Trajectory characteristic parameter establishment submodule: If the target's trackability is true, the target's trajectory characteristic parameters are recorded; if the target's trackability is false, the target is considered lost and is not recorded.
6. The video data structured processing system for intelligent video generation according to claim 4, characterized in that: As a commonly used data structure, adjacency list is suitable for storing the target's attribute feature set and trajectory feature set. The target's attribute feature set is used as the header node, and the target's trajectory feature set is used as the edge node. After connecting the header node with a linked list, this storage method structuredly represents the relationship between the attributes and running trajectories of each target. When regenerating a smart video, you only need to modify the value of a target's header node or edge node.
