Video target identification method and device based on artificial intelligence, and storage medium

Abstract

The invention relates to the technical field of image recognition, and provides a video target recognition method and device based on artificial intelligence and a storage medium, and the method comprises the steps: obtaining a video frame data sequence of a target video, carrying out the multi-dimensional analysis of the video frame data sequence, obtaining global video frame information and local region-of-interest information, and generating a target feature map based on the global video frame information and the local region-of-interest information, then carrying out time sequence mode analysis to obtain time sequence evolution features, combining the generated semantic representation vector, inputting the semantic representation vector into a preset adaptive Transform model to carry out target recognition, and obtaining a target recognition result. A semantic representation vector is generated through feature fusion, and a self-adaptive Transform model is used for target recognition, so that the target recognition precision in a complex scene is improved, and the problems of low detection precision and insufficient time sequence information utilization during complex scene processing, dynamic change and long-time sequence analysis are solved.

Description

Technical Field

[0001] The present application relates to the technical field of image recognition, and in particular to a method, device and storage medium for video target recognition based on artificial intelligence. Background Art

[0002] In recent years, with the rapid development of fields such as video surveillance, autonomous driving, and intelligent security, AI-based video object recognition technology has gained widespread attention and application. Video object recognition, a core technology in computer vision, can identify and extract valuable information from complex dynamic scenes, providing decision support and intelligent services for various industries.

[0003] Among related technologies, video object recognition methods are typically based on inter-frame difference analysis, background modeling, and object detection algorithms. These methods detect and recognize objects by analyzing pixel variations within video frames or the differences between static backgrounds and dynamic objects. Specifically, inter-frame difference analysis is primarily used to detect moving objects, while background modeling methods leverage stable background information to extract dynamic objects. These techniques have demonstrated high accuracy in static scenes and low-complexity object recognition tasks.

[0004] Regarding the above technical solutions, although basic video target recognition functions can be achieved through existing methods, there are problems of low detection accuracy and insufficient utilization of temporal information when dealing with complex scenes, dynamic changes and long time series analysis. Summary of the Invention

[0005] In order to improve the problems of low detection accuracy and insufficient utilization of timing information when processing complex scenes, dynamic changes and long time series analysis, the present application provides a video target recognition method, device and storage medium based on artificial intelligence.

[0006] The present application mentions a video target recognition method based on artificial intelligence, including: obtaining a video frame data sequence of a target video, performing a multi-dimensional analysis on the video frame data sequence to obtain global video frame information and local area of interest information, and generating a target feature map based on the global video frame information and the local area of interest information; performing a deep map construction on the target feature map to obtain a temporal association map, performing a topological structure analysis on the temporal association map to obtain dynamic structure nodes, and generating a node feature vector based on the dynamic structure nodes; performing a multi-layer attention mechanism processing on the node feature vector to obtain a weighted feature vector, performing a temporal pattern analysis on the weighted feature vector to obtain a temporal evolution feature; performing feature fusion on the global video frame information, the local area of interest information and the temporal evolution feature to obtain a semantic representation vector; inputting the semantic representation vector into a preset adaptive Transformer model for target recognition to obtain a target recognition result.

[0007] As a preferred solution, the steps of obtaining a video frame data sequence of a target video, performing multi-dimensional analysis on the video frame data sequence to obtain global video frame information and local region of interest information, and generating a target feature map based on the global video frame information and the local region of interest information include: obtaining a target video through a preset target video acquisition module, performing frame segmentation on the target video to obtain a video frame data sequence, performing spatial feature analysis and temporal continuity analysis on the video frame data sequence to obtain global video frame information and local region of interest information; performing contrast enhancement processing and noise suppression processing on the global video frame information and the local region of interest information to obtain enhanced video frame data, and performing feature extraction on the enhanced video frame data to obtain multiple frame-level feature vectors and inter-frame correlation information; constructing a frame feature mapping table using the frame-level feature vectors and the inter-frame correlation information, generating a frame correlation feature set and a temporal feature set based on the frame feature mapping table, constructing an initial feature map according to the frame correlation feature set, and performing structural optimization on the initial feature map using the temporal feature set to obtain a target feature map.

[0008] As a preferred solution, the step of constructing a deep graph of the target feature graph to obtain a temporal association graph, performing a topological structure analysis on the temporal association graph to obtain dynamic structure nodes, and generating a node feature vector based on the dynamic structure nodes includes: constructing a local feature subgraph and a global feature subgraph based on the target feature graph, performing structural relationship modeling on the local feature subgraph to obtain a feature node association matrix, performing spatial transformation modeling on the global feature subgraph to obtain a background feature mapping matrix; constructing a temporal association graph based on the feature node association matrix and the background feature mapping matrix, extracting the graph change pattern of the temporal association graph using a dynamic change analysis method to obtain a temporal feature set and a topological feature set; performing temporal segmentation analysis on the temporal feature set to extract key temporal patterns, performing topological optimization on the topological feature set to extract stable structure patterns, combining the key temporal patterns and the stable structure patterns to obtain dynamic structure nodes; performing feature aggregation analysis on the dynamic structure nodes to obtain node-level feature vectors and neighborhood feature vectors, and generating a node feature vector based on the node-level feature vectors and the neighborhood feature vectors.

[0009] As a preferred solution, the steps of performing multi-layer attention mechanism processing on the node feature vector to obtain a weighted feature vector, performing time series pattern analysis on the weighted feature vector to obtain a time series evolution feature include: constructing a node attention weight matrix based on the node feature vector, and using a multi-head attention mechanism to extract local attention information and global attention information of the node attention weight matrix; using the local attention information to calculate the relationship strength between nodes to obtain a node relationship matrix, and using the global attention information to calculate the node category correlation to obtain a node category weight matrix; performing feature fusion on the node relationship matrix and the node category weight matrix to obtain a weighted feature vector, performing time window segmentation on the weighted feature vector to generate a time window feature sequence, and extracting time series trend information and change rate information based on the time window feature sequence; generating a time series change pattern based on the time series trend information and change rate information, performing hierarchical clustering analysis on the time series change pattern to obtain a time series clustering feature, and performing feature transformation on the time series clustering feature to obtain a time series evolution feature.

[0010] As a preferred solution, the steps of constructing a node attention weight matrix based on the node feature vector and extracting the local attention information and global attention information of the node attention weight matrix using a multi-head attention mechanism include: constructing a node feature relationship matrix based on all nodes of the node feature vector, calculating the correlation between any two node feature vectors in the node feature relationship matrix through Euclidean distance measurement, and obtaining a node correlation matrix; standardizing the node correlation matrix to remove the influence of abnormal data points, and using a threshold screening method to remove low-correlation node pairs to obtain a screened node correlation matrix, and performing self-attention weighting on the screened node correlation matrix to obtain a node attention weight matrix; using a multi-channel weighted aggregation method to calculate the node feature weighting results of the node attention weight matrix under different attention heads, and extracting local attention information; applying a global feature transformation mapping method to calculate the global weight distribution of the node attention weight matrix under different attention heads, and extracting global attention information.

[0011] As a preferred solution, the step of fusing the global video frame information, the local region of interest information and the temporal evolution features to obtain a semantic representation vector includes: extracting global spatial features based on the global video frame information, extracting local edge features based on the local region of interest information, and matching the global spatial features and the local edge features using a feature alignment method to obtain spatial fusion features; extracting temporal correlation features based on the temporal evolution features, and correcting the temporal correlation features using temporal dependencies to obtain temporal correction features; hierarchically fusing the spatial fusion features and the temporal correction features to obtain a fused feature vector, and performing dimensionality reduction processing on the fused feature vector to obtain a semantic representation vector.

[0012] As a preferred solution, the step of inputting the semantic representation vector into a preset adaptive Transformer model for target recognition to obtain a target recognition result includes: after inputting the semantic representation vector into the preset adaptive Transformer model, in the adaptive Transformer model, using a multi-head attention mechanism to perform feature decoupling on the semantic representation vector to obtain category features and background features, performing feature normalization on the category features to obtain category feature vectors, performing background suppression processing on the background features to obtain background correction features; performing classification probability calculation on the category feature vectors and the background correction features to obtain category probability distribution and confidence scores, performing category decision based on the category probability distribution and the confidence scores to obtain target category labels and category confidences; and outputting the target category labels and the category confidences as target recognition results.

[0013] The present application also provides a video target recognition device based on artificial intelligence, including: an acquisition module, used to acquire a video frame data sequence of a target video, perform multi-dimensional analysis on the video frame data sequence, obtain global video frame information and local area of interest information, and generate a target feature map based on the global video frame information and the local area of interest information; a construction module, used to perform deep graph construction on the target feature map to obtain a temporal association map, perform topological structure analysis on the temporal association map to obtain dynamic structure nodes, and generate node feature vectors based on the dynamic structure nodes; an analysis module, used to perform multi-layer attention mechanism processing on the node feature vector to obtain a weighted feature vector, perform temporal pattern analysis on the weighted feature vector to obtain a temporal evolution feature; a fusion module, used to perform feature fusion on the global video frame information, the local area of interest information and the temporal evolution feature to obtain a semantic representation vector; a classification module, used to input the semantic representation vector into a preset adaptive Transformer model for target recognition to obtain a target recognition result.

[0014] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor executes the above-mentioned artificial intelligence-based video target recognition method.

[0015] Compared with the existing technology, the present application has the following beneficial effects: strong applicability and high detection accuracy. By using the video frame data sequence for multi-dimensional analysis, a target feature map of global video frame information and local area of interest information is generated, dynamic structure nodes are extracted through topological structure analysis of deep graph construction and temporal association graph, and node feature vectors are generated. The node feature expression is optimized through a multi-layer attention mechanism, and temporal evolution features are generated by combining temporal pattern analysis. Semantic representation vectors are generated through feature fusion, and an adaptive Transformer model is used for target recognition. It efficiently utilizes spatial, temporal and semantic information, significantly improving the target recognition accuracy in complex scenes. At the same time, it has real-time adaptability to dynamic changes, improving the problems of low detection accuracy and insufficient utilization of temporal information when processing complex scenes, dynamic changes and long time series analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] The structures, proportions, sizes, etc. depicted in the drawings of this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with this technology. They are not intended to limit the conditions under which the present invention can be implemented and therefore have no substantive technical significance. Any structural modifications, changes in proportional relationships, or adjustments in size should still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and objectives that can be achieved by the present invention.

[0018] Figure 1 1 is a flow chart of a method for video object recognition based on artificial intelligence provided by an embodiment of the present invention; Figure 2 It is a schematic block diagram of the structure of the video object recognition device based on artificial intelligence provided by an embodiment of the present invention.

[0019] Description of reference numerals: 10. Video target recognition device based on artificial intelligence; 11. Acquisition module; 12. Construction module; 13. Analysis module; 14. Fusion module; 15. Classification module. DETAILED DESCRIPTION

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0021] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0022] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0023] It should be further understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0024] The technical solution of the present invention will be further described below with reference to the accompanying drawings and through specific implementation methods.

[0025] Example 1: like Figure 1 As shown, the present application provides a video target recognition method based on artificial intelligence, including steps S100 to S500.

[0026] Step S100: Acquire a video frame data sequence of a target video, perform multi-dimensional analysis on the video frame data sequence to obtain global video frame information and local region of interest information, and generate a target feature map based on the global video frame information and the local region of interest information.

[0027] In this step, the quality of the video frame data sequence is ensured by performing image data preprocessing on the video frame data sequence of the target video, including image denoising, color space conversion, and resolution equalization. Specifically, the global feature information of each frame and the local area of interest information based on interest point detection are extracted through the algorithm to generate a target feature map containing spatial features and position features.

[0028] For example, when processing tracking videos captured by a camera, the video frame data sequence can be denoised to remove unnecessary background interference, while the vehicle area is detected as a local area of interest, ultimately generating a target feature map containing traffic scene information.

[0029] Step S200: construct a deep graph for the target feature graph to obtain a temporal association graph, perform a topological structure analysis on the temporal association graph to obtain dynamic structure nodes, and generate node feature vectors based on the dynamic structure nodes.

[0030] In this step, a deep map is constructed for the target feature map, and the temporal relationship is used to define nodes and construct edges to generate a temporal association map. Specifically, the image sequence feature clustering method is used to dynamically group the map nodes, and then the dynamic structure nodes are extracted through spatial topological structure analysis, and the node feature vector is generated based on the spatial features and temporal characteristics of each dynamic structure node.

[0031] For example, when constructing a graph of human activity in a tracking video, nodes can be defined for the person's motion trajectory, the trajectories between activity areas can be used as edges, and the positions of associated nodes can be dynamically adjusted to generate node feature vectors.

[0032] Step S300: Perform multi-layer attention mechanism processing on the node feature vector to obtain a weighted feature vector, perform temporal pattern analysis on the weighted feature vector, and obtain temporal evolution features.

[0033] In this step, the node feature vector is processed through a multi-layer attention mechanism, and the attention weights between nodes are calculated to emphasize key information and generate weighted feature vectors. Specifically, the weight distribution is optimized through temporal correlation analysis between nodes, and the accuracy of attention processing is improved based on graph context information, ultimately extracting the temporal evolution features in dynamic changes.

[0034] For example, when analyzing tracking videos, we can perform weighted processing on node feature vectors to highlight key moments of dynamic activities and form temporal evolution features to identify important events in the game.

[0035] Step S400: performing feature fusion on the global video frame information, the local region of interest information and the temporal evolution features to obtain a semantic representation vector.

[0036] In this step, the global video frame information, local region of interest information and temporal evolution features are fused, and a multi-scale fusion algorithm is used to semantically merge features at different levels to generate a semantic representation vector. Specifically, the complementary relationship between features is used to construct a comprehensive vector containing spatial, temporal and semantic information.

[0037] For example, in surveillance video applications, the human activities in the global video frame information, the abnormal activities in the local area of interest information, and the dynamic evolution in the temporal evolution features can be integrated to generate a comprehensive semantic representation vector.

[0038] Step S500: Input the semantic representation vector into a preset adaptive Transformer model to perform target recognition and obtain a target recognition result.

[0039] In this step, the semantic representation vector is input into the preset adaptive Transformer model, and the semantic representation is deeply analyzed through the self-attention mechanism to achieve effective modeling of target features and generate target recognition results; specifically, the model can automatically adjust the network parameters according to the target semantic features to enhance the recognition effect, while supporting the accurate detection of multiple targets in complex scenes.

[0040] For example, in personnel flow monitoring, the adaptive Transformer model can accurately identify and track the characteristics, speed, and trajectory of people.

[0041] In this embodiment, by obtaining a video frame data sequence of the target video and performing a multi-dimensional analysis on the video frame data sequence, global video frame information and local region of interest information are generated. On this basis, a target feature map is generated based on the global video frame information and the local region of interest information, and a depth map is constructed on the target feature map to generate a temporal association map. Subsequently, a topological structure analysis is performed on the temporal association map to extract dynamic structure nodes, and a node feature vector is generated based on the dynamic structure nodes. The node feature vector is then processed with a multi-layer attention mechanism to generate a weighted feature vector, and temporal evolution features are obtained through temporal pattern analysis. Finally, the global video frame information, local region of interest information, and temporal evolution features are feature fused to obtain a semantic representation vector, and the semantic representation vector is input into a preset adaptive Transformer model to complete target recognition and generate a target recognition result. It realizes multi-dimensional analysis of video frame data sequences to obtain global video frame information and local area of interest information, further generates target feature maps and constructs deep maps, fully utilizes the topological structure characteristics of temporal association maps, generates node feature vectors through dynamic structure nodes, and improves the accuracy of feature expression through multi-layer attention mechanism processing. Combined with the feature fusion of temporal evolution features, it realizes more efficient and accurate semantic representation, and performs target recognition through the preset adaptive Transformer model to improve the accuracy and reliability of target recognition results, especially showing obvious advantages in complex scenes and dynamically changing scenes, and improving the problems of low detection accuracy and insufficient utilization of temporal information when processing complex scenes, dynamic changes and long time series analysis.

[0042] Example 2: In step S100, the target video is acquired through a preset target video acquisition module, and the target video is frame segmented to obtain a video frame data sequence. The video frame data sequence is subjected to spatial feature analysis and temporal continuity analysis to obtain global video frame information and local area of interest information.

[0043] By processing the target video frame by frame, the global spatial distribution features of each frame are extracted. A motion detection-based algorithm is also used to identify local regions of interest (ROIs) to preserve areas related to the target's dynamic changes. Specifically, a gradient-based edge detection algorithm is applied to the video frame data sequence to extract spatial features. A frame-by-frame difference algorithm is used to analyze temporal continuity, identify the target's motion trajectory, and mark the spatial boundaries of the region of interest. Furthermore, a multi-scale image pyramid technique is used to optimize the frame resolution at multiple levels to improve the fidelity of spatial feature details.

[0044] For example, in video processing for monitoring pedestrian activities, edge detection algorithms can be used to extract the contour information of buildings and roads in the scene as spatial features of the global video frame information; at the same time, inter-frame difference technology is used to mark the moving pedestrian areas. These areas are identified as local area of interest information and annotated in the form of rectangular bounding boxes.

[0045] Contrast enhancement and noise suppression are performed on the global video frame information and the local region of interest information to obtain enhanced video frame data, and feature extraction is performed on the enhanced video frame data to obtain multiple frame-level feature vectors and inter-frame correlation information.

[0046] The system applies histogram equalization to global video frame information and local regions of interest (ROI) to enhance contrast, and employs median filtering to effectively suppress noise, generating optimized enhanced video frame data. Specifically, a feature extraction algorithm quantifies key feature points and feature descriptors within the enhanced video frame data, generating multiple frame-level feature vectors. Furthermore, temporal analysis calculates the degree of correlation between adjacent frames to generate inter-frame correlation information. This inter-frame correlation information is calculated using an optical flow algorithm, enabling accurate capture of the motion patterns of the target area.

[0047] For example, in a scene where people are monitored on the road, histogram equalization is used to enhance the brightness of video frames in low-light environments at night, and median filtering is used to remove sensor noise; then the edge feature points of the people are extracted, and the displacement and speed of the people between different frames are calculated using the optical flow algorithm to obtain inter-frame correlation information.

[0048] A frame feature map is constructed using frame-level feature vectors and inter-frame correlation information. A frame correlation feature set and a temporal feature set are generated based on the frame feature map. An initial feature map is constructed based on the frame correlation feature set. The temporal feature set is used to optimize the structure of the initial feature map to obtain a target feature map.

[0049] By constructing a frame feature mapping table with frame-level feature vectors as nodes and inter-frame correlation information as edges, an association clustering algorithm is used to aggregate similar nodes into a frame association feature set, and the temporal change information in the association set is extracted as a temporal feature set; specifically, when constructing the initial feature graph, the spatial position of the frame association feature set is mapped to the graph node, and the graph structure is optimized based on the dynamic relationship in the temporal feature set, and redundant nodes and invalid edges are removed, thereby improving the representation ability of the target feature graph.

[0050] For example, in video tracking applications, the human activity features in indoor surveillance videos are clustered through frame feature mapping tables to generate frame-related feature sets. At the same time, the time intervals of intermittent movements are quantitatively analyzed using temporal feature sets to optimize the temporal structure of feature maps and improve the detection accuracy of dynamic targets.

[0051] In step S200, a local feature subgraph and a global feature subgraph are constructed based on the target feature graph, a structural relationship model is performed on the local feature subgraph to obtain a feature node association matrix, and a spatial transformation model is performed on the global feature subgraph to obtain a background feature mapping matrix.

[0052] By dividing the target feature map into several sub-regions, local feature sub-graphs and global feature sub-graphs are generated. The local feature sub-graph uses a node nesting method to model the relationships between feature nodes within the sub-region to establish a feature node association matrix. The global feature sub-graph generates a background feature mapping matrix through coordinate space transformation and feature mapping methods to represent the spatial correlation of global background features. Specifically, the modeling of the local feature sub-graph includes calculating the connection relationships of feature nodes, such as using the connected component algorithm to extract connected nodes within the region, and performing coordinate normalization on the global feature sub-graph to enhance feature consistency.

[0053] For example, in the scenario of monitoring people, the local feature subgraph can represent the characteristics of people at road intersections, while the global feature subgraph represents the spatial background information of the entire road network. By establishing the association matrix of local personnel nodes and the feature mapping matrix of the road network background, road traffic behavior modeling can be achieved.

[0054] A temporal correlation graph is constructed based on the feature node association matrix and the background feature mapping matrix. The graph change pattern of the temporal correlation graph is extracted using the dynamic change analysis method to obtain the temporal feature set and the topological feature set.

[0055] By performing temporal analysis on the feature node association matrix, the data of local feature subgraphs and global feature subgraphs are integrated into the temporal association graph. At the same time, combined with the dynamic pattern generated by the background feature mapping matrix, the changing characteristics of the graph in the temporal dimension are extracted. Specifically, a dynamic change detection algorithm based on differential analysis is used to update the feature nodes and edges of the temporal association graph to generate a temporal feature set, and a structural clustering algorithm is used to optimize the topological relationship of the nodes in the graph to generate a topological feature set.

[0056] For example, in tracking and detection applications, the dynamic changes in the positions of people in real-time images are detected through temporal correlation graphs to generate temporal feature sets. At the same time, the topological feature sets are extracted by analyzing the topological characteristics of fixed structures in the scene (such as door frames or furniture).

[0057] The timing feature set is subjected to timing segmentation analysis to extract key timing patterns. The topology feature set is subjected to topology optimization to extract stable structural patterns. The key timing patterns and stable structural patterns are combined to obtain dynamic structural nodes.

[0058] Key timing patterns are extracted by applying a sliding window segmentation method to the timing feature set, from which the dynamic change information of key nodes is extracted; redundant edges are cleaned and the structure is simplified on the topological feature set to extract stable structural patterns in the global scene; key timing patterns and stable structural patterns are mapped and integrated through the association matrix to form dynamic structural nodes; specifically, key timing patterns are used to determine the time tags of dynamic structural nodes, and the spatial positions of dynamic nodes are optimized based on the stable structural patterns to ensure spatiotemporal consistency.

[0059] For example, when using cameras to monitor industrial equipment, temporal feature set analysis can mark the time points of the equipment's operating status, while topological feature sets represent the spatial patterns of the equipment's layout. By combining the two, dynamic structural nodes are generated to detect spatiotemporal correlations when equipment is abnormal.

[0060] Feature aggregation analysis is performed on dynamic structure nodes to obtain node-level feature vectors and neighborhood feature vectors, and node feature vectors are generated based on the node-level feature vectors and neighborhood feature vectors.

[0061] By aggregating and analyzing the local and global characteristics of dynamic structure nodes respectively, the node-level feature vector is generated by extracting the distribution characteristics of its time dimension and spatial dimension, and the neighborhood feature vector is generated by aggregating the collective characteristics of adjacent nodes; specifically, the neighborhood feature expression is optimized by calculating the feature weights between nodes, and the feature fusion technology is used to combine the node-level feature vector and the neighborhood feature vector to generate the final node feature vector.

[0062] For example, in personnel identification and tracking, the node-level feature vector can represent the speed and position characteristics of a single person, while the neighborhood feature vector aggregates the behavior patterns of adjacent persons, and global modeling of personnel flow dynamics is achieved by generating node feature vectors.

[0063] In step S300, a node attention weight matrix is constructed based on the node feature vector, and the local attention information and global attention information of the node attention weight matrix are extracted using the multi-head attention mechanism.

[0064] By performing multidimensional matrix analysis on node feature vectors, an initial node feature relationship matrix is constructed based on their distribution in the temporal and spatial dimensions. The correlation between feature vectors between nodes is calculated using the Euclidean distance measure to generate an inter-node association matrix. Subsequently, a multi-head attention mechanism is employed, using the node feature relationship matrix as input, to extract local and global attention information. Local attention captures the dynamic relationships between nodes over short distances, while global attention reflects the overall association between feature nodes globally. Specifically, local attention is generated by focusing on the weight distribution of neighboring nodes, while global attention is calculated using the weighted distribution of all nodes.

[0065] For example, in a monitoring system, the node feature vector can represent the spatial characteristics of multiple pedestrian positions, local attention information is used to determine the dynamic interaction relationship between adjacent pedestrians, and global attention information shows the overall distribution and change characteristics of the pedestrian group in the entire scene.

[0066] The local attention information is used to calculate the relationship strength between nodes to obtain the node relationship matrix, and the global attention information is used to calculate the node category correlation to obtain the node category weight matrix.

[0067] By calculating the feature similarity and temporal dynamic consistency between each pair of nodes based on local attention information, a node relationship matrix is generated. The correlation of node categories is statistically analyzed in combination with global attention information to generate a node category weight matrix. Specifically, the node relationship matrix is generated using a weight threshold algorithm to eliminate low-intensity connections. At the same time, the node category weight matrix is optimized using frequency analysis and distribution center calculation between categories to improve the matrix's discriminative ability.

[0068] For example, when monitoring a large flow of people, the node relationship matrix can represent the interaction intensity of key target personnel in the area, such as the frequency of interaction between pedestrians at a certain intersection, while the node category weight matrix can represent the distribution weights and correlations of specific categories (such as vehicles and pedestrians) in the area.

[0069] The node relationship matrix and the node category weight matrix are fused to obtain weighted feature vectors, which are then divided into time windows to generate time window feature sequences. Time series trend information and change rate information are extracted based on the time window feature sequences.

[0070] By mapping and fusing the dynamic interaction characteristics of nodes in the node relationship matrix with the category information in the node category weight matrix one by one, the weighted feature vector of the node is generated. Then, the sliding time window method is used to divide the weighted feature vector into a time window feature sequence. Based on the time window feature sequence, the feature change rate and trend direction in the time dimension are calculated. Specifically, the change rate analysis algorithm is used to quantify the feature increment within each time window, and the long-term trend is extracted through multidimensional curve fitting.

[0071] For example, when using cameras to monitor industrial equipment, weighted feature vectors can integrate the interactive information and category characteristics of various parts of the equipment, analyze the rate of change of the equipment's working status through time window segmentation, and extract operating trends to predict the equipment's health status.

[0072] A time series change pattern is generated according to the time series trend information and the change rate information. A hierarchical clustering analysis is performed on the time series change pattern to obtain the time series clustering features. The time series clustering features are then transformed to obtain the time series evolution features.

[0073] By combining time series trend information and change rate information to generate time series change patterns, the hierarchical clustering algorithm is used to analyze the heterogeneity and similarity of time series change patterns to generate preliminary time series clustering features; the time series clustering features are subjected to feature transformation processing, such as normalization and dimensionality reduction operations, to eliminate redundant features and highlight key features; finally, time series evolution features are generated for dynamic target recognition; specifically, the time series change pattern ensures that the key features of different time periods are fully modeled through a multi-scale clustering method, while the feature transformation further optimizes the efficiency of feature representation through principal component analysis.

[0074] Among them, the steps of constructing a node attention weight matrix based on the node feature vector and extracting the local attention information and global attention information of the node attention weight matrix using the multi-head attention mechanism include: constructing a node feature relationship matrix based on all nodes of the node feature vector, calculating the correlation between any two node feature vectors in the node feature relationship matrix through the Euclidean distance measure, and obtaining the node correlation matrix.

[0075] By taking the node feature vector as input, a node feature relationship matrix is constructed, and the feature similarity between nodes is analyzed using the Euclidean distance measure to form an inter-node correlation matrix. Specifically, the multidimensional characteristics of the node feature vector are mapped to the high-dimensional space of the relationship matrix, and the descriptive ability of the matrix is improved by dynamically updating the threshold.

[0076] For example, in environmental monitoring, the node feature vector includes the temperature and humidity of the monitoring point. The similarity of different monitoring points can be quantified through the correlation matrix. For example, the temperature and humidity features with closer distances have higher correlation.

[0077] The node correlation matrix is standardized to remove the influence of abnormal data points, and the threshold screening method is used to remove low-correlation node pairs to obtain the screened node correlation matrix. The self-attention weighted matrix is then applied to the screened node correlation matrix to obtain the node attention weight matrix.

[0078] The correlation matrix is normalized to eliminate the interference of outliers, and a threshold screening method based on the correlation distribution characteristics is used to delete node pairs with low correlation. Subsequently, different weighted values are assigned to the screened matrix through the self-attention mechanism to form a node attention weight matrix. Specifically, the weight matrix assigns weights through the soft maximum function, so that the characteristics of important node pairs can be highlighted.

[0079] For example, in a system for monitoring personnel, the node correlation matrix represents the correlation strength between monitoring points. After standardization and screening, key monitoring point pairs are retained, and the expression ability of key points is improved through self-attention weighting.

[0080] The multi-channel weighted aggregation method is used to calculate the node feature weighting results of the node attention weight matrix under different attention heads to extract local attention information.

[0081] The features of different attention heads are decomposed through a multi-channel weighted mechanism. Each attention head generates a set of weighted features, and then the local attention information is extracted by combining the results of all channels. Specifically, the local attention further refines the relationship between neighboring nodes through the neighborhood weighting method while ensuring the independence of the feature components.

[0082] For example, in image and video target detection, local attention information can be used to locate the interactive characteristics of nearby target objects in the video frame, thereby improving detection accuracy.

[0083] The global feature transformation mapping method is applied to calculate the global weight distribution of the node attention weight matrix under different attention heads and extract the global attention information.

[0084] By performing feature transformation on different attention heads globally, the weight matrix is mapped to the global feature distribution; specifically, the global attention information is evenly distributed among all nodes through normalization to capture the overall dynamic characteristics.

[0085] For example, in the panoramic monitoring of personnel competitions, global attention information can reflect the distribution dynamics of personnel in the entire game and be used to analyze game tactics.

[0086] In step S400, global spatial features are extracted based on global video frame information, local edge features are extracted based on local region of interest information, and feature alignment method is used to perform feature matching on the global spatial features and the local edge features to obtain spatial fusion features.

[0087] By applying a multi-scale feature extraction algorithm to the global video frame information, a feature set containing the global spatial distribution of the scene is generated. At the same time, an edge detection algorithm is applied based on the information of the local region of interest to extract the edge features of the local area. Subsequently, a feature alignment method is used to accurately match the global spatial features with the local edge features through a feature point matching algorithm to generate spatial fusion features. Specifically, the alignment process adopts a feature matching algorithm based on key point descriptors, and improves the matching accuracy through an error correction model to ensure the integrity and consistency of the spatial fusion features.

[0088] For example, in an autonomous driving scenario, global spatial features can represent the overall layout of the road, while local edge features represent the vehicle outline. Spatial fusion features are generated through feature matching to analyze the relative position relationship between the vehicle and the road.

[0089] The time series correlation features are extracted based on the time series evolution features, and the time series correlation features are modified using the time series dependency relationship to obtain the time series correction features.

[0090] By performing segmented time series modeling on the time series evolution features, time series correlation features related to dynamic changes are extracted; then, the abnormal points in the time series correlation features are corrected through the time series dependency analysis algorithm, the feature performance in the time dimension is optimized, and the time series correction features are generated; specifically, the correction process adopts a dependency correction model based on distribution laws, and improves the accuracy of time series features through the constraints of time intervals and dynamic directions.

[0091] For example, in the human activity detection scenario, temporal correlation features can describe the continuous changes in human movements, and use temporal dependencies to correct abnormal points in the motion trajectory, such as sudden stops or accelerations, and ultimately generate optimized temporal correction features.

[0092] The spatial fusion features and temporal correction features are hierarchically fused to obtain a fused feature vector, which is then subjected to dimensionality reduction to obtain a semantic representation vector.

[0093] By using a feature fusion algorithm to perform multi-level integration of spatial fusion features and temporal correction features, a comprehensive fusion feature vector containing spatial and temporal information is generated; then the principal component analysis (PCA) method is applied to the fused feature vector for dimensionality reduction to reduce redundant features, improve the efficiency of feature expression, and finally generate a semantic representation vector; specifically, the dimensionality reduction process ensures that the semantic representation vector retains core information while reducing computational complexity by selecting the main feature components and suppressing low-contribution features.

[0094] For example, in video surveillance, the fused feature vector can simultaneously represent the spatial information of building layout and the temporal information of personnel flow. The semantic representation vector generated after dimensionality reduction is used to identify high-traffic areas and their dynamic change trends.

[0095] In step S500, after the semantic representation vector is input into the preset adaptive Transformer model, the multi-head attention mechanism is used in the adaptive Transformer model to perform feature decoupling on the semantic representation vector to obtain category features and background features, the category features are normalized to obtain category feature vectors, and the background features are subjected to background suppression processing to obtain background correction features.

[0096] The multi-head attention mechanism is used to assign weights to each feature dimension in the semantic representation vector, separating the category features related to classification and the background features related to background information. The category features are normalized by the normalization method to reduce noise interference and generate category feature vectors; the background features are suppressed by background suppression to reduce the influence of background information and generate background correction features; specifically, the background suppression processing adopts the feature weighting method to dynamically adjust the weight of the background features to a lower value, thereby highlighting the recognition of the category features.

[0097] For example, in image recognition scenarios, the semantic representation vector contains mixed features of people and backgrounds. After decoupling through the multi-head attention mechanism, the category features represent the morphological information of the person, while the background features represent the environmental information. After normalization and suppression processing, it can ensure more accurate recognition of the person.

[0098] The classification probability is calculated for the category feature vector and background correction feature to obtain the category probability distribution and confidence score. Category decision is made based on the category probability distribution and confidence score to obtain the target category label and category confidence.

[0099] By performing classification probability calculation based on the Softmax function on the category feature vector, the probability distribution of each category is generated, and the confidence score is obtained by normalizing the background correction features. Combining the category probability distribution and confidence score, the final target category label and category confidence are determined through a weighted decision method. Specifically, the category decision process adopts the maximum a posteriori probability (MAP) criterion to select the highest probability category from the category probability distribution as the target category label, and at the same time combines the confidence score to improve the reliability of the classification.

[0100] For example, in vehicle type recognition, the category feature vector can correspond to different categories such as cars, trucks, and buses. The probability of each vehicle is determined by calculating the probability distribution, and the background confidence is combined to ensure the robustness of the classification results to complex backgrounds.

[0101] Output target category label and category confidence as target recognition result.

[0102] By taking the target category label and category confidence as the final results of the model output, the results are formatted or integrated through further post-processing steps for use in specific application scenarios; specifically, the output target recognition results can be presented in text form or visual graphical form as needed, so that users can intuitively understand the recognition content, and at the same time support integration into larger application systems for automated processing.

[0103] For example, in a personnel recognition system, the target category label displays "personnel" and the category confidence score is 0.95, indicating that a person is recognized in the image with high confidence. The output result can be directly used for real-time alarm or behavior analysis.

[0104] In this embodiment, a target video is acquired and frame-segmented to generate a video frame data sequence. Global video frame information and local region of interest information are extracted, and the quality of the initial video frame is improved through feature contrast enhancement and noise suppression. A frame-related feature set and a temporal feature set are then generated based on the frame feature map. The initial feature map is then structurally optimized to obtain a target feature map. Based on the target feature map, local and global feature subgraphs are constructed. A temporal correlation graph is established using a feature node association matrix and a background feature map matrix. Temporal and topological feature sets are extracted through dynamic change analysis. Dynamic structure nodes are generated through temporal segmentation analysis and topology optimization. Dynamic structure nodes are further subjected to feature aggregation analysis to generate node feature vectors. A multi-head attention mechanism is used to construct a node attention weight matrix for the node feature vectors, extracting local and global attention information. A node relationship matrix and a category weight matrix are calculated to achieve weighted feature fusion and extract temporal trend information and change patterns, ultimately generating temporal evolution features. A fused feature vector is generated by fusing global spatial features with local edge features and combining them with temporal correction features. A semantic representation vector is then obtained through dimensionality reduction. The semantic representation vector is input into the adaptive Transformer model, which generates a category feature vector and background correction features by decoupling features. Classification probability calculations and category decisions are then performed, ultimately outputting the target category label and category confidence. This embodiment achieves high-precision and robust object recognition in complex scenarios through a multi-dimensional, multi-stage feature extraction and optimization approach.

[0105] Example 3: like Figure 2 As shown, the present application also provides an artificial intelligence-based video target recognition device 10, including an acquisition module 11, a construction module 12, an analysis module 13, a fusion module 14 and a classification module 15.

[0106] The acquisition module 11 is mainly used to acquire the video frame data sequence of the target video, perform multi-dimensional analysis on the video frame data sequence, obtain global video frame information and local area of interest information, and generate a target feature map based on the global video frame information and local area of interest information.

[0107] The construction module 12 is mainly used to construct a deep map of the target feature map to obtain a temporal correlation map, perform topological structure analysis on the temporal correlation map to obtain dynamic structure nodes, and generate node feature vectors based on the dynamic structure nodes.

[0108] The analysis module 13 is mainly used to perform multi-layer attention mechanism processing on the node feature vector to obtain a weighted feature vector, and perform time series pattern analysis on the weighted feature vector to obtain time series evolution characteristics.

[0109] The fusion module 14 is mainly used to fuse the global video frame information, the local region of interest information and the temporal evolution features to obtain a semantic representation vector.

[0110] The classification module 15 is mainly used to input the semantic representation vector into the preset adaptive Transformer model for target recognition to obtain the target recognition result.

[0111] In this embodiment, the acquisition module 11 performs multidimensional analysis on the target video frame data sequence, extracting global video frame information and local region of interest information to generate a target feature map, providing a key feature foundation for subsequent processing. The construction module 12 constructs a deep graph of the target feature map to generate a temporal correlation map. It also extracts dynamic structure nodes through topological structure analysis and further generates node feature vectors, achieving comprehensive modeling of time and space dimensions. The analysis module 13 uses a multi-layer attention mechanism to weight the node feature vectors, extract key information, generate weighted feature vectors, and generate temporal evolution features reflecting the target's dynamic characteristics through temporal pattern analysis. The fusion module 14 fuses the global video frame information, local region of interest information, and temporal evolution features to generate a semantic representation vector with spatial and temporal semantics, ensuring the integrity and relevance of the information. The classification module 15 inputs the semantic representation vector into a preset adaptive Transformer model, leveraging its adaptive capabilities and multi-head attention mechanism to complete target recognition and output the target recognition results, achieving high-precision target detection and recognition in complex scenarios.

[0112] It should be noted that those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described device and each module can refer to the corresponding processes in the aforementioned embodiment 1 and will not be repeated here.

[0113] Example 4: The present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor executes the video target recognition method based on artificial intelligence as described in Example 1.

[0114] In this embodiment, a computer program on a computer-readable storage medium is used to enable the processor to execute the artificial intelligence-based video target recognition method in Example 1, completing the acquisition and multi-dimensional analysis of the video frame data sequence, the generation of the target feature map, the construction of the deep map and the topological analysis of the temporal association map, and the extraction of dynamic structure nodes and node feature vectors. The program further processes the node feature vectors through a multi-layer attention mechanism to generate weighted feature vectors and temporal evolution features, and generates a semantic representation vector through feature fusion. Finally, the target is recognized through an adaptive Transformer model, and the target category label and category confidence are output. This computer program implements the entire recognition process through process flow, significantly improving the efficiency and accuracy of target recognition. It is suitable for a variety of application scenarios, such as intelligent monitoring, autonomous driving, and video analysis, providing users with reliable target recognition services.

[0115] The structures, proportions, sizes, etc. depicted in the drawings of this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with this technology. They are not intended to limit the conditions under which the present invention can be implemented and therefore have no substantive technical significance. Any structural modifications, changes in proportional relationships, or adjustments in size should still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and objectives that can be achieved by the present invention.

[0116] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video target recognition method based on artificial intelligence, characterized in that: include: Acquire a video frame data sequence of a target video, perform multidimensional analysis on the video frame data sequence to obtain global video frame information and local region of interest information, and generate a target feature map based on the global video frame information and the local region of interest information; Performing a depth map construction on the target feature map to obtain a time series association map, performing a topological structure analysis on the time series association map to obtain dynamic structure nodes, and generating a node feature vector based on the dynamic structure nodes; Performing multi-layer attention mechanism processing on the node feature vector to obtain a weighted feature vector, and performing time series pattern analysis on the weighted feature vector to obtain a time series evolution feature; Performing feature fusion on the global video frame information, the local region of interest information, and the temporal evolution features to obtain a semantic representation vector; The semantic representation vector is input into a preset adaptive Transformer model for target recognition to obtain a target recognition result.

2. The method for video target recognition based on artificial intelligence according to claim 1, characterized in that: The steps of obtaining a video frame data sequence of a target video, performing multi-dimensional analysis on the video frame data sequence to obtain global video frame information and local region of interest information, and generating a target feature map based on the global video frame information and the local region of interest information include: Acquire a target video through a preset target video acquisition module, perform frame segmentation on the target video to obtain a video frame data sequence, perform spatial feature analysis and temporal continuity analysis on the video frame data sequence to obtain global video frame information and local area of interest information; Performing contrast enhancement and noise suppression processing on the global video frame information and the local region of interest information to obtain enhanced video frame data, and performing feature extraction on the enhanced video frame data to obtain multiple frame-level feature vectors and inter-frame correlation information; A frame feature mapping table is constructed using the frame-level feature vector and the inter-frame association information, a frame association feature set and a time series feature set are generated based on the frame feature mapping table, an initial feature map is constructed according to the frame association feature set, and the initial feature map is structurally optimized using the time series feature set to obtain a target feature map.

3. The method for video target recognition based on artificial intelligence according to claim 1, characterized in that: The steps of constructing a depth map of the target feature map to obtain a time series association map, performing a topological structure analysis on the time series association map to obtain dynamic structure nodes, and generating a node feature vector based on the dynamic structure nodes include: Constructing a local feature subgraph and a global feature subgraph based on the target feature graph, performing structural relationship modeling on the local feature subgraph to obtain a feature node association matrix, and performing spatial transformation modeling on the global feature subgraph to obtain a background feature mapping matrix; Constructing a temporal correlation map based on the feature node association matrix and the background feature mapping matrix, extracting a map change pattern of the temporal correlation map using a dynamic change analysis method, and obtaining a temporal feature set and a topological feature set; Performing a time segmentation analysis on the time sequence feature set to extract key time sequence patterns, performing a topology optimization on the topology feature set to extract stable structure patterns, and combining the key time sequence patterns with the stable structure patterns to obtain dynamic structure nodes; Performing feature aggregation analysis on the dynamic structure nodes to obtain a node-level feature vector and a neighborhood feature vector, and generating a node feature vector based on the node-level feature vector and the neighborhood feature vector.

4. The method for video target recognition based on artificial intelligence according to claim 1, characterized in that: The step of performing multi-layer attention mechanism processing on the node feature vector to obtain a weighted feature vector, and performing time series pattern analysis on the weighted feature vector to obtain a time series evolution feature includes: Constructing a node attention weight matrix based on the node feature vector, and extracting local attention information and global attention information of the node attention weight matrix using a multi-head attention mechanism; Calculating the relationship strength between nodes using the local attention information to obtain a node relationship matrix, and calculating the node category relevance using the global attention information to obtain a node category weight matrix; Performing feature fusion on the node relationship matrix and the node category weight matrix to obtain a weighted feature vector, performing time window segmentation on the weighted feature vector to generate a time window feature sequence, and extracting time series trend information and change rate information based on the time window feature sequence; A time series change pattern is generated according to the time series trend information and the change rate information, a hierarchical clustering analysis is performed on the time series change pattern to obtain a time series clustering feature, and a feature transformation is performed on the time series clustering feature to obtain a time series evolution feature.

5. The method for video target recognition based on artificial intelligence according to claim 4, characterized in that: The step of constructing a node attention weight matrix based on the node feature vector and extracting local attention information and global attention information of the node attention weight matrix using a multi-head attention mechanism includes: Constructing a node feature relationship matrix based on all nodes of the node feature vector, calculating the correlation between any two node feature vectors in the node feature relationship matrix by using the Euclidean distance measure to obtain an inter-node correlation matrix; The inter-node correlation matrix is normalized to remove the influence of abnormal data points, and low-correlation node pairs are removed using a threshold screening method to obtain a screened inter-node correlation matrix. The screened inter-node correlation matrix is weighted by self-attention to obtain a node attention weight matrix; The node feature weighting results of the node attention weight matrix under different attention heads are calculated using a multi-channel weighted aggregation method to extract local attention information; The global feature transformation mapping method is applied to calculate the global weight distribution of the node attention weight matrix under different attention heads and extract the global attention information.

6. The method for video target recognition based on artificial intelligence according to claim 1, characterized in that: The step of fusing the global video frame information, the local region of interest information, and the temporal evolution features to obtain a semantic representation vector includes: Extracting global spatial features based on the global video frame information, extracting local edge features based on the local region of interest information, and performing feature matching on the global spatial features and the local edge features using a feature alignment method to obtain spatial fusion features; Extracting a time series correlation feature based on the time series evolution feature, and performing feature correction on the time series correlation feature using a time series dependency relationship to obtain a time series correction feature; The spatial fusion feature and the temporal correction feature are hierarchically fused to obtain a fusion feature vector, and the fusion feature vector is subjected to dimensionality reduction processing to obtain a semantic representation vector.

7. The method for video target recognition based on artificial intelligence according to claim 1, characterized in that: The step of inputting the semantic representation vector into a preset adaptive Transformer model for target recognition to obtain a target recognition result includes: After inputting the semantic representation vector into a preset adaptive Transformer model, in the adaptive Transformer model, a multi-head attention mechanism is used to perform feature decoupling on the semantic representation vector to obtain category features and background features, the category features are feature normalized to obtain a category feature vector, and background suppression processing is performed on the background features to obtain background correction features; Performing classification probability calculation on the category feature vector and the background correction feature to obtain a category probability distribution and a confidence score, performing category decision based on the category probability distribution and the confidence score to obtain a target category label and category confidence; The target category label and the category confidence are output as the target recognition result.

8. A video target recognition device based on artificial intelligence, characterized in that: include: an acquisition module, configured to acquire a video frame data sequence of a target video, perform multi-dimensional analysis on the video frame data sequence to obtain global video frame information and local region of interest information, and generate a target feature map based on the global video frame information and the local region of interest information; A construction module is used to construct a deep map of the target feature map to obtain a time series association map, perform a topological structure analysis on the time series association map to obtain dynamic structure nodes, and generate a node feature vector based on the dynamic structure nodes; An analysis module is configured to perform a multi-layer attention mechanism on the node feature vector to obtain a weighted feature vector, and perform a temporal pattern analysis on the weighted feature vector to obtain a temporal evolution feature; A fusion module, configured to fuse the global video frame information, the local region of interest information, and the temporal evolution features to obtain a semantic representation vector; The classification module is used to input the semantic representation vector into a preset adaptive Transformer model for target recognition to obtain a target recognition result.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the processor is enabled to execute the video target recognition method based on artificial intelligence according to any one of claims 1 to 7.

Images