A park intelligent behavior recognition image processing method based on a large model

Abstract

The application discloses a park intelligent behavior recognition image processing method based on a large model, which comprises the following steps: acquiring a video stream containing multiple image frames, constructing a high-dimensional attribute matrix to represent the video stream, wherein each dimension represents different features extracted from the video; using a large model of multiple cross-time resolution modules to perform park behavior recognition, automatically determining the splitting requirement of the high-dimensional attribute matrix, and splitting it into a first attribute matrix and a second attribute matrix, which respectively contain different categories of feature dimensions; intelligently adjusting the number of time pooling layers to perform time down-sampling on the second attribute matrix when necessary; processing the two attribute matrices in parallel, respectively extracting spatial features and time features; and outputting the prediction result about the park behavior. The application can accurately and efficiently recognize personnel behavior, vehicle behavior and abnormal behavior, provide intelligent support for park safety management, resource scheduling and abnormal monitoring, and improve the park management efficiency and intelligent level.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to an image processing method for intelligent behavior recognition in a park based on a large model. Background Technology

[0002] With the rapid development of information technology, artificial intelligence, and image processing technology, intelligent management of industrial parks has gradually become an important part of modern urban management. The demand for intelligent behavior recognition, monitoring, and management within industrial parks is increasing, especially in areas such as security monitoring, resource scheduling, personnel management, and environmental monitoring, which have become key focuses for park managers. Image processing technology, as one of the core supporting technologies for intelligent behavior recognition in industrial parks, has applications covering multiple areas such as personnel behavior recognition, vehicle monitoring, and environmental detection. To improve the intelligence and automation level of industrial park management, intelligent behavior recognition technology based on video image streams has received widespread attention and has been gradually applied to practical systems. However, existing intelligent behavior recognition methods for industrial parks still face many technical challenges.

[0003] Traditional image processing techniques primarily rely on feature extraction, classification, and recognition algorithms, such as traditional computer vision methods based on hand-designed features, including HOG (Histogram of Oriented Gradients), SIFT (Scale-Invariant Feature Transform), and SURF.

[0004] Image recognition techniques, such as those using speed-up robust features and template matching, are characterized by the need for complex manual feature design, low computational efficiency, and high susceptibility to environmental changes such as variations in lighting intensity, viewing angle, and occlusion. However, with the rapid rise of deep learning technologies, particularly the widespread application of convolutional neural networks (CNNs) and recurrent neural networks (RNNs), image recognition technology has made significant progress in both accuracy and speed, especially in behavior recognition and dynamic scene analysis.

[0005] However, despite the outstanding performance of deep learning technology in image recognition, its practical applications still face challenges such as enormous computational demands and strong data dependence. In the practical application of intelligent behavior recognition in industrial parks, video image streams are typically continuous and high-dimensional. Extracting effective features from these streams and performing efficient recognition while ensuring computational efficiency remains a current technological challenge.

[0006] Realizing behavior recognition within a park requires processing massive amounts of continuous video stream data in real time, especially in large parks where the video data volume is enormous and structurally complex. The key to achieving real-time behavior recognition lies in ensuring recognition accuracy while avoiding computational delays caused by excessive data volume. Spatiotemporal feature extraction from video streams is particularly important in this process. Spatiotemporal feature extraction involves the comprehensive analysis of spatial information (such as object shape, color, and position) and temporal information (such as dynamic changes and motion trajectories of objects) within video frames. Most existing methods employ single spatiotemporal feature extraction techniques, making it difficult to simultaneously achieve efficient extraction of multi-dimensional spatiotemporal features and accurate behavior recognition.

[0007] Furthermore, traditional methods fall short in handling changes in people, objects, and scenes within video streams, particularly in terms of scale and speed. For instance, within a park, the temporal and hierarchical characteristics of human behavior and vehicle movement are complex, making it difficult for traditional recognition methods to capture multi-scale, multi-temporal resolution information, resulting in poor performance in identifying fine-grained behaviors. For example, in park surveillance, there are significant behavioral differences between fast-moving and slow-moving targets (such as pedestrians and fast-moving vehicles), and existing methods have limited effectiveness in distinguishing these behaviors. Summary of the Invention

[0008] To address the aforementioned problems in the existing technology, this invention proposes a campus intelligent behavior recognition image processing method based on a large model, the method comprising:

[0009] Acquire a video stream containing multiple image frames;

[0010] A high-dimensional attribute matrix is ​​constructed to represent the received video stream. The high-dimensional attribute matrix has multiple dimensions, where each dimension represents a different feature extracted from the video stream of the park.

[0011] A large model with multiple time-resolution modules is used to perform park behavior recognition. The large model automatically determines the splitting requirement of the high-dimensional attribute matrix and splits the high-dimensional attribute matrix according to the determination result, resulting in two independent attribute matrices, wherein:

[0012] The first attribute matrix contains the first set of dimensions;

[0013] The second attribute matrix contains a second set of dimensions;

[0014] The large model intelligently adjusts the number of time pooling layers for the second attribute matrix, and performs time downsampling only when necessary, based on actual needs.

[0015] The large model processes the two attribute matrices in parallel, processing the first attribute matrix and the second attribute matrix after time downsampling, respectively.

[0016] Output predictions about the behavior of the park in the received video stream.

[0017] The video stream is a dynamic video stream captured in real time by multiple cameras within the park, and the video stream is transmitted at a predetermined frame rate.

[0018] Each dimension of the high-dimensional attribute matrix represents spatial features, temporal features, or a combination of both extracted from the video stream. The spatial features include a person's appearance, posture, color, and texture, while the temporal features include a person's movement trajectory, speed, and behavioral changes.

[0019] The large model uses a deep convolutional neural network (CNN) to extract spatial features and a recurrent neural network (RNN) or a 3D convolutional neural network to process temporal features.

[0020] The first attribute matrix after splitting contains spatial features related to human behavior, and the second attribute matrix contains motion features related to vehicle behavior.

[0021] The time pooling layer intelligently adjusts the number of pools to flexibly adjust the compression level of the time dimension according to the actual needs of the video content.

[0022] The parallel processing includes using a convolutional neural network (CNN) to process spatial features in the first attribute matrix and using a recurrent neural network (RNN) to process time-series features in the second attribute matrix.

[0023] The output prediction results of park behavior include the identification results of personnel behavior, vehicle behavior, and abnormal behavior detection results within the park.

[0024] The behavior recognition results are used for decision support in the park management system, including security early warning, resource scheduling, abnormal behavior monitoring, and personnel and equipment management.

[0025] The behavior recognition system further supports self-learning and optimization.

[0026] Beneficial effects:

[0027] This invention achieves accurate behavior recognition of video streams by efficiently constructing a high-dimensional attribute matrix and combining it with the automatic judgment and dynamic adjustment capabilities of a large model. Through intelligent decomposition and parallel processing of the high-dimensional attribute matrix, spatial and temporal features can be extracted separately, improving recognition accuracy and processing efficiency. The number of temporal pooling layers is intelligently adjusted, and temporal downsampling is dynamically optimized based on behavioral complexity, ensuring effective information retention and reducing computational burden. Ultimately, this invention can quickly output park behavior prediction results, providing intelligent support for security monitoring, resource scheduling, and abnormal behavior detection, significantly improving the level of intelligence in park management. Attached Figure Description

[0028] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to unduly limit the invention. In the drawings:

[0029] Figure 1 The method steps of the present invention are shown in the diagram. Detailed Implementation

[0030] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The illustrative embodiments and descriptions are only used to explain the present invention and are not intended to limit the present invention.

[0031] The present invention will be described in detail below with reference to embodiments. The image stream used in the embodiments is a dynamic video stream of the park captured by the park's video surveillance system. Through the intelligent behavior recognition method based on the video stream, the behavioral features of people, vehicles and other objects in the park can be effectively extracted from the video stream, and behavior recognition and prediction can be performed.

[0032] As attached Figure 1 As shown in the figure, this embodiment details a park intelligent behavior recognition image processing method based on a high-dimensional attribute matrix and a cross-temporal resolution module. This method can accurately extract spatiotemporal features from the park's video stream and perform behavior recognition through an intelligent behavior recognition model. It is applicable to various scenarios such as park monitoring, personnel management, vehicle dispatching, and security monitoring.

[0033] Step 1: Obtain a video stream containing multiple image frames

[0034] In this embodiment, multiple high-resolution surveillance cameras are deployed within the park to capture dynamic video streams in real time. Each camera's field of view covers different areas of the park, ensuring comprehensive monitoring of all activities within the park. Each video stream consists of multiple consecutive image frames, each corresponding to a specific point in time, showcasing static or dynamic information of the park scene. The frame rate of the video stream is typically set to 30 frames per second (FPS), but can be adjusted according to actual needs to ensure smooth, latency-free images and adaptability to different park monitoring scenarios.

[0035] Each frame of the image may contain multiple moving targets, including but not limited to pedestrians, vehicles, goods handling equipment, robots, etc. The behavior and movement of these targets within the park are the key focus of the intelligent behavior recognition system of this invention. In addition, the video stream may also contain additional environmental information, such as temperature, humidity, light intensity, and noise levels. This information is crucial for understanding the overall environmental state and behavioral patterns within the park. For example, at night or in low-light conditions, the image quality of the camera may be affected, thus impacting the effectiveness of target detection and behavior recognition. Therefore, the system needs powerful image enhancement and compensation techniques to ensure accurate target recognition under various environmental conditions.

[0036] In practical applications, the video stream captured by the camera, after certain processing, becomes the input data source for the intelligent behavior recognition system of this invention. This video stream provides rich spatiotemporal information for subsequent behavior analysis, covering features such as the target's movement trajectory, speed, and behavioral changes, serving as the foundation for various behavior recognition and prediction tasks within the park. By acquiring this high-quality video stream data in real time, the system can respond promptly to various activities within the park, thereby providing accurate and real-time decision support for park managers.

[0037] Step 2: Construct a high-dimensional attribute matrix to represent the video stream

[0038] The acquired video stream, after preprocessing, is used as input data to construct a high-dimensional attribute matrix. This process aims to effectively organize spatiotemporal information into a high-dimensional matrix by extracting multidimensional features from each frame of the image, in order to perform further behavior analysis and prediction.

[0039] First, each frame in a video stream requires preprocessing, typically including denoising, image enhancement, and normalization. These preprocessing steps aim to improve image quality, providing clearer and more representative image data for subsequent feature extraction. Image enhancement techniques can improve image quality in low-light environments, while denoising can eliminate interference caused by poor shooting conditions (such as high-noise environments).

[0040] On the preprocessed image data, a deep convolutional neural network (CNN) is first used to extract the spatial features of the images. A convolutional neural network is a deep learning model based on local receptive fields and weight sharing, which can effectively and automatically learn key spatial features from the original images. Through the stacking of multiple convolutional layers, CNNs can abstract low-level features (such as edges, corners, and textures) and high-level features (such as the shape, appearance, and category of objects) from the image layer by layer. In the campus video stream, deep CNNs can extract important spatial information from each frame of the image, such as:

[0041] Character appearance: The shape, outline, and clothing characteristics of each target.

[0042] Posture: The target's posture and movement.

[0043] Color and texture: The color distribution and texture details of the target, which are especially important for distinguishing different types of targets.

[0044] In this way, deep CNNs can not only extract the static spatial features of images, but also identify the shape, action and relative position of different targets.

[0045] Next, combining the temporal dimension of the video stream, recurrent neural networks (RNNs) or 3D convolutional networks are used to extract temporal information. Temporal information includes changes in the target along the time axis, such as the target's trajectory, velocity, acceleration, and changes in behavioral patterns. In park scenarios, temporal information is crucial for recognizing dynamic behaviors (such as people walking, vehicles moving, and objects being moved).

[0046] Recurrent Neural Networks (RNNs): Primarily used to capture temporal dependencies in sequential data, suitable for modeling the sequential relationships between each time point in a video stream. RNNs can predict the temporal features of the current frame based on input from historical frames. For park behavior recognition, RNNs can effectively capture changes in the motion trajectory of people or objects, as well as transitions in behavioral states.

[0047] 3D Convolutional Networks: For video streams, 3D convolutional networks can extract spatiotemporal features simultaneously by incorporating temporal processing into the convolution operation. The advantage of this method is that it can perform convolution operations directly on both the spatial and temporal dimensions of the video data, making it suitable for processing continuous action recognition in dynamic scenes.

[0048] In the process of extracting temporal information, the output of RNN or 3D convolutional networks usually includes information such as the target's trajectory, velocity, and acceleration over a period of time. This information is important for tasks such as determining the type of behavior and predicting future actions.

[0049] Through the feature extraction steps described above, the spatial and temporal information of each frame is converted into a set of feature vectors. These feature vectors are then used to construct a high-dimensional attribute matrix based on their different dimensions. Specifically, each dimension of the matrix represents different feature information from the video stream, including:

[0050] The first dimension represents spatial information such as the shape, edges, and contours of the image, and is usually extracted by a convolutional neural network.

[0051] The second dimension represents the color and texture information of the image, used to describe the color distribution and surface features of the target.

[0052] The third dimension represents the target's trajectory and velocity information, which is usually extracted by temporal modeling networks (such as RNNs or 3D convolutional networks).

[0053] The fourth dimension represents the temporal characteristics of changes in target behavior, including the acceleration of the target and behavioral shifts.

[0054] In this way, the spatial and temporal features of all image frames are effectively organized into a high-dimensional matrix, with each dimension's feature vector carrying different types of information. For example, the high-dimensional attribute matrix of a particular image frame may contain color information, texture features, target position changes, motion direction, etc., and these features combined can comprehensively represent the image frame's performance in both spatial and temporal dimensions.

[0055] The size of a high-dimensional attribute matrix typically varies with the length of the video stream and the complexity of the images. For example, in a scene containing a 10-second video stream (30 frames per second), each frame might be 224x224 pixels in size, and the feature vector of each image could be as high as 2048 dimensions. If a high-dimensional attribute matrix is ​​used to represent all features, the total size of the matrix could reach hundreds of gigabytes. Therefore, efficient data storage and processing methods are essential for handling large-scale video data.

[0056] In this embodiment, distributed computing and GPU acceleration technologies are employed to optimize storage and computational efficiency. All feature matrix data is first stored in a distributed database and then loaded and processed on demand. The feature extraction process for each frame of image is processed in parallel by the GPU, which significantly improves computational efficiency and reduces processing time.

[0057] The constructed high-dimensional attribute matrix provides rich feature information for subsequent behavior recognition, target detection, and video analysis. In the park's intelligent behavior recognition system, based on these high-dimensional matrices, the system can perform multi-level feature fusion and behavior analysis to identify different behavioral patterns within the park (such as crowd gathering, abnormal movement, and illegal parking). Furthermore, utilizing the spatiotemporal features in the high-dimensional attribute matrix, the system can also predict behavior, identify potential abnormal behaviors in advance, and trigger alarms, further improving the park's security and intelligent management level.

[0058] By employing this efficient spatiotemporal feature construction method, this invention can extract richer and more accurate behavioral information from park video streams, solving the problem of insufficient spatiotemporal feature extraction capabilities in traditional video analysis methods, and providing strong technical support for intelligent management and behavior prediction in parks.

[0059] Step 3: Automatically determine the need for splitting the high-dimensional attribute matrix in the large model.

[0060] After constructing the high-dimensional attribute matrix, the next step is to split the matrix. Since park behavior recognition involves various types of behavior (such as walking, running, cycling, parking, etc.), the spatiotemporal characteristics of these behaviors may differ significantly. Therefore, based on the spatiotemporal characteristics required for different types of behavior, the high-dimensional attribute matrix must be reasonably split to optimize subsequent behavior recognition performance. To this end, this invention employs an intelligent large-scale model, which, based on deep learning methods, can automatically determine whether the high-dimensional attribute matrix needs to be split and decide how to split it.

[0061] The core of this intelligent large-scale model is a deep learning-based behavior recognition model. It automatically analyzes the spatiotemporal features in a high-dimensional attribute matrix, determines which features are significant for the current behavior recognition task, and makes splitting decisions based on this information. Specifically, the working principle of the intelligent large-scale model can be summarized in the following steps:

[0062] Feature Importance Analysis: The model first analyzes the spatial and temporal features of each dimension in the high-dimensional attribute matrix to evaluate the contribution of each feature to behavior recognition. In park behavior recognition, some behaviors (such as walking and running) rely more on spatial features (such as a person's shape and posture), while other behaviors (such as the rapid movement of a vehicle) may rely more on temporal features (such as speed and acceleration). Therefore, the model assigns a weight to each dimension based on the importance of the feature in the recognition.

[0063] Behavior type recognition: Based on the input video stream, the model automatically determines which features are most critical according to different behavior types (such as pedestrians, vehicles, objects, etc.). For example, spatial features may be more important for walking behavior, while temporal features may dominate for fast-moving behaviors (such as vehicle movement). The model automatically and selectively splits specific dimensions in the matrix based on the currently identified behavior type.

[0064] Splitting Decision Generation: After analyzing the importance of features and the requirements of behavior, the model generates splitting decisions. If the model identifies a type of behavior that requires processing multiple dimensions simultaneously (such as both spatial and temporal features), it will automatically decide whether to split the high-dimensional attribute matrix and divide it into multiple sub-matrices according to specific needs. The split sub-matrices will focus on different feature dimensions for independent processing, avoiding interference between different features and improving recognition accuracy.

[0065] Under the control of the intelligent large-scale model, the entire splitting process will be carried out automatically according to the different characteristic requirements of the behavior. The specific splitting process is as follows:

[0066] First, the model determines whether the high-dimensional attribute matrix needs to be split. If splitting is required, the model automatically generates a splitting strategy based on the following conditions:

[0067] Behavior Type: For complex park behaviors (such as fast-moving vehicles and slow-walking pedestrians), the model determines which features need to be separated based on the current behavior type. For example, pedestrian behavior features typically include relatively stable spatial information (such as shape and posture) and relatively smooth temporal information (such as gait and walking speed), while vehicle behavior features include more dynamic temporal information (such as acceleration and driving trajectory). The model decides whether to split the matrix based on these differences to process these features separately.

[0068] Spatiotemporal feature differences: The model analyzes the differences in spatiotemporal features. For example, there are significant differences in the temporal information such as speed and acceleration between fast-moving vehicles and slow-moving pedestrians within a park. In such cases, the model decides to split the original matrix into two parts: one part specifically handles features that change rapidly over time, and the other part handles features that change slowly over time.

[0069] After determining that splitting is necessary, the model decides on the dimensionality allocation of the resulting sub-matrix based on the feature type and importance of each dimension. Specifically, the split high-dimensional attribute matrix will be divided into two independent sub-matrices, each representing different types of feature information. The structure of the split matrix is ​​as follows:

[0070] The first submatrix specifically handles spatial features and some temporal features. In park behavior recognition, spatial features (such as the target's shape, color, and posture) are crucial for the recognition of most behaviors. For behaviors such as walking, spatial features can usually accurately characterize behavioral patterns; therefore, this submatrix will focus on processing spatial features and some low-frequency temporal information.

[0071] The second submatrix specifically handles temporal features, especially those that change frequently. For example, in the case of fast-moving vehicles, temporal features (such as speed, acceleration, and trajectory) change rapidly and are closely related to behavior recognition. The split second submatrix will focus on processing this rapidly changing temporal information to accurately identify dynamic behaviors.

[0072] The intelligent large-scale model employs an adaptive dimensional selection and allocation strategy. During the decomposition process, the model not only considers behavior type but also dynamically adjusts the allocation strategy for each dimension. For specific scenarios or behaviors, the model may increase the weight of certain dimensions or decrease the weight of others. For example, when identifying walking behavior in a park, the model may increase the weight of spatial features while decreasing the weight of temporal features; conversely, for fast-moving targets (such as vehicles), the weight of temporal features will be increased, while the weight of spatial features may be appropriately decreased.

[0073] The two sub-matrices after splitting will be processed separately. The dimension of each sub-matrix focuses on different types of features, enabling the model to perform deep learning and behavior analysis specifically for a certain type of feature, avoiding unnecessary feature interference, and thus improving the accuracy and efficiency of recognition.

[0074] The first submatrix will be fed into a deep learning network for further spatial feature processing. This process may involve multiple convolutional and pooling layers to extract more abstract spatial information, which will then be used for behavior classification and prediction.

[0075] The second submatrix will be fed into a temporal analysis network (such as an RNN or a 3D convolutional network) for processing. Through temporal modeling, the system can analyze the target's motion trajectory and behavioral changes, thereby accurately identifying dynamic behavior.

[0076] Through this automated decomposition process, the intelligent large model can flexibly adjust the processing method of the high-dimensional attribute matrix according to the actual scene and behavior type, thereby achieving optimal results in different types of behavior recognition. This process not only improves the accuracy of the park behavior recognition system, but also optimizes processing efficiency. Especially when facing large-scale video streams and complex behavior scenarios, it can effectively reduce the consumption of computing resources and improve the real-time performance and intelligence level of the system.

[0077] Step 4: Split the high-dimensional attribute matrix into two independent attribute matrices

[0078] In step 3, the intelligent big data model automatically determines whether to split the high-dimensional attribute matrix based on the analysis of behavior type and spatiotemporal features, and decides on the split dimension based on the determination result. Next, the system splits the high-dimensional attribute matrix into two independent attribute matrices, each used to process feature information of different categories. This splitting process effectively separates the features of different behavior types, thereby improving the recognition accuracy of various behaviors within the park. The following is a detailed technical implementation plan for this step.

[0079] A high-dimensional attribute matrix contains information from different feature dimensions, including spatial features (such as the target's shape, position, and posture) and temporal features (such as the target's velocity, acceleration, and trajectory). The goal of campus behavior recognition is to identify and classify various behavior types, such as walking, running, cycling, and vehicle movement, from video streams in real time. Different behavior types may depend on different feature categories.

[0080] In this context, the system splits the high-dimensional attribute matrix into two independent matrices to process features related to different behavior categories separately, ensuring that each feature category can be processed and analyzed independently. This prevents interference between features of different categories, allowing the model to more accurately identify different types of behavior.

[0081] Based on the judgment results of the intelligent large model in step 3, the system has identified which feature dimensions are closely related to the current behavior type. Therefore, the dimensions to be split will be selected based on the following rules:

[0082] Spatial Feature Dimension: For features that rely on spatial information for behavioral types (such as pedestrians, robots, etc.), the system will include these spatial features as part of the first attribute matrix. These spatial features include, but are not limited to, the target's shape, posture, gait, and positional changes. For example, a pedestrian's movement trajectory, gait, and head posture are all spatial features, and they are closely related to the target's static appearance and movement patterns.

[0083] Temporal Feature Dimension: For features that depend on temporal information for behavior types (such as vehicles, fast-moving objects, etc.), the system will include these temporal features as part of the second attribute matrix. Temporal features include the target's speed, acceleration, direction change, and trajectory. For example, vehicle speed, acceleration, and steering angle are temporal features, and they are key factors in distinguishing fast-moving targets.

[0084] The split high-dimensional attribute matrix will consist of two independent attribute matrices, each containing a dimension related to a feature category. The structure of the split attribute matrix can be described as follows:

[0085] First attribute matrix: This matrix contains the dimensions of spatial features and is mainly used to represent the static and dynamic spatial information of the target. For example, for a pedestrian, the first attribute matrix might include the following dimensions:

[0086] The physical characteristics of a person (such as outline, shape, posture, etc.);

[0087] A person's gait characteristics (such as stride length, stride frequency, posture changes, etc.);

[0088] The location information of the person (such as their coordinate position in the image, relative motion trajectory, etc.).

[0089] The dimensions of the first attribute matrix will be based on these spatial features, and features will be extracted through a deep convolutional neural network (CNN) to provide spatial information for the behavior recognition model.

[0090] Second attribute matrix: This matrix contains the temporal feature dimension and is primarily used to represent the dynamic behavior information of the target. For example, for a vehicle, the second attribute matrix might include the following dimensions:

[0091] Vehicle speed (e.g., distance traveled per unit time);

[0092] Acceleration (such as a vehicle's acceleration, deceleration, turning, etc.);

[0093] Trajectory of motion (such as the vehicle's driving route, steering angle, etc.).

[0094] The second attribute matrix will be based on temporal features, processed using recurrent neural networks (RNNs), long short-term memory networks (LSTMs), or 3D convolutional neural networks to capture the dynamic changes of the target over time.

[0095] After splitting the matrix, the two independent attribute matrices will handle spatial and temporal features separately. Ensuring that features of different categories are processed independently has the following advantages:

[0096] Avoiding feature interference: By splitting spatial and temporal features into different matrices, interference between features is avoided. For certain behaviors (such as fast-moving vehicles), temporal features may change very rapidly, while spatial features change relatively slowly. Processing them independently can reduce the computational complexity of the model and improve accuracy.

[0097] Improving recognition accuracy: Specializing the processing of different types of features can avoid the influence of certain irrelevant features on the behavior recognition results. For example, walking behavior usually relies on stable spatial features, while fast-moving vehicles require more sensitive temporal features. Mixing these features may lead to a decrease in recognition accuracy.

[0098] Optimizing computational efficiency: The split matrices can be processed using the most suitable network architecture based on their respective feature types. For example, spatial features are typically processed using CNNs, while temporal features are processed using RNNs or LSTMs. In this way, the system can select the optimal processing method according to different feature types, further improving computational efficiency.

[0099] The two independent attribute matrices after splitting will be sent to the subsequent behavior recognition module for processing. The specific processing flow is as follows:

[0100] Processing of the first attribute matrix: The first attribute matrix primarily handles spatial features, which are processed using a deep convolutional neural network (CNN). This network extracts high-order representations of spatial features through multiple convolutional layers and compresses the features using pooling layers. Finally, fully connected layers map the spatial features to specific behavior categories (such as walking, standing, running, etc.).

[0101] Processing of the Second Attribute Matrix: The second attribute matrix primarily handles temporal features, employing Long Short-Term Memory (LSTM) networks or 3D Convolutional Neural Networks (3D-CNN) to process temporal information. These networks can learn the changing patterns of the target along the time axis, analyze the target's trajectory, velocity, acceleration, and other features, and output the corresponding behavior category.

[0102] By splitting the high-dimensional attribute matrix into two independent attribute matrices, the system can process features of different behavior types separately, thereby improving the recognition accuracy of various behaviors within the park. This splitting process not only optimizes the feature processing method and avoids interference between different features, but also improves computational efficiency and enhances the system's real-time recognition capability in complex scenarios. The split matrices are then processed using the most suitable network for each, further ensuring the accuracy and reliability of the recognition results.

[0103] Step 5: Intelligently adjust the number of time pooling layers and perform time downsampling.

[0104] In intelligent behavior recognition within a park, the accurate extraction and processing of temporal features is crucial to the accuracy of behavior recognition. Especially when dealing with temporal features such as motion trajectories, gait changes, and object movements, effectively processing temporal information and optimizing the balance between computational complexity and accuracy becomes a core issue. To achieve this goal, this invention employs an intelligent pooling mechanism in the temporal feature processing of the second attribute matrix. This mechanism dynamically adjusts the number of temporal pooling layers and performs temporal downsampling to adapt to different behavior patterns and video content. The detailed implementation plan for this step is as follows.

[0105] In deep learning models, pooling layers are primarily used to reduce the dimensionality of input data while retaining key information, especially in temporal data processing. For temporal features in video streams (such as changes in a person's gait or the trajectory of a vehicle), temporal pooling layers can effectively reduce redundant data and improve the computational efficiency of the model.

[0106] Pooling operations typically take two forms: max pooling and average pooling. In the processing of time-series features, pooling layers can extract data within a time window, retaining the maximum (or average) value within that window, thereby achieving dimensionality reduction of the time features.

[0107] However, in practical applications, the complexity of time-series data can vary depending on the type of behavior. For simple behavior patterns (such as standing or walking slowly), too many pooling layers may result in the loss of excessive temporal detail; while for complex behavior patterns (such as fast-moving vehicles or multi-target interactions), increasing the number of pooling layers can effectively compress redundant data and improve processing efficiency.

[0108] The method for intelligently adjusting the number of pooling layers proposed in this invention dynamically adjusts the number of pooling layers based on the complexity of video content and the needs of behavioral patterns, in order to achieve temporal downsampling and computational optimization. Its core principle is as follows:

[0109] Dynamic Behavior Complexity Assessment: The system first evaluates the changes in temporal features in the video stream using a deep learning model. If the behavior in the video is relatively simple (e.g., slow gait changes or low-speed moving targets), the temporal feature changes are considered small, and the number of pooling layers can be appropriately reduced to retain more detailed information. If the behavior is more complex (e.g., fast-moving targets or complex interactions between multiple targets), the number of pooling layers can be increased to compress redundant information in the temporal dimension and improve the model's processing efficiency.

[0110] Intelligent pooling layer number adjustment: Based on the assessment of behavioral complexity, the model intelligently adjusts the number of pooling layers. The specific adjustment method is as follows:

[0111] If the behavior being identified is simple and the temporal features change relatively smoothly (such as static or slowly moving targets), reduce the number of pooling layers to avoid over-compressing temporal features and retain more details;

[0112] If the behavior being identified is complex and the temporal features change rapidly (such as fast-moving vehicles or complex crowd behavior), the number of pooling layers can be increased to reduce the time dimension and computational load.

[0113] The key to this intelligent mechanism is to evaluate the complexity of behavior through real-time analysis of temporal features by the model, and to dynamically adjust the pooling layer according to the needs.

[0114] The first step in intelligently adjusting the number of pooling layers is to perform pattern analysis on the behavior in the video stream. This analysis is based on the frequency and amplitude of changes in temporal features, as well as information such as the speed and trajectory of the target motion.

[0115] Speed ​​variation: The speed of the target in the video (such as pedestrians walking or running, vehicles moving) is an important factor in judging the complexity of the behavior. Higher speed variations are often accompanied by more complex temporal features, thus requiring more pooling layers for dimensionality reduction.

[0116] Behavioral pattern recognition: Based on convolutional neural networks (CNN) and long short-term memory networks (LSTM), the model can learn behavioral patterns in videos (such as walking, running, cycling, parking, etc.). If the behavior changes drastically, it indicates high temporal feature fluctuations, and the number of pooling layers should be appropriately increased.

[0117] Target interaction: If multiple targets interact frequently in the video (e.g., vehicles and pedestrians in traffic, multiple pedestrians in a park), this will result in more complex temporal features. The model needs to perform effective pooling processing on the time dimension to avoid excessive computation.

[0118] Based on the behavioral complexity assessment results, the system will dynamically adjust the number of pooling layers according to the following strategies:

[0119] Simple Behavior Patterns: For simple behavior patterns, such as slow walking or standing, the system reduces the number of pooling layers. For example, when the gait is relatively gentle, the model will choose fewer pooling layers to retain more temporal information and avoid excessive downsampling.

[0120] Complex Behavioral Patterns: For complex behavioral patterns, such as running or vehicle movement, the model increases the number of pooling layers. Each pooling operation extracts input within a specific time window, retaining the most important temporal information and reducing redundant data, thus achieving temporal downsampling. In this process, the system effectively compresses unnecessary time dimensions, reducing the model's computational burden while preserving key temporal features.

[0121] The number of pooling layers is adjusted using an adaptive algorithm. Specifically, if the temporal features change little within a certain time window, the number of pooling layers will be reduced; if the temporal features change drastically, the number of pooling layers will be increased.

[0122] By intelligently adjusting the number of pooling layers, the system can flexibly downsample the temporal dimension based on the complexity of the video content and the needs of behavioral patterns. For complex behavioral patterns, more pooling layers help the system reduce redundant data and optimize computing resources; while for simple behavioral patterns, it can retain more temporal details and improve recognition accuracy.

[0123] The technique of intelligently adjusting the number of time pooling layers and performing time downsampling optimizes computational efficiency while ensuring the accuracy of behavior recognition by intelligently adjusting the number of pooling layers according to changes in temporal characteristics. This flexible pooling layer adjustment mechanism, combined with the complexity of video content and the diversity of behavior types, can provide accurate and efficient intelligent behavior recognition solutions for parks in different scenarios.

[0124] Step 6: Process the two attribute matrices in parallel

[0125] In this step, by inputting the split first and second attribute matrices into the parallel processing module of the large model, the system can efficiently process and analyze different types of spatiotemporal features, thereby improving the overall performance of park behavior recognition. Specifically, the first attribute matrix is ​​mainly used for spatial feature extraction and analysis, while the second attribute matrix focuses on processing temporal information. The implementation strategy of this step is to adopt different processing strategies for these two matrices under a parallel computing framework, thereby accelerating data processing and improving the accuracy of behavior recognition.

[0126] This invention employs a parallel processing architecture, inputting the two split attribute matrices into two separate processing paths. Each path uses a specially designed deep learning model for processing; one path primarily handles spatial feature extraction, while the other focuses on processing time-series features. The core advantage of this parallel processing architecture lies in its ability to simultaneously perform deep analysis on two different types of features, avoiding the information processing bottlenecks that may exist in traditional single-feature processing methods, thereby improving processing efficiency.

[0127] The first attribute matrix mainly contains spatial features, such as the shape, color, and pose of objects in the park. To effectively extract these spatial features, the system employs a convolutional neural network (CNN). CNN is a deep learning architecture specifically designed for image processing, capable of automatically extracting local features from images through convolutional layers.

[0128] Specifically, before entering the CNN module, the first attribute matrix undergoes standardization and preprocessing steps (such as noise reduction and resizing) to ensure that the data meets the network's input requirements. The CNN network extracts multi-level features of the image through multiple convolutional and pooling layers, progressively representing them from low-level texture features (such as edges and corners) to high-level semantic features (such as human poses and object classifications).

[0129] Convolutional Layer: Each convolutional layer performs convolution operations on the input data through a set of filters (convolutional kernels), generating different feature maps. Convolutional operations can capture features of local regions in spatial information, and then extract more complex spatial information through stacked convolutions.

[0130] Pooling Layer: Pooling operations are used for downsampling, reducing the size of the feature map while retaining the most important information. Max pooling or average pooling are typically used to reduce the dimensionality of features and decrease computational cost.

[0131] After multiple processing steps through convolution and pooling layers, the spatial features extracted by the CNN will provide efficient input for subsequent behavior recognition.

[0132] The second attribute matrix contains temporal information from the video stream, such as the dynamic behavior of people and the trajectory of vehicles. To effectively process these temporal features, the system uses a recurrent neural network (RNN) for time series modeling. Unlike CNNs, RNNs can process and analyze information in the time dimension, capturing dynamic changes in time series data.

[0133] RNN Structure: RNNs use their internal recurrent structure to take the output of the previous time step as the input of the current time step, thus preserving the temporal dependencies of the time-series data. This allows RNNs to effectively handle the temporal features in video streams and capture the dynamic changes of targets over time (such as walking gait, vehicle acceleration, deceleration, etc.).

[0134] Long Short-Term Memory (LSTM) Units: To overcome the vanishing gradient problem encountered by traditional RNNs during long sequence training, the system can employ LSTM units. LSTMs control the flow of information through gating mechanisms (such as input gates, forget gates, and output gates), thereby better capturing dependencies over long time spans and making them suitable for long-term memory and change patterns of temporal features.

[0135] When processing time-series information, the second attribute matrix is ​​first preprocessed (e.g., normalized time-series data) and then input into an RNN or LSTM model. The network processes each time step in the time series sequentially through multiple layers of RNN or LSTM units, capturing temporal dependencies.

[0136] The parallel processing architecture allows spatial and temporal information to be processed simultaneously and independently. The two processing paths do not interfere with each other, maximizing the efficiency of computational resource utilization. Furthermore, after completing their respective feature extraction tasks, the two attribute matrices output feature representations processed by multiple convolutional layers (CNN) or recurrent neural network layers (RNN / LSTM). These feature representations are then further passed to the subsequent fusion module for final behavior recognition and prediction.

[0137] After the two attribute matrices have been processed separately, the system will perform data fusion. The specific fusion method may vary depending on the specific requirements; the following are some common fusion strategies:

[0138] Feature-level fusion involves concatenating or weighting spatial and temporal features extracted from two paths to form a unified feature vector. This feature vector is then input into a subsequent classifier or behavior recognition module for final behavior determination. fused .

[0139] The formula is expressed as:

[0140]

[0141] Among them, F spatial F represents the spatial eigenvectors. temporal Represents the temporal feature vector. This indicates a splicing operation.

[0142] Decision-level fusion: After the spatial and temporal feature processing paths output their respective prediction results, the outputs of the two can be weighted and averaged or voted to obtain the final behavior recognition result P. final .

[0143] The formula is expressed as:

[0144] P final =w1P spatial +w2P temporal

[0145] Among them, P spatial and P temporal These are the output prediction results for the spatial feature path and the temporal feature path, respectively, with w1 and w2 being the weight coefficients.

[0146] In some complex application scenarios, it may be necessary to consider more input features, such as environmental features (temperature, humidity, lighting, etc.). In this case, the system may need to introduce cross-modal fusion technology to integrate feature information from different sources to achieve more accurate behavior recognition.

[0147] By using parallel processing, the system can efficiently extract spatial and temporal information, improving computational efficiency and reducing time overhead. In traditional single-channel processing systems, the extraction of spatial and temporal features is often performed serially, which can lead to prolonged processing time. However, in a parallel architecture, the processing tasks for both feature types can be performed simultaneously, greatly improving the overall processing capacity of the system, especially in real-time video surveillance and rapid response scenarios.

[0148] By processing two attribute matrices in parallel, the system achieves efficient parallel computation in the extraction and analysis of spatial and temporal features. This processing architecture not only improves computational efficiency but also ensures high accuracy in behavior recognition. The use of parallel structures of CNN and RNN enables the system to consider both spatiotemporal features when handling complex behavior recognition tasks, exhibiting strong application adaptability and real-time performance.

[0149] Step 7: Output the park behavior recognition results

[0150] Following the aforementioned image processing and behavior analysis, the system will ultimately generate behavior recognition results for various targets within the park (such as pedestrians and vehicles). These results will reflect the specific behavioral state of the targets at each time point or in each time series within the video stream, and will be categorized and output according to different target types (such as human behavior, vehicle behavior, etc.). Through accurate behavior recognition of targets within the park, the system can not only monitor the dynamic changes of the park in real time, but also make corresponding management decisions based on the recognition results, providing effective support for intelligent park management.

[0151] Based on the input video stream, the system can output behavior predictions for each target based on the processing results of each frame of the image or video sequence. These predictions are comprehensively judged based on the spatiotemporal features in the video stream and the spatial and temporal features extracted in the preceding steps. The output behavior recognition results may include, but are not limited to, the following common types:

[0152] Personnel Behavior Recognition: The system can identify and classify specific behavioral patterns of people within the park, such as walking, running, standing, talking, and sitting. By analyzing the movement trajectories of people in the video stream, combined with the recognition of their physical characteristics and temporal behavior, the system can effectively distinguish different types of behavior.

[0153] Vehicle Behavior Recognition: For vehicles within the park, the system can identify and classify their behaviors, such as driving, parking, accelerating, braking, and U-turns. By analyzing vehicle trajectories, speed changes, and interactions with the surrounding environment, the system can accurately extract vehicle behavioral characteristics.

[0154] Abnormal Behavior Detection: The system can not only identify routine behaviors, but also automatically identify and flag potential abnormal behaviors. For example, if a target suddenly stops moving, experiences abnormal speed changes, or deviates from the planned route, the system can automatically identify and issue an alarm, prompting the administrator to take further action.

[0155] These behavioral recognition results, through further processing, can provide detailed analysis of real-time dynamics within the park, offering decision support for the park management system.

[0156] To present the recognition results more intuitively, the system typically visualizes the behavior prediction results and displays them to park management personnel in graphical, textual, or other easily understandable ways. These visualizations not only help management personnel understand the dynamics within the park in real time but also serve as important data analysis and decision support for subsequent processes.

[0157] For example, the system can display the recognition results in the following ways:

[0158] Behavior type labeling: Each target in the video stream is labeled with its current behavior type (e.g., "walking", "driving", "parking"). This label is usually displayed dynamically on the video screen for easy observation and understanding by managers.

[0159] Behavioral Timeline: For each target, the system can generate a behavioral timeline diagram, showing its behavioral changes over different time periods. This timeline diagram helps analyze the target's behavioral patterns and further optimize park management strategies.

[0160] Anomaly Warning Notification: The system can also automatically determine whether abnormal behavior has occurred during the behavior recognition process and promptly push notifications to management personnel. For example, if it detects that someone has entered a restricted area during an abnormal time period, or that a vehicle is parked in a no-parking zone, the system will automatically trigger an alert and send a notification to management personnel.

[0161] The final behavior recognition results not only provide park managers with a real-time monitoring view, but also directly impact the park's intelligent decision-making system. Below are some typical application scenarios for behavior recognition results:

[0162] Automatic security alerts: When the system detects potential security threats or abnormal behavior (such as unusual lingering of people or illegal parking of vehicles), it will automatically generate a security alert to remind relevant personnel to take necessary actions. For example, the system can automatically issue a security alarm when crowds gather or other unusual activities occur within the park.

[0163] Intelligent resource scheduling: The system can intelligently schedule park resources (such as parking spaces, routes, and cleaning staff) by monitoring the distribution of people and vehicles in real time. By analyzing behavior recognition results, the park management system can predict which areas may experience resource shortages and then optimize scheduling accordingly.

[0164] Abnormal Behavior Monitoring and Analysis: The system can monitor abnormal behaviors within the park over a long period and accumulate and analyze the data. For example, for frequently occurring violations within the park, the system can generate behavior reports, analyze the patterns of these behaviors, and help managers develop effective management measures and improvement plans.

[0165] Personnel and Equipment Management: Based on behavioral recognition results, the park management system can accurately manage personnel and equipment within the park. For example, the system can assign work tasks to employees or optimize workflows based on personnel behavior data, and can also promptly detect equipment malfunctions or changes in demand by monitoring equipment usage status.

[0166] In addition to outputting real-time behavior recognition results for the park, the system also supports further analysis and optimization of these results. Through machine learning and data mining techniques, the system can learn from historical data and continuously optimize its behavior recognition algorithm, improving the accuracy and intelligence of the recognition.

[0167] Data Analysis and Feedback: The system can periodically summarize and analyze the behavioral recognition results within the park, identifying potential problems or risks in park management. Based on this data analysis, managers can adjust park management strategies or improve existing management processes.

[0168] Self-learning and optimization: As the amount of data increases, the system can continuously improve its behavior recognition model through a self-learning mechanism. For example, the system can analyze the types of behaviors that are misidentified and adjust the model parameters according to the actual situation, thereby continuously improving the accuracy and robustness of recognition.

[0169] Through the above steps, the park's intelligent behavior recognition system can not only monitor various behaviors within the park in real time, but also automatically react based on the recognition results, providing real-time decision support. Whether in security monitoring, resource scheduling, abnormal behavior detection, or personnel and equipment management, behavior recognition results can effectively improve the level of intelligence in park management. With continuous technological optimization and development, the park behavior recognition system of this invention will demonstrate its powerful value in a wider range of application scenarios.

[0170] The intelligent behavior recognition image processing method for industrial parks of this invention can significantly improve processing efficiency while ensuring high-precision recognition. By employing a high-dimensional attribute matrix and its decomposition processing technology, the spatiotemporal features of different targets within the park can be effectively extracted, and personalized processing strategies can be adopted for different targets. Simultaneously, the dynamic adjustment of the number of intelligent pooling layers effectively avoids unnecessary computation and improves the system's response speed. The parallel processing approach enables multi-target behavior recognition to be performed simultaneously, thereby further improving the system's efficiency and real-time performance.

[0171] The technical solution described in this embodiment can be widely applied to security monitoring, behavior analysis, and resource scheduling within the park, greatly improving the level of intelligent park management and providing park managers with accurate and real-time behavior recognition data.

[0172] The above description is only a preferred embodiment of the present invention. Therefore, all equivalent changes or modifications made to the structure, features and principles described in the claims of this patent application are included in the scope of this patent application.

Claims

1. A method for intelligent behavior recognition image processing in a park based on a large model, characterized in that: The method includes: Obtain a video stream containing multiple image frames; A high-dimensional attribute matrix is ​​constructed to represent the received video stream. The high-dimensional attribute matrix has multiple dimensions, where each dimension represents a different feature extracted from the video stream of the park. A large model with multiple time-resolution modules is used to perform park behavior recognition. The large model automatically determines the splitting requirement of the high-dimensional attribute matrix and splits the high-dimensional attribute matrix according to the determination result, resulting in two independent attribute matrices, wherein: The first attribute matrix contains the first set of dimensions; The second attribute matrix contains a second set of dimensions; The large model intelligently adjusts the number of time pooling layers for the second attribute matrix, and performs time downsampling only when necessary, based on actual needs. The large model processes the two attribute matrices in parallel, processing the first attribute matrix and the second attribute matrix after time downsampling, respectively. Output predictions about the behavior of the park in the received video stream; The first attribute matrix after splitting contains spatial features related to human behavior, and the second attribute matrix contains motion features related to vehicle behavior; The time pooling layer intelligently adjusts the number of pooling operations to flexibly adjust the compression level of the time dimension according to the actual needs of the video content. The time-based pooling layer intelligently adjusts the number of pooling operations, including: If the behavior being identified is simple and the temporal features change relatively smoothly, reduce the number of pooling layers; If the behavior being identified is complex and the temporal features change rapidly, increase the number of pooling layers. The large model automatically determines the splitting requirements of the high-dimensional attribute matrix, including: Feature importance analysis: Analyze the spatial and temporal features of each dimension in the high-dimensional attribute matrix to evaluate the contribution of each feature to behavior recognition; Behavior type recognition: Based on the input video stream, key features are automatically determined according to different behavior types; Split Decision Generation: After analyzing the importance of features and the needs of behavior, split decisions are generated. For example, if it is identified that a certain type of behavior needs to be processed in multiple dimensions at the same time, it will automatically decide whether to split the high-dimensional attribute matrix and split the matrix into multiple sub-matrices according to specific needs. The split sub-matrices will focus on different feature dimensions.

2. The image processing method for intelligent behavior recognition in a park based on a large model as described in claim 1, characterized in that: The video stream is a dynamic video stream captured in real time by multiple cameras within the park, and the video stream is transmitted at a predetermined frame rate.

3. The image processing method for intelligent behavior recognition in a park based on a large model as described in claim 1, characterized in that: Each dimension of the high-dimensional attribute matrix represents spatial features, temporal features, or a combination of both extracted from the video stream. The spatial features include a person's appearance, posture, color, and texture, while the temporal features include a person's movement trajectory, speed, and behavioral changes.

4. The image processing method for intelligent behavior recognition in a park based on a large model as described in claim 1, characterized in that: The large model uses a deep convolutional neural network (CNN) to extract spatial features and a recurrent neural network (RNN) or a 3D convolutional neural network to process temporal features.

5. The image processing method for intelligent behavior recognition in a park based on a large model as described in claim 1, characterized in that: The parallel processing includes using a convolutional neural network (CNN) to process spatial features in the first attribute matrix and using a recurrent neural network (RNN) to process time-series features in the second attribute matrix.

6. The image processing method for intelligent behavior recognition in a park based on a large model as described in claim 1, characterized in that: The output prediction results of park behavior include the identification results of personnel behavior, vehicle behavior, and abnormal behavior detection results within the park.

7. The image processing method for intelligent behavior recognition in a park based on a large model as described in claim 1, characterized in that: The behavior recognition results are used for decision support in the park management system, including security early warning, resource scheduling, abnormal behavior monitoring, and personnel and equipment management.

8. The image processing method for intelligent behavior recognition in a park based on a large model as described in claim 1, characterized in that: The behavior recognition system further supports self-learning and optimization.

Images