Method, device and equipment for dynamic scene recognition of objects based on AI vision

Abstract

The present invention relates to the technical field of AI vision, and discloses a method and device for dynamic scene recognition of objects based on AI vision. The present invention collects video streams of dynamic scenes to generate video frame sequences, inputs the video frame sequences into an AI vision dynamic recognition model composed of a convolutional neural network (CNN) and an LSTM network, performs model training, and adds an attention mechanism and a multi-task mechanism to generate a final optimized training version to achieve accurate recognition of objects in dynamic scenes through AI vision. The present solution combines CNN and LSTM networks, and the model can effectively extract spatiotemporal features in video streams and improve the ability to recognize dynamic objects. The attention mechanism can automatically focus on key areas in the video to improve recognition accuracy. The multi-task mechanism enables the model to complete multiple tasks at the same time, such as object detection and action recognition, thereby enhancing the versatility and adaptability of the system and solving the problem of difficulty in accurately identifying changes in objects in dynamic scenes in the prior art.

Description

Technical Field

[0001] The present invention relates to the technical field of AI vision, and in particular to a method, device and equipment for dynamic scene recognition of objects based on AI vision. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, computer vision has become one of the core technologies in many application fields. Especially in the field of dynamic scene recognition of objects, AI vision systems can realize efficient recognition and analysis of scenes and objects by analyzing image information in video streams. Dynamic scene recognition of objects has a wide range of applications in the fields of autonomous driving, security monitoring, smart homes, industrial automation, etc. It can not only improve the intelligence level of machines, but also significantly improve the accuracy of scene understanding.

[0003] Existing visual recognition methods can solve the static recognition problem of objects and scenes to a certain extent, but in dynamic scenes, it is difficult to accurately identify objects and handle complex temporal and spatial changes. Summary of the invention

[0004] The purpose of the present invention is to provide a method, device and equipment for dynamic scene recognition of objects based on AI vision, aiming to solve the problem in the prior art that it is difficult to accurately identify changes in objects in dynamic scenes.

[0005] The present invention is implemented as follows. In a first aspect, the present invention provides an object dynamic scene recognition method based on AI vision, comprising:

[0006] Collecting a dynamic scene video stream, and performing frame preprocessing on the collected dynamic scene video stream to obtain a video stream frame sequence;

[0007] Substituting the video stream frame sequence into an AI visual dynamic recognition model composed of a convolutional neural network model and an LSTM network model, so that the AI ​​visual dynamic recognition model performs model training according to the video stream sequence to obtain a basic training version of the AI ​​visual dynamic recognition model;

[0008] An attention mechanism and a multi-task mechanism are set on the basic training version of the AI ​​visual dynamic recognition model to obtain an optimized training version of the AI ​​visual dynamic recognition model for realizing dynamic scene recognition of objects through AI vision.

[0009] In a second aspect, the present invention provides an object dynamic scene recognition device based on AI vision, which is used to implement an object dynamic scene recognition method based on AI vision as described in any one of the first aspects, including:

[0010] A data acquisition module is used to acquire dynamic scene video streams and perform frame preprocessing on the acquired dynamic scene video streams to obtain video stream frame sequences;

[0011] A model training module, used for substituting the video stream frame sequence into an AI visual dynamic recognition model composed of a convolutional neural network model and an LSTM network model, so that the AI ​​visual dynamic recognition model is trained according to the video stream sequence to obtain a basic training version of the AI ​​visual dynamic recognition model;

[0012] The model optimization module is used to set an attention mechanism and a multi-task mechanism on the basic training version of the AI ​​visual dynamic recognition model to obtain an optimized training version of the AI ​​visual dynamic recognition model for realizing dynamic scene recognition of objects through AI vision.

[0013] In a third aspect, the present invention provides an AI vision-based object dynamic scene recognition device, which is used to implement an AI vision-based object dynamic scene recognition method as described in any one of the first aspects.

[0014] The present invention provides an object dynamic scene recognition method based on AI vision, which has the following beneficial effects:

[0015] The present invention collects video streams of dynamic scenes to generate video frame sequences, inputs the video frame sequences into an AI visual dynamic recognition model composed of a convolutional neural network (CNN) and an LSTM network, performs model training, and adds an attention mechanism and a multi-task mechanism to generate a final optimized training version to achieve accurate recognition of objects in dynamic scenes through AI vision. This solution combines CNN and LSTM networks, and the model can effectively extract spatiotemporal features in video streams and improve the ability to recognize dynamic objects. The attention mechanism can automatically focus on key areas in the video to improve recognition accuracy. The multi-task mechanism enables the model to complete multiple tasks at the same time, such as object detection and action recognition, thereby enhancing the versatility and adaptability of the system and solving the problem in the prior art that it is difficult to accurately recognize changes in objects in dynamic scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 1 is a schematic diagram of the steps of a method for identifying dynamic scenes of objects based on AI vision provided by an embodiment of the present invention;

[0017] Figure 2 It is a structural schematic diagram of an AI vision-based object dynamic scene recognition device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0019] The implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0020] Reference Figure 1 , Figure 2 As shown, a preferred embodiment of the present invention is provided.

[0021] In a first aspect, the present invention provides a method for identifying dynamic scenes of objects based on AI vision, comprising:

[0022] S1: collecting a dynamic scene video stream, and performing frame preprocessing on the collected dynamic scene video stream to obtain a video stream frame sequence;

[0023] S2: Substituting the video stream frame sequence into an AI visual dynamic recognition model composed of a convolutional neural network model and an LSTM network model, so that the AI ​​visual dynamic recognition model performs model training according to the video stream sequence to obtain a basic training version of the AI ​​visual dynamic recognition model;

[0024] S3: Setting an attention mechanism and a multi-task mechanism on the basic training version of the AI ​​visual dynamic recognition model to obtain an optimized training version of the AI ​​visual dynamic recognition model for realizing dynamic scene recognition of objects through AI vision.

[0025] Specifically, in step S1 of the embodiment provided by the present invention, capturing a dynamic scene video stream and performing frame preprocessing on it to obtain a video stream frame sequence is an important basic step in computer vision and video analysis. Selecting a suitable camera device (such as a camera, monitoring equipment, drone, etc.) to capture dynamic scenes, parameters such as the resolution, frame rate and stability of the device have a great influence on the video quality and subsequent analysis results. A higher resolution can provide more details and help with object detection and recognition. A high frame rate video can more accurately capture fast-moving objects or changes, and usually requires a frame rate of 30FPS (30 frames per second) or higher.

[0026] More specifically, the video stream can be in common video formats (such as MP4, AVI, MKV, etc.), and usually needs to be converted into a standard format for model processing (such as frame sequence) to ensure that the quality and frame rate of the video stream meet the task requirements, which helps with subsequent frame extraction and dynamic scene analysis. High-definition video streams can provide more accurate visual features, especially in complex scenes or low-light environments.

[0027] More specifically, the captured video stream is decomposed into frames to obtain a video frame sequence. This is the basic step for subsequent computer vision tasks and usually includes the following sub-steps: Use appropriate video decoding tools (such as OpenCV, FFmpeg, etc.) to decode the video stream into continuous image frames. Each frame represents a still image in the video. These frames are arranged in chronological order to form a video frame sequence.

[0028] More specifically, each frame of the image is processed to ensure that the data format is suitable for subsequent deep learning models or analysis tasks. Common preprocessing steps include: Resizing: resizing each frame of the image to a uniform size to meet the input requirements of the deep learning model, color space conversion: converting each frame of the image from BGR (OpenCV default color space) or RGB to other color spaces (such as grayscale, HSV, etc.) to improve processing efficiency or highlight certain features, and normalizing pixel values ​​to the 0-1 range or standardizing to a mean of 0 and a variance of 1. This helps speed up model convergence and avoid gradient explosion or disappearance.

[0029] More specifically, if each frame of video needs to be time-tagged, the timestamp information of the frame is usually saved. For example, the time of each frame of the video relative to the start of the video or the index of the frame can be recorded for subsequent timing analysis.

[0030] Through the above processing steps, the video stream is converted into a series of image frames, each of which represents a still image in the video. The video frame sequence can be regarded as a high-dimensional dataset (usually a sequence containing multiple 3D tensors), each frame is an image data (such as the height, width, and channel dimensions of an RGB image), and these image frame sequences will be used as input for subsequent deep learning models for tasks such as target detection, object tracking, and dynamic scene analysis.

[0031] It can be understood that the frame preprocessing of the video stream converts the video data into a structured image sequence, which is convenient for the deep learning model to process. Through the processed frame sequence, the spatiotemporal features in the video can be fully mined. Preprocessing steps such as image enhancement and data enhancement can effectively improve the generalization ability of the model and prevent the model from overfitting.

[0032] Specifically, in step S2 of the embodiment provided by the present invention, before performing model training, it is first necessary to prepare a frame sequence of the video stream. This sequence is obtained through the frame preprocessing process mentioned above, has been adjusted to a suitable size, normalized and denoised, and arranged in chronological order. Each frame usually represents a static image in the video stream. The preprocessed frame sequence can ensure the consistency and high quality of the data and reduce the impact of noise on subsequent training.

[0033] More specifically, the model consists of two parts: a convolutional neural network (CNN) and a long short-term memory network (LSTM). CNN is used to extract the spatial features of each frame, while LSTM is used to capture the temporal dependencies between frames and process the dynamic information of the video. The usual network architecture includes the following key steps: Input layer: Each frame of the image (such as a 224x224 RGB image) is used as the input of the CNN. Since the video is dynamic, multiple consecutive frames are usually taken as an input sequence. Convolution layer: Feature extraction is performed on each frame of the image through multiple convolution layers. The convolution layer can automatically learn the spatial features of the edge, texture, shape, etc. in the image. Pooling layer: The pooling layer is used to downsample the convolution features, reduce the amount of data and calculation, while retaining the most important features. Fully connected layer: The last layer of CNN is usually a fully connected layer, which is used to convert high-dimensional convolution features into low-dimensional feature vectors for subsequent time series model (LSTM) processing.

[0034] It is understandable that CNN can effectively extract the spatial features of each frame and extract key information (such as object shape, color, position, etc.) from the image, which is crucial for tasks such as video classification and object detection.

[0035] More specifically, the feature vector of each frame processed by CNN will be input into the LSTM network in chronological order. LSTM can process and model the dependencies of long time series and capture the dynamic changes and time series information in the video. LSTM selectively remembers or forgets the previous information through its unique gating mechanism (such as input gate, forget gate, output gate), so that it can effectively capture the long-term dependencies and temporal patterns in the video. The final output of LSTM is usually a vector representing the dynamic features of the entire video sequence, which is usually associated with the label of the video (such as behavior category, object category, etc.). This output layer can be a classification layer (for example, softmax layer, for multi-classification tasks) or a regression layer (for regression tasks).

[0036] It is understandable that LSTM is able to model temporal information in videos and effectively learn the dependencies between video frames, which is crucial for understanding dynamic changes in videos. After the video frame sequence is input into the CNN and LSTM networks, the goal of model training is usually to optimize the parameters of the network so that the model can accurately predict the category or behavior of the video.

[0037] More specifically, the back-propagation algorithm is used to calculate the gradient and the network weights in CNN and LSTM are updated through optimizers (such as Adam, SGD, etc.). During the training process, the network parameters are continuously adjusted to gradually enable the model to achieve higher accuracy in tasks such as video classification or behavior recognition. The training set and validation set are divided into different data sets to evaluate the training effect and generalization ability of the model. The video frame sequence is divided into multiple small batches, and the model weights are updated through iterative training. At the end of each training cycle (epoch), the validation set is used to evaluate the performance of the model, and hyperparameters such as the learning rate are adjusted to prevent overfitting.

[0038] It is understandable that by optimizing the parameters in the network, CNN and LSTM can work together, making full use of spatial and temporal information, and improving the model's ability to understand dynamic scenes. As the training progresses, the performance of the model will gradually improve, and eventually a basic version of the AI ​​visual dynamic recognition model will be obtained. After the training is completed, the model is evaluated using the test set to calculate its accuracy, recall rate, F1-score and other indicators. Through these evaluation indicators, we can understand the performance of the model in practical applications and further optimize the model.

[0039] It can be seen that the combination of CNN and LSTM enables the model to not only extract spatial features in video frames (such as object shape and texture), but also capture the temporal dependencies between video frames (such as continuity of actions and timing changes). This combined model is particularly suitable for the recognition of dynamic scenes and can process complex spatiotemporal data. It is suitable for tasks such as behavior recognition, motion detection, and video analysis. After training, the AI ​​visual dynamic recognition model can make accurate predictions for dynamic video data and obtain a basic version of the model, which can serve as the basis for further optimization and tuning.

[0040] Specifically, in step S3 of the embodiment provided by the present invention, an attention mechanism is added to the basic AI visual dynamic recognition model (CNN+LSTM). Common attention mechanisms include spatial attention and temporal attention: Spatial attention mechanism: For each frame of the image, the model can automatically learn which areas in the image (such as objects, background, etc.) are more important through the attention mechanism, so as to give more attention to these areas in the feature extraction stage. A common practice is to generate a spatial attention map, which is generated by a convolution operation and is used to weight the contribution of different spatial positions in the input feature map. Temporal attention mechanism: In the LSTM part, the temporal attention mechanism can help the model determine which time steps in the video sequence are more important. By generating a temporal attention weight, the weight will weight the feature sequence of the LSTM input, thereby helping the model focus on the key time steps related to the recognition of the target object.

[0041] More specifically, when adding a spatial attention mechanism to a CNN network, an attention module can be added after the convolutional layer, which generates a spatial weight map based on the input feature map and then multiplies it with the original feature map at the element level to focus on important areas in the image. When adding a temporal attention mechanism to an LSTM network, a mechanism similar to "self-attention" can be used to apply a weighted average to the output of the LSTM to highlight the time steps in the sequence that contribute most to the final recognition task.

[0042] It is understandable that through the spatial attention mechanism, the model can focus on the parts of the image related to the target object and reduce the interference of irrelevant areas on the final judgment. Through the temporal attention mechanism, the model can identify key moments in the video (such as turning points of object movement) and improve the accuracy of dynamic recognition. The attention mechanism enables the model to be more efficient when processing complex dynamic scenes, reduce the amount of calculation, and improve the accuracy of recognition.

[0043] More specifically, multi-task learning (MTL) is a model training strategy that aims to improve the overall performance of the model by simultaneously learning multiple related tasks by sharing part of the model's network structure. In object dynamic scene recognition, multi-task learning enables the model to perform multiple tasks simultaneously, such as object detection, action classification, scene analysis, etc. On the basis of the basic visual dynamic recognition model, multiple tasks are supported by introducing multiple output branches: Task 1: Object recognition and localization (object detection): Identify objects in the video and give their positions (for example, bounding box). The task goal is to accurately locate the objects appearing in the video. Task 2: Action classification: Identify the dynamic behavior or action of objects in the video (for example, movement, rotation, grasping, etc.). Task 3: Scene classification: Identify the video scene type, such as indoor, outdoor, street, office, etc. For each task, the model will generate a different loss function and optimize the loss functions of all tasks during the training process in order to share knowledge and learn common features.

[0044] It can be seen that by sharing network parameters and features, the model's ability to understand multiple tasks is improved, thereby improving the model's overall performance. Multi-task learning can help the model learn multiple related tasks at the same time, thereby improving the comprehensive ability to identify objects and scenes. Combining the attention mechanism and the multi-task mechanism, the training process is optimized so that the model can not only accurately identify the static features of objects, but also understand the dynamic behavior of objects and adapt to complex dynamic scene recognition tasks. Through the introduction of these two mechanisms, the optimized training version of the AI ​​visual dynamic recognition model can show higher accuracy and robustness in object dynamic scene recognition tasks.

[0045] The present invention provides an object dynamic scene recognition method based on AI vision, which has the following beneficial effects:

[0046] The present invention collects video streams of dynamic scenes to generate video frame sequences, inputs the video frame sequences into an AI visual dynamic recognition model composed of a convolutional neural network (CNN) and an LSTM network, performs model training, and adds an attention mechanism and a multi-task mechanism to generate a final optimized training version to achieve accurate recognition of objects in dynamic scenes through AI vision. This solution combines CNN and LSTM networks, and the model can effectively extract spatiotemporal features in video streams and improve the ability to recognize dynamic objects. The attention mechanism can automatically focus on key areas in the video to improve recognition accuracy. The multi-task mechanism enables the model to complete multiple tasks at the same time, such as object detection and action recognition, thereby enhancing the versatility and adaptability of the system and solving the problem in the prior art that it is difficult to accurately recognize changes in objects in dynamic scenes.

[0047] Preferably, the step of collecting a dynamic scene video stream and performing frame preprocessing on the collected dynamic scene video stream to obtain a video stream frame sequence includes:

[0048] S11: performing continuous video acquisition of the target recognition scene according to a video acquisition device pre-set in the target recognition scene to obtain a dynamic scene video stream corresponding to the target recognition scene;

[0049] S12: performing video decoding processing on the dynamic scene video stream to obtain a plurality of static image frames corresponding to the dynamic scene video stream and arranged in time sequence;

[0050] S13: performing frame extraction processing on a plurality of static image frames according to a preset time interval standard and a key frame identification standard to obtain a video stream frame sequence.

[0051] Specifically, according to the target recognition scenario, we first need to select appropriate video acquisition devices (such as cameras, surveillance cameras or mobile devices). These devices should have the ability to continuously acquire high-quality video streams and configure the video acquisition devices to continuously record the target recognition scenario. The video acquisition process usually generates a continuous dynamic scene video stream, containing tens to hundreds of frames of image data per second, forming a video stream.

[0052] More specifically, the captured dynamic scene video stream is generally stored in a compressed format (such as H.264, H.265, etc.). For further processing, the video stream needs to be decoded. The decoding process converts the compressed video stream into visual raw image data. Common decoding tools or libraries include FFmpeg, OpenCV, etc. They can extract each frame in the video stream in chronological order to obtain several static image frames. Each frame is a static image (such as JPEG, PNG format), which can be further analyzed in subsequent processing steps. In the decoded static image frame, ensure that the image frames are arranged in chronological order to maintain the timing information of the video stream. This is because each frame in the video corresponds to a specific time point, and the understanding of dynamic scenes depends on the relationship between these frames in the time dimension.

[0053] More specifically, when extracting video frames, they need to be screened according to a preset time interval standard. The time interval standard determines which frames are selected for processing from the decoded continuous image frames. The time interval can usually be set to a fixed time (such as collecting 30 frames per second) or flexibly adjusted according to actual scene requirements (for example: adjusting the interval according to the speed of object movement or the frequency of scene changes). For example, if the goal is to analyze fast-moving objects, a shorter time interval (more frames) may be required; if the goal is to analyze slowly changing scenes, a longer time interval (fewer frames) can be used.

[0054] More specifically, in addition to frame extraction based on time intervals, key frame recognition criteria can also be introduced. Key frames usually refer to frames with large content changes or important information in the video. For example, key actions of objects in the video (such as starting to move, stopping, changing direction, etc.) or key moments of scene changes (such as entering or leaving the scene) may become key frames. Key frame recognition usually relies on image similarity metrics (such as SSIM, PSNR, etc.) or computer vision methods (such as optical flow analysis, background modeling, motion detection, etc.) to detect scene mutation points or important actions. According to the above time interval criteria and key frame recognition criteria, the required static image frames are extracted from the decoded video stream. The extracted image frames constitute a video stream frame sequence for subsequent processing.

[0055] More specifically, the extracted static image frames are arranged in chronological order to form a frame sequence of the video stream. Each frame retains the timing information in the video stream, which can provide key input for the subsequent dynamic scene recognition model. The frame sequence is usually stored as a set of image files, a video file (re-encoded in frame sequence) or directly stored in memory as an image array for subsequent processing.

[0056] It is understandable that by decoding the video and arranging the static image frames in chronological order, the timing information in the frame sequence is ensured to be complete, which is crucial for the analysis of dynamic scenes, because the recognition of objects and actions depends on the temporal relationship between consecutive frames. Setting reasonable time interval standards and key frame recognition standards can ensure that while ensuring timeliness, unnecessary frames are minimized as much as possible, thereby improving processing efficiency. This helps avoid wasting computing resources by processing too many irrelevant frames, especially in scenes with little change in the video stream. Key frame extraction can help the model focus on the most informative frames. For example, the key action frames of objects or the change frames of scenes are extracted, which can help the model understand the changes in dynamic scenes more accurately, reduce the interference of irrelevant information, and improve the accuracy of subsequent tasks such as dynamic object recognition and action classification.

[0057] Preferably, the step of substituting the video stream frame sequence into an AI visual dynamic recognition model composed of a convolutional neural network model and an LSTM network model so that the AI ​​visual dynamic recognition model performs model training according to the video stream sequence to obtain a basic training version of the AI ​​visual dynamic recognition model includes:

[0058] S21: constructing the AI ​​visual dynamic recognition model based on a preset convolutional neural network model and an LSTM network model;

[0059] S22: Substituting the video stream frame sequence into the convolutional neural network model, and allowing the convolutional neural network model to perform model training according to the video stream frame sequence to obtain the spatiotemporal features of the video stream frame sequence;

[0060] S23: Substitute the spatiotemporal features into the LSTM network model, and train the LSTM network model according to the spatiotemporal features to obtain dynamic change features of the object, so as to obtain a basic version of the AI ​​visual dynamic recognition model.

[0061] Specifically, convolutional neural network is a classic deep learning model for processing image data. It can extract spatial features in images, such as edges, shapes, textures, etc., through convolutional layers. For video stream frame sequences, CNN can be used to extract spatial features of each frame. Usually, each frame of the video is input into the convolutional neural network. After being processed by multiple convolutional layers and pooling layers, the spatial information of each frame is extracted. The temporal feature extraction part is constructed based on the long short-term memory (LSTM) network: LSTM is a recursive neural network model specifically used to process sequence data (such as time series), which can effectively capture long-term dependencies in the data. Since the video stream frame sequence is time-ordered, LSTM can handle the changes in the time dimension in this process and capture the temporal features in the video frame sequence.

[0062] More specifically, in this process, the input of LSTM will be the spatiotemporal features extracted by CNN. LSTM will learn how these spatiotemporal features change over time and be able to capture the movement and change patterns of dynamic objects in the temporal dimension. CNN and LSTM are combined into a joint model. CNN is used to extract the spatial features of each frame, while LSTM performs temporal modeling on these spatial features. The output of the combined model can be the dynamic changes of objects in the video, behavior patterns, or event predictions.

[0063] More specifically, before inputting the video stream frame sequence into the model, the video frame sequence needs to be preprocessed into a data format suitable for CNN input (such as adjusting the image size, normalizing, converting to tensor, etc.). Each frame of the image needs to be preprocessed and input into CNN to extract the spatial features of the image. Each frame of the image input into the convolutional neural network extracts spatial information through multiple convolutional layers.

[0064] More specifically, CNN usually contains multiple convolutional layers and pooling layers to extract low-level (such as edges, corners) and high-level (such as objects, people) features. The convolutional layer extracts local features of the image (for example, the outline and texture of the object) through convolution operations. The pooling layer reduces the spatial dimension, retains important feature information, and reduces computational complexity. The last layer of CNN usually generates a spatiotemporal feature vector through a fully connected layer or convolutional layer. This vector represents the spatial information of each frame of the image. The spatial features of all frames are combined into a feature sequence to provide input for subsequent LSTM processing.

[0065] More specifically, the spatiotemporal feature sequence extracted by CNN is input into the LSTM network. LSTM can capture the temporal changes of the input data through its memory mechanism, that is, the changes from the previous frame to the next frame, the trajectory of the object, the scene changes, etc. Each time step of LSTM receives a spatiotemporal feature vector and updates its state based on the previous memory and the current input. Through multi-layer LSTM units, the model can learn the dynamic changes of objects, the continuity of actions, and other time-related behavior patterns.

[0066] More specifically, the LSTM network is trained based on the input features of the time series to identify the dynamic changes of objects. These features include the position, speed, direction, action category, etc. of the object in the video. The LSTM model is trained through the back-propagation algorithm, which enables it to extract meaningful dynamic change patterns from the input spatiotemporal features.

[0067] More specifically, after the spatiotemporal features extracted by CNN are processed by LSTM, dynamic features of objects (such as the object's motion trajectory, behavior pattern, etc.) can be generated. These dynamic change features will become the basic version of the AI ​​visual dynamic recognition model for subsequent model evaluation, optimization and application. By training the CNN and LSTM combined model, the basic version of the AI ​​visual dynamic recognition model obtained will be able to recognize the dynamic changes and behaviors of objects based on video stream sequences. This version of the model already has the preliminary capabilities for tasks such as object recognition and action classification in dynamic scenes.

[0068] It is understandable that CNN extracts visual features of each frame in the spatial dimension, while LSTM processes dynamic changes in the temporal dimension. The combination of the two can effectively extract spatiotemporal features in video data and provide rich information for subsequent dynamic recognition. The introduction of LSTM enables the model to not only understand the spatial information in the image, but also capture the changes in objects or scenes over time, improving the modeling ability of dynamic scenes. It can identify the movement trajectory and behavior patterns of objects and adapt to more complex time series data. Through the combination of CNN and LSTM, the AI ​​visual dynamic recognition model can learn the dynamic change characteristics of objects, including the movement, action, interaction, etc. of objects. The training of the model enables it to analyze the behavior of objects at different time points and accurately identify their dynamic state.

[0069] Preferably, the step of substituting the video stream frame sequence into the convolutional neural network model and allowing the convolutional neural network model to perform model training according to the video stream frame sequence to obtain the spatiotemporal features of the video stream frame sequence includes:

[0070] S221: Construct a convolutional neural network model consisting of an input layer, a convolutional layer, a pooling layer, and several fully connected layers;

[0071] S222: Substituting the video stream frame sequence into the input layer;

[0072] S223: The input layer receives the video stream frame sequence, and transmits the video stream frame sequence to the convolution layer, and the convolution layer is used to collect the spatiotemporal features of each frame image of the video stream frame sequence to obtain the spatiotemporal features of the video stream frame sequence;

[0073] S224: the pooling layer is used to perform a pooling operation on the spatiotemporal features of the video stream frame sequence extracted by the convolution layer to reduce the size of the spatiotemporal features of the video stream frame sequence and retain key features of the spatiotemporal features of the video stream frame sequence;

[0074] S225: Each of the fully connected layers is used to perform continuous vector flattening processing on the spatiotemporal features of the video stream frame sequence after processing by the pooling layer, so as to flatten the spatiotemporal features of the video stream frame sequence into one-dimensional vector features; wherein the one-dimensional vector features are used to perform basic graphical expression on the spatiotemporal features of the video stream frame sequence.

[0075] Specifically, the input layer is responsible for receiving a sequence of video stream frames. For video data, the input is usually a set of continuous frame images. Each frame image is usually composed of a pixel matrix. Each frame image can be three-channel (RGB) or single-channel (grayscale), depending on the processing method of the preprocessing stage. The dimension of the input layer is generally [batch size, number of frames, height, width, number of channels], which is adapted to the input video frame sequence.

[0076] More specifically, the convolution layer is the core part of CNN, which is used to extract spatial features in video frames. Each convolution kernel (filter) scans the video frame image and performs a convolution operation to extract local features of the image (such as edges, textures, shapes, etc.). For video streams, the convolution layer extracts spatiotemporal features in each frame. During the convolution process, the convolution kernel performs a dot multiplication operation with the image to extract specific spatiotemporal features from each frame, such as the boundaries of moving objects, changes in movements, etc. The convolution operation is usually accompanied by an activation function (such as ReLU) to introduce nonlinearity, thereby improving the expressiveness of the model.

[0077] More specifically, the role of the pooling layer is to reduce the dimension of the features extracted by the convolutional layer. Common pooling methods include maximum pooling and average pooling. It reduces the dimension of the data by aggregating the features of the local area, thereby reducing the amount of calculation. In the processing of video stream frame sequences, the pooling operation helps to reduce the spatial size (height and width) of the image while retaining the key spatiotemporal features in the video stream sequence. The pooling operation not only reduces the computational complexity, but also makes the features more robust to changes in position.

[0078] More specifically, the fully connected layers are the last few layers in CNN, which are used to convert the features extracted by the pooling layer into a one-dimensional vector. This process usually includes several fully connected layers. Each fully connected layer expands the pooled feature map and performs weighted summation, and then performs nonlinear transformation through activation functions (such as ReLU). In the context of video stream sequences, after each frame of the image is processed by the convolution layer and the pooling layer, its spatiotemporal features will be flattened into a one-dimensional vector. In the fully connected layer, these flattened vectors will be further processed to form the final feature representation of the video stream sequence.

[0079] More specifically, a video stream frame sequence (multiple continuous frame images) is input to the input layer of the model. Each frame of the image enters the convolution layer through forward propagation for feature extraction. The spatial features of each frame of the image (such as object edges, motion trajectories, etc.) are gradually extracted in the convolution layer. In the convolution layer, the model performs sliding operations according to the set convolution kernel to extract the spatiotemporal features of each frame of the image. These features include local features within the frame (such as object contours, action boundaries) and time-dependent features across frames (such as object motion trajectories). The pooling layer downsamples the output of the convolution layer, retains important information through aggregation operations, reduces spatial resolution, reduces the size of the feature map, and reduces computational complexity, enhancing the generalization ability of the model. After the pooling layer, the resulting feature map (two-dimensional matrix) will be flattened into a one-dimensional vector. The flattening operation arranges the values ​​of all feature maps in order into a long vector as the input of the subsequent fully connected layer. The key to this step is to convert the spatiotemporal features from two-dimensional feature maps into one-dimensional feature vectors, retain the high-order information of the image, and enable the neural network to better process the spatiotemporal features. The flattened spatiotemporal feature vectors will be processed by several fully connected layers. The fully connected layer calculates the output through weighted summation and generates a basic graphical expression of the spatiotemporal features of the video stream frame sequence. These graphical representations can capture the dynamic changes of objects, motion, scenes, etc. in the video stream. During the image expression process, the model will learn the effective pattern of spatiotemporal features from the training data and generate a one-dimensional feature vector, which is convenient for the subsequent model to perform classification, regression or other tasks. The final output one-dimensional vector feature is the basic graphical expression of the spatiotemporal features of the video stream frame sequence. These feature vectors can not only be used for subsequent dynamic recognition tasks (such as action recognition, object tracking, etc.), but can also be used as training targets for models to help AI recognition systems understand the spatiotemporal changes in video streams.

[0080] It can be understood that CNN can effectively extract spatiotemporal features from video stream frame sequences, the convolution layer extracts spatial features from each frame of the image, and the pooling layer helps to reduce unnecessary details while retaining key feature information. The whole process enables the model to effectively capture dynamic changes in the image. The pooling layer and flattening operation reduce the dimension of the feature space, avoid overfitting, improve computational efficiency, and retain the core features in the video stream frame sequence. The flattening operation converts high-dimensional features into one-dimensional feature vectors, which is convenient for subsequent fully connected layers to perform graphical expression and further processing.

[0081] Preferably, the step of substituting the spatiotemporal features of the video stream frame sequence into the LSTM network model, training the LSTM network model according to the spatiotemporal features of the video stream frame sequence, and obtaining the dynamic change features of the object includes:

[0082] S231: Construct an LSTM network model consisting of an input layer, an LSTM layer and several fully connected layers;

[0083] S232: Substituting the spatiotemporal features of the video stream frame sequence into the input layer;

[0084] S233: The input layer receives the spatiotemporal features of the video stream frame sequence, and transmits the spatiotemporal features of the video stream frame sequence to the LSTM layer, and the LSTM layer is used to collect features of the length dependency relationship of the spatiotemporal features of the video stream frame sequence in time sequence, so as to obtain the dynamic change features of the spatiotemporal features of the video stream frame sequence;

[0085] S234: Each of the fully connected layers is used to perform continuous vector flattening processing on the dynamic change features of the item extracted by the LSTM layer, so as to flatten the dynamic change features of the item into one-dimensional vector features; wherein the one-dimensional vector features are used to perform basic graphical expression on the dynamic change features of the item.

[0086] Specifically, the input layer is responsible for receiving the spatiotemporal features of the video stream frame sequence. These features can be the spatial feature vectors of each frame extracted by the convolutional neural network (CNN), which already have the spatial information of the image (such as object boundaries, textures, etc.). These spatiotemporal features will serve as the input data of the LSTM model.

[0087] More specifically, the LSTM layer can process time series data and capture the long-term and short-term dependencies of video stream frame sequences. In video data, the dynamic changes of objects usually have time correlation. LSTM processes the dependencies of long time series through its special memory units (including forget gate, input gate and output gate), which can better understand the temporal changes between video frames. After receiving the spatiotemporal features extracted from the convolutional layer or pooling layer, the LSTM layer processes the features of each frame in time order and learns the dynamic associations between frames. The LSTM layer will capture the dynamic change characteristics of objects in the video stream in the time dimension (for example, the movement path, change speed, interaction, etc. of the object). These dynamic features are very critical for video analysis (such as action recognition, event detection, etc.).

[0088] More specifically, the fully connected layer is used to further process and flatten the dynamic change features of the items extracted by the LSTM layer into a one-dimensional vector. The output of the LSTM is generally a multi-dimensional tensor, which represents the features corresponding to each time step (i.e., each frame or each group of video frames). These features are converted into a one-dimensional vector through the fully connected layer as a representation of the dynamic change features of the items in the video stream frame sequence. In the fully connected layer, the features are weighted, summed, and processed by an activation function (such as ReLU) to finally form a feature representation suitable for subsequent tasks.

[0089] More specifically, the spatiotemporal features extracted by the convolutional neural network (such as the boundaries of objects, movement paths, etc.) will be passed as input to the LSTM model. The input features are multidimensional vectors, representing the spatiotemporal information of each frame of the video stream. These spatiotemporal features are input into the LSTM layer in chronological order. The LSTM layer will process each frame of the image in sequence according to the chronological order to capture the temporal dependencies between frames.

[0090] More specifically, the unique advantage of the LSTM network is that it can effectively process long-term and short-term dependencies in time series data. In the case of video streams, the dynamic changes of objects are usually continuous and time-dependent. LSTM can capture the temporal associations between frames, such as the motion trajectory of objects, the interaction between objects, the changes in scenes, etc. LSTM uses its memory units to decide which information needs to be retained (memorized) and which information needs to be forgotten, thereby jointly influencing the feature extraction of subsequent frames based on the information of the current frame and the information of the historical frames.

[0091] More specifically, in the LSTM layer, the dynamic change features of the object will be extracted. These features characterize the movement laws and change patterns of the object in the video stream. By processing the video stream frame sequence, LSTM can extract dynamic information such as the object's speed, direction, and position. The features output by the LSTM layer are usually multi-dimensional tensors, representing the feature vectors of each time step. In order to facilitate subsequent processing and analysis, these features will be flattened to flatten the features of each time step into a one-dimensional vector. The flattening operation converts the multi-dimensional representation of spatiotemporal features into a one-dimensional vector, so that these features can be passed to the subsequent fully connected layer for further learning.

[0092] More specifically, in the fully connected layer, the flattened dynamic change features of the object will be further processed. The fully connected layer generates the final representation of the dynamic change of the object through weighted summation, activation function and other operations. These one-dimensional vector features can be used for graphical expression to represent the dynamic change of the object in the video stream. This graphical expression can be further used for tasks such as classification, regression, action recognition, and event detection.

[0093] It is understandable that LSTM can effectively capture the long-term and short-term dependencies in the video stream frame sequence, helping to identify the dynamic changes of objects in the video. The change patterns of objects at different time steps can be learned through the time memory mechanism of the LSTM layer, thereby improving the accuracy of video analysis. Through LSTM, the model can understand the dynamic behavior of objects in the video. Whether it is the movement of a single object or the interaction between multiple objects, LSTM can model in the time dimension and capture the laws of dynamic changes. Since LSTM can extract information from time series, the model is particularly effective for processing video streams, surveillance videos, action recognition and other tasks. It can understand the time dependency in the video and make accurate inferences.

[0094] Preferably, the step of setting an attention mechanism and a multi-task mechanism on the basic training version of the AI ​​visual dynamic recognition model to obtain an optimized training version of the AI ​​visual dynamic recognition model includes:

[0095] S31: adding a spatial attention module after the convolutional neural network model, and performing back-propagation training on the spatiotemporal features of the video stream frame sequence obtained by the convolutional neural network model through the spatial attention module, so as to allocate attention weights of the spatial region of the video stream frame sequence, thereby realizing the setting of the attention mechanism;

[0096] S32: adding a multi-task learning module to the LSTM network model, and performing feature sharing and multi-task diversion on model parameters of the LSTM network model through the multi-task learning module, so that the LSTM network model can simultaneously perform model training corresponding to multiple tasks, thereby realizing a multi-task mechanism;

[0097] S33: By adding an attention mechanism and a multi-task mechanism to the basic training version of the AI ​​visual dynamic recognition model, an optimized version of the AI ​​visual dynamic recognition model is obtained.

[0098] Specifically, the basic CNN network is responsible for extracting spatiotemporal features from the video stream frame sequence. These features usually include spatial information such as the position, shape, color, texture, etc. of the object, as well as the timing information between each frame.

[0099] More specifically, the spatial attention mechanism is used to strengthen the model's attention to key areas in the video frame. In the video stream, different objects or areas contribute differently to the task. The spatial attention module learns the importance of each area, assigns more weight to key areas, and suppresses the influence of unimportant areas, so that the model can better focus on meaningful information. This module usually adopts the channel-spatial attention mechanism to adjust the spatial dimension of the convolutional feature map by weighting the features of the spatial position. Common methods include spatial attention based on convolution operations and spatial attention based on self-attention mechanisms.

[0100] More specifically, the spatial attention module optimizes the attention weights through back-propagation training. This enables the model to adaptively adjust its attention focus during training so that it can focus on different parts of the video frame at each time step. During training, the spatial attention module updates the weights of each spatial region according to the gradient and focuses on important areas in the video stream, enhancing the model's feature extraction capabilities.

[0101] More specifically, multi-task learning (MTL) is a training strategy that allows the model to share features and model parameters on multiple related tasks while training on one task. Introducing a multi-task learning module in the LSTM network allows the LSTM to handle multiple tasks (e.g., object recognition, action classification, trajectory prediction, etc.) at the same time, thereby improving the overall capabilities of the model. The key to the multi-task learning module is to share some model parameters, and at the same time make the output layers or decision layers of different tasks independent through the task diversion strategy. In the LSTM network, sharing parameters means that some layers of the network (such as the LSTM layer) share feature representations in multiple tasks, and each task uses a different output layer for optimization according to specific needs. For example, the output of the LSTM layer can be shared by multiple tasks (such as behavior classification and target tracking), and then optimized by different task-specific loss functions.

[0102] More specifically, the time series features learned by the LSTM layer will be shared by multiple tasks, so that the model can learn common features that are useful in different tasks. This sharing can improve the generalization ability of the model and avoid overfitting to a certain extent. Each task usually has an independent output layer. Through these task-specific output layers, the model can be trained for different tasks separately. For example, the model may perform action recognition tasks, target detection tasks, and trajectory prediction tasks at the same time, and the output layer of each task will be optimized according to the task requirements.

[0103] More specifically, adding the spatial attention module and the multi-task learning module to the basic training version at the same time can enable the AI ​​visual dynamic recognition model to focus on important information in the space while optimizing multiple tasks at the same time when processing video streams. In this way, the model not only has better performance on a single task, but also can improve the efficiency and generalization of learning under the joint training of multiple tasks. The spatial attention module is connected after the convolutional neural network to optimize spatial feature extraction, and the attention weight is adjusted using back propagation training. Then, these spatiotemporal features are input into the LSTM layer, and the tasks are shared and diverted through the multi-task learning module, and finally the output results of multiple tasks are obtained. By adding the attention mechanism, the model can better focus on important spatiotemporal areas in the video, thereby improving the accuracy of feature extraction. By introducing multi-task learning, the model not only solves a single task, but can handle multiple related tasks at the same time, which improves the versatility and adaptability of the model.

[0104] Preferably, the LSTM network model has an LSTM layer for extracting dynamic change characteristics of items, and the multi-task learning model is used to insert a number of task execution channels into the LSTM layer, each of which receives the dynamic change characteristics of items extracted by the LSTM layer and executes the corresponding model learning task, so as to realize feature sharing and multi-task diversion of model parameters of the LSTM network model; wherein the task execution channel includes object detection, scene classification and action recognition.

[0105] Specifically, the LSTM (Long Short-Term Memory) layer is used to extract dynamic features of objects from video streams or time series data. LSTM can capture dependencies in long time series through its memory units, so it is particularly suitable for processing dynamic features with time dependencies. In the video dynamic recognition task, the LSTM layer can extract dynamic information such as the movement, position change, and action evolution of objects from video frames.

[0106] More specifically, the core of the multi-task learning (MTL) module is to share the features extracted by the LSTM layer for multiple tasks, while separating the learning objectives and outputs of each task. Specifically, the dynamic change features of objects extracted by the LSTM layer will be sent as input to multiple task execution channels. Each task execution channel will process these dynamic features differently, and perform tasks such as object detection, scene classification, and action recognition respectively, and finally obtain the prediction results of each task.

[0107] More specifically, this channel is used to detect objects or targets in the video. Object detection tasks may involve object localization (such as through bounding box regression) and category recognition. The dynamic change features of objects extracted by the LSTM layer will be used to help detect different objects or activity areas in the video and classify them.

[0108] More specifically, the goal of the scene classification task is to identify the scene type based on the temporal information in the video stream. The dynamic features extracted by LSTM not only contain the temporal information of the object movement, but may also contain global changes in the scene (such as the scene switching from indoor to outdoor). This channel will make classification predictions for the scene based on these features.

[0109] More specifically, the action recognition task is used to identify the actions of objects or people in a video. The dynamic change features extracted by LSTM can be used to analyze the motion patterns in the time series, thereby identifying various actions or behaviors in the video. This task requires modeling the long-term dependency of the dynamics of objects in the video frames to determine whether a specific action (such as running, jumping, etc.) has occurred.

[0110] More specifically, based on the LSTM network, through the design of the multi-task learning module, the dynamic features extracted by the LSTM layer can be shared by multiple tasks. This means that each task channel can extract information from the same LSTM layer, so that the model can use the same set of time series features to perform different tasks at the same time. Feature sharing can promote the flow of information between tasks, especially when there are similarities between tasks, it helps to improve the performance of the model on each task. For example, object detection and action recognition may share some similar features (such as the position change and speed of the object), so sharing the dynamic features extracted by LSTM can improve the effect of multi-task learning.

[0111] More specifically, each task execution channel independently processes the task objectives associated with it. The task execution channels perform predictions for their respective tasks through dedicated output layers. For example, the object detection task outputs the bounding box and category label of the target, the scene classification task outputs the scene category, and the action recognition task outputs the action label. This split design ensures that the learning objectives of each task are not interfered with by other tasks, thereby improving the focus and accuracy of the task.

[0112] More specifically, in multi-task learning, the loss function of each task is defined according to the characteristics of the task. Usually, the loss function of each task is weighted and summed to obtain the overall loss function. The training process optimizes the loss of all tasks through back propagation and updates the parameters of the LSTM layer and task execution channel. Through this sharing-diverging mechanism, the LSTM network can improve its understanding of spatiotemporal features through the joint training of multiple tasks, while avoiding overfitting and improving generalization capabilities.

[0113] Reference Figure 2 As shown, in a second aspect, the present invention provides an object dynamic scene recognition device based on AI vision, which is used to implement an object dynamic scene recognition method based on AI vision as described in any one of the first aspects, including:

[0114] A data acquisition module is used to acquire dynamic scene video streams and perform frame preprocessing on the acquired dynamic scene video streams to obtain video stream frame sequences;

[0115] A model training module, used for substituting the video stream frame sequence into an AI visual dynamic recognition model composed of a convolutional neural network model and an LSTM network model, so that the AI ​​visual dynamic recognition model is trained according to the video stream sequence to obtain a basic training version of the AI ​​visual dynamic recognition model;

[0116] The model optimization module is used to set an attention mechanism and a multi-task mechanism on the basic training version of the AI ​​visual dynamic recognition model to obtain an optimized training version of the AI ​​visual dynamic recognition model for realizing dynamic scene recognition of objects through AI vision.

[0117] In this embodiment, for the specific implementation of each module in the above-mentioned device embodiment, please refer to the above-mentioned method embodiment, which will not be described in detail here.

[0118] In a third aspect, the present invention provides an AI vision-based object dynamic scene recognition device, which is used to implement an AI vision-based object dynamic scene recognition method as described in any one of the first aspects.

[0119] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for object dynamic scene recognition based on AI vision, characterized in that: include: Collecting a dynamic scene video stream, and performing frame preprocessing on the collected dynamic scene video stream to obtain a video stream frame sequence; Substituting the video stream frame sequence into an AI visual dynamic recognition model composed of a convolutional neural network model and an LSTM network model, so that the AI ​​visual dynamic recognition model performs model training according to the video stream frame sequence to obtain a basic training version of the AI ​​visual dynamic recognition model; Setting an attention mechanism and a multi-task mechanism on the basic training version of the AI ​​visual dynamic recognition model to obtain an optimized training version of the AI ​​visual dynamic recognition model for realizing object dynamic scene recognition through AI vision; The steps of collecting a dynamic scene video stream and performing frame preprocessing on the collected dynamic scene video stream to obtain a video stream frame sequence include: According to the video acquisition device pre-set in the target recognition scene, the target recognition scene is continuously acquired to obtain a dynamic scene video stream corresponding to the target recognition scene; Performing video decoding processing on the dynamic scene video stream to obtain a plurality of static image frames corresponding to the dynamic scene video stream and arranged in time sequence; Performing frame extraction processing on a plurality of static image frames according to a preset time interval standard and a key frame identification standard to obtain a video stream frame sequence; Substituting the video stream frame sequence into an AI visual dynamic recognition model composed of a convolutional neural network model and an LSTM network model, so that the AI ​​visual dynamic recognition model performs model training according to the video stream frame sequence, and the steps of obtaining a basic training version of the AI ​​visual dynamic recognition model include: Constructing the AI ​​visual dynamic recognition model based on the preset convolutional neural network model and LSTM network model; Substituting the video stream frame sequence into the convolutional neural network model, allowing the convolutional neural network model to perform model training according to the video stream frame sequence to obtain the spatiotemporal characteristics of the video stream frame sequence; Substitute the spatiotemporal features into the LSTM network model, and train the LSTM network model according to the spatiotemporal features to obtain the dynamic change features of the object, so as to obtain the basic version of the AI ​​visual dynamic recognition model.

2. The method for dynamic scene recognition of objects based on AI vision according to claim 1, characterized in that: Substituting the video stream frame sequence into the convolutional neural network model, and allowing the convolutional neural network model to perform model training according to the video stream frame sequence to obtain the spatiotemporal features of the video stream frame sequence includes: Construct a convolutional neural network model consisting of an input layer, a convolutional layer, a pooling layer, and several fully connected layers; Substituting the video stream frame sequence into the input layer; The input layer receives the video stream frame sequence and transmits the video stream frame sequence to the convolution layer, and the convolution layer is used to collect the spatiotemporal features of each frame image of the video stream frame sequence to obtain the spatiotemporal features of the video stream frame sequence; The pooling layer is used to perform a pooling operation on the spatiotemporal features of the video stream frame sequence extracted by the convolution layer to reduce the size of the spatiotemporal features of the video stream frame sequence and retain key features of the spatiotemporal features of the video stream frame sequence; Each of the fully connected layers is used to perform continuous vector flattening processing on the spatiotemporal features of the video stream frame sequence after processing by the pooling layer, so as to flatten the spatiotemporal features of the video stream frame sequence into one-dimensional vector features; wherein the one-dimensional vector features are used to perform basic graphical expression on the spatiotemporal features of the video stream frame sequence.

3. The method for dynamic scene recognition of objects based on AI vision according to claim 1, characterized in that: Substituting the spatiotemporal features of the video stream frame sequence into the LSTM network model, training the LSTM network model according to the spatiotemporal features of the video stream frame sequence, and obtaining the dynamic change features of the object comprises: Construct an LSTM network model consisting of an input layer, an LSTM layer, and several fully connected layers; Substituting the spatiotemporal features of the video stream frame sequence into the input layer; The input layer receives the spatiotemporal features of the video stream frame sequence, and transmits the spatiotemporal features of the video stream frame sequence to the LSTM layer, and the LSTM layer is used to collect features of the length dependency relationship of the spatiotemporal features of the video stream frame sequence in time sequence, so as to obtain the dynamic change features of the spatiotemporal features of the video stream frame sequence; Each of the fully connected layers is used to perform continuous vector flattening processing on the dynamic change features of the item extracted by the LSTM layer, so as to flatten the dynamic change features of the item into one-dimensional vector features; wherein the one-dimensional vector features are used to perform basic graphical expression on the dynamic change features of the item.

4. The method for object dynamic scene recognition based on AI vision as claimed in claim 1, characterized in that: The steps of setting an attention mechanism and a multi-task mechanism on the basic training version of the AI ​​visual dynamic recognition model to obtain an optimized training version of the AI ​​visual dynamic recognition model include: A spatial attention module is added after the convolutional neural network model, and the spatial attention module is used to perform back-propagation training on the spatiotemporal features of the video stream frame sequence obtained by the convolutional neural network model, so as to allocate attention weights of the spatial region of the video stream frame sequence, thereby realizing the setting of the attention mechanism; A multi-task learning module is added to the LSTM network model, and the model parameters of the LSTM network model are subjected to feature sharing and multi-task diversion through the multi-task learning module, so that the LSTM network model can simultaneously perform model training corresponding to multiple tasks, thereby realizing a multi-task mechanism; By adding attention mechanism and multi-task mechanism to the basic training version of the AI ​​visual dynamic recognition model, an optimized version of the AI ​​visual dynamic recognition model is obtained.

5. The method for object dynamic scene recognition based on AI vision as claimed in claim 4, characterized in that: The LSTM network model has an LSTM layer for extracting dynamic change features of items. The multi-task learning model is used to insert a plurality of task execution channels into the LSTM layer. Each of the task execution channels receives the dynamic change features of items extracted by the LSTM layer and executes a corresponding model learning task, so as to realize feature sharing and multi-task diversion of model parameters of the LSTM network model; wherein the task execution channels include object detection, scene classification and action recognition.

6. An object dynamic scene recognition device based on AI vision, characterized in that: A method for realizing an object dynamic scene recognition method based on AI vision as described in any one of claims 1 to 5, comprising: A data acquisition module is used to acquire dynamic scene video streams and perform frame preprocessing on the acquired dynamic scene video streams to obtain video stream frame sequences; A model training module, used for substituting the video stream frame sequence into an AI visual dynamic recognition model composed of a convolutional neural network model and an LSTM network model, so that the AI ​​visual dynamic recognition model performs model training according to the video stream frame sequence to obtain a basic training version of the AI ​​visual dynamic recognition model; The model optimization module is used to set an attention mechanism and a multi-task mechanism on the basic training version of the AI ​​visual dynamic recognition model to obtain an optimized training version of the AI ​​visual dynamic recognition model for realizing dynamic scene recognition of objects through AI vision.

Images