Motion detection method and device, electronic equipment and storage medium
By collecting video frames in real time and using sparse encoding and convolutional neural networks for feature extraction, the problem of insufficient processing speed and accuracy of motion detection algorithms in the prior art is solved, and more efficient and accurate recognition of moving objects is achieved.
Patent Information
- Application Number
- CN202410930422.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-08
AI Technical Summary
Existing motion detection algorithms have shortcomings in real-time processing speed and accuracy of motion trajectory analysis, especially in complex scenarios and dynamic contexts.
The video frame is collected in real time and converted into sparse representations through sparse encoding. The target convolutional neural network model is used for feature extraction, and the sparse encoding difference value is used to determine whether there are moving objects in the video frame, and then the motion trajectory is analyzed.
It greatly improves the recognition speed and accuracy of moving objects, improves the user experience, and maintains efficient and accurate motion detection performance in complex scenarios.
Smart Images

Figure CN118918504A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of video processing and computer vision technology, and in particular to a motion detection method, device, electronic device and storage medium. Background Art
[0002] Motion detection algorithm is a technology used to analyze dynamic objects in video images. It can detect the movement trajectory and position of the target object by comparing the differences between consecutive frames. The camera can detect the slight movement and shaking of the camera, and then adjust and stabilize the image accordingly to reduce the effects of image blur and shaking, and provide a clearer image. Motion detection algorithm has a wide range of applications in many fields, such as security monitoring, intelligent transportation, human-computer interaction, etc.
[0003] In the related technologies, motion detection algorithms mainly include motion detection based on video and audio, motion detection based on acceleration sensors, etc., such as background subtraction, frame difference, optical flow, motion vector analysis, etc. However, these algorithms mainly rely on pixel changes and regional motion vectors to detect motion, and have problems such as insufficient real-time processing speed and inaccurate motion trajectory analysis. When faced with complex scenes and dynamic backgrounds, they often show low robustness and accuracy.
[0004] Therefore, how to improve the recognition speed and accuracy of motion detection methods is a problem that needs to be solved urgently. Summary of the invention
[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides a motion detection method, device, electronic device and storage medium.
[0006] In a first aspect, the present application provides a motion detection method, the method comprising:
[0007] Real-time acquisition of video frames;
[0008] Performing sparse coding on the video frame to convert the image of the video frame into a sparse representation;
[0009] Using a target convolutional neural network model to extract features from the sparse representation to obtain target features;
[0010] It is determined whether there is a moving object in the video frame according to the target feature.
[0011] As an optional implementation manner of the embodiment of the present invention, after judging whether there is a moving object in the video frame according to the target feature, the method further includes:
[0012] If the moving object exists, the moving trajectory of the moving object is analyzed to obtain the moving information of the moving object.
[0013] As an optional implementation manner of the embodiment of the present invention, after judging whether there is a moving object in the video frame according to the target feature, the method further includes:
[0014] If the moving object exists, the moving trajectory of the moving object is analyzed to obtain the moving information of the moving object.
[0015] As an optional implementation of the embodiment of the present invention, the use of the target convolutional neural network model to extract features from the sparse representation to obtain target features includes:
[0016] Inputting the motion feature matrix into the target convolutional neural network model to extract regional features and obtain a regional feature vector;
[0017] Convolution and pooling operations are performed on the regional feature vector to obtain target features.
[0018] As an optional implementation manner of the embodiment of the present invention, judging whether there is a moving object in the video frame according to the target feature includes:
[0019] Performing convolution calculation on the target feature to obtain weights and offsets;
[0020] Calculating a sparse coding difference value of a video frame within a preset time according to the weight and the offset;
[0021] If the sparse coding difference value is greater than a preset threshold, it is determined that there is a moving object in the video frame.
[0022] As an optional implementation of the embodiment of the present invention, the target convolutional neural network model is trained by the following method:
[0023] Acquire a training sample set; the training sample set includes: a sparse representation of each original video frame and a target sparse coefficient corresponding to each original video frame;
[0024] Inputting the sparse representation of each original video frame into the initial convolutional neural network model to obtain a sparse coefficient of each original video frame;
[0025] A loss function is determined based on the sparse coefficient of each original video frame and the target sparse coefficient corresponding to each original video frame, and the weight parameters of the initial convolutional neural network model are adjusted according to the loss function to obtain the target convolutional neural network model.
[0026] As an optional implementation of the embodiment of the present invention, the obtaining of the training sample set includes:
[0027] Real-time acquisition of video frame sequences;
[0028] The video data of each video frame is preprocessed, and the moving objects in each video frame are marked and classified.
[0029] In a second aspect, the present application provides a motion detection device, comprising:
[0030] Video acquisition module, used to acquire video frames in real time;
[0031] A sparse coding module, used to perform sparse coding on the video frame and convert the image of the video frame into a sparse representation;
[0032] A feature extraction module, used to extract features from the sparse representation using a target convolutional neural network model to obtain target features;
[0033] The feature judgment module is used to judge whether there is a moving object in the video frame according to the target feature.
[0034] As an optional implementation of the embodiment of the present invention, the device further includes:
[0035] The information acquisition module is used to analyze the motion trajectory of the moving object if the moving object exists, and obtain the motion information of the moving object.
[0036] As an optional implementation manner of the embodiment of the present invention, the sparse coding module is specifically used for:
[0037] Based on the singular value decomposition algorithm, obtain the dictionary matrix;
[0038] Performing sparse coding on the dictionary matrix to obtain a sparse coding matrix;
[0039] The sparse coding matrix is inversely transformed to obtain a motion feature matrix.
[0040] As an optional implementation of the embodiment of the present invention, the feature extraction module is specifically used to:
[0041] Inputting the motion feature matrix into the target convolutional neural network model to extract regional features and obtain a regional feature vector;
[0042] Convolution and pooling operations are performed on the regional feature vector to obtain target features.
[0043] As an optional implementation of the embodiment of the present invention, the feature judgment module is specifically used to:
[0044] Performing convolution calculation on the target feature to obtain weights and offsets;
[0045] Calculating a sparse coding difference value of a video frame within a preset time according to the weight and the offset;
[0046] If the sparse coding difference value is greater than a preset threshold, it is determined that there is a moving object in the video frame.
[0047] As an optional implementation of the embodiment of the present invention, the feature extraction module includes:
[0048] An acquisition unit is used to acquire a training sample set; the training sample set includes: a sparse representation of each original video frame and a target sparse coefficient corresponding to each original video frame;
[0049] An input unit, used to input the sparse representation of each original video frame into the initial convolutional neural network model to obtain a sparse coefficient of each original video frame;
[0050] A determination unit is used to determine a loss function based on the sparse coefficients of each original video frame and the target sparse coefficients corresponding to each original video frame, and adjust the weight parameters of the initial convolutional neural network model according to the loss function to obtain the target convolutional neural network model.
[0051] As an optional implementation manner of the embodiment of the present invention, the acquisition unit is specifically used to:
[0052] Real-time acquisition of video frame sequences;
[0053] The video data of each video frame is preprocessed, and the moving objects in each video frame are marked and classified.
[0054] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method provided in the first aspect of the present application are implemented.
[0055] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method provided in the first aspect of the present application are implemented.
[0056] Compared with the prior art, the technical solution provided by this application has the following advantages:
[0057] The present application provides a motion detection method, which collects video frames in real time, performs sparse coding on the video frames, converts the images of the video frames into sparse representations, uses a target convolutional neural network model to extract regional features from the sparse representations, obtains target features, and determines whether there are moving objects in the video frames based on the target features. Sparse coding can accurately detect whether a scene has changed, and neural networks can quickly detect whether an object has moved. The combination of the two greatly improves the recognition speed and accuracy of moving objects, further improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0059] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0060] Figure 1 An application scenario diagram of a motion detection method in an embodiment;
[0061] Figure 2 is a schematic flow chart of a motion detection method in one embodiment;
[0062] Figure 3 is a schematic structural diagram of a motion detection device in one embodiment;
[0063] Figure 4 FIG. 1 is a schematic diagram of the structure of an electronic device in an embodiment. DETAILED DESCRIPTION
[0064] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.
[0065] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only part of the embodiments of the present application, rather than all of the embodiments.
[0066] At present, the mainstream motion detection algorithm technologies include the following:
[0067] Background subtraction is a commonly used motion detection method. Its basic principle is to compare the current frame with the background frame to extract the moving target. This method requires accurate removal of background noise to extract the moving target, and noise has a greater impact on this detection method.
[0068] The inter-frame difference method is a method for extracting moving targets by calculating the difference between the current frame and the historical frame. The main advantage of this method is that the algorithm is simple to implement, but it has high requirements for the matching of sequence frames. The inter-frame difference method is suitable for scenes with fast or small moving targets, but for scenes with slow or large moving targets, this method may result in missed detection or false detection.
[0069] Optical flow is another common motion detection method, which uses optical flow calculation to estimate the characteristic parameters of motion, such as speed, position, etc. The optical flow method requires high computational complexity, but has high requirements on hardware equipment. The optical flow method is suitable for scenes with slow or large moving targets, but for scenes with fast or small moving targets, this method may miss detection or make false detections.
[0070] Motion vector analysis is a relatively new motion detection method that detects moving targets by analyzing motion vector information such as acceleration and speed. Compared with the optical flow method, this method has higher real-time performance and hardware feasibility. Motion vector analysis is suitable for scenes with fast or large moving targets, but for scenes with slow or small moving targets, this method may miss detection or make false detections.
[0071] Existing motion detection algorithms usually rely on expensive hardware equipment and complex algorithms, which limits their popularity and use in practical applications. Therefore, in order to improve the accuracy and reliability of motion detection, it is necessary to seek a more efficient, low-cost and automated motion detection method to achieve a high-efficiency, low-energy, and high-precision motion detection algorithm, thereby providing more accurate and reliable detection results for the required areas of the market.
[0072] The present application provides a motion detection method that can be applied to electronic devices with a camera function. For example, the electronic device can be, but is not limited to, a smart phone, a tablet, a smart camera, a smart watch, a smart bracelet, a smart glasses, and other electronic devices with a camera function. Among them, the operating systems of various electronic devices may include, but are not limited to, Android operating system, IOS operating system, Symbian operating system, Black Berry operating system, Windows Phone 8 operating system, etc., which are not limited in the embodiments of the present invention. Figure 1FIG. 1 is an application scenario diagram of a motion detection method in an embodiment. For example, during a selfie, after a face moves, AF (Automatic Focus) needs to refocus, and at this time AF needs to obtain whether the object has changed. In addition, it can also be applied to HDR, super-resolution, video surveillance, alarm systems and other similar scenarios.
[0073] In one embodiment, Figure 2 As shown, a motion detection method is provided. This application embodiment takes the method applied to an electronic device as an example to illustrate, and the method includes the following steps:
[0074] S21. Collect video frames in real time.
[0075] Specifically, a high-definition camera is used to collect video frames in real time.
[0076] S22. Perform sparse coding on the video frame to convert the image of the video frame into a sparse representation.
[0077] Specifically, sparse coding is a representation learning method that aims to represent the input signal as a small number of non-zero activities. The basic idea of sparse coding is to represent the signal by selecting a minimum number of basis functions and a linear combination of the input signal, so as to better capture the structure and characteristics of the signal. In other words, sparse coding represents the input data as a sparse linear combination in an overcomplete dictionary.
[0078] In some embodiments, the above step S22 (sparsely encoding the video frame and converting the image of the video frame into a sparse representation) can be implemented as follows:
[0079] A. Based on the singular value decomposition algorithm, obtain the dictionary matrix.
[0080] Among them, singular value decomposition (SVD) is a matrix factorization method that decomposes any given matrix into the product of three matrices: an orthogonal left singular vector matrix, a diagonal singular value matrix, and an orthogonal right singular vector matrix. The singular value representations of the data set are arranged according to importance, and unimportant eigenvectors are discarded to achieve the purpose of dimensionality reduction, thereby finding the main components in the data.
[0081] Specifically, based on the singular value decomposition algorithm, given a set of input data D, the matrix D is represented as the product of the dictionary matrix A and the sparse coding matrix X, that is, D = AX. The dictionary matrix A is composed of atoms, each atom is a fixed vector. The sparse coding matrix X is a matrix composed of sparse coefficients, each coefficient corresponds to an atom. The dictionary matrix stores the features in the original sample, while the sparse coding matrix represents the way the original sample uses the features to combine into its own way.
[0082] The reconstruction error is minimized by updating the dictionary matrix A and the sparse coding matrix X, i.e., ||D-AX||. The reconstruction error refers to the difference or loss between the reconstructed data and the original input when the input data is reconstructed (or restored) by the learned model. The reconstruction error can be used to evaluate the model's ability to restore the input data. A smaller reconstruction error means that the model can accurately reconstruct the input data, while a larger reconstruction error means that the model cannot completely restore the original data.
[0083] In the singular value decomposition algorithm, the optimal solution is gradually approached by iteratively updating the dictionary matrix A and the sparse coding matrix X. Specifically, the sparse coding matrix X is first fixed, and then the dictionary matrix A is updated. In each iteration, the dictionary matrix A is decomposed into the product of U, Σ, and V by singular value decomposition (SVD), that is, A = UΣV T . Among them, U and V are orthogonal matrices, Σ is a diagonal matrix, and the elements on the diagonal are called singular values. Then the dictionary matrix A and the sparse coding matrix X are updated according to the size of the singular values. For small singular values, they are set to zero to achieve the sparsity of the dictionary; for large singular values, they are retained and the elements in the dictionary matrix A and the sparse coding matrix X are updated accordingly.
[0084] By iteratively updating the dictionary matrix A and the sparse coding matrix X, the K-SVD algorithm can gradually approach the optimal solution, making the atoms in the dictionary fit the training data as much as possible while maintaining the sparsity of the dictionary. The final dictionary matrix A can be used as the output data of sparse coding for subsequent motion detection tasks.
[0085] B. Perform sparse coding on the dictionary matrix to obtain a sparse coding matrix.
[0086] Specifically, the dictionary matrix A is sparsely encoded to obtain a sparse coding matrix X. In the sparse coding process, the Matching Pursuit (MP) algorithm can be used to automatically learn a sparse representation according to the characteristics of the data, thereby obtaining a more accurate motion feature representation.
[0087] C. Perform an inverse transformation on the sparse coding matrix to obtain a motion feature matrix.
[0088] Specifically, the sparse coding matrix X is inversely transformed to obtain the motion feature matrix Y. In this step, the inverse transformation method, that is, the commonly used least squares method, can be used to convert the sparse coding matrix X back to the original data space, thereby obtaining a more intuitive and easy-to-understand motion feature representation, and the motion feature matrix Y is used as the input parameter of the CNN for convolution.
[0089] S23. Use the target convolutional neural network model to extract features from the sparse representation to obtain target features.
[0090] In some embodiments, the above step S23 (using the target convolutional neural network model to extract features from the sparse representation to obtain target features) can be implemented as follows:
[0091] The motion feature matrix is input into the target convolutional neural network model to extract regional features and obtain a regional feature vector.
[0092] Convolution and pooling operations are performed on the regional feature vector to obtain target features.
[0093] Among them, regional feature extraction refers to local feature extraction. For example, through weighted processing, a picture is divided into 30*30 pixels, and the 5*5 in the middle position is the required core area.
[0094] Specifically, regional features are extracted through a region-based method to obtain a set of regional feature vectors; the regional feature vectors are used as input, and through multi-layer convolution and pooling operations, more advanced feature representations, namely target features, are learned and extracted.
[0095] Optionally, the target convolutional neural network model is trained by the following method:
[0096] (1) Obtain a training sample set.
[0097] The training sample set includes: a sparse representation of each original video frame and a target sparse coefficient corresponding to each original video frame.
[0098] In some embodiments, the above step (1) can be implemented as follows:
[0099] Real-time acquisition of video frame sequences;
[0100] The video data of each video frame is preprocessed, and the moving objects in each video frame are marked and classified.
[0101] Specifically, after a high-definition camera is used to capture the video sequence in real time, the data is preprocessed. The preprocessing process includes: first, each frame of the video sequence is cleaned to remove invalid or noise data; then each frame of the video is enhanced to enhance the diversity and generalization ability of the data through operations such as rotation, translation, and scaling; finally, each frame of the video data is labeled, and the moving targets are marked and classified.
[0102] (2) Inputting the sparse representation of each original video frame into the initial convolutional neural network model to obtain the sparse coefficients of each original video frame.
[0103] The initial convolutional neural network model includes convolutional layers, pooling layers, flattening layers, fully connected layers, and output layers. The dimension of the convolution kernel matrix is 1, the padding size is 0.5, and the stride is 0.01. The pooling standard is MAX, which is used to extract deep motion feature information. The flattening layer outputs one-dimensional vector features, and the flattened feature vector is connected to the fully connected layer for further feature integration and mapping. The gradient iteration parameter iteration step of the fully connected layer is 0.01, and the stop iteration threshold is 500. The output layer is used to output sparse coefficients.
[0104] (3) Determine a loss function based on the sparse coefficient of each original video frame and the target sparse coefficient corresponding to each original video frame, and adjust the weight parameters of the initial convolutional neural network model according to the loss function to obtain the target convolutional neural network model.
[0105] Specifically, a training group is selected, and 2048 samples are randomly selected from the sample set as the training group; the weight matrix x and offset threshold z of each flat layer and output layer are set to random values close to 0, and then the learning rate and control parameter accuracy are initialized. An input pattern is taken from the training group and added to the neural network model, and its target output vector is given; the output vector of the flat layer and the actual output vector of the neural network are calculated. The output Y of the current scene in the actual state is compared with the output Y of the current scene in the ideal state, and the output error is calculated; the error of the hidden unit of the flat layer also needs to be calculated; the adjustment amount of each weight and the adjustment amount of the threshold are calculated in turn, and then the weight value and the adjustment threshold are adjusted; after multiple rounds of training, it is judged whether the indicator meets the accuracy requirement. If not, it returns and continues to iterate; if it meets, it goes to the next step; after the training is completed, the weight matrix x and the offset threshold z are saved in the file. At this time, it can be considered that each weight coefficient has reached stability and the scene classifier has been formed. Training is performed again, and the weight matrix coefficient x and threshold z are directly exported from the file for training without initialization.
[0106] S24. Determine whether there is a moving object in the video frame according to the target feature.
[0107] In some embodiments, the above step S24 (determining whether there is a moving object in the video frame according to the target feature) can also be implemented in the following manner:
[0108] Performing convolution calculation on the target feature to obtain weights and offsets;
[0109] Calculating a sparse coding difference value of a video frame within a preset time according to the weight and the offset;
[0110] If the sparse coding difference value is greater than a preset threshold, it is determined that there is a moving object in the video frame.
[0111] Specifically, the change in the sparse value of the video frame in time t is calculated by Δf(x,z)=|f(x,z,t)-f(x,z,t-Δt)|. When Δf(x,z) is greater than the set threshold, it means that the current object is moving. Among them, Δf(x,z) represents the sparse coding difference value of the video frame, f(x,z,t) represents the pixel value at point (x,z) and time t, and Δt represents the time interval.
[0112] A hybrid model of sparse coding and convolutional neural networks is used to segment the foreground and background of video frames, and motion is detected by judging whether the background changes. At the same time, foreground features are extracted to detect motion trajectories.
[0113] In some embodiments, after executing the above step S24 (determining whether there is a moving object in the video frame according to the target feature), the following steps may also be executed:
[0114] If the moving object exists, the moving trajectory of the moving object is analyzed to obtain the moving information of the moving object.
[0115] The motion information includes but is not limited to the motion direction, motion speed, etc.
[0116] Specifically, if there is a moving object, further analysis is performed based on the motion trajectory of the object. By detecting and analyzing the motion trajectory of the object, more information about the object, such as the direction of motion, speed, etc., can be obtained.
[0117] The present application provides a motion detection method, which collects video frames in real time, performs sparse coding on the video frames, converts the images of the video frames into sparse representations, uses a target convolutional neural network model to extract regional features from the sparse representations, obtains target features, and determines whether there are moving objects in the video frames based on the target features. Sparse coding can accurately detect whether a scene has changed, and neural networks can quickly detect whether an object has moved. The combination of the two greatly improves the recognition speed and accuracy of moving objects, further improving the user experience.
[0118] In one embodiment, Figure 3 As shown, a motion detection device 300 is provided, comprising:
[0119] The video acquisition module 310 is used to acquire video frames in real time;
[0120] A sparse coding module 320, configured to perform sparse coding on the video frame, and convert the image of the video frame into a sparse representation;
[0121] A feature extraction module 330 is used to extract features from the sparse representation using a target convolutional neural network model to obtain target features;
[0122] The feature judgment module 340 is used to judge whether there is a moving object in the video frame according to the target feature.
[0123] As an optional implementation of the embodiment of the present invention, the device further includes:
[0124] The information acquisition module is used to analyze the motion trajectory of the moving object if the moving object exists, and obtain the motion information of the moving object.
[0125] As an optional implementation manner of the embodiment of the present invention, the sparse coding module is specifically used for:
[0126] Based on the singular value decomposition algorithm, obtain the dictionary matrix;
[0127] Performing sparse coding on the dictionary matrix to obtain a sparse coding matrix;
[0128] The sparse coding matrix is inversely transformed to obtain a motion feature matrix.
[0129] As an optional implementation of the embodiment of the present invention, the feature extraction module is specifically used to:
[0130] Inputting the motion feature matrix into the target convolutional neural network model to extract regional features and obtain a regional feature vector;
[0131] Convolution and pooling operations are performed on the regional feature vector to obtain target features.
[0132] As an optional implementation of the embodiment of the present invention, the feature judgment module is specifically used to:
[0133] Performing convolution calculation on the target feature to obtain weights and offsets;
[0134] Calculating a sparse coding difference value of a video frame within a preset time according to the weight and the offset;
[0135] If the sparse coding difference value is greater than a preset threshold, it is determined that there is a moving object in the video frame.
[0136] As an optional implementation of the embodiment of the present invention, the feature extraction module includes:
[0137] An acquisition unit is used to acquire a training sample set; the training sample set includes: a sparse representation of each original video frame and a target sparse coefficient corresponding to each original video frame;
[0138] An input unit, used to input the sparse representation of each original video frame into the initial convolutional neural network model to obtain a sparse coefficient of each original video frame;
[0139] A determination unit is used to determine a loss function based on the sparse coefficients of each original video frame and the target sparse coefficients corresponding to each original video frame, and adjust the weight parameters of the initial convolutional neural network model according to the loss function to obtain the target convolutional neural network model.
[0140] As an optional implementation manner of the embodiment of the present invention, the acquisition unit is specifically used to:
[0141] Real-time acquisition of video frame sequences;
[0142] The video data of each video frame is preprocessed, and the moving objects in each video frame are marked and classified.
[0143] The present application provides a motion detection device that collects video frames in real time, performs sparse coding on the video frames, converts the images of the video frames into sparse representations, uses a target convolutional neural network model to extract regional features from the sparse representations, obtains target features, and determines whether there are moving objects in the video frames based on the target features. Sparse coding can accurately detect whether a scene has changed, and neural networks can quickly detect whether an object has moved. The combination of the two greatly improves the recognition speed and accuracy of moving objects, further improving the user experience.
[0144] For the specific definition of the motion detection device, please refer to the definition of the motion detection method applied to the electronic device above, which will not be repeated here. Each module in the above motion detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the electronic device in the form of hardware, or can be stored in the memory in the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0145] In one embodiment, an electronic device is provided, whose internal structure diagram can be as follows: Figure 4 As shown. The electronic device includes a processor 41 and a memory 42 connected via a system bus. Among them, the processor 41 of the electronic device is used to provide computing and control capabilities. The memory 42 of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the computer program is executed by the processor, a motion detection method is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.
[0146] Those skilled in the art will understand that Figure 4 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.
[0147] In one embodiment, the motion detection method provided by the present application can be implemented in the form of a computer program. The computer program can be Figure 4 The memory of the electronic device may store various program modules constituting the mobile terminal of the electronic device, for example, Figure 3 The video acquisition module 310, sparse coding module 320, feature extraction module 330 and feature judgment module 340 are shown. The computer program composed of various program modules enables the processor to execute the steps of the motion detection method of the electronic device of each embodiment of the present application described in this specification.
[0148] For example, Figure 4 The electronic device shown can be Figure 3The video acquisition module 310 in the motion detection device shown in the figure performs step S21. The electronic device may perform step S22 through the sparse coding module 320. The electronic device may perform step S23 through the feature extraction module 330. The electronic device may perform step S24 through the feature determination module 340.
[0149] In one embodiment, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0150] Real-time acquisition of video frames;
[0151] Performing sparse coding on the video frame to convert the image of the video frame into a sparse representation;
[0152] Using a target convolutional neural network model to extract features from the sparse representation to obtain target features;
[0153] It is determined whether there is a moving object in the video frame according to the target feature.
[0154] As an optional implementation manner of the embodiment of the present invention, after judging whether there is a moving object in the video frame according to the target feature, the method further includes:
[0155] If the moving object exists, the moving trajectory of the moving object is analyzed to obtain the moving information of the moving object.
[0156] As an optional implementation manner of the embodiment of the present invention, after judging whether there is a moving object in the video frame according to the target feature, the method further includes:
[0157] If the moving object exists, the moving trajectory of the moving object is analyzed to obtain the moving information of the moving object.
[0158] As an optional implementation of the embodiment of the present invention, the use of the target convolutional neural network model to extract features from the sparse representation to obtain target features includes:
[0159] Inputting the motion feature matrix into the target convolutional neural network model to extract regional features and obtain a regional feature vector;
[0160] Convolution and pooling operations are performed on the regional feature vector to obtain target features.
[0161] As an optional implementation manner of the embodiment of the present invention, judging whether there is a moving object in the video frame according to the target feature includes:
[0162] Performing convolution calculation on the target feature to obtain weights and offsets;
[0163] Calculating a sparse coding difference value of a video frame within a preset time according to the weight and the offset;
[0164] If the sparse coding difference value is greater than a preset threshold, it is determined that there is a moving object in the video frame.
[0165] As an optional implementation of the embodiment of the present invention, the target convolutional neural network model is trained by the following method:
[0166] Acquire a training sample set; the training sample set includes: a sparse representation of each original video frame and a target sparse coefficient corresponding to each original video frame;
[0167] Inputting the sparse representation of each original video frame into the initial convolutional neural network model to obtain a sparse coefficient of each original video frame;
[0168] A loss function is determined based on the sparse coefficient of each original video frame and the target sparse coefficient corresponding to each original video frame, and the weight parameters of the initial convolutional neural network model are adjusted according to the loss function to obtain the target convolutional neural network model.
[0169] As an optional implementation of the embodiment of the present invention, the obtaining of the training sample set includes:
[0170] Real-time acquisition of video frame sequences;
[0171] The video data of each video frame is preprocessed, and the moving objects in each video frame are marked and classified.
[0172] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
[0173] Real-time acquisition of video frames;
[0174] Performing sparse coding on the video frame to convert the image of the video frame into a sparse representation;
[0175] Using a target convolutional neural network model to extract features from the sparse representation to obtain target features;
[0176] It is determined whether there is a moving object in the video frame according to the target feature.
[0177] As an optional implementation manner of the embodiment of the present invention, after judging whether there is a moving object in the video frame according to the target feature, the method further includes:
[0178] If the moving object exists, the moving trajectory of the moving object is analyzed to obtain the moving information of the moving object.
[0179] As an optional implementation manner of the embodiment of the present invention, after judging whether there is a moving object in the video frame according to the target feature, the method further includes:
[0180] If the moving object exists, the moving trajectory of the moving object is analyzed to obtain the moving information of the moving object.
[0181] As an optional implementation of the embodiment of the present invention, the use of the target convolutional neural network model to extract features from the sparse representation to obtain target features includes:
[0182] Inputting the motion feature matrix into the target convolutional neural network model to extract regional features and obtain a regional feature vector;
[0183] Convolution and pooling operations are performed on the regional feature vector to obtain target features.
[0184] As an optional implementation manner of the embodiment of the present invention, judging whether there is a moving object in the video frame according to the target feature includes:
[0185] Performing convolution calculation on the target feature to obtain weights and offsets;
[0186] Calculating a sparse coding difference value of a video frame within a preset time according to the weight and the offset;
[0187] If the sparse coding difference value is greater than a preset threshold, it is determined that there is a moving object in the video frame.
[0188] As an optional implementation of the embodiment of the present invention, the target convolutional neural network model is trained by the following method:
[0189] Acquire a training sample set; the training sample set includes: a sparse representation of each original video frame and a target sparse coefficient corresponding to each original video frame;
[0190] Inputting the sparse representation of each original video frame into the initial convolutional neural network model to obtain a sparse coefficient of each original video frame;
[0191] A loss function is determined based on the sparse coefficient of each original video frame and the target sparse coefficient corresponding to each original video frame, and the weight parameters of the initial convolutional neural network model are adjusted according to the loss function to obtain the target convolutional neural network model.
[0192] As an optional implementation of the embodiment of the present invention, the obtaining of the training sample set includes:
[0193] Real-time acquisition of video frame sequences;
[0194] The video data of each video frame is preprocessed, and the moving objects in each video frame are marked and classified.
[0195] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0196] Those of ordinary skill in the art can understand that the implementation of all or part of the process in the example method described above can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the process of the embodiments of the above methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory may include ROM (Read-Only Memory), tape, floppy disk, flash memory or optical storage, etc. Volatile memory may include RAM (Random Access Memory) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as SRAM (Static Random Access Memory) and DRAM (Dynamic Random Access Memory).
[0197] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0198] The above is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features applied for herein.
Claims
1. A motion detection method, characterized in that: The method comprises: Real-time acquisition of video frames; Performing sparse coding on the video frame to convert the image of the video frame into a sparse representation; Using a target convolutional neural network model to extract features from the sparse representation to obtain target features; It is determined whether there is a moving object in the video frame according to the target feature.
2. The method according to claim 1, characterized in that After determining whether there is a moving object in the video frame according to the target feature, the method further includes: If the moving object exists, the moving trajectory of the moving object is analyzed to obtain the moving information of the moving object.
3. The method according to claim 1, characterized in that The step of performing sparse coding on the video frame to obtain a sparse representation includes: Based on the singular value decomposition algorithm, obtain the dictionary matrix; Performing sparse coding on the dictionary matrix to obtain a sparse coding matrix; The sparse coding matrix is inversely transformed to obtain a motion feature matrix.
4. The method according to claim 3, characterized in that The method of extracting features from the sparse representation using a target convolutional neural network model to obtain target features includes: Inputting the motion feature matrix into the target convolutional neural network model to extract regional features and obtain a regional feature vector; Convolution and pooling operations are performed on the regional feature vector to obtain target features.
5. The method according to claim 4, characterized in that The determining, according to the target feature, whether there is a moving object in the video frame comprises: Performing convolution calculation on the target feature to obtain weights and offsets; Calculating a sparse coding difference value of a video frame within a preset time according to the weight and the offset; If the sparse coding difference value is greater than a preset threshold, it is determined that there is a moving object in the video frame.
6. The method according to claim 1, characterized in that The target convolutional neural network model is trained by the following method: Acquire a training sample set; the training sample set includes: a sparse representation of each original video frame and a target sparse coefficient corresponding to each original video frame; Inputting the sparse representation of each original video frame into the initial convolutional neural network model to obtain a sparse coefficient of each original video frame; A loss function is determined based on the sparse coefficient of each original video frame and the target sparse coefficient corresponding to each original video frame, and the weight parameters of the initial convolutional neural network model are adjusted according to the loss function to obtain the target convolutional neural network model.
7. The method according to claim 6, characterized in that The step of obtaining a training sample set includes: Real-time acquisition of video frame sequences; The video data of each video frame is preprocessed, and the moving objects in each video frame are marked and classified.
8. A motion detection device, characterized in that: include: Video acquisition module, used to acquire video frames in real time; A sparse coding module, used to perform sparse coding on the video frame and convert the image of the video frame into a sparse representation; A feature extraction module, used to extract features from the sparse representation using a target convolutional neural network model to obtain target features; The feature judgment module is used to judge whether there is a moving object in the video frame according to the target feature.
9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the motion detection method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the motion detection method according to any one of claims 1 to 7 is implemented.
Citation Information
Cited By
Inter-frame macro block motion vector determination method and device based on neural network
CN121173965A
Neural network based inter-frame macroblock motion vector determination method and apparatus
CN121173965B
