Action recognition method, device and computer readable storage medium

Abstract

The present invention provides an action recognition method, device and computer-readable storage medium. The method extracts a first spatial cumulative image collection and a first temporal cumulative image collection from a video to be processed, and then extracts first target action information and second target action information corresponding to the first spatial cumulative image collection and the first temporal cumulative image collection respectively. Since the first target action information and the second target action information respectively reflect the spatial and temporal target action information of the video to be processed, the stability of action recognition based on the first target action information and the second target action information is enhanced, and when an arbitrary pixel point in the first spatial cumulative image collection or an arbitrary pixel point in the first temporal cumulative image collection moves to a preset boundary area, the pixel point regression convergence of the video to be processed can be quickly determined, thereby accelerating the processing speed of the video to be processed, and thereby improving the recognition efficiency during target action recognition.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of video image processing, and in particular to a motion recognition method, device and computer-readable storage medium. Background Art

[0002] At present, in the motion recognition technology for moving targets in videos, it is usually necessary to perform target detection, target classification, motion tracking and feature description on the moving targets in the video, and finally identify the motion of the moving targets in the video through the acquired feature description. However, the technical problems of the existing technology for motion recognition of moving targets in videos are: the existing technology has a complex motion recognition process and low recognition efficiency. Summary of the invention

[0003] The present invention is intended to provide a motion recognition method, device and computer-readable storage medium to improve the efficiency of motion recognition of moving objects in a video.

[0004] The technical solution of the present invention is achieved in this way:

[0005] An embodiment of the present invention provides a method for motion recognition, comprising:

[0006] Get the video to be processed;

[0007] Extracting a first spatial accumulated image collection and a first temporal accumulated image collection from the video to be processed;

[0008] Performing pixel convergence processing on the first spatial accumulation image collection and the first temporal accumulation image collection respectively to obtain a first target image corresponding to the first spatial accumulation image collection and a second target image corresponding to the first temporal accumulation image collection;

[0009] extracting first target action information and second target action information from the first target image and the second target image respectively;

[0010] Based on the first target action information, the second target action information and the reference action information, a target action in the video to be processed is determined.

[0011] The embodiment of the present invention further provides a motion recognition device, comprising:

[0012] A video acquisition unit, used to acquire the video to be processed;

[0013] A first processing unit, configured to extract a first spatially accumulated image collection and a first temporally accumulated image collection from the video to be processed;

[0014] A second processing unit is used to perform pixel convergence processing on the first spatial accumulation image collection and the first temporal accumulation image collection, respectively, to obtain a first target image and a second target image corresponding to the first spatial accumulation image collection and the first temporal accumulation image collection, respectively;

[0015] An information extraction unit, configured to extract first target action information and second target action information from the first target image and the second target image respectively;

[0016] The identification unit is used to determine the target action in the video to be processed based on the first target action information, the second target action information and the reference action information.

[0017] An embodiment of the present invention further provides a motion recognition device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps in the above method when executing the program.

[0018] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method are implemented.

[0019] The embodiment of the present invention obtains a video to be processed; extracts a first spatial cumulative image collection and a first temporal cumulative image collection from the video to be processed, performs pixel convergence processing on the first spatial cumulative image collection and the first temporal cumulative image collection respectively, and obtains a first target image corresponding to the first spatial cumulative image collection and a second target image corresponding to the first temporal cumulative image collection; extracts first target action information and second target action information from the first target image and the second target image respectively. Since the first target action information and the second target action information are extracted based on the first spatial cumulative image collection and the first temporal cumulative image collection respectively, the stability of action recognition based on the first target action information and the second target action information is enhanced, and when an arbitrary pixel point in the first spatial cumulative image collection or an arbitrary pixel point in the first temporal cumulative image collection moves to the boundary of a region composed of eight points of equal distance around the pixel point, the pixel point regression convergence in the first spatial cumulative image collection or the first temporal cumulative image collection can be quickly determined, thereby obtaining the first target image and the second target image, thereby improving the recognition efficiency during target action recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A process of an action recognition method provided by an embodiment of the present invention Figure 1 ;

[0021] Figure 2aA schematic diagram of a first spatial accumulated image collection provided by an embodiment of the present invention Figure 1 ;

[0022] Figure 2b A second schematic diagram of a first spatial accumulated image collection provided by an embodiment of the present invention;

[0023] Figure 2c A schematic diagram of a first spatial accumulated image collection provided by an embodiment of the present invention Figure 3 ;

[0024] Figure 2d A schematic diagram of a first spatial accumulated image collection provided by an embodiment of the present invention Figure 4 ;

[0025] Figure 2e A schematic diagram of a first spatial accumulated image collection provided by an embodiment of the present invention Figure 5 ;

[0026] Figure 2f A schematic diagram of a first spatial accumulated image collection provided by an embodiment of the present invention Figure 6 ;

[0027] Figure 3 A schematic diagram of a first time accumulated image collection provided by an embodiment of the present invention;

[0028] Figure 4 Flow chart 2 of a method for motion recognition provided by an embodiment of the present invention;

[0029] Figure 5 A process of an action recognition method provided by an embodiment of the present invention Figure 3 ;

[0030] Figure 6 A process of an action recognition method provided by an embodiment of the present invention Figure 4 ;

[0031] Figure 7 A process of an action recognition method provided by an embodiment of the present invention Figure 5 ;

[0032] Figure 8 A process of an action recognition method provided by an embodiment of the present invention Figure 6 ;

[0033] Fig. 9 A process of an action recognition method provided by an embodiment of the present invention Figure 7 ;

[0034] Fig.10 A process of an action recognition method provided by an embodiment of the present invention Figure 8 ;

[0035] Fig.11 A process of an action recognition method provided by an embodiment of the present invention Figure 9 ;

[0036] Fig.12 A schematic diagram of the structure of a motion recognition device provided by an embodiment of the present invention;

[0037] Fig.13 A schematic diagram of a motion recognition device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0039] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0040] If similar descriptions of "first / second" appear in the invention document, the following description is added. In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the invention described herein can be implemented in an order other than that illustrated or described herein.

[0041] Figure 1 The present invention provides a method for motion recognition. Figure 1 , will combine Figure 1 The steps shown are explained.

[0042] S01, obtaining a video to be processed.

[0043] In the disclosed embodiment of the present invention, the motion recognition device obtains the video to be processed from the external device. The external device establishes a communication line with the motion recognition device in advance. After the external device collects the video to be processed, it sends the video to be processed to the motion recognition device through the communication line. The video to be processed may include: street view video, indoor scene video and security inspection video. The external device may be a camera, monitoring equipment, etc.

[0044] In the disclosed embodiment of the present invention, the video to be processed includes an object whose action needs to be recognized, such as a person, an animal, or a machine, etc. If a certain part of the object whose action needs to be recognized moves, then the part is the target for recognition by the action recognition device.

[0045] In the disclosed embodiment of the present invention, the external device acquires video images in real time, and when objects such as people or animals appear in the acquired video images, the external device sends the video including the objects such as people or animals as a video to be processed to the motion recognition device.

[0046] In the disclosed embodiment of the present invention, after the action recognition device obtains the video to be processed, the action recognition device performs binarization processing on the video to be processed, and the action recognition device performs binarization processing on the pixel points on the video to be processed to obtain a grayscale video.

[0047] S02: extracting a first spatially accumulated image collection and a first temporally accumulated image collection from the video to be processed.

[0048] In the disclosed embodiment of the present invention, the action recognition device extracts a first spatial cumulative image collection and a first temporal cumulative image collection from the video to be processed. The first spatial cumulative image collection includes: cumulative images of objects in the video to be processed on the spatial plane. The first temporal cumulative image collection includes: cumulative images of objects in the video to be processed along the time axis. The action recognition device sorts each video frame of the video to be processed in time order, and obtains video frames including objects. The action recognition device sorts the video frames including objects on the same spatial plane to obtain the first spatial cumulative image collection. The action recognition device sorts each video frame of the video to be processed in time order, and obtains video frames including objects. The action recognition device sorts the video frames including objects along the time axis to obtain the first temporal cumulative image collection.

[0049] In the disclosed embodiment of the present invention, the action recognition device can extract the first spatial accumulated image collection and the first temporal accumulated image collection from the grayscale video by the same method as above.

[0050] In the disclosed embodiment of the present invention, the action recognition device extracts a first spatial accumulated image collection from the video to be processed. Figure 2a-2f , the first spatial cumulative image collection includes tiles sorted on the spatial plane Figure 2a , Figure 2b , Figure 2c , Figure 2d , Figure 2e and Figure 2f . Figure 2a , Figure 2b , Figure 2c , Figure 2d, Figure 2e and Figure 2f The person in the figure is the object to be identified, and the person's waving arm is the target.

[0051] In the disclosed embodiment of the present invention, the action recognition device extracts a first time accumulated image collection from the video to be processed. Figure 3 , the first time accumulated image collection includes 6 pictures sorted along the time axis. Among them, the person waving his arm is the target. Figure 3 It can be seen that the action of a person waving his arms is the target action of the video to be processed.

[0052] S03, performing pixel convergence processing on the first spatial accumulation image collection and the first temporal accumulation image collection respectively to obtain a first target image corresponding to the first spatial accumulation image collection and a second target image corresponding to the first temporal accumulation image collection.

[0053] In the disclosed embodiment of the present invention, the action recognition device operates multiple images in the first spatial cumulative image collection through the nine-point differential Poisson equation. If the pixel points of the multiple images in the first spatial cumulative image collection finally regress and converge, the first target image corresponding to the first spatial cumulative image collection is obtained by solving the nine-point differential Poisson equation.

[0054] In the disclosed embodiment of the present invention, the action recognition device operates multiple images in the first time accumulated image collection through the nine-point differential Poisson equation. If the pixel points of the multiple images in the first time accumulated image collection finally regress and converge, by solving the nine-point differential Poisson equation, a second target image corresponding to the first time accumulated image collection will be obtained.

[0055] In the disclosed embodiment of the present invention, the first target image and the second target image include the moving part of the object, which is the target, wherein the target may be a person's hand, a person's head, a dog's tail, a car door, etc.

[0056] S04, extracting first target action information and second target action information from the first target image and the second target image respectively.

[0057] In the disclosed embodiment of the present invention, the action recognition device can obtain the first target action information and the second target action information corresponding to the first target image and the second target image respectively through classification and regression processing according to the size of the pixel values ​​of several pixel points on the first target image and the second target image, and the position corresponding to each pixel point.

[0058] In the disclosed embodiment of the present invention, the first target action information and the second target action information may include vector information corresponding to the target image, matrix information corresponding to the target image, etc.

[0059] In the disclosed embodiment of the present invention, the action recognition device divides the first target image into a plurality of first cells. The action recognition device calculates the action information corresponding to each first cell for the pixel points of each first cell in the plurality of first cells, and fuses the action information corresponding to each first cell to obtain the first target action information.

[0060] In the disclosed embodiment of the present invention, the action recognition device divides the second target image into a plurality of second cells. The action recognition device calculates the action information corresponding to each second cell for the pixel points of each second cell in the plurality of second cells, and fuses the action information corresponding to each second cell to obtain the second target action information.

[0061] In the disclosed embodiment of the present invention, the action recognition device obtains the pixel value of each pixel on the first target image, converts the pixel value of each pixel to generate action information corresponding to each pixel. The action recognition device fuses the action information corresponding to each pixel to obtain the first target action information.

[0062] In the disclosed embodiment of the present invention, the action recognition device obtains the pixel value of each pixel on the second target image, converts the pixel value of each pixel to generate action information corresponding to each pixel. The action recognition device fuses the action information corresponding to each pixel to obtain the second target action information.

[0063] S05, determining a target action in the video to be processed based on the first target action information, the second target action information and the reference action information.

[0064] In the disclosed embodiment of the present invention, the motion recognition device matches the first target motion information and the second target motion information obtained with the reference motion information to obtain matching information. If the matching information is greater than the matching threshold, it means that the target motion in the to-be-processed video corresponding to the first target motion information and the second target motion information is similar to the motion in the reference video corresponding to the reference motion information. It can be confirmed that the target motion in the to-be-processed video is the motion in the reference video.

[0065] In the disclosed embodiment of the present invention, the action recognition device will match the acquired first target action information and second target action information with the reference action information to obtain matching information. If the matching information is not greater than the matching threshold, it means that the target action in the to-be-processed video corresponding to the first target action information and the second target action information is less similar to the action in the reference video corresponding to the reference action information. It can be confirmed that the target action in the to-be-processed video is not the action in the reference video. The action recognition device will obtain the reference action information of other reference videos and repeat the above steps until the matching information obtained is greater than the matching threshold, and the target action of the to-be-processed video is confirmed.

[0066] In the disclosed embodiment of the present invention, the reference action information is action information obtained by processing a video in which the target action has been confirmed through the same method as described above.

[0067] In the disclosed embodiment of the present invention, a video to be processed is obtained; a first spatial cumulative image collection and a first temporal cumulative image collection are extracted from the video to be processed, and pixel convergence processing is performed on the first spatial cumulative image collection and the first temporal cumulative image collection respectively, so as to obtain a first target image and a second target image corresponding to the first spatial cumulative image collection and the first temporal cumulative image collection respectively; first target action information and second target action information are extracted from the first target image and the second target image respectively. Since the first target action information and the second target action information are extracted based on the first spatial cumulative image collection and the first temporal cumulative image collection respectively, the first target action information and the second target action information enhance the stability during action recognition; and when an arbitrary pixel point in the first spatial cumulative image collection or an arbitrary pixel point in the first temporal cumulative image collection moves to the boundary of a region composed of eight points of equal distance around the pixel point, the pixel point regression convergence in the first spatial cumulative image collection or the first temporal cumulative image collection can be quickly determined, so as to obtain the first target image and the second target image, thereby greatly improving the recognition efficiency of the target action.

[0068] In some embodiments, see Figure 4 , Figure 4 is an optional flow chart of the method provided in an embodiment of the present invention, Figure 1 The illustrated S02 can also be implemented through S06-S08, which will be described in conjunction with each step.

[0069] S06, obtaining the pixel value to be processed of the pixel point on the video to be processed.

[0070] In the disclosed embodiment of the present invention, the motion recognition device obtains the pixel value of each pixel point on the image to be processed as the pixel value to be processed.

[0071] In the disclosed embodiment of the present invention, the action recognition device sorts each video frame in the image to be processed in time sequence, and obtains the pixel value of each pixel point on each video frame as the pixel value to be processed.

[0072] S07, binarizing the pixel values ​​to be processed to obtain a grayscale video.

[0073] In the disclosed embodiment of the present invention, the action recognition device obtains the pixel value to be processed on the video to be processed. If the pixel value to be processed is greater than the pixel threshold, the action recognition device increases the pixel value to be processed to a first pixel extreme value; if the pixel value to be processed is not greater than the pixel threshold, the action recognition device decreases the pixel value to be processed to a second pixel extreme value, and the action recognition device obtains a grayscale video.

[0074] In the disclosed embodiment of the present invention, the grayscale video includes an object that needs to be action recognized.

[0075] In the disclosed embodiment of the present invention, the action recognition device can also perform image denoising and / or image enhancement processing on the video to be processed. After performing image denoising and / or image enhancement processing on the video to be processed, the action recognition device can perform binarization processing on the video to be processed.

[0076] S08, extracting a first spatial accumulated image collection and a first temporal accumulated image collection from the grayscale video.

[0077] In the disclosed embodiment of the present invention, the action recognition device sorts each video frame in the grayscale video in time order to obtain multiple video frames, and the action recognition device sorts the video frames including the object in the multiple video frames in time order on the spatial plane to obtain a first spatial cumulative image collection. The action recognition device sorts the video frames including the object in the multiple video frames along the time axis to obtain a first time cumulative image collection.

[0078] In the disclosed embodiment of the present invention, the motion recognition device sorts each video frame in the grayscale video in time order to obtain multiple video frames. The motion recognition device subtracts the pixel value of each pixel point on the previous video frame from the pixel value of each pixel point on the latter video frame in the multiple video frames to obtain multiple difference video frames. The motion recognition device extracts multiple object video frames including objects from the multiple difference video frames. The motion recognition device spatially sorts the multiple object video frames in time order on the spatial plane to obtain a first spatial cumulative image collection. The motion recognition device sorts the multiple object video frames along the time axis to obtain a first time cumulative image collection.

[0079] In the disclosed embodiment of the present invention, the action recognition device extracts multiple object video frames including objects from multiple video frames in the grayscale video by using the target detection method. The action recognition device spatially sorts the multiple object video frames in time order on the spatial plane to obtain a first spatial cumulative image collection. The action recognition device sorts the multiple object video frames along the time axis to obtain a first time cumulative image collection.

[0080] In the disclosed embodiment of the present invention, the motion recognition device stores the first spatially accumulated image collection and the first temporally accumulated image collection in a database of the motion recognition device.

[0081] In some embodiments, see Figure 5 , Figure 5 is an optional flow chart of the method provided in an embodiment of the present invention, Figure 4 The illustrated S08 can also be implemented through S10-S15, which will be described in conjunction with each step.

[0082] S10, sorting the video frames of the grayscale video in chronological order to obtain n video frames.

[0083] In the disclosed embodiment of the present invention, the action recognition device sorts the video frames in the grayscale video in time order to obtain n video frames.

[0084] In the disclosed embodiment of the present invention, the action recognition device sorts the key video frames in the grayscale video in chronological order to obtain n video frames, wherein the key video frame is the video frame with the largest pixel value among multiple video frames within one second in the grayscale video.

[0085] S11, subtract the pixel value of the pixel point on the previous video frame from the pixel value of the pixel point on the next video frame among the n video frames, to obtain n-1 difference video frames.

[0086] In the disclosed embodiment of the present invention, the action recognition device forms an image by subtracting the pixel value of the pixel at the corresponding position on the previous video frame from the pixel value of the pixel at each position on the next video frame in time sequence, thereby obtaining n-1 difference video frames.

[0087] In the disclosed embodiment of the present invention, the video frames include: a first video frame, a second video frame, a third video frame, a fourth video frame and a fifth video frame. The action recognition device subtracts the pixel value of the pixel at the corresponding position on the first video frame from the pixel value of the pixel at each position on the second video frame, and then forms the first difference video frame. The action recognition device subtracts the pixel value of the pixel at the corresponding position on the second video frame from the pixel value of the pixel at each position on the third video frame, and then forms the second difference video frame. The action recognition device subtracts the pixel value of the pixel at the corresponding position on the third video frame from the pixel value of the pixel at each position on the fourth video frame, and then forms the third difference video frame. The action recognition device subtracts the pixel value of the pixel at the corresponding position on the fourth video frame from the pixel value of the pixel at each position on the fifth video frame, and then forms the fourth difference video frame. Finally, the action recognition device forms four difference video frames. The number of difference video frames is 1 less than the number of video frames.

[0088] S12, obtaining n-1 object video frames including the object from the n-1 difference video frames.

[0089] In the disclosed embodiment of the present invention, the action recognition device can extract n-1 object video frames that only include the object from n-1 difference video frames through a target detection method or a convolutional neural network. The object video frame may include a portion of the difference video frame, or the object video frame may include all image content of the difference video frame.

[0090] S13, adjusting the sizes of n-1 object video frames to a predetermined size.

[0091] In the disclosed embodiment of the present invention, the motion recognition device adjusts n-1 object video frames to a predetermined size, wherein the predetermined size may be the size of a picture that can be recognized by the motion recognition device.

[0092] In the disclosed embodiment of the present invention, the motion recognition device adjusts n-1 object video frames to a predetermined size without changing the pixel values ​​of the pixel points on the multiple object video frames.

[0093] In the disclosed embodiment of the present invention, the motion recognition device adjusts the n-1 object video frames to a size of L×K, where L is the height of the n-1 object video frames and K is the width of the n-1 object video frames.

[0094] S14, spatially sorting the n-1 object video frames to form a first spatial accumulated image collection.

[0095] In the disclosed embodiment of the present invention, the action recognition device sorts n-1 object video frames on the same spatial plane in time order to obtain a first spatial cumulative image collection.

[0096] S15, sorting the n-1 object video frames along the time axis to form a first time accumulated image collection.

[0097] In the disclosed embodiment of the present invention, the action recognition device sorts n-1 object video frames along the time axis to obtain a first time accumulated image collection.

[0098] In some embodiments, see Figure 6 , Figure 6 is an optional flow chart of the method provided in an embodiment of the present invention, Figure 1 The illustrated S03 can also be implemented through S21-S26, which will be described in conjunction with each step.

[0099] S21, when the spatial pixel points on the n-1 object video frames move to a predetermined boundary, the convergence equations of the spatial pixel points on the n-1 object video frames are solved to obtain n-1 intermediate matrix information corresponding to the n-1 object video frames.

[0100] In the disclosed embodiment of the present invention, the motion recognition device uses the nine-point differential Poisson equation to perform convergence operations on multiple object video frames respectively. First, a target area is assumed in each of the n-1 object video frames, and the outline of the target area is X. When the spatial pixel point moves from any point in the target area to the outline X of the target area, it means that the spatial pixel point on the object video frame is converged in the nine-point differential Poisson equation. The motion recognition device solves the nine-point differential Poisson equation to obtain n-1 intermediate matrix information corresponding to the n-1 object video frames.

[0101] In the disclosed embodiment of the present invention, the motion recognition device uses the nine-point differential Poisson equation to perform convergence operations on multiple object video frames respectively. First, a spatial pixel point is assumed in each of the n-1 object video frames. Eight points are equidistantly distributed around the spatial pixel point, and the eight points form a target area, and the outline of the target area is X. When the spatial pixel point moves from any point in the target area to the outline X of the target area, it means that the spatial pixel point on the object video frame is converged in the nine-point differential Poisson equation. The motion recognition device solves the nine-point differential Poisson equation to obtain n-1 intermediate matrix information corresponding to the n-1 object video frames.

[0102] S22, adding n-1 intermediate matrix information to obtain first matrix information.

[0103] In the disclosed embodiment of the present invention, the action recognition device fuses and adds the obtained n-1 intermediate matrix information to obtain the first matrix information. The first matrix information is the matrix information corresponding to the first spatial accumulated image collection. The first matrix information reflects the pixel point information of the target image on the n-1 object video frames in the first spatial accumulated image collection.

[0104] S23, converting the first matrix information into a first target image corresponding to the first spatial accumulated image collection.

[0105] In the disclosed embodiment of the present invention, the motion recognition device converts the first matrix information into an image, namely, a first target image, through a cross-platform computer program. The first target image includes multiple moving targets on the first object video frame, for example, a human arm.

[0106] In the disclosed embodiment of the present invention, the first target image may be a black and white image.

[0107] S24, adding pixel values ​​of pixel points on n-1 object video frames to obtain an intermediate video frame.

[0108] In the disclosed embodiment of the present invention, the action recognition device adds pixel values ​​of pixels at corresponding positions on n-1 object video frames to obtain an intermediate video frame. The intermediate video frame reflects pixel information of the target image on the n-1 object video frames in the first time accumulated image collection.

[0109] In the disclosed embodiment of the present invention, the intermediate video frame may be a black and white image.

[0110] S25, when a temporal pixel point on an intermediate video frame moves to a predetermined boundary, a convergence equation of a temporal pixel point on an intermediate video frame is solved to obtain second matrix information.

[0111] In the disclosed embodiment of the present invention, the motion recognition device uses the nine-point differential Poisson equation to operate on the intermediate video frame. First, a predetermined target area is assumed in the intermediate video frame, and the outline of the target area is Y. When the time pixel point moves from any point in the target area to the outline Y of the target area, it means that the time pixel point on the intermediate video frame is convergent in the nine-point differential Poisson equation. The motion recognition device solves the nine-point differential Poisson equation to obtain the second matrix information corresponding to the intermediate video frame.

[0112] In the disclosed embodiment of the present invention, the motion recognition device uses the nine-point differential Poisson equation to operate on the intermediate video frame. First, a time pixel point is assumed in the intermediate video frame. Eight points are equidistantly distributed around the time pixel point, and the eight points form a target area, and the outline of the target area is Y. When the time pixel point moves from any point in the target area to the outline Y of the target area, it means that the time pixel point on the intermediate video frame is convergent in the nine-point differential Poisson equation. The motion recognition device solves the nine-point differential Poisson equation to obtain the second matrix information corresponding to the intermediate video frame.

[0113] S26, converting the second matrix information into a second target image corresponding to the first time accumulated image collection.

[0114] In the disclosed embodiment of the present invention, the motion recognition device converts the second matrix information into an image, namely, the second target image, through a cross-platform computer program.

[0115] In some embodiments, see Figure 7 , Figure 7 is an optional flow chart of the method provided in an embodiment of the present invention, Figure 1 The illustrated S04 can also be implemented through S27-S30, which will be described in conjunction with each step.

[0116] S27, dividing the first target image into a plurality of first cells, and dividing the second target image into a plurality of second cells.

[0117] In the disclosed embodiment of the present invention, the motion recognition device divides the first target image into a plurality of first cells, and the motion recognition device divides the second target image into a plurality of second cells. The size of each first cell is the same as the size of each second cell.

[0118] In the disclosed embodiment of the present invention, the action recognition device can also divide the first target image into multiple first cells according to the number of pixels of the first target image. The sizes of the multiple first cells divided by the action recognition device are large and small, with larger first cells having more pixels and smaller first cells having fewer pixels. The action recognition device can also divide the second target image into multiple second cells according to the number of pixels of the second target image of the second target unit. The sizes of the multiple second cells divided by the action recognition device are large and small, with larger second cells having more pixels and smaller second cells having fewer pixels.

[0119] S28, dividing the pixel values ​​of the pixel points of the first target image into a plurality of first pixel intervals, and dividing the pixel values ​​of the pixel points of the second target image into a plurality of second pixel intervals.

[0120] In the disclosed embodiment of the present invention, the motion recognition device divides the pixel values ​​of the pixels on the first target image into a plurality of first pixel intervals of equal value. Usually, the range of the pixel values ​​of the pixels is fixed, and the motion recognition device can also pre-set the first pixel interval. In the disclosed embodiment of the present invention, the motion recognition device divides the pixel values ​​of the pixels on the second target image into a plurality of second pixel intervals of equal value. Usually, the range of the pixel values ​​of the pixels is fixed, and the motion recognition device can also pre-set the second pixel interval.

[0121] In the disclosed embodiment of the present invention, the action recognition device obtains the pixel value of the pixel point on the first target image in the range of 0-255. The action recognition device divides the range of pixel value 0-255 into five first pixel intervals of 0-50, 51-100, 101-150, 151-200 and 201-255. The division of the second pixel interval is the same as the division principle of the first pixel interval, which will not be repeated here.

[0122] S29, obtaining first target action information corresponding to the first target image based on the pixel values ​​of the pixel points on the plurality of first cells and the first pixel intervals corresponding to the plurality of first cells.

[0123] In the disclosed embodiment of the present invention, the action recognition device calculates the histogram corresponding to each first cell according to the first pixel interval in which the pixel value of the pixel point in each first cell is located, and then calculates the vector of each histogram according to the pixel value of the pixel point on each first cell. The vectors of each histogram are fused to obtain the first target action information corresponding to the first target image.

[0124] S30, obtaining second target action information corresponding to the second target image based on the pixel values ​​of the pixel points on the plurality of second cells and the second pixel intervals corresponding to the plurality of second cells.

[0125] In the disclosed embodiment of the present invention, the action recognition device calculates the histogram corresponding to each second cell according to the second pixel interval in which the pixel values ​​of the pixel points in each second cell are located, and then calculates the vector of each histogram according to the pixel values ​​of the pixel points on each second cell. The vectors of each histogram are fused to obtain the second target action information corresponding to the second target image.

[0126] In some embodiments, see Figure 8 , Figure 8 is an optional flow chart of the method provided in an embodiment of the present invention, Figure 7 The illustrated S29-S30 can also be implemented through S31-S34, which will be described in conjunction with each step.

[0127] S31 , determining a histogram vector of each of the plurality of first cells based on pixel values ​​of pixel points on the plurality of first cells and first pixel intervals corresponding to the plurality of first cells.

[0128] In the disclosed embodiment of the present invention, the action recognition device first obtains the pixel values ​​of the pixel points on the plurality of first cells. The action recognition device obtains the histogram on each first cell by calculating the histogram of oriented gradients (HOG) according to the pixel values ​​of the pixel points on the plurality of first cells. The action recognition device calculates the histogram vector of the histogram corresponding to each first cell.

[0129] S32, combining the histogram vectors of each first cell to obtain first target action information corresponding to the first target image.

[0130] In the disclosed embodiment of the present invention, the action recognition device normalizes the histogram vector corresponding to each first cell to obtain the histogram vectors corresponding to multiple first cells. The histogram vectors corresponding to the multiple first cells are the first target action information corresponding to the first target image.

[0131] S33, determining a histogram vector of each second cell in the plurality of second cells based on the pixel values ​​of the pixel points on the plurality of second cells and the second pixel intervals corresponding to the plurality of second cells.

[0132] In the disclosed embodiment of the present invention, the action recognition device first obtains the pixel values ​​of the pixel points on the plurality of second cells. The action recognition device obtains the histogram on each second cell by calculating the histogram of oriented gradients (HOG) according to the pixel values ​​of the pixel points on the plurality of second cells. The action recognition device calculates the histogram vector of the histogram corresponding to each second cell.

[0133] S34, combining the histogram vectors of each second cell to obtain second target action information corresponding to the second target image.

[0134] In the disclosed embodiment of the present invention, the action recognition device normalizes the histogram vector corresponding to each second cell to obtain the histogram vectors corresponding to multiple second cells. The histogram vectors corresponding to the multiple second cells are the second target action information corresponding to the second target image.

[0135] In some embodiments, see Fig. 9 , Fig. 9 is an optional flow chart of the method provided in an embodiment of the present invention, Figure 1The step S05 shown may also include S35 , which will be described in conjunction with each step.

[0136] S35, obtaining first reference action information and second reference action information corresponding to the reference video in the reference video.

[0137] In the disclosed embodiment of the present invention, the action recognition device processes the reference video in which the target action has been confirmed through the above steps to obtain two histogram feature vectors corresponding to the first reference image and the second reference image of the reference video. The histogram feature vector corresponding to the first reference image is the first reference action information. The histogram feature vector corresponding to the second reference image is the second reference action information.

[0138] In some embodiments, see Fig.10 , Fig.10 is an optional flow chart of the method provided in an embodiment of the present invention, Fig. 9 The illustrated S35 can also be implemented through S36-S38, which will be described in conjunction with each step.

[0139] S36, extracting a second spatially accumulated image collection and a second temporally accumulated image collection from the reference video.

[0140] In the disclosed embodiment of the present invention, the action recognition device extracts a second spatial cumulative image collection and a second temporal cumulative image collection from a reference video. The second spatial cumulative image collection includes: cumulative images of objects in the reference video on a spatial plane. The second temporal cumulative image collection includes: cumulative images of objects in the reference video along a temporal axis. The action recognition device sorts each video frame of the reference video in time order to obtain video frames including objects. The action recognition device sorts the video frames including objects on the same spatial plane to obtain a second spatial cumulative image collection. The action recognition device sorts each video frame of the reference video in time order to obtain video frames including objects. The action recognition device sorts the video frames including objects along the time axis to obtain a second temporal cumulative image collection.

[0141] S37, performing pixel convergence processing on the second spatial accumulation image collection and the second temporal accumulation image collection respectively to obtain a first reference image and a second reference image corresponding to the second spatial accumulation image collection and the second temporal accumulation image collection respectively.

[0142] In the disclosed embodiment of the present invention, the action recognition device respectively operates the multiple images in the second spatial cumulative image collection and the multiple images in the second temporal cumulative image collection through the nine-point differential Poisson equation. If the pixel points of the multiple images in the second spatial cumulative image collection and the multiple images in the second temporal cumulative image collection finally regress and converge, the action recognition device solves the nine-point differential Poisson equation. It will obtain the first reference image and the second reference image corresponding to the second spatial cumulative image collection and the second temporal cumulative image collection respectively.

[0143] In the disclosed embodiment of the present invention, the action recognition device respectively performs convergence operations on multiple images in the second spatial cumulative image and multiple images in the second temporal cumulative image collection through the five-point differential Poisson equation. If the pixel points of the multiple images in the second spatial cumulative image collection and the multiple images in the second temporal cumulative image collection finally regress and converge, the action recognition device solves the five-point differential Poisson equation. It will obtain the first reference image and the second reference image corresponding to the second spatial cumulative image collection and the second temporal cumulative image collection respectively.

[0144] S38, extracting first reference action information and second reference action information from the first reference image and the second reference image respectively.

[0145] In the disclosed embodiment of the present invention, the action recognition device can obtain first reference action information and second reference action information corresponding to the first reference image and the second reference image respectively through classification and regression processing based on the size of pixel values ​​of several pixel points on the first reference image and the second reference image, and the position corresponding to each pixel point.

[0146] In the disclosed embodiment of the present invention, the action recognition device divides the first reference image into a plurality of third cells. The action recognition device calculates the action information corresponding to each third cell for the pixel points of each third cell in the plurality of third cells, and fuses the action information corresponding to each third cell to obtain the first reference action information.

[0147] In the disclosed embodiment of the present invention, the action recognition device divides the second reference image into a plurality of fourth cells. The action recognition device calculates the action information corresponding to each fourth cell for the pixel points of each fourth cell in the plurality of fourth cells, and fuses the action information corresponding to each third cell to obtain the second reference action information.

[0148] In some embodiments, see Fig.11 , Fig.11 is an optional flow chart of the method provided in an embodiment of the present invention, Fig.10 The shown S05 can also be implemented through S39-S41, which will be explained in combination with each step.

[0149] S39: Match the first target action information with the first reference action information to obtain first matching information.

[0150] In the disclosed embodiment of the present invention, the first target action information acquired by the action recognition device includes A×B data. A is the number of first cells on the first target image. B is the number of direction units in the first action information. Each data in the first target action information is compared with each data in the first reference action information. The matching degree between the data on the first target action information and the data on the first reference action information is calculated as the first matching information.

[0151] S40: Match the second target action information with the second reference action information to obtain second matching information.

[0152] In the disclosed embodiment of the present invention, the second target action information acquired by the action recognition device includes A×B data. A is the number of first cells on the second target image. B is the number of direction cells in the second action information. Each data in the second target action information is compared with each data in the second reference action information. The matching degree between the data on the second target action information and the data on the second reference action information is calculated as the second matching information.

[0153] S41: If the first matching information is greater than the first matching threshold and the second matching information is greater than the second matching threshold, determining that the target action in the video to be processed is the target action in the reference video.

[0154] In the disclosed embodiment of the present invention, if the action recognition device detects that the first matching information is greater than the first matching threshold and the second matching information is greater than the second matching threshold, it is determined that the target action in the video to be processed is the target action in the reference video. If the action recognition device detects that the first matching information is not greater than the first matching threshold or the second matching information is not greater than the second matching threshold, it is determined that the target action in the video to be processed is not the target action in the reference video.

[0155] In the disclosed embodiment of the present invention, the first matching threshold and the second matching threshold are a value reflecting the similarity of the target action in the video to be processed and the reference video. The greater the similarity of the target action in the video to be processed and the reference video, the greater the value of the first matching threshold and the second threshold. The first matching threshold and the second matching threshold can be values ​​such as 0.7 or 0.9.

[0156] See also Fig.12The disclosed embodiment of the present invention further provides an action recognition device 800, including: a video acquisition unit 803, a first processing unit 804, a second processing unit 805, an information extraction unit 806 and a recognition unit 807.

[0157] The video acquisition unit 803 is used to acquire the video to be processed;

[0158] A first processing unit 804 is used to extract a first spatial accumulated image collection and a first temporal accumulated image collection from the video to be processed;

[0159] The second processing unit 805 is used to perform pixel convergence processing on the first spatial accumulation image collection and the first temporal accumulation image collection respectively to obtain a first target image corresponding to the first spatial accumulation image collection and a second target image corresponding to the first temporal accumulation image collection;

[0160] An information extraction unit 806 is used to extract first target action information and second target action information from the first target image and the second target image respectively;

[0161] The identification unit 807 is used to determine the target action in the video to be processed based on the first target action information, the second target action information and the reference action information.

[0162] In the disclosed embodiment of the present invention, after acquiring the video to be processed, the action recognition device obtains the pixel values ​​to be processed of the pixel points on the video to be processed. The action recognition device performs binarization processing on the pixel values ​​to be processed to obtain a grayscale video. The action recognition device extracts the first spatial cumulative image collection and the first temporal cumulative image collection from the grayscale video through the first processing unit 804.

[0163] In the disclosed embodiment of the present invention, the action recognition device uses the first processing unit 804 to sort the video frames of the grayscale video in time order to obtain n video frames; subtract the pixel value of the pixel point on the next video frame from the pixel value of the previous video frame in the n video frames to obtain n-1 difference video frames; obtain n-1 object video frames including the object in the n-1 difference video frames; adjust the size of the n-1 object video frames to a predetermined size. The first processing unit 804 sorts the n-1 object video frames in space to form a first spatial cumulative image collection. The first processing unit 804 sorts the n-1 object video frames along the time axis to form a first time cumulative image collection.

[0164] In the disclosed embodiment of the present invention, the motion recognition device processes n-1 object video frames through the second processing unit 805. When the spatial pixel points on the n-1 object video frames move to a predetermined boundary, the convergence equation of the spatial pixel points on the n-1 object video frames is solved to obtain n-1 intermediate matrix information corresponding to the n-1 object video frames; the n-1 intermediate matrix information is added to obtain the first matrix information; the first matrix information is converted into a first target image corresponding to the first spatial cumulative image collection; the motion recognition device adds the pixel values ​​of the pixel points on the n-1 object video frames through the second processing unit 805 to obtain an intermediate video frame; when the time pixel points on an intermediate video frame move to a predetermined boundary, the convergence equation of the time pixel points on the intermediate video frame is solved to obtain the second matrix information; the second matrix information is converted into a second target image corresponding to the first time cumulative image collection.

[0165] In the disclosed embodiment of the present invention, the action recognition device divides the first target image into multiple first cells and the second target image into multiple second cells through the information extraction unit 806; divides the pixel values ​​of the pixels of the first target image into multiple first pixel intervals, and divides the pixel values ​​of the pixels on the second target image into multiple second pixel intervals; obtains first target action information corresponding to the first target image based on the pixel values ​​of the pixels on the multiple first cells and the first pixel intervals corresponding to the multiple first cells; obtains second target action information corresponding to the second target image based on the pixel values ​​of the pixels on the multiple second cells and the second pixel intervals corresponding to the multiple second cells.

[0166] In the disclosed embodiment of the present invention, the action recognition device determines the histogram vector of each of the multiple first cells based on the pixel values ​​of the pixel points on the multiple first cells and the first pixel intervals corresponding to the multiple first cells through the information extraction unit 806; the histogram vectors of each first cell are combined to obtain the first target action information corresponding to the first target image.

[0167] In the disclosed embodiment of the present invention, the action recognition device determines the histogram vector of each of the multiple second cells based on the pixel values ​​of the pixel points on the multiple second cells and the second pixel intervals corresponding to the multiple second cells through the information extraction unit 806; the histogram vectors of each second cell are combined to obtain the second target action information corresponding to the second target image.

[0168] In the disclosed embodiment of the present invention, the action recognition device extracts the second spatial cumulative image collection and the second temporal cumulative image collection from the reference video through the first processing unit 804; the action recognition device performs pixel convergence processing on the second spatial cumulative image collection and the second temporal cumulative image collection respectively through the second processing unit 805 to obtain a first reference image corresponding to the second spatial cumulative image collection and a second reference image corresponding to the second temporal cumulative image collection; the action recognition device extracts first reference action information and second reference action information from the first reference image and the second reference image respectively through the information acquisition unit 806.

[0169] In the disclosed embodiment of the present invention, the action recognition device matches the first target action information with the first reference action information through the recognition unit 807 to obtain first matching information; matches the second target action information with the second reference action information to obtain second matching information; if the first matching information is greater than the first matching threshold and the second matching information is greater than the second matching threshold, it is determined that the target action in the video to be processed is the target action in the reference video.

[0170] In the disclosed embodiment of the present invention, the action recognition device obtains a video to be processed through a video acquisition unit; extracts a first spatial cumulative image collection and a first temporal cumulative image collection from the video to be processed through a first processing unit, and performs pixel convergence processing on the first spatial cumulative image collection and the first temporal cumulative image collection respectively through a second processing unit to obtain a first target image and a second target image corresponding to the first spatial cumulative image collection and the first temporal cumulative image collection respectively; extracts first target action information and second target action information from the first target image and the second target image respectively through an information extraction unit. Since the first target action information and the second target action information are extracted based on the first spatial cumulative image collection and the first temporal cumulative image collection respectively, the first target action information and the second target action information enhance the stability during action recognition; and when an arbitrary pixel point in the first spatial cumulative image collection or an arbitrary pixel point in the first temporal cumulative image collection moves to the boundary of a region composed of eight points of equal distance around the pixel point, the pixel point regression convergence in the first spatial cumulative image collection or the first temporal cumulative image collection can be quickly determined, thereby obtaining the first target image and the second target image, thereby greatly improving the recognition efficiency of the target action.

[0171] It should be noted that, in the embodiment of the present invention, if the above-mentioned action recognition method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present invention can be embodied in the form of a software product in essence or in other words, the part that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable the action recognition device 800 (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk.

[0172] Correspondingly, an embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method are implemented.

[0173] Correspondingly, an embodiment of the present invention provides a motion recognition device 800, including a memory 802 and a processor 801, wherein the memory 802 stores a computer program that can be executed on the processor 801, and the processor 801 implements the steps in the above method when executing the program.

[0174] It should be noted that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of the present invention, please refer to the description of the method embodiments of the present invention for understanding.

[0175] It should be noted that Fig.13 A schematic diagram of a hardware entity of a motion recognition device provided by an embodiment of the present invention, such as Fig.13 As shown, the hardware entity of the motion recognition device 800 includes: a processor 801 and a memory 802, wherein:

[0176] The processor 801 generally controls the overall operation of the motion recognition device 800 .

[0177] The memory 802 is configured to store instructions and applications executable by the processor 801, and can also cache data to be processed or processed by the processor 801 and each module in the motion recognition device 800 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0178] The above is only an embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for motion recognition, characterized in that: include: Get the video to be processed; Sorting the video frames including the object in the video to be processed on the same spatial plane to determine a first spatial cumulative image collection, and sorting the video frames including the object in the video to be processed in time order to determine a first time cumulative image collection; Performing a final regression convergence operation on pixel points of multiple images in the first spatial cumulative image collection through the nine-point differential Poisson equation to determine the corresponding first target image, and performing a final regression convergence operation on pixel points of multiple images in the first temporal cumulative image collection through the nine-point differential Poisson equation to determine the corresponding second target image; extracting first target action information and second target action information from the first target image and the second target image respectively; Based on the first target action information, the second target action information and the reference action information, a target action in the video to be processed is determined.

2. The action recognition method according to claim 1, characterized in that: After obtaining the video to be processed, the method further includes: Obtaining a pixel value to be processed of a pixel point on the video to be processed; Binarizing the pixel values ​​to be processed to obtain a grayscale video; Correspondingly, the video frames including the object in the video to be processed are sorted on the same spatial plane to determine a first spatial cumulative image collection, and the video frames including the object in the video to be processed are sorted in time order to determine a first time cumulative image collection, including: The first spatially accumulated image collection and the first temporally accumulated image collection are extracted from the grayscale video.

3. The action recognition method according to claim 2, characterized in that: The extracting the first spatial accumulated image collection and the first temporal accumulated image collection from the grayscale video includes: Sorting the video frames of the grayscale video in chronological order to obtain n video frames; Subtract the pixel value of a pixel point on a previous video frame from the pixel value of a pixel point on a subsequent video frame among the n video frames to obtain n-1 difference video frames; Acquire n-1 object video frames including the object from the n-1 difference video frames; spatially sorting the n-1 object video frames to form the first spatial accumulated image collection; Sort the n-1 object video frames along the time axis to form the first time-accumulated image collection; Accordingly, before spatially sorting the n-1 object video frames to form the first spatial accumulated image collection, the method further includes: The sizes of the n-1 object video frames are adjusted to a predetermined size.

4. The action recognition method according to claim 3, characterized in that: The method further comprises: performing a final regression convergence operation on pixel points of a plurality of images in the first spatial cumulative image collection by using a nine-point differential Poisson equation to determine a corresponding first target image, and performing a final regression convergence operation on pixel points of a plurality of images in the first temporal cumulative image collection by using a nine-point differential Poisson equation to determine a corresponding second target image, including: When the spatial pixel points on the n-1 object video frames move to a predetermined boundary, solving the convergence equations of the spatial pixel points on the n-1 object video frames to obtain n-1 intermediate matrix information corresponding to the n-1 object video frames; Adding the n-1 intermediate matrix information to obtain first matrix information; Converting the first matrix information into a first target image corresponding to the first spatial accumulated image collection; Adding pixel values ​​of pixels on the n-1 object video frames to obtain an intermediate video frame; When a temporal pixel point on the one intermediate video frame moves to a predetermined boundary, solving a convergence equation of the temporal pixel point on the one intermediate video frame to obtain second matrix information; The second matrix information is converted into a second target image corresponding to the first time-accumulated image collection.

5. The action recognition method according to claim 4, characterized in that: The extracting the first target action information and the second target action information from the first target image and the second target image respectively comprises: Dividing the first target image into a plurality of first cells, and dividing the second target image into a plurality of second cells; Dividing the pixel values ​​of the pixels of the first target image into a plurality of first pixel intervals, and dividing the pixel values ​​of the pixels of the second target image into a plurality of second pixel intervals; Obtaining first target action information corresponding to the first target image based on pixel values ​​of the pixel points on the plurality of first cells and the first pixel intervals corresponding to the plurality of first cells; Based on the pixel values ​​of the pixel points on the plurality of second cells and the second pixel intervals corresponding to the plurality of second cells, the second target action information corresponding to the second target image is obtained.

6. The action recognition method according to claim 5, characterized in that: The obtaining the first target action information corresponding to the first target image based on the pixel values ​​of the pixel points on the plurality of first cells and the first pixel intervals corresponding to the plurality of first cells includes: Determine a histogram vector of each of the plurality of first cells based on pixel values ​​of the pixel points on the plurality of first cells and the first pixel intervals corresponding to the plurality of first cells; The histogram vectors of each of the first cells are combined to obtain the first target action information corresponding to the first target image.

7. The action recognition method according to claim 6, characterized in that: The obtaining the second target action information corresponding to the second target image based on the pixel values ​​of the pixel points on the plurality of second cells and the second pixel intervals corresponding to the plurality of second cells includes: Determine a histogram vector of each second cell in the plurality of second cells based on the pixel values ​​of the pixel points on the plurality of second cells and the second pixel intervals corresponding to the plurality of second cells; The histogram vectors of each second cell are combined to obtain the second target action information corresponding to the second target image.

8. The action recognition method according to any one of claims 5 to 7, characterized in that: Before determining the target action in the video to be processed based on the first target action information, the second target action information and the reference action information, the method further includes: Acquiring first reference action information and second reference action information corresponding to the reference video in the reference video; The reference video is a video in which the target action has been confirmed; Correspondingly, the acquiring the first reference action information and the second reference action information corresponding to the reference video in the reference video includes: Extracting a second spatially accumulated image collection and a second temporally accumulated image collection from the reference video; Performing pixel convergence processing on the second spatial accumulation image collection and the second temporal accumulation image collection respectively to obtain a first reference image corresponding to the second spatial accumulation image collection and a second reference image corresponding to the second temporal accumulation image collection; extracting first reference action information and second reference action information from the first reference image and the second reference image respectively; Correspondingly, determining the target action in the image to be processed based on the first target action information, the second target action information and the reference action information includes: matching the first target action information with the first reference action information to obtain first matching information; Matching the second target action information with the second reference action information to obtain second matching information; If the first matching information is greater than a first matching threshold and the second matching information is greater than a second matching threshold, it is determined that the target action in the to-be-processed video is the target action in the reference video.

9. A motion recognition device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the steps in the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 8 are implemented.

Images