Motion detection method, apparatus, electronic device and storage medium

Abstract

A motion detection method, apparatus, electronic device and storage medium, the motion detection method including: acquiring a video stream of a scene area (S101); detecting a motion direction of a target object in the scene area based on a plurality of image frames in the video stream (S102); detecting skeletal key points of the target object in the plurality of image frames (S103); and identifying motion information of the target object based on a geometric relationship between target key points corresponding to the motion direction in the detected skeletal key points (S104).

Description

[Technical field]

[0001] <CROSS REFERENCE TO RELATED APPLICATIONS> This disclosure claims priority to a Chinese patent application bearing application number 202210346615.4, filed with the China National Intellectual Property Office on March 31, 2022, the entire contents of which are incorporated herein by reference.

[0002] The present disclosure relates to the technical field of image detection, and in particular to a motion detection method, apparatus, electronic device and storage medium. [Background technology]

[0003] With the continuous development of artificial intelligence technology, the types of image and video detection are becoming more and more numerous, and the effects are becoming more and more excellent. In particular, detection technology can be applied to the field of safety protection to help users avoid danger. Taking the scene of a vehicle cabin as an example, image processing technology can be used to detect the vehicle cabin environment and human safety. By collecting images and videos of the vehicle interior during the driving process or parking state, it can be detected whether the person in the vehicle is performing dangerous actions, thereby improving driving and riding safety. However, in the conventional technology, the detection of human actions is prone to erroneous judgment, resulting in a poor user experience. Summary of the Invention [Means for solving the problem]

[0004] The present disclosure provides a motion detection method, apparatus, device and storage medium to address the deficiencies of the prior art.

[0005] According to a first aspect of the present disclosure, there is provided a motion detection method, the motion detection method comprising: Obtaining a video stream of a scene region; detecting a motion direction of a target object within the scene region based on a plurality of image frames in the video stream; detecting skeletal keypoints of the target object in the plurality of image frames; determining motion information of the target object based on a geometric relationship between object keypoints corresponding to the motion directions in the detected skeletal keypoints.

[0006] In one embodiment, detecting a direction of motion of a target object within the scene region based on a plurality of image frames in the video stream comprises: extracting features of the target object in each image frame of the plurality of image frames; determining optical flow information for the target object in the video stream based on features of the target object in each image frame of the plurality of image frames; Determining a direction of motion of the target object within the scene region based on optical flow information of the target object in the video stream.

[0007] In one embodiment, each of the movement directions corresponds to a movement of an object; Determining motion information of the target object based on a geometric relationship between object key points corresponding to the motion directions of the detected skeletal key points includes: If a geometric relationship between target key points corresponding to the movement direction in the detected skeletal key points satisfies a first preset condition corresponding to the movement direction, the target object is identified as having a target movement corresponding to the movement direction; and identifying the target object as having no target motion corresponding to the motion direction in the target object if a geometric relationship between target key points corresponding to the motion direction in the detected skeletal key points does not satisfy a first preset condition corresponding to the motion direction.

[0008] In one embodiment, the skeletal key points include a left shoulder key point, a right shoulder key point, a left wrist key point, a right wrist key point, a left elbow key point, a right elbow key point, a left ear key point, and a right ear key point; Determining the motion information of the target object based on the geometric relationship between object key points corresponding to the motion directions of the detected skeleton key points includes at least one of the following: If the movement direction is left, the tangent value of the angle between a first target vector from a right shoulder key point to a left shoulder key point and a standard vector in a horizontal rightward direction is a positive number and the absolute value of the tangent value is greater than a first threshold value, the target object is determined to have a leftward body tilt movement; If the movement direction is right, the tangent value of the angle between a first target vector from a right shoulder key point to a left shoulder key point and a standard vector in a horizontal rightward direction is negative and the absolute value of the tangent value is greater than a first threshold value, the target object is determined to have a rightward body tilt movement; If the movement direction is up, the target object is identified as holding his / her chest with his / her hand according to the following first or second situation, where the first situation includes: an angle between a second target vector, which is a vector from a left elbow keypoint to a left wrist keypoint, and a third target vector, which is a vector from the left elbow keypoint to a left shoulder keypoint, is greater than a second threshold value; the left wrist keypoint is lower than a right shoulder keypoint, and the vertical distance between the left wrist keypoint and the right shoulder keypoint is greater than a third threshold value; the second situation includes: an angle between a fourth target vector, which is a vector from a right elbow keypoint to a right wrist keypoint, and a fifth target vector, which is a vector from the right elbow keypoint to the right shoulder keypoint, is greater than the second threshold value; the right wrist keypoint is lower than a left shoulder keypoint, and the vertical distance between the right wrist keypoint and the left shoulder keypoint is greater than a third threshold value; or If the movement direction is downward, the target object is identified as having a forward leaning prone movement if the left ear key point is lower than the left shoulder key point and the vertical distance between the left ear key point and the left shoulder key point is greater than a fourth threshold, and / or if the right ear key point is lower than the right shoulder key point and the vertical distance between the right ear key point and the right shoulder key point is greater than a fourth threshold.

[0009] In one embodiment, detecting a direction of motion of a target object within the scene region based on a plurality of image frames in the video stream comprises: detecting whether a target object in the video stream is in motion; If a target object in the video stream is moving, detecting a direction of movement of the target object in the scene region based on a plurality of image frames in the video stream.

[0010] In one embodiment, determining motion information of the target object based on a geometric relationship between object keypoints corresponding to the motion directions of the detected skeletal keypoints includes: For each image frame in the plurality of image frames in the video stream, determining motion information of the target object in the image frame based on a geometric relationship between object key-points corresponding to the motion directions of the skeletal key-points detected in the image frame; smoothing motion information of the target object in the plurality of image frames to obtain motion information of the target object identified based on the plurality of image frames.

[0011] In one embodiment, the motion detection method further comprises: In the process of acquiring the video stream of the scene area, a preset number of the most recent image frames in the video stream are cached in real time; Smoothing the motion information of the target object in the plurality of image frames to obtain the motion information of the target object identified based on the plurality of image frames, The method includes smoothing motion information of the target object in the cached predetermined number of the image frames to obtain motion information of the target object identified based on the cached predetermined number of the image frames.

[0012] In one embodiment, caching the pre-defined number of most recent image frames in real time comprises: Detecting whether predetermined key information, including at least one of face, at least a part of body, and skeletal key points of a target object, is present in each image frame in the video stream; The method includes caching a preset number of the latest image frames in real time among the image frames in which preset key information of the target object in the video stream exists.

[0013] In one embodiment, the scene area includes a vehicle cabin scene area; Obtaining a video stream of the scene region includes: The method includes capturing a video stream of the scene area when the vehicle doors are locked and / or the vehicle speed reaches a preset speed threshold.

[0014] In one embodiment, further comprising: detecting a plurality of objects in the video stream; Identifying the target object from the plurality of objects based on a position within the vehicle cabin of each of the plurality of objects and / or facial information of each of the plurality of objects.

[0015] In one embodiment, further comprising: and sending alarm information to a service platform when the target object motion detection result is that a target motion exists in the target object.

[0016] According to a second aspect of the embodiment of the present disclosure, there is provided a motion detection device, the motion detection device comprising: an acquisition module for acquiring a video stream of a scene region; a direction module for detecting a direction of motion of a target object within the scene region based on a plurality of image frames in the video stream; a detection module for detecting skeletal key points of the target object in the plurality of image frames; and an identifying module for identifying motion information of the target object based on a geometric relationship between object key points corresponding to the motion directions in the detected skeletal key points.

[0017] In one embodiment, the direction module specifically: extracting features of the target object in each image frame of the plurality of image frames; determining optical flow information for the target object in the video stream based on features of the target object in each image frame of the plurality of image frames; The target object is to determine a direction of motion of the target object within the scene region based on optical flow information in the video stream.

[0018] In one embodiment, each of the movement directions corresponds to a movement of an object; Specifically, the specific module is Determining motion information of the target object based on a geometric relationship between object key points corresponding to the motion directions of the detected skeletal key points includes: If a geometric relationship between target key points corresponding to the movement direction in the detected skeletal key points satisfies a first preset condition corresponding to the movement direction, the target object is identified as having a target movement corresponding to the movement direction; If the geometric relationship between the target key points corresponding to the movement direction in the detected skeletal key points does not satisfy a first preset condition corresponding to the movement direction, it is determined that the target object does not have a target movement corresponding to the movement direction.

[0019] In one embodiment, the skeletal key points include a left shoulder key point, a right shoulder key point, a left wrist key point, a right wrist key point, a left elbow key point, a right elbow key point, a left ear key point, and a right ear key point; The specific module is specifically used in at least one of the following: If the movement direction is left, the tangent value of the angle between a first target vector from a right shoulder key point to a left shoulder key point and a standard vector in a horizontal rightward direction is a positive number and the absolute value of the tangent value is greater than a first threshold value, the target object is determined to have a leftward body tilt movement; If the movement direction is right, the tangent value of the angle between a first target vector from a right shoulder key point to a left shoulder key point and a standard vector in a horizontal rightward direction is negative and the absolute value of the tangent value is greater than a first threshold value, the target object is determined to have a rightward body tilt movement; If the movement direction is up, the target object is identified as holding his / her chest with his / her hand according to the following first or second situation, where the first situation includes: an angle between a second target vector, which is a vector from a left elbow keypoint to a left wrist keypoint, and a third target vector, which is a vector from the left elbow keypoint to a left shoulder keypoint, is greater than a second threshold value; the left wrist keypoint is lower than a right shoulder keypoint, and the vertical distance between the left wrist keypoint and the right shoulder keypoint is greater than a third threshold value; the second situation includes: an angle between a fourth target vector, which is a vector from a right elbow keypoint to a right wrist keypoint, and a fifth target vector, which is a vector from the right elbow keypoint to the right shoulder keypoint, is greater than the second threshold value; the right wrist keypoint is lower than a left shoulder keypoint, and the vertical distance between the right wrist keypoint and the left shoulder keypoint is greater than a third threshold value; or If the movement direction is downward, the target object is identified as having a forward leaning prone movement if the left ear key point is lower than the left shoulder key point and the vertical distance between the left ear key point and the left shoulder key point is greater than a fourth threshold, and / or if the right ear key point is lower than the right shoulder key point and the vertical distance between the right ear key point and the right shoulder key point is greater than a fourth threshold.

[0020] In one embodiment, the direction module specifically: Detecting whether a target object in the video stream is in motion; and to detect, if a target object in the video stream is moving, a motion direction of the target object in the scene region based on a plurality of image frames in the video stream.

[0021] In one embodiment, the specific module specifically includes: for each image frame in the plurality of image frames in the video stream, determining motion information of the target object in the image frame based on a geometric relationship between object key-points corresponding to the motion directions of the skeletal key-points detected in the image frame; The motion information of the target object in the plurality of image frames is smoothed to obtain motion information of the target object identified based on the plurality of image frames.

[0022] In one embodiment, the cache module further comprises: During the process of acquiring the video stream of the scene area, caching a preset number of the latest image frames in the video stream in real time; When the identification module smoothes the motion information of the target object in the plurality of image frames to obtain the motion information of the target object identified based on the plurality of image frames, specifically, The present invention is intended to smooth motion information of the target object in a predetermined number of the cached image frames, and to obtain motion information of the target object identified based on the predetermined number of the cached image frames.

[0023] In one embodiment, the cache module specifically: Detecting whether predetermined key information, including at least one of face, at least a part of body, and skeletal key points, of a target object is present in each image frame in the video stream; The present invention is for caching a preset number of the latest image frames in real time among the image frames in which preset key information of the target object in the video stream exists.

[0024] In one embodiment, the scene area includes a vehicle cabin scene area; The acquisition module specifically includes: and to obtain a video stream of the scene area when the vehicle doors are locked and / or the vehicle speed reaches a preset speed threshold.

[0025] In one embodiment, the method further includes a target module, the target module comprising: Detecting a plurality of objects in the video stream; The target object is identified from the plurality of objects based on a position of each of the plurality of objects within the vehicle cabin and / or facial information of each of the plurality of objects.

[0026] In one embodiment, the device further includes an alarm module, the alarm module comprising: and sending alarm information to a service platform when the target object motion detection result is that a target motion exists in the target object.

[0027] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, the device comprising a memory and a processor, the memory for storing computer instructions executable by the processor, and for performing a method as described in the first aspect when the processor executes the computer instructions.

[0028] According to a fourth aspect of the present disclosure, there is provided a computer-readable storage medium having a computer program stored therein, the computer program being executed by a processor to perform the method according to the first aspect.

[0029] As can be seen from the above embodiment, by obtaining a video stream of a scene area, the motion direction of a target object in the scene area can be detected based on a number of image frames in the video stream, and then the skeleton key points of the target object in the image frames can be detected, and finally the motion information of the target object can be determined based on the geometric relationship between the target key points corresponding to the motion direction in the skeleton key points. Because the geometric relationship between the target key points is objective and accurate, it can accurately detect whether the target object has a dangerous motion, and further, before determining the motion information, the detection result of the motion direction is introduced to analyze the geometric relationship between the target key points, thereby reducing the spatio-temporal complexity of motion detection, making the motion detection directional, and further improving the detection accuracy. When the detection method is applied in a vehicle, it can accurately detect whether a danger occurs to the driver and passengers in the vehicle, thereby improving the safety of the ride and improving the user's experience.

[0030] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

[0031] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure. [Brief description of the drawings]

[0032] [Figure 1] 1 shows a flowchart of a motion detection method according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 shows a schematic diagram of the key points of the framework of one embodiment of the present disclosure. [Diagram 3] 1 shows a complete flowchart of a method for detecting motion in a scene of vehicle travel according to one embodiment of the present disclosure. [Figure 4] 1 shows a schematic diagram of a configuration of a motion detection device according to an embodiment of the present disclosure. [Diagram 5] 1 shows a schematic diagram of a configuration of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0033] Exemplary embodiments will now be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, like numerals in different drawings represent like or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as described in the appended claims.

[0034] The terms used in this disclosure are for the purpose of describing particular embodiments only and are not intended to limit the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "the," and "said" are intended to include the plural forms unless the context clearly indicates otherwise. It is also to be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0035] Terms such as first, second, third, etc. may be used in this disclosure to describe various information, but it will be understood that the information should not be limited to these terms. These terms are used only to distinguish the same type of information from each other. For example, the first information can be called the second information, and similarly, the second information can be called the first information, without departing from the scope of this disclosure. Depending on the context, the word "when" used in this specification can be interpreted as "at the time" or "when" or "depending on a particular".

[0036] As a first aspect, at least one embodiment of the present disclosure provides a motion detection method, and referring to FIG. 1, FIG. 1 shows a flow of the method, including steps S101 to S103.

[0037] The method can also be used to perform motion detection on a target object in a video stream of a scene region. For example, it is detected whether a target object in a video stream has a target motion, and the target motion may be a dangerous motion, that is, the method can be used to detect whether a target object in a video stream of a scene region has a dangerous motion. The method can be applied to a scene such as a vehicle driving, and can detect whether a driver or a passenger has a dangerous motion in the scene. The dangerous motion may be predefined, and for example, the dangerous motion may be a left lean of the body, a right lean of the body, a chest covering with hands, or a forward leaning face down.

[0038] The video stream is a video recorded by an image collecting device for a scene area. For example, in a scene of a vehicle driving, the scene area may be a scene area of ​​a vehicle cabin. The video stream of the scene area may be a video collected by a surveillance camera installed in the vehicle cabin, and the video stream may be a video collected of a driver in the vehicle cabin, or a video collected of a passenger in the vehicle cabin.

[0039] The method may also be performed by an electronic device such as a terminal device or a server, and the terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a mobile phone, a cordless phone, a personal digital assistant (PDA) handheld device, a computing device, an in-vehicle device, a wearable device, etc., and the method may be realized in a manner in which a processor calls a computer-readable instruction stored in a memory. Or, the method may be performed by a server, and the server may be a local server, a cloud server, etc. In a vehicle driving scene, the method may be performed by an artificial intelligence emergency call system, and the system is connected to a surveillance camera in the vehicle cabin, and thus can obtain a video stream of a scene area in the vehicle cabin collected by the surveillance camera.

[0040] In step S101, a video stream of a scene area is obtained.

[0041] Also, the video stream of the scene area may be a video recorded by an image collecting device, and the image collecting device may be an electronic device having an image collecting function such as a mobile phone, a camera, a security camera, etc. The video stream of the scene area has a target object, the target object being a person whose action needs to be detected in the video stream, and the area of ​​the video stream other than the target object may be a background area and other objects (other objects may not exist), for example, the target object in a scene of a vehicle running may be a driver or a specific passenger. Therefore, after obtaining the video stream of the scene area, a plurality of objects in the video stream are detected, and the target object is identified from the plurality of objects based on the position of each object in the vehicle cabin and / or the face information of each object. Exemplarily, an object in the driver's seat in the vehicle cabin may be identified as the target object, i.e., the driver may be identified as the target object, or an object whose face information matches the features of a reference face registered in advance may be identified as the target object, i.e., a specific person such as the owner of the vehicle, a registered driver, etc. may be identified as the target object. In the following steps, the operation of the target object may be performed based on the target object identified in this step.

[0042] As can be understood, the target object in the video stream may be one or more. When there are multiple target objects in the video stream, each target object can be processed according to the method provided by the embodiment, or each target object can be processed simultaneously according to the method provided by the embodiment. For example, in a video of a scene area of ​​a vehicle cabin, the target object can include one or more of multiple target objects such as a driver, a passenger in a front seat, and a passenger in a rear seat.

[0043] In one possible embodiment, when the scene area is a vehicle cabin scene area, the activation condition of this step can be preset, for example, the activation condition can be that the vehicle door is locked and / or the vehicle speed reaches a preset speed threshold, etc., and when the vehicle meets the above activation condition, i.e., the vehicle door is locked and / or the vehicle speed reaches a preset speed threshold, the video stream of the scene area can be acquired. In this way, the video stream can be acquired and detected in the scene where the vehicle has the need for danger detection, so that the detection method has a directionality, saves computation power, memory, and reduces energy consumption.

[0044] In step S102, a motion direction of a target object in the scene region is detected based on a plurality of image frames in the video stream.

[0045] In addition, the plurality of image frames in the video stream may be a preset number of image frames, for example, two frames, four frames, five frames, etc. Exemplarily, in the process of acquiring a video stream of a scene area, a preset number of latest image frames in the video stream are cached in real time. For example, each frame image frame is acquired from the video stream and is cached, or an image frame is extracted from the video stream at a certain interval and is cached, or an image frame is extracted from the video stream under a certain cache condition and is cached, and for example, it is possible to detect whether preset key information of a target object exists in each image frame in the video stream, and the preset key information includes at least one of a face, at least a part of a body, and a skeleton key point, and the part of a body may be a part such as a left shoulder, a right shoulder, a left ear, or a right ear. Then, a preset number of latest image frames in the video stream in which preset key information of the target object exists are cached in real time. Since the image frames in the cache are image frames containing preset key information, the success rate of acquiring the required skeleton key points during subsequent motion detection of the target object based on the cached image frames is improved. In the step of caching the image frames, coarse-grained selection is performed on the images in the video stream based on the key information, which is advantageous in reducing resources such as time and memory required for detecting missing key skeleton key points during the motion detection process, thereby improving the efficiency of motion detection.

[0046] Also, the motion direction of the target object may be left, right, up, down, etc. In one possible embodiment, the feature of the target object in each image frame of the multiple image frames can be extracted first, and illustratively, the feature points of the target object are extracted based on the basic principle that the brightness of the same target does not change when it moves between different frames, and then, in the multiple image frames, optical flow information of the target object in the video stream is determined based on the feature of the target object in each image frame, and the optical flow information can indicate that the target moves between different frames, and finally, the motion direction of the target object in the scene area is determined based on the optical flow information of the target object in the video stream. Exemplarily, the LucasKanade algorithm is used to determine the motion direction of the target object from being relatively still to moving.

[0047] Optionally, a trigger condition can be set for this step, so that this step is performed with directionality. For example, in the process of acquiring a video stream of a scene area, it can detect whether a target object in the video stream is moving, and if the target object in the video stream is moving, this step can be performed, that is, the motion direction of the target object in the scene area can be detected based on multiple image frames in the video stream. Thus, the effectiveness and directionality of this step can be improved.

[0048] In step S103, skeletal key points of the target object in the multiple image frames are detected.

[0049] The image frames can be processed using a pre-trained neural network, and thus the skeletal keypoints of the target object can be obtained. The skeletal keypoints can indicate the joint parts in the skeletal structure of the human body, and the skeletal structure diagram of the human body can be drawn by these joint parts. For example, the neural network can detect the obtained skeletal keypoints and the skeletal structure diagram drawn by the skeletal keypoints, as shown in Figure 2, and as can be seen from Figure 2, the skeletal keypoints include: nose keypoint 0, left eye keypoint 1, right eye keypoint 2, left ear keypoint 3, right ear keypoint 4, left shoulder keypoint 5, right shoulder keypoint 6, left elbow keypoint 7, right elbow keypoint 8, left wrist keypoint 9, right wrist keypoint 10, left hip keypoint 11, right hip keypoint 12, left knee keypoint 13, right knee keypoint 14, left ankle keypoint 15 and right ankle keypoint 16. In addition, since the video stream is recorded by a surveillance camera facing the target object directly or at a certain angle toward the target object, the target object in the image frame and the target object in the actual scene are mirror images, i.e., the left side of the target object in the actual scene is the right side of the target object in the image frame and the right side of the target object in the actual scene is the left side of the target object in the image frame, and the origin of the coordinate system in the image frame may be in the upper left corner, or may be in the positive direction of the horizontal axis (e.g., the x-axis) along the horizontal side to the right, or in the positive direction of the vertical axis (e.g., the y-axis) along the vertical side downward.

[0050] In addition, the skeleton key points of the target object can be detected, and the skeleton key points included in the part where the target object appears in the image frame can be detected, for example, when the driver's upper body appears in the image frame, only the skeleton key points of the upper body can be detected in this step. In other words, in this step, the skeleton key points of the target object can be detected and obtained, and all the key points shown in Figure 2 can be detected and obtained, or some of the key points shown in Figure 2 can be detected.

[0051] The detected skeletal keypoints may be indicated by their coordinate locations in the image frame, and the skeletal keypoints may be marked at corresponding locations on the image frame.

[0052] In step S104, motion information of the target object is determined based on the geometric relationships between object key points corresponding to the motion directions of the skeletal key points.

[0053] In addition, the motion information of the target object may be the presence or absence of the target motion in the target object, and the target motion may be a dangerous motion that needs to be detected, and the dangerous motion that needs to be detected may be preset. Each motion direction may correspond to one or more preset target motions, and each target motion has a corresponding number of target key points, and in each target motion, the corresponding number of target key points satisfy the constraint of the corresponding geometric relationship. Therefore, each target motion may be preset with a first preset condition, and the first preset condition can be set for the geometric relationship satisfied between the target key points corresponding to the target motion. Therefore, when identifying the motion information of the target object, for each target motion in the motion direction detected in step S102, it can be determined whether the target key points corresponding to the target motion among the key points of the skeleton of the target object detected in step S103 satisfy the first preset condition corresponding to the target motion, and if it is satisfied, the target motion is present in the target object, and if it is not satisfied, the target motion is not present in the target object.

[0054] For example, each motion direction corresponds to the motion of one target, and a target key point and a first preset condition corresponding to the motion direction can be set. Then, when identifying the motion information of the target target, if the geometric relationship between the target key points corresponding to the motion direction in the detected skeletal key points satisfies the first preset condition corresponding to the motion direction, the target target is identified as having a target motion corresponding to the motion direction; otherwise (i.e., the geometric relationship between the target key points does not satisfy the corresponding first preset condition), the target target is identified as not having a target motion corresponding to the motion direction. The detection range of the target motion is reduced based on the motion direction, thereby further saving energy consumption and memory and improving detection efficiency. And the target key points are detected based on the motion direction, which provides directionality to motion detection and further improves detection accuracy.

[0055] In one possible embodiment, object movements corresponding to four movement directions, i.e., left, right, up and down, are preset as tilting the body to the left (corresponding to the direction "left"), tilting the body to the right (corresponding to the direction "right"), covering the chest with hands (corresponding to the direction "up") and leaning forward and prone (corresponding to the direction "down"), and for each object movement (i.e., each movement direction), a corresponding object key point and a first preset condition are set.

[0056] The target key points corresponding to the left lean of the body can be set to the right shoulder key point and the left shoulder key point, the vector formed by the connecting line from the right shoulder key point to the left shoulder key point is called the first target vector, the vector in the horizontal rightward direction (i.e., parallel to the horizontal side of the image to be detected and oriented rightward) is called the standard vector, and the corresponding first preset condition is set as follows: the tangent value of the angle between the target vector and the standard vector is a positive number, and the absolute value of the tangent value is greater than a first threshold (e.g., the first threshold is 0.4). Taking the right shoulder key point 6 and the left shoulder key point 5 shown in FIG. 2 as an example, the first preset condition corresponding to the left lean of the body can be expressed as tan(vec(6,5))>0.4. In other words, when the movement direction is to the left, if the tangent value of the angle between the first target vector from the right shoulder key point to the left shoulder key point and the standard vector in the horizontal rightward direction is a positive number and the absolute value of the tangent value is greater than a first threshold value, it is determined that the target object is experiencing a left body tilt movement.

[0057] The target key points corresponding to the right lean of the body can be set to the right shoulder key point and the left shoulder key point, the vector formed by the connecting line from the right shoulder key point to the left shoulder key point is called the target vector, the vector in the horizontal rightward direction (i.e., parallel to the horizontal side of the image to be detected and oriented rightward) is called the standard vector, and the corresponding first preset condition is set as follows: the tangent value of the angle between the target vector and the standard vector is a negative number, and the absolute value of the tangent value is greater than a first threshold (e.g., the first threshold is 0.4). Taking the right shoulder key point 6 and the left shoulder key point 5 shown in FIG. 2 as an example, the first preset condition corresponding to the right lean of the body can be expressed as tan(vec(6,5))<-0.4. In other words, when the movement direction is to the right, if the tangent value of the angle between the first target vector from the right shoulder key point to the left shoulder key point and the standard vector in the horizontal rightward direction is negative and the absolute value of the tangent value is greater than a first threshold value, it is determined that the target object is experiencing a rightward body tilt movement.

[0058] The target key points corresponding to the hand covering the chest can be set as a left elbow key point, a left wrist key point, a left shoulder key point, a right elbow key point, a right wrist key point, and a right shoulder key point, and a vector formed by a connecting line from the left elbow key point to the left wrist key point is called a second target vector, a vector formed by a connecting line from the left elbow key point to the left shoulder key point is called a third target vector, a vector formed by a connecting line from the right elbow key point to the right wrist key point is called a fourth target vector, and a vector formed by a connecting line from the right elbow key point to the right shoulder key point is called a fifth target vector, a cosine value of an angle between the second target vector and the third target vector is called a first cosine value, and a cosine value of an angle between the fourth target vector and the fifth target vector is called a second cosine value, and a corresponding first preset condition is set as a condition in which the first cosine value is equal to or larger than a second threshold (for example, the second threshold is 0 2), and the vertical distance between the left wrist keypoint and the right shoulder keypoint is greater than a third threshold (e.g., the third threshold is 100); and / or the second cosine value is greater than a second threshold (e.g., the second threshold is 0.2), and the vertical distance between the right wrist keypoint and the left shoulder keypoint is greater than a third threshold (e.g., the third threshold is 100). Then, the left elbow keypoint 7 and the left wrist keypoint 8 shown in FIG. Taking keypoint 9 on the left shoulder, keypoint 5 on the right elbow, keypoint 8 on the right wrist, and keypoint 6 on the right shoulder as an example, the first preset condition corresponding to the hands covering the chest can be expressed as cos(vec(7,5), vec(7,9))>0.2 and y(9)-y(6)>100, and / or the first preset condition can be expressed as cos(vec(8,6), vec(8,10))>0.2 and y(10)-y(5)>100.That is, when the movement direction is upward, the target object is identified as holding his / her chest with his / her hand according to the following first or second situation, the first situation includes that the angle between the second target vector and the third target vector is greater than a second threshold, the left wrist keypoint is lower than the right shoulder keypoint, and the vertical distance between the left wrist keypoint and the right shoulder keypoint is greater than a third threshold, the second target vector is a vector from the left elbow keypoint to the left wrist keypoint, and the third target vector is a vector from the left elbow keypoint to the left shoulder keypoint, the second situation includes that the angle between the fourth target vector and the fifth target vector is greater than the second threshold, the right wrist keypoint is lower than the left shoulder keypoint, and the vertical distance between the right wrist keypoint and the left shoulder keypoint is greater than a third threshold, the fourth target vector is a vector from the right elbow keypoint to the right wrist keypoint, and the fifth target vector is a vector from the right elbow keypoint to the right shoulder keypoint.

[0059] The target keypoints corresponding to the prone position can be set to the left ear keypoint, the left shoulder keypoint, the right ear keypoint, and the right shoulder keypoint, and the corresponding first preset condition is set to the vertical distance between the left ear keypoint and the left shoulder keypoint being greater than a fourth threshold (e.g., the fourth threshold is 50) and / or the vertical distance between the right ear keypoint and the right shoulder keypoint being greater than a fourth threshold (e.g., the fourth threshold is 50). Taking the left ear keypoint 3, the left shoulder keypoint 5, the right ear keypoint 4, and the right shoulder keypoint 6 shown in FIG. 2 as examples, the first preset condition corresponding to the prone position can be expressed as y(3)-y(5)>50 and / or y(4)-y(6)>50. In other words, when the movement direction is downward, the target object is identified as having a forward leaning prone movement if the left ear key point is lower than the left shoulder key point and the vertical distance between the left ear key point and the left shoulder key point is greater than a fourth threshold value, and / or if the right ear key point is lower than the right shoulder key point and the vertical distance between the right ear key point and the right shoulder key point is greater than a fourth threshold value.

[0060] As can be seen, when the motion detection result of the target object is that the target object has a target motion, alarm information can be sent to the service platform. For example, if the target motion is a dangerous motion that needs to be detected, alarm information can be sent to the service platform when the target object has a dangerous motion. In a vehicle driving scene, the service platform can be a service platform that operates vehicles, such as an internet-based ride-hailing service platform, and the service platform can be a medical platform, and after the service platform receives the alarm information, emergency treatment measures can be taken, thereby improving the efficiency of emergency rescue for the driver who has encountered a danger, further improving the emergency effect, and better protecting the life safety of the person inside the vehicle. Also, as can be seen, along with recording of the video stream of the scene in the area, the motion information of the target object can be identified in real time or at a certain frequency, and thus, within a preset period, the motion information is continuously that the target object has a target motion, and alarm information can be sent to the service platform, thereby reducing erroneous transmission of alarm information due to fluctuations in the detection result of the motion information.

[0061] As can be seen from the above embodiment, by obtaining a video stream of a scene area, the motion direction of a target object in the scene area can be detected based on a number of image frames in the video stream, and then the skeleton key points of the target object in the image frames can be detected, and finally, the motion information of the target object can be determined based on the geometric relationship between the skeleton key points and the target key points corresponding to the motion direction. Because the geometric relationship between the target key points is objective and accurate, it can accurately detect whether the target object has a dangerous motion, and the target key points are detected according to the motion direction, so that the motion detection is directional and the detection accuracy is further improved. When the detection method is applied in a vehicle, it can accurately detect whether a danger occurs to the driver and passengers in the vehicle, thereby improving the safety of the ride and improving the user's experience.

[0062] In some embodiments of the present disclosure, step S103 shown in FIG. 1 detects skeletal key points of a target object in one image frame among a plurality of image frames of a video stream, i.e., detects skeletal key points of a target object in one image frame among a plurality of image frames in step S102, and the specific image frame to be detected may be preset, such as detecting the first frame, the last frame, or an intermediate frame, etc. Exemplarily, when a preset number of latest image frames in the video stream are cached in real time, step S103 can detect skeletal key points of a target object in one image frame among the preset number of latest image frames.

[0063] Based on this, in step S104 shown in FIG. 1, the motion information of the target object can be identified directly based on the geometric relationship between the target key points corresponding to the motion directions in the detected skeletal key points. For specific identification methods, please refer to the detailed description of step S104 in the above embodiment.

[0064] In some other embodiments of the present disclosure, step S103 shown in Fig. 1 may detect skeletal key points of the target object for each image frame in a plurality of image frames of a video stream, i.e., detect skeletal key points of the target object for each image frame in a plurality of image frames in step S102. Exemplarily, when a preset number of latest image frames of the video stream are cached in real time, step S103 detects skeletal key points of the target object in each image frame of the preset number of latest image frames.

[0065] Based on this, in step S104 shown in Fig. 1, first, among the multiple image frames in the video stream, the motion information of the target object in the corresponding image frame can be identified based on the geometric relationship between the target key points corresponding to the motion direction in the skeletal key points detected in each image frame, and then the motion information of the target object in the multiple image frames can be smoothed to identify the motion information of the target object identified based on the multiple image frames. Exemplarily, when a preset number of latest image frames in the video stream are cached in real time, the motion information of the target object in the cached preset number of image frames is smoothed to obtain the motion information of the target object identified based on the cached preset number of image frames.

[0066] In addition, the motion information detected in each image frame is input into a smoothing queue, and a smoothing window is set, the size of which can be the number of image frames in the video stream, for example the number of cached image frames, and the smoothing window is moved along with the update of the smoothing queue, and after the smoothing window is moved each time, the motion information of the target object identified based on the plurality of image frames is identified based on the plurality of motion information in the smoothing window, thereby realizing a smoothing process for the motion information and improving the validity and stability of the motion information.

[0067] If the motion information detected in each image frame in a plurality of image frames is the same, it can be identified as the motion information of the target object identified based on the plurality of image frames. For example, if the motion information detected in each image frame among five image frames is a leftward tilt of the body, the leftward tilt of the body can be identified as the motion information of the target object identified based on the five image frames.

[0068] If the motion information detected in the multiple image frames does not match, the motion information with the largest number of occurrences is determined as the motion information of the target object identified based on the multiple image frames. For example, if the motion information detected in four of the five image frames is a leftward inclination of the body, and the motion information detected in one image frame is no motion of any target, the leftward inclination of the body can be determined as the motion information of the target object identified based on the five image frames. Optionally, if the detection result of the motion information with the largest number of occurrences does not exceed a preset ratio (e.g., 50%) in the detection results of the motion information of the multiple image frames in a smooth window, the detection result in the smooth window can be discarded, further improving the reliability of the motion detection result.

[0069] In order to accurately identify the motion information of the target object identified based on the multiple image frames, the number of the multiple image frames can be set to an odd number such as 3, 5, 7, etc. When the number is set to an even number and multiple types of motion information with the same number appear, the latest motion information among them is regarded as the motion information of the target object identified based on the multiple image frames.

[0070] Referring to Fig. 3, Fig. 3 shows a complete flow of a motion detection method in a vehicle driving scene as an example. As can be seen from Fig. 3, first, execute step S11, and if the vehicle state meets the activation condition of the method, activate the vehicle cabin emergency rescue function, then execute step S12, and the monitoring camera in the vehicle collects the vehicle cabin passenger information, i.e., the video stream of the passenger in the vehicle cabin, then execute step S13, and cache the most recent images more than two frames, then execute step S14, and use the optical flow method to determine the motion direction, then execute step S15, and perform motion detection within the space corresponding to the motion direction (i.e., the range of the object motion corresponding to the motion direction), that is, detect whether the target object has the object motion corresponding to the motion direction, then execute step S16, and smooth the multiple detection results obtained in step S15, and finally execute step S17, and send a rescue request signal when a dangerous motion is detected based on the smoothing result of S15 and the dangerous motion continues for a certain period of time.

[0071] The motion detection method provided in this embodiment combines the geometric characteristics of motion with optical flow detection processing and some objective circumstances, which can more accurately and comprehensively evaluate the current state of passenger motion information, and at the same time use smoothing algorithm and cache processing to effectively handle some jumps and fluctuations in the results, provide important reference data for rental car companies and traffic supervision and management departments, and direct the customization of safety plans and operation management, thereby improving the life, health and safety of passengers in the vehicle.

[0072] According to a second aspect of the present disclosure, a motion detection device is provided. Referring to FIG. 4, the device includes: an acquisition module 401, a direction module 402, a detection module 403 and an identification module 404; The acquisition module 401 is for acquiring a video stream of a scene area; a direction module 402 for detecting a direction of motion of a target object within the scene region based on a plurality of image frames in the video stream; a detection module 403 for detecting skeletal key points of the target object in the plurality of image frames; The identification module 404 is for identifying motion information of the target object based on the geometric relationship between object key points corresponding to the motion directions in the detected skeletal key points.

[0073] In some embodiments of the present disclosure, the direction module specifically includes: extracting features of the target object in each image frame of the plurality of image frames; determining optical flow information for the target object in the video stream based on features of the target object in each image frame of the plurality of image frames; The target object is to determine a direction of motion of the target object within the scene region based on optical flow information in the video stream.

[0074] In some embodiments of the present disclosure, each movement direction corresponds to a movement of one object.

[0075] Specifically, the specific module is If the geometric relationship between the target key points corresponding to the movement direction in the detected skeletal key points satisfies a first preset condition corresponding to the movement direction, the target object is identified as having a target movement corresponding to the movement direction, and if not, the target object is identified as not having a target movement corresponding to the movement direction.

[0076] In some embodiments of the present disclosure, the skeletal key points include a left shoulder key point, a right shoulder key point, a left wrist key point, a right wrist key point, a left elbow key point, a right elbow key point, a left ear key point, and a right ear key point; The specific module is specifically used in at least one of the following: If the movement direction is left, the tangent value of the angle between a first target vector from a right shoulder key point to a left shoulder key point and a standard vector in a horizontal rightward direction is a positive number and the absolute value of the tangent value is greater than a first threshold value, the target object is determined to have a leftward body tilt movement; If the movement direction is right, the tangent value of the angle between a first target vector from a right shoulder key point to a left shoulder key point and a standard vector in a horizontal rightward direction is negative and the absolute value of the tangent value is greater than a first threshold value, the target object is determined to have a rightward body tilt movement; If the movement direction is up, the target object is identified as holding his / her chest with his / her hand according to the following first or second situation, where the first situation includes: an angle between a second target vector, which is a vector from a left elbow keypoint to a left wrist keypoint, and a third target vector, which is a vector from the left elbow keypoint to a left shoulder keypoint, is greater than a second threshold value; the left wrist keypoint is lower than a right shoulder keypoint, and the vertical distance between the left wrist keypoint and the right shoulder keypoint is greater than a third threshold value; the second situation includes: an angle between a fourth target vector, which is a vector from a right elbow keypoint to a right wrist keypoint, and a fifth target vector, which is a vector from the right elbow keypoint to the right shoulder keypoint, is greater than the second threshold value; the right wrist keypoint is lower than a left shoulder keypoint, and the vertical distance between the right wrist keypoint and the left shoulder keypoint is greater than a third threshold value; or If the movement direction is downward, the target object is identified as having a forward leaning prone movement if the left ear key point is lower than the left shoulder key point and the vertical distance between the left ear key point and the left shoulder key point is greater than a fourth threshold, and / or if the right ear key point is lower than the right shoulder key point and the vertical distance between the right ear key point and the right shoulder key point is greater than a fourth threshold.

[0077] In some embodiments of the present disclosure, the direction module specifically includes: Detecting whether a target object in the video stream is in motion; and to detect, if a target object in the video stream is moving, a motion direction of the target object in the scene region based on a plurality of image frames in the video stream.

[0078] In some embodiments of the present disclosure, the specific module specifically includes: for each image frame in the plurality of image frames in the video stream, determining motion information of the target object in the image frame based on a geometric relationship between object key-points corresponding to the motion directions of the skeletal key-points detected in the image frame; The motion information of the target object in the plurality of image frames is smoothed to obtain motion information of the target object identified based on the plurality of image frames.

[0079] In some embodiments of the present disclosure, the method further includes a cache module, the cache module comprising: During the process of acquiring the video stream of the scene area, a preset number of the latest image frames in the video stream are cached in real time.

[0080] The identification module smoothes the motion information of the target object in the plurality of image frames to obtain the motion information of the target object identified based on the plurality of image frames, specifically: The present invention is to smooth motion information of a target object in a predetermined number of cached image frames, and obtain motion information of the target object identified based on the predetermined number of cached image frames.

[0081] In some embodiments of the present disclosure, the cache module specifically includes: Detect whether pre-defined key information, including at least one of face, at least a part of body, and skeletal key points of a target object, is present in each image frame in the video stream; The present invention is for caching a preset number of the latest image frames in real time among the image frames in which preset key information of the target object in the video stream exists.

[0082] In some embodiments of the present disclosure, the scene area includes a vehicle interior scene area; The acquisition module specifically includes: To obtain a video stream of the scene area when the vehicle doors are locked and / or the vehicle speed reaches a preset speed threshold.

[0083] In some embodiments of the present disclosure, the method further includes a target module, the target module comprising: Detecting a plurality of objects in the video stream; The target object is identified from the plurality of objects based on a position of each of the plurality of objects within the vehicle cabin and / or facial information of each of the plurality of objects.

[0084] In some embodiments of the present disclosure, the method further includes an alarm module, the alarm module comprising: and sending alarm information to a service platform when the target object motion detection result is that a target motion exists in the target object.

[0085] The specific manner in which each module of the apparatus in the above embodiment performs operations has already been described in detail in the embodiment of the method in the third aspect, so it will not be described in detail here.

[0086] In a third aspect, at least one embodiment of the present disclosure provides an apparatus, and referring to FIG. 5, which shows a configuration of the apparatus, the apparatus includes a memory and a processor, the memory is for storing computer instructions executable by the processor, and the processor, upon executing the computer instructions, is for detecting an operation based on the method according to any one of the first aspect.

[0087] In a fourth aspect, at least one embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored therein, the computer program performing a method according to any one of the first aspects when executed by a processor.

[0088] In this disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying a relative importance. The term "plurality" means two or more, unless otherwise specified.

[0089] Other embodiments of the present disclosure will be readily suggested to those skilled in the art from consideration of this specification and practice of the disclosure disclosed herein. This disclosure follows the general principles of this disclosure and includes common general knowledge or practice in the art that is not disclosed by this disclosure. It is intended to cover any modifications, uses, or adaptations of this disclosure. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0090] It should be understood that the present disclosure is not limited to the exact configuration described above and illustrated in the accompanying drawings, and various modifications and changes may be made thereto without departing from the scope of the present disclosure, which is limited only by the scope of the appended claims.

Claims

1. 1. A motion detection method, comprising: Obtaining a video stream of a scene region; detecting a motion direction of a target object within the scene region based on a plurality of image frames in the video stream; detecting skeletal keypoints of the target object in the plurality of image frames; determining motion information of the target object based on a geometric relationship between object key points corresponding to the motion directions in the detected skeletal key points.

2. Detecting a motion direction of a target object within the scene region based on a plurality of image frames in the video stream includes: extracting features of the target object in each image frame of the plurality of image frames; determining optical flow information for the target object in the video stream based on features of the target object in each image frame of the plurality of image frames; 2. The method of claim 1, further comprising: determining a direction of motion of the target object within the scene region based on optical flow information in the video stream.

3. Each of the movement directions corresponds to a movement of one object, Determining motion information of the target object based on a geometric relationship between object key points corresponding to the motion directions of the detected skeletal key points includes: If a geometric relationship between target key points corresponding to the movement direction in the detected skeletal key points satisfies a first preset condition corresponding to the movement direction, the target object is identified as having a target movement corresponding to the movement direction; 2. The method of claim 1, further comprising: identifying the target object as having no target motion corresponding to the motion direction in the target object if a geometric relationship between target key points corresponding to the motion direction in the detected skeletal key points does not satisfy a first preset condition corresponding to the motion direction.

4. The skeleton key points include a left shoulder key point, a right shoulder key point, a left wrist key point, a right wrist key point, a left elbow key point, a right elbow key point, a left ear key point, and a right ear key point; Determining the motion information of the target object based on the geometric relationship between object key points corresponding to the motion directions of the detected skeletal key points includes at least one of the following: If the movement direction is to the left, a tangent value of an angle between a first target vector from a right shoulder key point to a left shoulder key point and a standard vector in a horizontal rightward direction is a positive number and the absolute value of the tangent value is greater than a first threshold value, the target object is determined to have a leftward body tilt movement; If the movement direction is right, the tangent value of the angle between a first target vector from a right shoulder key point to a left shoulder key point and a standard vector in a horizontal rightward direction is negative and the absolute value of the tangent value is greater than a first threshold value, the target object is determined to have a rightward body tilt movement; If the motion direction is upward, the target object is identified as holding his / her chest with his / her hand according to the following first or second situation, where the first situation includes: an angle between a second target vector, which is a vector from a left elbow keypoint to a left wrist keypoint, and a third target vector, which is a vector from the left elbow keypoint to a left shoulder keypoint, is greater than a second threshold value; the left wrist keypoint is lower than a right shoulder keypoint, and a vertical distance between the left wrist keypoint and the right shoulder keypoint is greater than a third threshold value; the second situation includes: an angle between a fourth target vector, which is a vector from a right elbow keypoint to a right wrist keypoint, and a fifth target vector, which is a vector from the right elbow keypoint to the right shoulder keypoint, is greater than a second threshold value; the right wrist keypoint is lower than a left shoulder keypoint, and a vertical distance between the right wrist keypoint and the left shoulder keypoint is greater than a third threshold value; or 4. The method for detecting a motion according to claim 1, further comprising the steps of: determining that the target object is performing a prone motion while the motion direction is downward; and / or determining that the target object is performing a prone motion while the left ear key point is lower than the left shoulder key point and the vertical distance between the left ear key point and the left shoulder key point is greater than a fourth threshold value; and / or determining that the right ear key point is lower than the right shoulder key point and the vertical distance between the right ear key point and the right shoulder key point is greater than a fourth threshold value.

5. Detecting a motion direction of a target object within the scene region based on a plurality of image frames in the video stream includes: detecting whether a target object in the video stream is in motion; 5. The method of claim 1, further comprising: detecting a direction of movement of a target object in the scene area based on a plurality of image frames in the video stream when the target object in the video stream is moving.

6. Determining motion information of the target object based on a geometric relationship between object key points corresponding to the motion directions of the detected skeletal key points includes: For each image frame in the plurality of image frames in the video stream, determining motion information of the target object in the image frame based on a geometric relationship between object key-points corresponding to the motion directions of the skeletal key-points detected in the image frame; The motion detection method according to claim 1, 3 or 4, further comprising: smoothing motion information of the target object in the plurality of image frames to obtain motion information of the target object identified based on the plurality of image frames.

7. The motion detection method further includes: In the process of acquiring the video stream of the scene area, a preset number of the most recent image frames in the video stream are cached in real time; Smoothing the motion information of the target object in the plurality of image frames to obtain the motion information of the target object identified based on the plurality of image frames, The motion detection method of claim 6, further comprising smoothing motion information of the target object in the cached predetermined number of the image frames to obtain motion information of the target object identified based on the cached predetermined number of the image frames.

8. Caching a preset number of the most recent image frames in real time Detecting whether predetermined key information, including at least one of face, at least a part of body, and skeletal key points of a target object, is present in each image frame in the video stream; The motion detection method of claim 7, further comprising: caching a preset number of the latest image frames in real time among the image frames in which preset key information of the target object in the video stream is present.

9. the scene area includes a vehicle interior scene area; Obtaining a video stream of the scene region includes:

9. The method of claim 1, further comprising: capturing a video stream of the scene area when the vehicle doors are locked and / or the vehicle speed reaches a preset speed threshold.

10. The motion detection method further includes: detecting a plurality of objects in the video stream; and identifying the target object from the plurality of objects based on a position of each of the plurality of objects in the vehicle interior and / or face information of each of the plurality of objects.

11. The motion detection method further includes: The motion detection method according to any one of claims 1 to 10, further comprising: sending alarm information to a service platform when the motion detection result of the target object is that a target motion exists in the target object.

12. A motion detection device, comprising: an acquisition module for acquiring a video stream of a scene region; a direction module for detecting a direction of motion of a target object within the scene region based on a plurality of image frames in the video stream; a detection module for detecting skeletal key points of the target object in the plurality of image frames; and an identification module for identifying motion information of the target object based on a geometric relationship between object key points corresponding to the motion directions in the detected skeletal key points.

13. 1. An electronic device comprising: A memory and a processor are included. An electronic device, characterized in that the memory is for storing computer instructions to be executed by a processor, and when the processor executes the computer instructions, the method according to any one of claims 1 to 11 is performed.

14. 1. A computer-readable storage medium, comprising: A computer readable storage medium having a computer program stored thereon, the computer program being adapted to perform the method according to any one of claims 1 to 11 when executed by a processor.

Images