Methods, devices, wearable devices and storage media for motion detection

Abstract

This application provides a method, apparatus, wearable device, and storage medium for motion detection. The motion detection method includes: obtaining first angle data collected by an angle sensor at a first moment and a first image frame collected by a camera at the same moment; obtaining second angle data collected by the angle sensor at a second moment and a second image frame collected by the camera at the same moment; determining an angle change vector of the wearable device based on the first and second angle data; determining a comprehensive motion change vector of the wearable device based on the first and second image frames; and obtaining a position change vector of the wearable device based on the angle change vector and the comprehensive motion change vector. The position change vector of the wearable device is used for motion detection. This application embodiment can improve the accuracy of motion detection.

Description

Technical Field

[0001] This application relates to the field of wearable device technology, specifically to a method, apparatus, wearable device, and storage medium for motion detection. Background Technology

[0002] Currently, motion monitoring in wearable devices typically incorporates accelerometers to determine the user's arm posture and achieve motion detection. Generally, a built-in accelerometer continuously collects data during the user's walking process, and motion detection is performed based on the collected acceleration data. However, the accuracy of current motion detection methods is relatively low. Summary of the Invention

[0003] This application provides a method, apparatus, wearable device, and storage medium for motion detection, which can improve the accuracy of motion detection.

[0004] A first aspect of this application provides a motion detection method, the method being applied to a wearable device, the method comprising:

[0005] Obtain first angle data collected by the angle sensor at a first moment and first image frame collected by the camera at the first moment; obtain second angle data collected by the angle sensor at a second moment and second image frame collected by the camera at the second moment.

[0006] The angle change vector of the wearable device is determined based on the first angle data and the second angle data, and the comprehensive motion change vector of the wearable device is determined based on the first image frame and the second image frame.

[0007] The position change vector of the wearable device is obtained based on the angle change vector and the comprehensive motion change vector, and the position change vector of the wearable device is used for motion detection.

[0008] A second aspect of this application provides a motion detection device, which is applied to a wearable device, the device comprising:

[0009] The acquisition unit is used to acquire first angle data collected by the angle sensor at a first moment and first image frame collected by the camera at the first moment, and to acquire second angle data collected by the angle sensor at a second moment and second image frame collected by the camera at the second moment.

[0010] The processing unit is configured to determine the angle change vector of the wearable device based on the first angle data and the second angle data, and to determine the comprehensive motion change vector of the wearable device based on the first image frame and the second image frame.

[0011] The processing unit is further configured to obtain the position change vector of the wearable device based on the angle change vector and the comprehensive motion change vector, and the position change vector of the wearable device is used for motion detection.

[0012] A third aspect of this application provides a wearable device including a processor and a memory, the memory being used to store a computer program, the computer program including program instructions, and the processor being configured to invoke the program instructions to execute the step instructions as described in the first aspect of this application.

[0013] A fourth aspect of this application provides a computer-readable storage medium storing a computer program for electronic data interchange, the computer program including program instructions that, when executed by a processor, cause the processor to perform the step instructions as described in the first aspect of this application.

[0014] A fifth aspect of this application provides a computer program product, wherein the computer program product includes a computer program, the computer program including program instructions, and the program instructions, when executed by a processor, cause the processor to perform the step instructions as described in the first aspect of this application.

[0015] In this embodiment, first angle data collected by an angle sensor at a first moment and a first image frame collected by a camera at the first moment are obtained; second angle data collected by the angle sensor at a second moment and a second image frame collected by the camera at the second moment are obtained; an angle change vector of the wearable device is determined based on the first angle data and the second angle data; a comprehensive motion change vector of the wearable device is determined based on the first image frame and the second image frame; and a position change vector of the wearable device is obtained based on the angle change vector and the comprehensive motion change vector, which is used for motion detection. The motion detection method of this embodiment, because the position change vector of the wearable device considers the angle changes at two moments and the comprehensive motion change between the two image frames at those two moments, can obtain a more accurate position change vector. Using the position change vector of the wearable device for motion detection can improve the accuracy of motion detection. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic flowchart of a motion detection method provided in an embodiment of this application;

[0018] Figure 2 This is a schematic diagram illustrating the angle change, translation position change, and combined change of a camera according to an embodiment of this application;

[0019] Figure 3 This is a schematic diagram illustrating a method for calculating the comprehensive motion change vector of a wearable device, as provided in an embodiment of this application.

[0020] Figure 4 This is a schematic diagram illustrating a method for defining a three-dimensional coordinate system according to an embodiment of this application;

[0021] Figure 5 This is a schematic diagram illustrating how a position change vector of a wearable device is obtained by subtracting a comprehensive motion change vector from a planar motion vector, as provided in an embodiment of this application.

[0022] Figure 6 This is a schematic flowchart of another motion detection method provided in an embodiment of this application;

[0023] Figure 7 This is a schematic flowchart of another motion detection method provided in an embodiment of this application;

[0024] Figure 8 This is a schematic diagram of a watch frame for motion detection provided in an embodiment of this application;

[0025] Figure 9 This is a schematic diagram of the structure of a motion detection device provided in an embodiment of this application;

[0026] Figure 10 This is a schematic diagram of the structure of a wearable device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0029] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0030] The wearable devices involved in the embodiments of this application can be devices with data processing and communication functions, including an angle sensor and at least one camera. For example, wearable devices can include: smartwatches, smart bracelets, smart helmets, smart glasses, etc.

[0031] Please see Figure 1 , Figure 1 This is a schematic flowchart of a motion detection method provided in an embodiment of this application. Figure 1 As shown, the motion detection method may include the following steps.

[0032] 101. The wearable device obtains first angle data collected by the angle sensor at a first moment and first image frame collected by the camera at a first moment, and obtains second angle data collected by the angle sensor at a second moment and second image frame collected by the camera at a second moment.

[0033] In this embodiment, the wearable device may include an angle sensor and a camera. The first moment and the second moment are two different moments. The angle sensor may be a geomagnetic sensor. The angle sensor and the camera may simultaneously perform periodic data acquisition. For example, the angle sensor may acquire angle data every 0.1 seconds, and the camera may acquire an image frame every 0.1 seconds. The data acquired by the angle sensor and the camera at the same moment are then analyzed. For example, the first moment and the second moment may be two adjacent periods, with a 0.1-second interval between them.

[0034] The angle sensor and camera can be configured to have the same duty cycle. When motion detection is required, both the angle sensor and camera enter their duty cycles and begin acquiring data simultaneously. The angle sensor can transmit the periodically acquired angle data to the wearable device's processing module (e.g., a processor), and the camera can transmit the periodically acquired image frames to the wearable device's processing module. The wearable device's processing module can obtain the first angle data acquired by the angle sensor at a first moment and the first image frame acquired by the camera at a first moment, and then obtain the second angle data acquired by the angle sensor at a second moment and the second image frame acquired by the camera at a second moment.

[0035] 102. The wearable device determines the angle change vector of the wearable device based on the first angle data and the second angle data, and determines the comprehensive motion change vector of the wearable device based on the first image frame and the second image frame.

[0036] In this embodiment, the first angle data may include a first angle vector a1, and the second angle data may include a second angle vector a2. The angle change vector A = a1 - a2. The angle change vector A may include the magnitude of the angle and the direction of the angle change, and can be understood as a vector generated by the rotation of the camera. For example, if the wearable device is worn on the user's wrist, the first angle vector a1 detected by the geomagnetic sensor at the first moment may be a line along the central axis of the camera at the first moment, and the second angle vector a2 detected by the geomagnetic sensor at the second moment may be a line along the central axis of the camera at the second moment. The angle between the first angle vector a1 and the second angle vector a2 is the angle of the angle change vector A. The second angle vector a2 points in the direction of the first angle vector a1, that is, the direction of change of the angle change vector A.

[0037] For example, the processing module of the wearable device can determine the comprehensive motion change vector of the wearable device based on the motion vector of at least one target object in the first image frame and the second image frame. For instance, at least one target object can be identified that exists in both the first and second image frames. For each target object, its positional change between the first and second image frames can be calculated, thereby obtaining the motion change vector of each target object. The comprehensive motion change vector of the wearable device can be obtained by averaging the motion change vectors of each target object.

[0038] For example, the processing module of the wearable device can perform image registration between the first image frame and the second image frame, and calculate the average of the vectors between all registered feature points to obtain the comprehensive motion change vector of the wearable device.

[0039] It should be noted that the position and angle of the camera may change at both the first and second moments. Therefore, the comprehensive motion change vector of the wearable device determined based on the first and second image frames reflects the result of the comprehensive motion change of the wearable device under changes in angle and position.

[0040] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating the angle change, translation position change, and combined change of a camera according to an embodiment of this application. For example... Figure 2 As shown, the left side displays the camera's initial angle, translation position, and combined position, while the middle and right sides display the camera's angle, translation position, and combined position at two different moments. The combined change is the superposition of the angle and translation position changes.

[0041] from Figure 2 As can be seen, in terms of angle changes, the left camera faces directly upwards, the middle camera faces slightly to the left of directly upwards, and the right camera faces slightly to the right of directly upwards. In terms of position changes, the left camera is in its initial position, the middle camera has been translated to the lower left relative to the left, and the right camera has been translated to the lower right relative to the left. In terms of overall changes, the left camera's overall position is the initial position, the middle camera's overall position has been rotated and translated to the lower left relative to the left, and the right camera's position has been rotated and translated to the lower right relative to the left.

[0042] Optionally, in step 102, the wearable device determines the comprehensive motion change vector of the wearable device based on the first image frame and the second image frame, which may include the following steps:

[0043] The wearable device determines the position change vector of the first object based on the first position information of the first object in the first image frame and the second position information of the first object in the second image frame, and uses the position change vector of the first object as the comprehensive motion change vector of the wearable device.

[0044] In this embodiment, the first object is the same object present in both the first and second image frames. The wearable device can determine that the same object present in both the first and second image frames is the first object. The first and second position information can be two-dimensional coordinates. For example, please refer to... Figure 3 , Figure 3 This is a schematic diagram illustrating the calculation of the comprehensive motion change vector of a wearable device according to an embodiment of this application. Figure 3As shown, the first position information can be the first two-dimensional coordinate P1(x1, y1), and the second position information can be the second two-dimensional coordinate P2(x2, y2). Then, the position change vector of the first object is B = (x2-x1, y2-y1), and the position change vector B of the first object can be used as the comprehensive motion change vector of the wearable device.

[0045] This application provides an accurate and effective method for calculating the overall motion change vector of a wearable device. The method can determine the position change vector of the same object by the position change of the same object in two image frames, and use the position change vector as the overall motion change vector of the wearable device. The overall motion change vector of the wearable device can be accurately reflected by the position change of the same object in two adjacent image frames captured by the camera.

[0046] 103. The wearable device obtains the position change vector of the wearable device based on the angle change vector and the comprehensive motion change vector. This position change vector of the wearable device is used for motion detection.

[0047] In this embodiment, the comprehensive motion change vector reflects the combined motion change of the wearable device under changes in angle and position, while the angle change vector reflects the change in angle of the wearable device. The portion reflecting angle change can be removed from the comprehensive motion change vector to obtain the position change vector of the wearable device. The position change vector of the wearable device can be a translational position change vector.

[0048] Motion detection can include the detection of motion data, such as step count data and running data.

[0049] After obtaining the position change vector of the wearable device, motion detection can be performed based on the position change vector, the angle change vector, the position change vector and the acceleration data collected by the accelerometer, or the position change vector, acceleration data collected by the accelerometer and the angle change vector of the wearable device.

[0050] Optionally, in step 103, the wearable device obtains its position change vector based on the angle change vector and the combined motion change vector, including:

[0051] (11) The wearable device obtains the planar motion vector based on the angle change vector;

[0052] (12) The wearable device obtains the position change vector of the wearable device based on the planar motion vector and the comprehensive motion change vector.

[0053] In this embodiment, the angle change vector may include the magnitude and direction of the angle change; however, the angle change vector cannot reflect the impact of the angle change on the displacement of the object's planar position within an image frame. The angle change vector can be converted into a displacement vector of the same object in adjacent image frames due to the angle change: a planar motion vector.

[0054] Since the comprehensive motion change vector is the position change vector of the same object in two adjacent image frames captured by the camera (including the effects of camera angle changes and translation), the angle change vector is converted into a displacement vector of the same object in adjacent image frames due to angle changes: a planar motion vector. The comprehensive motion change vector (including the effects of camera angle changes and translation) and the planar motion vector (including the effects of camera angle changes) are calculated according to the same standard. Based on the planar motion vector and the comprehensive motion change vector, the position change vector of the wearable device (including the effects of camera translation) can be accurately obtained, improving the accuracy of the wearable device's position change vector calculation.

[0055] Optionally, in step (11), the wearable device obtains the planar motion vector based on the angle change vector, which may specifically include the following steps:

[0056] (111) The wearable device calculates the lateral angle change of the wearable device based on the first projection angle vector obtained by projecting the angle change vector onto the first plane, and calculates the longitudinal angle change of the wearable device based on the second projection angle vector obtained by projecting the angle change vector onto the second plane; the first plane includes a plane composed of the lateral direction of the image and a direction perpendicular to the image, and the second plane includes a plane composed of the longitudinal direction of the image and a direction perpendicular to the image.

[0057] (112) The wearable device obtains a horizontal motion vector based on the horizontal angle change, the horizontal viewing angle of the camera, the horizontal frame length and the first projection angle vector, obtains a vertical motion vector based on the vertical angle change, the vertical viewing angle of the camera, the vertical frame length and the second projection angle vector, and obtains a planar motion vector based on the horizontal motion vector and the vertical motion vector.

[0058] In this embodiment, the image frame may include the image frame size (e.g., the image frame size of a first image frame or the image frame size of a second image frame). The angle change vector can be understood as a sector-shaped region with an angle extending from the origin of the three-dimensional coordinate system in three-dimensional space. The included angle of the sector is the angle of the angle change vector, and the directions pointed by the two included sides of the sector are the directions of the angle change vector. The three-dimensional coordinate system may include: the horizontal direction of the image frame (e.g., the horizontal direction to the right), the vertical direction of the image frame (e.g., the vertical direction upwards), and the direction perpendicular to the image frame (e.g., the direction perpendicular to the image frame outwards). Please refer to [link to relevant documentation]. Figure 4 , Figure 4 This is a schematic diagram illustrating a method for defining a three-dimensional coordinate system according to an embodiment of this application, as shown below. Figure 4 As shown, the three-dimensional coordinate system can be defined by mutually perpendicular x-axis, y-axis, and z-axis. The positive direction of the x-axis is the horizontal direction to the right of the image frame, the positive direction of the y-axis is the vertical direction upwards of the image frame, and the positive direction of the z-axis is the direction perpendicular to the image frame outwards. It should be noted that once the three-dimensional coordinate system is defined, it will not change during the execution of steps 101 to 103.

[0059] Projecting the angle change vector onto a first plane yields a first projected angle vector, which is an angled first projected sector extending from the origin in three-dimensional space. Calculating the included angle of this first projected sector reveals the lateral angle change of the wearable device. It should be noted that if this first projected sector is a straight line, its included angle is 0. Similarly, projecting the angle change vector onto a second plane yields a second projected angle vector, which is an angled second projected sector extending from the origin in three-dimensional space. Calculating the included angle of this second projected sector reveals the longitudinal angle change of the wearable device. It should also be noted that if this second projected sector is a straight line, its included angle is 0.

[0060] In step (112), the wearable device obtains a lateral motion vector based on the lateral angle change, the lateral viewing angle of the camera, and the lateral frame length, including:

[0061] The wearable device obtains the magnitude of the lateral motion vector based on the lateral angle change, the lateral viewing angle of the camera, and the lateral frame length, and determines the direction of the lateral motion vector based on the direction of the first projection angle vector.

[0062] In this embodiment of the application, the magnitude of the lateral motion vector can be calculated according to the following formula:

[0063] L1 = α1 * L2 / α2;

[0064] Where L1 is the magnitude of the lateral motion vector, α1 is the lateral angle change, α2 is the lateral viewing angle of the camera, and L2 is the lateral frame length.

[0065] If a three-dimensional coordinate system can be defined as having mutually perpendicular x-axis, y-axis, and z-axis, then the horizontal direction to the right of the image frame is the positive direction of the x-axis, the vertical direction upwards is the positive direction of the y-axis, and the direction perpendicular to the image frame outwards is the positive direction of the z-axis. If the angle between the direction of the first projection angle vector and the positive direction of the x-axis of the three-dimensional coordinate system is less than 90 degrees, then the direction of the horizontal movement vector is the positive direction of the x-axis; if the angle between the direction of the first projection angle vector and the positive direction of the x-axis of the three-dimensional coordinate system is greater than 90 degrees, then the direction of the horizontal movement vector is the negative direction of the x-axis.

[0066] In step (112), the wearable device obtains a longitudinal motion vector based on the longitudinal angle change, the longitudinal viewing angle of the camera, and the longitudinal frame length, including:

[0067] The wearable device obtains the magnitude of the longitudinal motion vector based on the longitudinal angle change, the longitudinal viewing angle of the camera, and the longitudinal frame length, and determines the direction of the longitudinal motion vector based on the second projection angle vector.

[0068] In this embodiment of the application, the magnitude of the longitudinal motion vector can be calculated according to the following formula:

[0069] H1 = β1 * H2 / β2;

[0070] Where H1 is the magnitude of the longitudinal motion vector, β1 is the longitudinal angle change, β2 is the longitudinal viewing angle of the camera, and H2 is the longitudinal frame length.

[0071] If the three-dimensional coordinate system can be defined as having mutually perpendicular x-axis, y-axis, and z-axis, then the horizontal direction to the right of the image frame is the positive direction of the x-axis, the vertical direction upwards is the positive direction of the y-axis, and the direction perpendicular to the image frame outwards is the positive direction of the z-axis. If the angle between the direction of the second projection angle vector and the positive direction of the y-axis of the three-dimensional coordinate system is less than 90 degrees, then the direction of the vertical motion vector is the positive direction of the y-axis; if the angle between the direction of the second projection angle vector and the positive direction of the y-axis of the three-dimensional coordinate system is greater than 90 degrees, then the direction of the vertical motion vector is the negative direction of the y-axis.

[0072] In step (112), the wearable device obtains a planar motion vector based on the lateral motion vector and the longitudinal motion vector, which may include: adding the lateral motion vector and the longitudinal motion vector together to obtain the planar motion vector.

[0073] This application provides a method for converting an angle change vector into a planar motion vector, which can accurately calculate the magnitude and direction of the lateral motion vector and the magnitude and direction of the longitudinal motion vector, thereby obtaining an accurate planar motion vector.

[0074] For example, suppose the camera tilts diagonally upwards to the right, the frame of the image captured by the camera will move diagonally downwards to the left. In this case, the direction of the angle change vector A is diagonally upwards to the right, and it has a certain angle value. Assuming the camera's horizontal viewing angle is 80° and its vertical viewing angle is 90°, the angle change vector A can be projected onto the plane containing the horizontal direction of the frame (the first plane mentioned above) and the plane containing the vertical direction (the second plane mentioned above), respectively. The angles of the angle change vector A in the horizontal and vertical directions can then be calculated. Then, based on the ratio of the horizontal angle of the angle change vector A to 80° and the horizontal frame length, the magnitude of the planar motion vector in the horizontal and vertical directions can be calculated. Finally, the planar motion vector can be obtained from the horizontal and vertical directions of the planar motion vector.

[0075] Optionally, in step (12), the wearable device obtains the position change vector of the wearable device based on the planar motion vector and the comprehensive motion change vector, which may specifically include the following steps:

[0076] The wearable device performs a vector subtraction operation between the comprehensive motion change vector and the planar motion vector to obtain the position change vector of the wearable device.

[0077] In this embodiment, the comprehensive motion change vector (including the influence of camera angle change and translational motion) and the planar motion vector (including the influence of camera angle change) are vectors calculated according to the same standard. Subtracting the planar motion vector from the comprehensive motion change vector can eliminate the influence of angle change in the comprehensive motion change vector, thereby accurately obtaining the position change vector of the wearable device (including the influence of camera translational motion), and thus improving the accuracy of the calculation of the position change vector of the wearable device.

[0078] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating how to obtain the position change vector of a wearable device by subtracting a comprehensive motion change vector from a planar motion vector, as provided in an embodiment of this application. Figure 5 As shown, the overall motion change vector is B, the planar motion vector is A', and the position change vector of the wearable device is C, where C = B - A'.

[0079] In this embodiment, the position change vector of the wearable device can be obtained based on the angle change vector and the comprehensive motion change vector, and motion detection can be performed based on the position change vector and the angle change vector. Since the position change vector of the wearable device considers the angle change at two moments and the comprehensive motion change between the two image frames at those two moments, a more accurate position change vector can be obtained. Using the position change vector of the wearable device for motion detection can improve the accuracy of motion detection.

[0080] Please see Figure 6 , Figure 6 This is a schematic flowchart of another motion detection method provided in an embodiment of this application. Figure 6 As shown, the motion detection method may include the following steps.

[0081] 601, when the wearable device is detected to be in motion, the wearable device obtains first angle data collected by the angle sensor at a first moment and first image frame collected by the camera at a first moment, and obtains second angle data collected by the angle sensor at a second moment and second image frame collected by the camera at a second moment.

[0082] In this embodiment, the wearable device may include an accelerometer, which can be used to determine whether the wearable device is in motion by collecting acceleration data. The acceleration data may include numerical values ​​of acceleration; if the detected acceleration value is greater than a set threshold, it can be considered that the wearable device is in motion.

[0083] In this embodiment, the angle sensor and camera can start collecting data when the wearable device is detected to be in motion, which can avoid the angle sensor and camera working when the wearable device is not in motion, thereby reducing the power consumption of the wearable device.

[0084] 602, the wearable device determines the angle change vector of the wearable device based on the first angle data and the second angle data, and determines the comprehensive motion change vector of the wearable device based on the first image frame and the second image frame.

[0085] 603. The wearable device obtains its position change vector based on the angle change vector and the comprehensive motion change vector, and this position change vector is used for motion detection.

[0086] For details on the implementation of steps 602 to 603, please refer to [link / reference]. Figure 1 Steps 102 to 103 shown are not repeated here.

[0087] In this embodiment, when the wearable device is in motion, its position change vector can be obtained based on the angle change vector and the comprehensive motion change vector. Motion detection based on the position change vector and the angle change vector can reduce the power consumption of the wearable device. Since the position change vector of the wearable device considers the angle change at two moments and the comprehensive motion change between the two image frames at those two moments, a more accurate position change vector can be obtained. Using the position change vector of the wearable device for motion detection can improve the accuracy of motion detection.

[0088] Please see Figure 7 , Figure 7 This is a schematic flowchart of another motion detection method provided in an embodiment of this application. Figure 7 As shown, the motion detection method may include the following steps.

[0089] 701, The wearable device obtains first angle data collected by the angle sensor at a first moment and first image frame collected by the camera at a first moment, and obtains second angle data collected by the angle sensor at a second moment and second image frame collected by the camera at a second moment.

[0090] 702, the wearable device determines the angle change vector of the wearable device based on the first angle data and the second angle data, and determines the comprehensive motion change vector of the wearable device based on the first image frame and the second image frame.

[0091] For details on the implementation of steps 701 to 702, please refer to [link / reference]. Figure 1 Steps 101 to 102 shown are not repeated here.

[0092] 703. The wearable device obtains the position change vector of the wearable device based on the angle change vector and the comprehensive motion change vector, and performs motion detection based on the acceleration data collected by the accelerometer, the position change vector of the wearable device, and the angle change vector.

[0093] In this embodiment, motion detection takes step counting as an example. Whether a step counting event has occurred can be determined based on acceleration data at the same time, the position change vector of the wearable device, and the angle change vector. The acceleration change trend can be determined based on acceleration data periodically collected by the accelerometer, and the position change vector and angle change vector change trends can be determined based on angle data periodically collected by the angle sensor and image frames periodically collected by the camera. When the acceleration change trend matches the position change vector and angle change vector change trends, a step counting event can be considered to have occurred.

[0094] For example, a curve showing the change of acceleration data over time can be generated based on acceleration data periodically collected by an accelerometer, and curves showing the change of position vector over time and the change of angle vector over time can be generated based on angle data periodically collected by an angle sensor and image frames periodically collected by a camera. When the curve showing the change of acceleration data over time matches the curves showing the change of position vector over time and the curves showing the change of angle vector over time, a step counting event can be considered to have occurred.

[0095] This application embodiment can perform motion detection based on acceleration data collected by an accelerometer, the position change vector and angle change vector of the wearable device. Compared with motion detection based solely on acceleration data, this can improve the accuracy of motion detection. Since the position change vector of the wearable device considers the angle change at two moments and the comprehensive motion change between the two image frames at those two moments, a more accurate position change vector can be obtained. Motion detection based on acceleration data, position change vector and angle change vector can improve the accuracy of motion detection, thereby providing real-time and accurate feedback of the user's motion detection data (step count data).

[0096] The following diagram illustrates the framework for motion detection using a wearable device, specifically a watch. Please refer to [link / reference]. Figure 8 , Figure 8 This is a schematic diagram of a watch frame for motion detection provided in an embodiment of this application. Figure 8 As shown, the processing model can be a functional module of the watch's processor. The geomagnetic sensor inputs the collected geomagnetic sensor data into the processing model, and the camera inputs two consecutive frames of footage into the processing model. The processing model calculates the watch's angle change (e.g., the angle change vector of the wearable device mentioned above) and position change (e.g., the position change vector of the wearable device mentioned above) based on the geomagnetic sensor data and the two consecutive frames. The motion detection module can then assist in determining the motion status based on the changes in the watch's angle and position. The motion detection module can also be another functional module of the watch's processor.

[0097] In this embodiment, since the position change vector of the wearable device takes into account the angle change at two moments and the comprehensive motion change between the two image frames at the two moments, a more accurate position change vector can be obtained. Using the position change vector of the wearable device for motion detection can improve the accuracy of motion detection.

[0098] The above primarily describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, wearable devices include corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0099] This application embodiment can divide the wearable device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0100] Please see Figure 9 , Figure 9 This is a schematic diagram of a motion detection device 900 provided in an embodiment of this application. The motion detection device 900 is applied to a wearable device and may include an acquisition unit 901 and a processing unit 902, wherein:

[0101] The obtaining unit 901 is used to obtain the first angle data collected by the angle sensor at the first moment and the first image frame collected by the camera at the first moment, and to obtain the second angle data collected by the angle sensor at the second moment and the second image frame collected by the camera at the second moment.

[0102] The processing unit 902 is configured to determine the angle change vector of the wearable device based on the first angle data and the second angle data, and to determine the comprehensive motion change vector of the wearable device based on the first image frame and the second image frame.

[0103] The processing unit 902 is further configured to obtain the position change vector of the wearable device based on the angle change vector and the comprehensive motion change vector, wherein the position change vector of the wearable device is used for motion detection.

[0104] Optionally, the processing unit 902 determines the comprehensive motion change vector of the wearable device based on the first image frame and the second image frame, including: determining the position change vector of the first object based on the first position information of the first object in the first image frame and the second position information of the first object in the second image frame, and using the position change vector of the first object as the comprehensive motion change vector of the wearable device.

[0105] Optionally, the processing unit 902 obtains the position change vector of the wearable device based on the angle change vector and the comprehensive motion change vector, including: obtaining a planar motion vector based on the angle change vector, and obtaining the position change vector of the wearable device based on the planar motion vector and the comprehensive motion change vector.

[0106] Optionally, the processing unit 902 obtains a planar motion vector based on the angle change vector, including: calculating the lateral angle change of the wearable device based on a first projection angle vector obtained by projecting the angle change vector onto a first plane; calculating the longitudinal angle change of the wearable device based on a second projection angle vector obtained by projecting the angle change vector onto a second plane; the first plane includes a plane composed of the lateral direction of the image frame and a direction perpendicular to the image frame, and the second plane includes a plane composed of the longitudinal direction of the image frame and a direction perpendicular to the image frame; obtaining a lateral motion vector based on the lateral angle change, the lateral viewing angle of the camera, the lateral image frame length, and the first projection angle vector; obtaining a longitudinal motion vector based on the longitudinal angle change, the longitudinal viewing angle of the camera, the longitudinal image frame length, and the second projection angle vector; and obtaining a planar motion vector based on the lateral motion vector and the longitudinal motion vector.

[0107] Optionally, the processing unit 902 obtains the position change vector of the wearable device based on the planar motion vector and the comprehensive motion change vector, including: performing a vector subtraction operation between the comprehensive motion change vector and the planar motion vector to obtain the position change vector of the wearable device.

[0108] Optionally, the obtaining unit 901 is further configured to, when the wearable device is detected to be in motion, obtain first angle data collected by the angle sensor at a first moment and first image frame collected by the camera at a first moment, and obtain second angle data collected by the angle sensor at a second moment and second image frame collected by the camera at a second moment.

[0109] Optionally, the processing unit 902 is further configured to perform motion detection based on the acceleration data collected by the accelerometer, the position change vector of the wearable device, and the angle change vector after obtaining the position change vector of the wearable device based on the angle change vector and the comprehensive motion change vector.

[0110] In this embodiment, the obtaining unit 901 may be a communication interface in a wearable device, and the processing unit 902 may be a processor in a wearable device.

[0111] Figure 9 For a detailed implementation of the motion detection device 900 shown, please refer to [link to relevant documentation]. Figure 1 , Figure 6 and Figure 7 The method embodiments shown are not described in detail here.

[0112] In this embodiment, the position change vector of the wearable device can be obtained based on the angle change vector and the comprehensive motion change vector, and motion detection can be performed based on the position change vector and the angle change vector. Since the position change vector of the wearable device considers the angle change at two moments and the comprehensive motion change between the two image frames at those two moments, a more accurate position change vector can be obtained. Using the position change vector of the wearable device for motion detection can improve the accuracy of motion detection.

[0113] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of a wearable device provided in an embodiment of this application, as shown below. Figure 10 As shown, the wearable device 1000 includes a processor 1001 and a memory 1002, which are interconnected via a communication bus 1003. The communication bus 1003 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus 1003 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 10 The bus is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. Memory 1002 stores computer programs, which include program instructions. Processor 1001 is configured to invoke these program instructions, which include instructions for execution. Figures 2-5 Some or all of the steps in the method shown.

[0114] The processor 1001 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the above-mentioned program.

[0115] The memory 1002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0116] The wearable device 1000 may include an angle sensor 1004 and a camera module 1005, the camera module including at least one camera. The wearable device 1000 may also include sensors such as an accelerometer, an ambient light sensor, and a heart rate sensor.

[0117] In addition, the wearable device 1000 may also include a communication module, which may include a Bluetooth communication module. The wearable device 1000 may also include general components such as a communication interface and an antenna, which will not be described in detail here.

[0118] The wearable device 1000 may also include a display module, which may include a display screen for displaying motion data.

[0119] In this embodiment, the position change vector of the wearable device can be obtained based on the angle change vector and the comprehensive motion change vector, and motion detection can be performed based on the position change vector and the angle change vector. Since the position change vector of the wearable device considers the angle change at two moments and the comprehensive motion change between the two image frames at those two moments, a more accurate position change vector can be obtained. Using the position change vector of the wearable device for motion detection can improve the accuracy of motion detection.

[0120] This application also provides a computer-readable storage medium storing a computer program for electronic data interchange that causes a computer to perform some or all of the steps of any of the motion detection methods described in the above method embodiments.

[0121] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0122] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0123] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0124] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0125] Furthermore, the functional units in the various embodiments of the application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0126] If the integrated unit is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0127] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0128] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for motion detection, characterized in that, The method is applied to a wearable device, the wearable device including an angle sensor and a camera, and the method includes: Obtain first angle data collected by the angle sensor at a first moment and first image frame collected by the camera at the first moment; obtain second angle data collected by the angle sensor at a second moment and second image frame collected by the camera at the second moment. The angle change vector of the wearable device is determined based on the first angle data and the second angle data. The position change vector of the first object is determined based on the first position information of the first object in the first image frame and the second position information of the first object in the second image frame. The position change vector of the first object is used as the comprehensive motion change vector of the wearable device. The planar motion vector is obtained based on the angle change vector. The comprehensive motion change vector is subtracted from the planar motion vector to obtain the position change vector of the wearable device. The position change vector of the wearable device is used for motion detection. The motion detection includes the detection of motion data, which includes step count data or running data. The planar motion vector is the displacement vector of the same object in the first image frame and the second image frame due to the angle change.

2. The method according to claim 1, characterized in that, The step of obtaining the planar motion vector based on the angle change vector includes: The lateral angle change of the wearable device is calculated based on the first projection angle vector obtained by projecting the angle change vector onto the first plane, and the longitudinal angle change of the wearable device is calculated based on the second projection angle vector obtained by projecting the angle change vector onto the second plane; the first plane includes a plane composed of the lateral direction of the image frame and a direction perpendicular to the image frame, the second plane includes a plane composed of the longitudinal direction of the image frame and a direction perpendicular to the image frame, and the image frame includes the image frame of the first image frame or the image frame of the second image frame; A horizontal motion vector is obtained based on the horizontal angle change, the horizontal viewing angle of the camera, the horizontal frame length, and the first projection angle vector. A vertical motion vector is obtained based on the vertical angle change, the vertical viewing angle of the camera, the vertical frame length, and the second projection angle vector. A planar motion vector is obtained based on the horizontal motion vector and the vertical motion vector.

3. The method according to any one of claims 1 to 2, characterized in that, The method further includes: When the wearable device is detected to be in motion, the steps of obtaining the first angle data collected by the angle sensor at a first moment and the first image frame collected by the camera at the first moment are executed, and the second angle data collected by the angle sensor at a second moment and the second image frame collected by the camera at the second moment are executed.

4. The method according to any one of claims 1 to 2, characterized in that, After obtaining the position change vector of the wearable device based on the angle change vector and the comprehensive motion change vector, the method further includes: Motion detection is performed based on acceleration data collected by the accelerometer, the position change vector of the wearable device, and the angle change vector.

5. A motion detection device, characterized in that, The device is used in a wearable device, the wearable device including an angle sensor and a camera, and the device includes: The acquisition unit is used to acquire first angle data collected by the angle sensor at a first moment and first image frame collected by the camera at the first moment, and to acquire second angle data collected by the angle sensor at a second moment and second image frame collected by the camera at the second moment. The processing unit is configured to determine the angle change vector of the wearable device based on the first angle data and the second angle data, and to determine the comprehensive motion change vector of the wearable device based on the first image frame and the second image frame. The processing unit is further configured to obtain the position change vector of the wearable device based on the angle change vector and the comprehensive motion change vector. The position change vector of the wearable device is used for motion detection. The motion detection includes the detection of motion data, which includes step count data or running data. The processing unit determines the comprehensive motion change vector of the wearable device based on the first image frame and the second image frame, including: determining the position change vector of the first object based on the first position information of the first object in the first image frame and the second position information of the first object in the second image frame, and using the position change vector of the first object as the comprehensive motion change vector of the wearable device. The processing unit obtains the position change vector of the wearable device based on the angle change vector and the comprehensive motion change vector, including: obtaining a planar motion vector based on the angle change vector, and performing a vector subtraction operation between the comprehensive motion change vector and the planar motion vector to obtain the position change vector of the wearable device; the planar motion vector is the displacement vector of the same object in the first image frame and the second image frame due to the angle change.

6. A wearable device, characterized in that, The method includes a processor and a memory, the memory being used to store a computer program, the computer program including program instructions, and the processor being configured to invoke the program instructions to perform the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1 to 4.

Images