Human body detection method, human body detection device, and computer-readable storage medium

Abstract

The present application provides a human body detection method, a human body detection device and a computer readable storage medium. The method comprises: obtaining a plurality of image frames associated with a human body; detecting a plurality of joint coordinates in each image frame, and finding a plurality of specific image frames accordingly; obtaining an image area height corresponding to the human body in each specific image frame; obtaining a first joint coordinate of a first joint node in each specific image frame; obtaining a second joint coordinate of a second joint node in each specific image frame; and estimating an actual length between the first joint node and the second joint node based on the height of the human body, the image area height in each specific image frame, the first joint coordinate and the second joint coordinate. In this way, subsequent motion detection of the human body can be more accurate, thereby improving the detection effect.

Description

Technical Field

[0001] This invention relates to an image analysis technology, and more particularly to an image-based human body detection method, a human body detection device, and a computer-readable storage medium. Background Technology

[0002] In modern society, technologies that use specialized instruments to detect user motion are quite common. For example, existing technologies have proposed methods to obtain corresponding motion data by analyzing images of a user's movement process.

[0003] In some scenarios, estimating the actual lengths between certain joints of a user through image analysis techniques could further improve the accuracy of the obtained motion data and increase the diversity of the calculated motion data. Therefore, for those skilled in the art, designing a mechanism that can accurately estimate the actual lengths between certain joints using image analysis techniques is an important issue. Summary of the Invention

[0004] In view of the above, the present invention provides a human body detection method, a human body detection device, and a computer-readable storage medium, which can be used to solve the above-mentioned technical problems.

[0005] This invention provides a human body detection method, suitable for a human body detection device, comprising: acquiring multiple image frames associated with a human body; detecting multiple joint coordinates of the human body in each image frame, and thereby identifying multiple specific image frames in the multiple image frames, wherein the human body is determined to be in a specific posture in each specific image frame; acquiring the height of an image region corresponding to the human body in each specific image frame; acquiring the coordinates of a first joint point of the human body in each specific image frame; acquiring the coordinates of a second joint point of the human body in each specific image frame; and estimating an actual length between the first joint point and the second joint point based on the height of the human body, the height of the image region in each specific image frame, the first joint coordinate, and the second joint coordinate.

[0006] This invention provides a human body detection device, including an image acquisition circuit and a processor. The image acquisition circuit is used to acquire multiple image frames associated with a human body. The processor is coupled to the image acquisition circuit and configured to perform: detecting multiple joint coordinates of the human body in each image frame, and accordingly identifying multiple specific image frames in the multiple image frames, wherein the human body is determined to be in a specific posture in each specific image frame; obtaining the height of an image region corresponding to the human body in each specific image frame; obtaining a first joint coordinate of a first joint point of the human body in each specific image frame; obtaining a second joint coordinate of a second joint point of the human body in each specific image frame; and estimating an actual length between the first joint point and the second joint point based on the height of the human body, the height of the image region in each specific image frame, the first joint coordinate, and the second joint coordinate.

[0007] This invention provides a computer-readable storage medium recording an executable computer program, which is loaded by a human detection device to perform the following steps: detecting multiple joint coordinates of a human body in each image frame, and accordingly identifying multiple specific image frames in the multiple image frames, wherein the human body is determined to be in a specific posture in each specific image frame; obtaining the height of an image region corresponding to the human body in each specific image frame; obtaining a first joint coordinate of a first joint point of the human body in each specific image frame; obtaining a second joint coordinate of a second joint point of the human body in each specific image frame; and estimating an actual length between the first joint point and the second joint point based on the height of the human body, the height of the image region in each specific image frame, the first joint coordinate, and the second joint coordinate. Attached Figure Description

[0008] The accompanying drawings are included to further illustrate the invention, and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0009] Figure 1 This is a schematic diagram of a human body detection device illustrated according to an embodiment of the present invention.

[0010] Figure 2 This is a flowchart illustrating a human body detection method based on an embodiment of the present invention.

[0011] Figures 3A to 3C This is a schematic diagram illustrating the analysis of image frames according to an embodiment of the present invention.

[0012] Figure 4 This is a flowchart illustrating the determination of a human body performing a specific action, based on an embodiment of the present invention.

[0013] Figure 5 This is an application scenario diagram illustrated according to an embodiment of the present invention.

[0014] Figure 6 It is based on Figure 5 A schematic diagram illustrating posture change signals.

[0015] Figure 7 This is a flowchart illustrating a method for obtaining two-dimensional motion velocity according to an embodiment of the present invention.

[0016] Figure 8 This is a flowchart illustrating the estimation of three-dimensional motion velocity according to an embodiment of the present invention. Detailed Implementation

[0017] Reference will now be made in detail to exemplary embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same component symbols are used in the drawings and description to denote the same or similar parts.

[0018] Please refer to Figure 1 This is a schematic diagram of a human body detection device according to an embodiment of the present invention. In different embodiments, the human body detection device 100 can be implemented as various electronic devices, such as various smart devices / computer devices, including but not limited to smartphones, tablet computers, laptops, smart glasses, cameras, etc.

[0019] exist Figure 1 In this device, the human body detection device 100 includes an image-capturing circuit 102 and a processor 104. The image-capturing circuit 102 can be any camera with a charge-coupled device (CCD) camera, a complementary metal oxide semiconductor transistor (CMOS) camera, or an infrared camera.

[0020] Processor 104 is coupled to image-capturing circuit 102 and may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor, multiple microprocessors, one or more microprocessors incorporating a digital signal processor core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), any other type of integrated circuit, a state machine, an processor based on an advanced RISC machine (ARM), and the like.

[0021] In embodiments of the present invention, processor 104 can access specific modules and program code to implement the human body detection method proposed in this invention. In general, the method of the present invention can be used to estimate the actual length between a first joint point (e.g., wrist joint) and a second joint point (e.g., shoulder joint) on a human body A, thereby facilitating subsequent motion detection (e.g., push-ups). In other embodiments, the actual length between other first and second joint points can be estimated for other motion detection, such as the length between the hip or shoulder joint and the ankle joint for squats, the length between the shoulder joint and the hip joint for sit-ups, and the length between the elbow joint and the shoulder joint for pull-ups, but is not limited to these. Details of the method of the present invention will be described below.

[0022] Please refer to Figure 2 This is a flowchart illustrating a human body detection method according to an embodiment of the present invention. The method of this embodiment can be derived from... Figure 1 The human body detection device 100 is executed, and the following is paired with Figure 1 Component description shown Figure 2 Details of each step.

[0023] First, in step S210, the image capturing circuit 102 acquires multiple image frames associated with human body A. In one embodiment, human body A is, for example, the body of a user to be detected by the human body detection device 100, and the image capturing circuit 102 may, for example, capture several images of human body A as the multiple image frames according to the corresponding image capturing frame rate, but is not limited thereto.

[0024] In one embodiment, the human detection device 100 may be fixedly positioned at a certain location, and the user may move to the current imaging range of the imaging circuit 102 so that the imaging circuit 102 can acquire the plurality of image frames, but it is not limited to this.

[0025] Subsequently, in step S220, the processor 104 detects multiple joint coordinates of human body A in each image frame, and finds multiple specific image frames in the multiple image frames accordingly.

[0026] In one embodiment, the processor 104 may input one of the plurality of image frames (hereinafter referred to as the first image frame) into at least one pre-trained human detection model (such as various neural network / deep learning models), wherein the at least one human detection model can detect the plurality of joint coordinates of human body A in the first image frame, and output a probability that human body A is in a specific pose in the first image frame. To make the above concepts easier to understand, the following further explains... Figures 3A to 3C To further explain, among which Figures 3A to 3C This is a schematic diagram illustrating the analysis of image frames according to an embodiment of the present invention.

[0027] exist Figure 3A In this context, assuming image frame 310 is one of multiple image frames acquired by image acquisition circuit 102 from human body A, processor 104 can, for example, detect the skeleton of human body A (represented by dots marked in image frame 310 and line segments connecting the dots) using a first human body detection model, and accordingly obtain the coordinates of the joints on this skeleton (i.e., the dots on the skeleton) in image frame 310. In one embodiment, the aforementioned first human body detection model can be implemented as MediaPipe Pose, Openpose, or other similar pose detection models, but is not limited to these.

[0028] The processor 104 can then input the aforementioned joint coordinates into a second human detection model, which can, for example, output the probability that human A is in a specific pose in image frame 310. For ease of explanation, the specific pose considered below is assumed to be the natural standing pose of human A as shown in Figure 3, but it is not limited to this.

[0029] exist Figure 3B In this context, assuming the joint coordinates obtained by the first human detection model are as shown in skeleton diagram 320, the second human detection model can, for example, determine the probability that the human body corresponding to skeleton diagram 320 is in a specific posture (e.g., a natural standing posture) based on the relative positions between these joint coordinates.

[0030] In one embodiment, to enable the second human detection model to possess the aforementioned capabilities, during the training process of the second human detection model, the designer can feed specially designed training data into the second human detection model to allow it to learn accordingly. For example, after obtaining a set of joint coordinates (e.g., taken from a skeletal diagram of a human body in a specific posture) labeled as corresponding to a specific posture (e.g., a natural standing posture), the processor 104 can generate corresponding feature vectors based on the relative positions between these joint coordinates and feed them into the second human detection model. This allows the second human detection model to learn relevant features about a specific posture from these feature vectors. In this case, when the second human detection model receives a feature vector corresponding to a certain human posture in the future, it can accordingly determine the probability that this human posture is a specific posture, but this is not limited to this.

[0031] In one embodiment, in response to determining that the probability output by the second human detection model in response to the first image frame is higher than a probability threshold (e.g., 60%), the processor 104 may determine that the first image frame belongs to the plurality of specific image frames. On the other hand, in response to determining that the probability output by the second human detection model in response to the first image frame is not higher than the probability threshold, the processor 104 may determine that the first image frame does not belong to the plurality of specific image frames.

[0032] In short, the processor 104 can identify one or more image frames that include a human body in a specific posture as the specific image frame, but it is not limited to this.

[0033] In one embodiment, the processor 104 may maintain a queue (N is a positive integer) comprising N storage locations, wherein the first to Nth storage locations of this queue may be used to store indicators for the i-th to (i+N-1)-th image frames, respectively. In one embodiment, the indicators may be used, for example, to indicate whether the corresponding image frame is a specific image frame.

[0034] For example, if the processor 104 determines that the i-th image frame belongs to a specific image frame, the processor 104 may, for example, set the indicator at the first storage location in the queue to a first value (e.g., 1) to indicate that the i-th image frame is one of the specific image frames. On the other hand, if the processor 104 determines that the i-th image frame does not belong to a specific image frame, the processor 104 may, for example, set the indicator at the first storage location in the queue to a second value (e.g., 0) to indicate that the i-th image frame is not a specific image frame.

[0035] For another example, if the processor 104 determines that the (i+N-1)th image frame belongs to a specific image frame, the processor 104 may, for example, set the indicator at the Nth storage location in the queue to a first value (e.g., 1) to indicate that the (i+N-1)th image frame is one of the specific image frames. On the other hand, if the processor 104 determines that the (i+N-1)th image frame does not belong to a specific image frame, the processor 104 may, for example, set the indicator at the Nth storage location in the queue to a second value (e.g., 0) to indicate that the (i+N-1)th image frame is not a specific image frame.

[0036] After obtaining one or more specific image frames from the plurality of image frames, in step S230, the processor 104 obtains the image region height of human body A in each specific image frame.

[0037] exist Figure 3CIn this embodiment, assuming that human body A in image frame 310 is determined to be in a specific posture, processor 104 can determine image frame 310 as a specific image frame. In one embodiment, processor 104 can use a bounding box 330 to outline the image region of human body A in image frame 310, and then use the height of this bounding box 330 as the height of the image region of human body A in image frame 310. Based on a similar concept, processor 104 can define corresponding bounding boxes in each specific image frame and determine the height of the image region corresponding to each specific image frame accordingly. In one embodiment, assuming that the number of specific image frames is K (for example, K from the i-th image frame to the (i+N-1)-th image frame belong to specific image frames), processor 104 can obtain a total of K image region heights, but it is not limited to this.

[0038] As previously mentioned, the method of the present invention can be used to determine the actual length between the first joint point and the second joint point on human body A. The designer can select any two joint points on human body A as the first joint point and the second joint point as needed, but is not limited to this.

[0039] Therefore, in step S240, the processor 104 obtains the first joint coordinates of the first joint point of human body A in each specific image frame. For example, assuming the first joint point under consideration is the wrist joint of human body A, the processor 104 can obtain the joint coordinates of the wrist joint of human body A in each specific image frame as the aforementioned first joint coordinates. Assuming the number of specific image frames is K, the processor 104 can obtain a total of K first joint coordinates, but it is not limited to this.

[0040] Additionally, in step S250, the processor 104 obtains the second joint coordinates of the second joint point of human body A in each specific image frame. For example, assuming the second joint point under consideration is the shoulder joint of human body A, the processor 104 can obtain the joint coordinates of the shoulder joint of human body A in each specific image frame as the aforementioned second joint coordinates. Assuming the number of specific image frames is K, the processor 104 can obtain a total of K second joint coordinates, but it is not limited to this.

[0041] Subsequently, in step S260, the processor 104 estimates the actual length between the first joint point and the second joint point based on the height of the human body A, the height of the image region in each specific image frame, the first joint coordinate, and the second joint coordinate.

[0042] In one embodiment, between steps S220 and S260, the processor 104 may first determine, through a certain mechanism, whether the human body A has been stably in a specific posture (e.g., stably in a natural standing posture). For example, the processor 104 may first obtain a specific proportion of the aforementioned specific image frame in the aforementioned image frame, and determine whether this specific proportion is higher than a proportion threshold (e.g., 80%).

[0043] In one embodiment, the processor 104 may determine the specific ratio based on the contents of the queue. For example, the processor 104 may estimate the specific ratio by dividing the number of indicators in the queue that have a first value by the length of the queue (i.e., N), but is not limited to this.

[0044] In one embodiment, in response to determining that a specific ratio is higher than a ratio threshold, the processor 104 may continue to execute step S260. On the other hand, in response to determining that a specific ratio is not higher than a ratio threshold, the processor 104 may re-determine the specific image frame.

[0045] In one embodiment, during the process of re-determining a specific image frame, the processor 104 can use the (i+1)th to (i+N)th image frames as multiple image frames considered in step S210, and execute step S220 again to find the specific image frame among the (i+1)th to (i+N)th image frames. Then, the processor 104 can determine whether the current specific ratio is higher than a ratio threshold. If so, steps S220 to S260 can be executed; otherwise, the above-described re-determination of the specific image frame can be performed again, which will not be elaborated further here. In one embodiment, if the specific ratio of the (i+1)th to (i+N)th image frames is determined not to be higher than the ratio threshold, the processor 104 can use the (i+2)th (i+1+1)th to (i+N+1)th image frames as multiple image frames considered in step S210 to re-determine whether the specific ratio of the specific image frame is not higher than the ratio threshold.

[0046] In short, before executing step S260, the processor 104 can first determine whether human body A has been stably in a specific posture. If so, step S260 can be executed. Otherwise, the above-mentioned behavior of re-determining the specific image frame needs to be repeated until it is determined that human body A has been stably in a specific posture, but it is not limited to this.

[0047] In one embodiment, the height of human body A is, for example, the actual height of human body A in the real world. It can be provided to the human body detection device 100 by the user to be tested, or input to the human body detection device 100 by relevant personnel on behalf of the user, or obtained by the human body detection device 100 from the database based on user data, or obtained by the human body detection device 100 using specific camera hardware and height calculation program, but it is not limited to these.

[0048] In one embodiment, during the execution of step S260, the processor 104 may be configured to perform: obtaining an average height of the image region height in each specific image frame (e.g., the average of the K image region heights); obtaining a first average coordinate of the first joint coordinate in each specific image frame (e.g., the average coordinate of the K first joint coordinates); obtaining a second average coordinate of the second joint coordinate in each specific image frame (e.g., the average coordinate of the K second joint coordinates); and estimating the actual length between the first joint point and the second joint point based on the height, average height, first average coordinate, and second average coordinate of human body A.

[0049] In one embodiment, the actual length between the first joint and the second joint can be characterized as Where P1 av P2 is the first average coordinate. av H is the second average coordinate. av Let UH be the average height, and D(P1) be the height of body A. av P2 av ) represents the distance between the first average coordinate and the second average coordinate (e.g., Euclidean distance).

[0050] As can be seen from the above, the method of the present invention can estimate the actual length between the first and second joint points after determining that human body A has been stably positioned in a specific posture (e.g., a natural standing posture or other postures required by the designer). This allows for more accurate subsequent motion detection of the human body, thereby improving the detection effect.

[0051] In one embodiment, the actual length between the first and second joints obtained by the method taught in the above embodiments can be used, for example, to estimate the three-dimensional motion speed of human body A when performing a specific action, as will be further explained below.

[0052] In one embodiment, to achieve the above objective, the present invention further provides a method for a human body detection device 100 to determine whether a human body A has performed a specific action once (repetition). For ease of explanation, the specific action under consideration will be assumed to be a push-up, but the embodiments of the present invention are not limited thereto. In other embodiments, those skilled in the art should be able to understand the detection operations corresponding to other specific actions based on the following teachings, such as squats, sit-ups, crunches, pull-ups, and other equipment-based or bodyweight training exercises.

[0053] Please refer to Figure 4 This is a flowchart illustrating the determination of a human body performing a specific action, based on an embodiment of the present invention. The method of this embodiment can be derived from… Figure 1 The human body detection device 100 is executed, and the following is paired with Figure 1 Component description shown Figure 4 Details of each step. Additionally, to ensure... Figure 4 The concept is easier to understand; additional information is provided below. Figure 5 The content is explained, including Figure 5 This is an application scenario diagram illustrated according to an embodiment of the present invention.

[0054] In this embodiment, the processor 104 can first determine whether human body A is in a ready posture to perform a specific action. Figure 5 In this scenario, since the specific action under consideration is a push-up, the preparatory posture is, for example, the posture shown in image frame 510. In one embodiment, processor 104 can determine, using any known image recognition method, that human body A in image frame 510 has assumed the preparatory posture for a specific action. In another embodiment, processor 104 can first determine the preparatory posture of human body A, then determine the corresponding specific action based on the determined preparatory posture, and finally implement the method and steps corresponding to this specific action based on the determined specific action. For example, in Figure 5 In this scenario, after determining that the user is performing the action shown in the image frame 510, the processor 104 can determine that the specific action the user intends to perform is a push-up based on the action shown in the image frame 510, and then implement the corresponding push-up follow-up method and steps.

[0055] Therefore, in step S410, in response to the determination that human body A is in a ready posture to perform a specific action, processor 104 detects the first joint point J1, the second joint point J2, and the third joint point J3, as shown in image frame 520. In one embodiment, processor 104 may detect the first joint point J1, the second joint point J2, and the third joint point J3 within image frame 520 based on the previously mentioned technical means, but is not limited to this.

[0056] In one embodiment, the first joint J1 is connected to the second joint J2 via a third joint J3. Figure 5In this scenario, the first joint J1, the second joint J2, and the third joint J3 are, for example, the wrist joint, the shoulder joint, and the elbow joint, respectively. The position of the first joint J1 is substantially fixed during the process of human body A performing a specific action. That is, the position of the first joint J1 does not change as human body A performs the specific action (i.e., a push-up). Based on this, the present invention can determine whether human body A has left the preparatory posture for a specific action based on the position of the first joint J1.

[0057] Specifically, after detecting the first joint point J1, the processor 104 executes step S420 to define a fixed range R1 based on the position of the first joint point J1, wherein the position and size of the fixed range R1 are fixed after definition. In one embodiment, the processor 104 may determine the position of R1 by taking the position of the first joint point J1 as the center, and then determine the fixed range R1 based on a certain radius (e.g., the length of the palm of human body A or the height of human body A, or R1 is a fixed radius), but it is not limited to this.

[0058] In this embodiment, since the exercise is a push-up, the wrist joint is used as the first joint point J1 to define the fixed range R1. In other embodiments, if the exercise is a squat, the ankle joint can be used as the first joint point J1 to define the fixed range R1; if the exercise is a pull-up, the wrist joint can be used as the first joint point J1 to define the fixed range R1; if the exercise is a sit-up or crunch, the hip joint can be used as the first joint point J1 to define the fixed range R1, but it is not limited to these.

[0059] Then, in step S430, the processor 104 determines whether the first joint point J1 has left the fixed range R1. As previously mentioned, the position of the first joint point J1 is fixed during the process of human body A performing a specific action. Therefore, when the first joint point J1 is determined to have left the fixed range R1, the processor 104 can execute step S440 to determine that human body A has left the ready posture (e.g., temporarily not performing the specific action).

[0060] On the other hand, if the processor 104 determines that the first joint point J1 has not left the fixed range R1, the processor 104 can execute step S450 to obtain the initial value (e.g., 170 degrees) of the joint angle A1 of the third joint point J3. Figure 5 In this scenario, the joint angle A1 of the third joint point J3 (i.e., the elbow joint) can vary depending on the specific movement performed by human A. For example, when human A performs the eccentric phase (i.e., the descent) of a push-up, the joint angle A1 of the third joint point J3 can gradually decrease from the initial value mentioned above, while when human A performs the concentric phase (i.e., the ascent) of a push-up, the joint angle A1 of the third joint point J3 will increase accordingly.

[0061] Therefore, the method of the present invention can determine whether human body A has performed a specific action based on the change in joint angle A1 of the third joint point J3. In one embodiment, the processor 104 can determine whether the joint angle A1 of the third joint point J3 has changed from an initial value to a value less than an angle threshold and then returned to the initial value.

[0062] exist Figure 5 In this scenario, the change in joint angle A1 of the third joint point J3 in each image frame can be represented by the corresponding progress bar. For example, in image frame 520, assuming that the joint angle A1 of the third joint point J3 is the initial value, the progress bar B1 in image frame 520 can be displayed as 0%, to indicate that the joint angle A1 of the third joint point J3 has not changed.

[0063] Furthermore, during the eccentric phase of a specific movement, the progress bar B1 will increase as the joint angle A1 of the third joint point J3 gradually approaches the aforementioned angle threshold (e.g., 90 degrees). When the joint angle A1 of the third joint point J3 becomes no greater than the aforementioned angle threshold, the progress bar B1 may, for example, be displayed as 100%, as shown in image frame 530.

[0064] Furthermore, during the centripetal phase of a specific movement, the progress bar B1 will decrease as the joint angle A1 of the third joint point J3 gradually approaches its initial value. When the joint angle A1 of the third joint point J3 returns to a value no less than the initial value, the progress bar B1 may, for example, return to 0%, as shown in image frame 540.

[0065] As can be seen from the above, during the process of human body A performing a specific action (such as performing a push-up), the joint angle A1 of the third joint point J3 will change from its initial value to a value greater than the angle threshold and then return to its initial value. Based on this, in step S460, in response to the determination that the joint angle A1 of the third joint point J3 changes from its initial value to a value greater than the angle threshold and then returns to its initial value, the processor 104 can correspondingly determine that human body A has performed a specific action once and can return to step S430.

[0066] Based on the above principles, the processor 104 can estimate the number of times that human body A performs a specific action from the time it assumes a ready posture to the time it leaves the ready posture, as the number of times human body A performs a specific set of actions.

[0067] exist Figure 5In this scenario, assuming human body A has completed the required number of repetitions, human body A can stand up accordingly. In this case, processor 104 can determine that the human body has left the ready position by detecting that the position of the first joint J1 has moved away from the fixed range R1, as shown in image frames 550 and 560. Afterwards, processor 104 can determine that human body A has completed one set and record it accordingly, but this is not limited to this. In one embodiment, the number of sets of movements can be calculated and recorded by determining the number of times the human body enters and leaves the ready position.

[0068] In addition, by Figure 5 It can be seen that during the process of human body A performing a specific action, the distance between the second joint point J2 (i.e., the shoulder joint) and a certain reference position will change accordingly. In one embodiment, the reference position is, for example, the top of the image frame, the bottom of the image frame, R1, the user's ankle, etc., but is not limited to these.

[0069] In one embodiment, in response to determining that human body A is in a ready posture to perform a specific action, processor 104 can detect the change in distance between the second joint point J2 and the reference position.

[0070] For example, in image frame 520, when human body A is in a ready posture, processor 104 can detect the distance between the second joint point J2 and a reference position (e.g., the top of image frame 520). At this time, the distance between the second joint point J2 and the top of image frame 520 is relatively close. During the eccentric phase of human body A performing a specific action, the distance between the second joint point J2 and the top of the image frame will gradually increase. Conversely, during the centripetal phase of human body A performing a specific action, the distance between the second joint point J2 and the top of the image frame will gradually decrease. Based on this, processor 104 can record the change in distance between the second joint point J2 and the top of the image frame as a posture change signal when human body A performs a specific action. Subsequently, processor 104 can perform other analyses / processes based on this posture change signal, but is not limited to this.

[0071] Please refer to Figure 6 It is based on Figure 5 A schematic diagram illustrating the posture change signal. In this embodiment, the posture change signal 700 is illustrated, for example... Figure 5 The change in distance between the second joint point J2 and the top of the image frame. Figure 6 In this context, suppose human body A performs a specific action (e.g., 5 push-ups) a total of 5 times. Since processor 104 can determine the process of human body A performing each specific action based on previous teaching, processor 104 can correspondingly extract the posture change signal segment corresponding to each specific action from posture change signal 600.

[0072] For example, in Figure 6In this scenario, assuming the processor 104 determines that the first to 180th image frames correspond to the process of human body A performing a first specific action, the processor 104 can extract the portion of the posture change signal 600 corresponding to the first to 180th image frames as posture change signal segment 611 corresponding to the first specific action. As another example, assuming the processor 104 determines that the first to 280th image frames correspond to the process of human body A performing a second specific action, the processor 104 can extract the portion of the posture change signal 600 corresponding to the first to 280th image frames as posture change signal segment 612 corresponding to the second specific action. Based on a similar principle, the processor 104 can extract posture change signal segments 613 to 615 corresponding to the third, fourth, and fifth specific actions from the posture change signal 600, but is not limited to this.

[0073] Subsequently, the processor 104 can estimate the corresponding two-dimensional motion velocity based on each pose change signal segment 613-615. In one embodiment, the processor 104 can estimate the corresponding two-dimensional motion velocity based on each pose change signal segment 613-615. Figure 7 The method described above is used to obtain the two-dimensional motion velocity.

[0074] Please refer to Figure 7 This is a flowchart illustrating a method for obtaining two-dimensional motion velocity according to an embodiment of the present invention. The method of this embodiment can be derived from... Figure 1 The human body detection device 100 is executed, and the following is paired with Figure 1 Component description shown Figure 7 Details of each step.

[0075] First, in step S710, in response to the determination that human body A has performed a specific action, processor 104 obtains the posture change signal segment of human body A performing this specific action. For ease of explanation, the posture change signal under consideration is assumed below to be... Figure 6 The pose change signal segment 614 (which corresponds to a specific action in the fourth instance) is used, but is not limited to this. In other embodiments, those skilled in the art should be able to understand the operations performed by the processor 104 based on other pose change signal segments based on the following teachings.

[0076] In step S720, the processor 104 identifies a first extremum and a second extremum in the posture change signal segment 614. In one embodiment, during the execution of step S720, the processor 104 may identify multiple extrema 614a to 614e in the posture change signal segment 614. In one embodiment, the processor 104 may identify the extrema 614a to 614e, for example, by taking the second derivative of the posture change signal segment 614, but is not limited to this method.

[0077] Subsequently, the processor 104 can identify the first extreme value and the second extreme value from the extreme values ​​614a to 614e. In one embodiment, the absolute difference between the j-th extreme value and the (j+1)-th extreme value in the determination of extreme values ​​614a to 614e is greater than a preset threshold value, and the change between the j-th extreme value and the (j+1)-th extreme value corresponds to the centripetal phase of a specific action. In this push-up embodiment, this means that the distance between the (j+1)-th extreme value and the top of the image frame is less than the distance between the j-th extreme value and the top of the image frame. The processor 104 can determine that the j-th extreme value and the (j+1)-th extreme value are the first extreme value and the second extreme value, respectively, where j is an index value.

[0078] In different embodiments, the preset threshold value can be set by the designer to a sufficiently large value as needed, or it can be automatically set based on the height or actual height of the user's body bounding box 330, or based on the coordinate distance or actual length (e.g., the arm length between the shoulder joint and wrist joint) between the user's first and second joints, but it is not limited to these. In the posture change signal segment 614, assuming that the absolute difference between the extreme values ​​614d and 614e is greater than the preset threshold value, the processor 104 can determine that the extreme values ​​614d and 614e are the first extreme value and the second extreme value in the posture change signal segment 614, respectively.

[0079] After obtaining the first and second extreme values, in step S730, the processor 104 estimates the two-dimensional motion velocity based on the first and second extreme values. In one embodiment, the processor 104 may obtain a first frame and a second frame corresponding to the first and second extreme values, respectively; obtain the frame number difference between the first and second frames, and estimate the time difference between the first and second extreme values ​​based on a frame rate and this frame number difference; obtain an absolute difference between the first and second extreme values, and estimate the two-dimensional motion velocity based on this absolute difference and the aforementioned time difference.

[0080] exist Figure 6 In this context, extreme values ​​614d and 614e correspond, for example, to the 170th and 185th frames of the pose change signal segment 614, respectively. Therefore, processor 104 can consider the 170th and 185th frames of the pose change signal segment 614 as the first and second frames under consideration, respectively. Then, processor 104 can obtain the frame difference (e.g., 15 frames) between these first and second frames. Next, processor 104 can estimate the time difference between extreme values ​​614d and 614e based on a frame rate (e.g., the imaging frame rate of the imaging circuit 102) and this frame difference.

[0081] In one embodiment, the processor 104 may, for example, divide the frame rate difference to estimate the time difference between the extreme values ​​614d and 614e. Then, the processor 104 may obtain an absolute difference between the extreme values ​​614d and 614e, and estimate the two-dimensional motion velocity based on this absolute difference and the time difference. In one embodiment, the processor 104 may, for example, divide the absolute difference between the extreme values ​​614d and 614e by the time difference to estimate the two-dimensional motion velocity corresponding to the attitude change signal segment 614.

[0082] In one embodiment, after obtaining the two-dimensional motion velocity corresponding to the pose change signal segment 614, the processor 104 can further estimate the three-dimensional motion velocity corresponding to the pose change signal segment 614. Further details will follow. Figure 8 Further explanation is needed.

[0083] Please refer to Figure 8 This is a flowchart illustrating the estimation of three-dimensional motion velocity according to an embodiment of the present invention. The method of this embodiment can be derived from… Figure 1 The human body detection device 100 is executed, and the following is paired with Figure 1 Component description shown Figure 8 Details of each step.

[0084] In one embodiment, Figure 8 The method can be found Figure 1 The process is executed after step S260. In this embodiment, it is assumed that the specific action under consideration is as follows: Figure 5 The push-up shown is an example, but it is not limited to this.

[0085] First, in step S810, the processor 104 determines whether human body A has performed a specific action. If so, the processor 104 then proceeds to step S820 to obtain the two-dimensional motion speed of the human body performing this specific action. In this embodiment, the details of steps S810 and S820 can be found in the description of the previous embodiments, and will not be repeated here.

[0086] Furthermore, for ease of understanding, the following assumes that the two-dimensional motion velocity obtained in step S820 corresponds to... Figure 6 The two-dimensional motion velocity of the attitude change signal segment 614, but not limited to this.

[0087] Therefore, in step S830, the processor 104 obtains the first initial coordinates and the second initial coordinates of the first joint point J1 and the second joint point J2 when the human body A performs this specific action. For example, the processor 104 can obtain the first initial coordinates and the second initial coordinates of the first joint point J1 and the second joint point J2 in the corresponding image frame when the human body A performs the fourth specific action.

[0088] Subsequently, in step S840, the processor 104 estimates the three-dimensional motion speed of human body A performing a specific action based on the first initial coordinates, the second initial coordinates, the two-dimensional motion speed, and the actual length mentioned above.

[0089] In one embodiment, the three-dimensional motion velocity (in v) corresponding to the pose change signal segment 614 3D (Representation) For example, it can be characterized as Where v 2D For example, the two-dimensional motion velocity corresponding to the posture change signal segment 614, where L is the distance between the first initial coordinate and the second initial coordinate. 1,2 This is the actual length between the first joint J1 and the second joint J2.

[0090] In other embodiments, the processor 104 may estimate the two-dimensional and three-dimensional motion velocities corresponding to other pose change signal segments 611, 612, 614, 615 in a manner taught above, but is not limited thereto.

[0091] In other embodiments, the processor 104 may further estimate the energy consumed by the user, remaining physical strength, etc., based on the estimated two-dimensional and three-dimensional motion velocities of the posture change, with reference to other physiological parameters and / or other values ​​input by the user (e.g., weight training weight), but is not limited thereto.

[0092] The present invention also provides a computer-readable storage medium for performing a human detection method. This computer-readable storage medium comprises a plurality of program instructions (such as setup and deployment program instructions). These program instructions can be loaded into the human detection device 100 and executed to perform the aforementioned human detection method and the functions of the human detection device 100.

[0093] In summary, the method of the present invention can estimate the actual length between the first and second joint points after determining that the human body has stabilized in a specific posture (e.g., a natural standing posture or other postures required by the designer). This allows for more accurate subsequent motion detection of the human body (e.g., three-dimensional motion velocity during each specific action), thereby improving the detection effect.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A human body detection method, suitable for human body detection devices, characterized in that, include: Obtain multiple image frames associated with the human body; Multiple joint coordinates of the human body are detected in each of the image frames, and multiple specific image frames are identified in the multiple image frames, wherein the human body is determined to be in a specific posture in each of the specific image frames; the specific posture is a natural standing posture. Obtain the height of the image region corresponding to the human body in each of the specific image frames; Obtain the first joint coordinates of the first joint point of the human body in each of the specific image frames; Obtain the second joint coordinates of the human body in each of the specific image frames; The actual length between the first joint point and the second joint point is estimated based on the height of the human body, the height of the image region in each specific image frame, the first joint coordinate, and the second joint coordinate. Determine the specific action performed by the human body once, and obtain the two-dimensional motion velocity of the specific action performed by the human body once; The first initial coordinates and the second initial coordinates of the first joint point and the second joint point are obtained respectively when the human body performs the specific action. The three-dimensional motion speed of the human body performing the specific action is estimated based on the first initial coordinates, the second initial coordinates, the two-dimensional motion speed, and the actual length. The step of obtaining the two-dimensional motion speed of the human body performing the specific action includes: Obtain the posture change signal segment of the human body performing the specific action; Find the first and second extreme values ​​in the posture change signal segment; The two-dimensional motion velocity is estimated based on the first extreme value and the second extreme value.

2. The method of claim 1, wherein the step of detecting the plurality of joint coordinates of the human body in each of the image frames and thereby identifying the plurality of specific image frames in the plurality of image frames comprises: The first image frame of the plurality of image frames is input into at least one pre-trained human detection model, wherein the at least one human detection model detects the plurality of joint coordinates of the human body in the first image frame and outputs the probability that the human body is in the specific pose in the first image frame. In response to the determination that the probability is higher than a probability threshold, the first image frame is determined to belong to the plurality of specific image frames; In response to the determination that the probability is not higher than the probability threshold, it is determined that the first image frame does not belong to the plurality of specific image frames.

3. The method according to claim 1, prior to the step of estimating the actual length between the first joint point and the second joint point based on the height of the human body, the height of the image region in each of the specific image frames, the first joint coordinates, and the second joint coordinates, the method further includes: Obtain the specific proportion that the plurality of specific image frames occupy in the plurality of image frames; In response to the determination that the specific ratio is higher than the ratio threshold, the actual length between the first joint point and the second joint point is estimated based on the height of the human body, the height of the image region in each of the specific image frames, the first joint coordinates, and the second joint coordinates.

4. The method according to claim 3, wherein the plurality of image frames includes the i-th image frame to the (i+N-1)-th image frames, and the method further includes: In response to the determination that the specific ratio is not higher than the ratio threshold, the multiple specific image frames are re-determined based on the (i+1)th to (i+N)th image frames, where N is a positive integer.

5. The method of claim 1, wherein the step of estimating the actual length between the first joint point and the second joint point based on the height of the human body, the height of the image region in each of the specific image frames, the first joint coordinates, and the second joint coordinates comprises: Obtain the average height of the image region in each of the specific image frames; Obtain the first average coordinate of the first joint coordinate in each of the specific image frames; Obtain the second average coordinate of the second joint coordinate in each of the specific image frames; The actual length between the first joint point and the second joint point is estimated based on the height of the human body, the average height, the first average coordinate, and the second average coordinate.

6. The method according to claim 5, wherein the actual length between the first joint and the second joint is characterized as: ， in The first average coordinate. The second average coordinate. Where UH is the average height, and UH is the height of the human body. The distance between the first average coordinate and the second average coordinate is denoted as .

7. The method of claim 1, wherein the step of obtaining the posture change signal segment of the human body performing the subspecific action comprises: In response to determining that the human body is in a ready posture to perform the specific action, the change in distance between the second joint point and the reference position is detected; In response to the determination that the human body is performing the specific action, the corresponding distance change is obtained as the posture change signal segment of the human body performing the specific action.

8. The method of claim 1, wherein the step of finding the first extreme value and the second extreme value in the posture change signal segment comprises: Identify multiple extreme values ​​within the posture change signal segment; The reaction is based on the determination that the absolute difference between the j-th extreme value and the (j+1)-th extreme value among the plurality of extreme values ​​is greater than a preset threshold value, and the change between the j-th extreme value and the (j+1)-th extreme value corresponds to the centripetal phase of the specific action. The j-th extreme value and the (j+1)-th extreme value are determined to be the first extreme value and the second extreme value, respectively, where j is an index value.

9. The method of claim 1, wherein the step of estimating the two-dimensional motion velocity based on the first extreme value and the second extreme value comprises: Obtain the first frame and the second frame corresponding to the first extreme value and the second extreme value, respectively; Obtain the frame number difference between the first frame and the second frame, and estimate the time difference between the first extreme value and the second extreme value based on the frame rate and the frame number difference; Obtain the absolute difference between the first extreme value and the second extreme value, and estimate the two-dimensional motion velocity based on the absolute difference and the time difference.

10. The method according to claim 1, wherein the three-dimensional motion velocity is characterized as: ， in Let L be the two-dimensional motion velocity, and L be the distance between the first initial coordinate and the second initial coordinate. The actual length between the first joint and the second joint.

11. The method according to claim 1, further comprising: In response to determining that the human body is in a ready posture to perform the specific action, the system detects the first joint point, the second joint point, and the third joint point, wherein the first joint point is connected to the second joint point through the third joint point, and the joint angle of the third joint point changes in response to the human body performing the specific action. Obtain the initial value of the joint angle at the third joint point; The reaction is that the joint angle of the third joint point changes from the initial value to no greater than the angle threshold value and then returns to the initial value, and it is determined that the human body has performed the specific action.

12. The method of claim 11, wherein after the step of detecting the first joint, the second joint, and the third joint, the method further comprises: A fixed range is defined based on the position of the first joint point, wherein the position of the first joint point is fixed during the process of the human body performing the specific action; The reaction is to determine that the first joint point has left the fixed range, and to determine that the human body has left the ready posture.

13. A human body detection device, characterized in that, include: Image acquisition circuit, which is used to acquire multiple image frames associated with the human body; A processor, coupled to the image-capturing circuit, and configured to execute: Multiple joint coordinates of the human body are detected in each of the image frames, and multiple specific image frames are identified in the multiple image frames, wherein the human body is determined to be in a specific posture in each of the specific image frames; the specific posture is a natural standing posture. Obtain the height of the image region corresponding to the human body in each of the specific image frames; Obtain the first joint coordinates of the first joint point of the human body in each of the specific image frames; Obtain the second joint coordinates of the human body in each of the specific image frames; The actual length between the first joint point and the second joint point is estimated based on the height of the human body, the height of the image region in each specific image frame, the first joint coordinate, and the second joint coordinate. Determine the specific action performed by the human body once, and obtain the two-dimensional motion velocity of the specific action performed by the human body once; The first initial coordinates and the second initial coordinates of the first joint point and the second joint point are obtained respectively when the human body performs the specific action. The three-dimensional motion speed of the human body performing the specific action is estimated based on the first initial coordinates, the second initial coordinates, the two-dimensional motion speed, and the actual length. The step of obtaining the two-dimensional motion speed of the human body performing the specific action includes: Obtain the posture change signal segment of the human body performing the specific action; Find the first and second extreme values ​​in the posture change signal segment; The two-dimensional motion velocity is estimated based on the first extreme value and the second extreme value.

14. A computer-readable storage medium, characterized in that, The computer-readable storage media records an executable computer program, which is loaded by the human detection device to perform the following steps: Obtain multiple image frames associated with the human body; Multiple joint coordinates of the human body are detected in each of the image frames, and multiple specific image frames are identified in the multiple image frames, wherein the human body is determined to be in a specific posture in each of the specific image frames; the specific posture is a natural standing posture. Obtain the height of the image region corresponding to the human body in each of the specific image frames; Obtain the first joint coordinates of the first joint point of the human body in each of the specific image frames; Obtain the second joint coordinates of the human body in each of the specific image frames; The actual length between the first joint point and the second joint point is estimated based on the height of the human body, the height of the image region in each specific image frame, the first joint coordinate, and the second joint coordinate. Determine the specific action performed by the human body once, and obtain the two-dimensional motion velocity of the specific action performed by the human body once; The first initial coordinates and the second initial coordinates of the first joint point and the second joint point are obtained respectively when the human body performs the specific action. The three-dimensional motion speed of the human body performing the specific action is estimated based on the first initial coordinates, the second initial coordinates, the two-dimensional motion speed, and the actual length. The step of obtaining the two-dimensional motion speed of the human body performing the specific action includes: Obtain the posture change signal segment of the human body performing the specific action; Find the first and second extreme values ​​in the posture change signal segment; The two-dimensional motion velocity is estimated based on the first extreme value and the second extreme value.

Images