Behavior interaction triggering method based on human body contour

Abstract

The invention discloses a behavior interaction triggering method based on a human body contour, and the method uses a human body mask graph to generate an interaction space based on the human body contour, thereby completing an interaction triggering task under the condition of not interrupting the current motion keeping of a user, and aiming at the demand of encouraging the user to keep motion. An interaction space contraction strategy based on time self-adaptive adjustment is used, and the interaction space gradually fits the human body contour along with the extension of time, so that a user can complete task triggering more simply; multi-task interaction is realized by adopting three different space region division methods under multi-task control, so that the multi-task interaction requirement in an office scene is met. According to the method, the limitation of a traditional interaction mode in a healthy office scene is overcome, the office experience of the user is effectively improved, occupational diseases are prevented, meanwhile, positive contributions are made in the aspects of improving the interaction efficiency, protecting the health of the user and the like, and the method has important practical application value and popularization significance.

Description

Technical Field

[0001] The present invention relates to the technical field of human-computer interaction, and particularly relates to a behavior interaction triggering method based on a human body contour. Background Art

[0002] With the popularization of computer technology, people's work efficiency has been greatly improved. However, at the same time, related health problems have become increasingly prominent. The incidence of occupational diseases such as cervical spondylosis, lumbar spondylosis, and scapulohumeral periarthritis caused by long-term high-intensity office work is rising continuously, seriously affecting people's work and life quality.

[0003] Traditional interaction triggering technologies such as mice and keyboards are prone to causing discomfort in body parts during long-term use in high-intensity office scenarios. These traditional input devices require users to maintain a fixed posture, restricting the natural movement of the body and easily leading to muscle fatigue and chronic injuries. Although there are currently some new triggering schemes, such as gesture recognition triggering methods and human body posture triggering methods, most of these triggering methods require interrupting the existing actions to complete the interaction during the interaction process.

[0004] In the process of healthy interaction in an office scenario, a non-interruptive interaction triggering method is an important interaction method. Taking stretching exercises as an example, when a user is performing healthy interaction based on stretching exercises, they need to complete interaction tasks during the process of performing stretching exercises. The user cannot interrupt the current movement to use traditional interaction means. Therefore, a non-interruptive interaction method is needed to achieve the interaction triggering task, such as tasks like document page turning and volume adjustment.

[0005] Therefore, how to achieve non-interruptive interaction triggering in a high-intensity office scenario, and ensure the efficiency and accuracy of the interaction without affecting the office efficiency, is an urgent problem to be solved by the current technology.

[0006] Most of the current human-computer interaction methods do not consider the non-interruptive requirements in the scenario of healthy interaction, nor do they take into account the continuity and naturalness requirements of users during the interaction process. The patent application with the publication number CN105718878A uses a cascaded convolutional neural network to achieve hand detection and gesture classification. The focus is on the accuracy of recognition and does not consider the continuity problem of interaction; the patent with the publication number CN204945985U realizes interaction through holographic display and induction border, but pays more attention to the display effect and touch recognition and does not involve the implementation of non-interruptive interaction; the patent application with the publication number CN117555427A performs posture recognition and interaction control based on pressure map data, but mainly targets the sleep scenario and does not consider the non-interruptive requirements in healthy interaction; the patent applications with the publication numbers CN216908560U and CN110682267A involve human posture recognition interaction, but focus on specific application scenarios such as exercise and sedentary reminder respectively and fail to provide a general non-interruptive interaction solution. In summary, there is currently no method that can effectively solve the non-interruptive requirements in the healthy interaction scenario, that is, to meet the comprehensive requirements of interaction continuity, adaptability and multi-task control. Summary of the Invention

[0007] The purpose of the present invention is to provide a method for triggering behavior interaction based on human body contour to achieve non-interruptive interaction triggering in a high-intensity office scenario.

[0008] To achieve the above task, the present invention adopts the following technical solutions:

[0009] A method for triggering behavior interaction based on human body contour, comprising:

[0010] Real-time collect the human motion images during user interaction, and then transmit the human motion images in sequence;

[0011] Receive the human motion images, use a semantic segmentation algorithm to segment the human contour part in the human motion images to generate corresponding human mask images, and dynamically store them in the interaction space generation queue;

[0012] After the interaction space generation queue is updated, first perform superposition operation on multiple human mask images stored in the interaction space generation queue to obtain an accumulated mask image; then, perform binarization processing on the accumulated mask image to obtain a new binary mask image; finally, perform dilation processing on the binary mask image through dilation operation in morphology, and use a preset dilation convolution kernel to dilate the mask, so as to generate the final interaction space;

[0013] Determine the trigger behavior by real-time monitoring of the relationship between the human body contour in the user's human motion image and the interaction space, so as to use the generated interaction space to achieve the interaction trigger of the user's single-task or multi-task trigger scenario; when the trigger behavior is valid, execute the corresponding task; during the interaction trigger process, dynamically adjust the size of the interaction space according to the user's usage time;

[0014] Display the interaction trigger process and the task execution result.

[0015] Further, an interaction space generation queue and a mask update cache queue are set in the server; the interaction space generation queue stores the generated human mask images, and the mask update cache queue stores the latest images to be put into the interaction space generation queue; the interaction space generation queue and the mask update cache queue are updated in a first-in, first-out access mode. The newly generated human mask image first enters the tail of the mask update cache queue, then the human mask image at the head of the mask update cache queue is taken out and put into the tail of the interaction space generation queue, and then the human mask image at the head of the interaction space generation queue is removed, so as to update the interaction space generation queue and the mask update cache queue with the newly generated human mask image.

[0016] Further, perform binary processing on the accumulated mask image, specifically: set the pixel points greater than zero in the accumulated mask image to 1, and other pixel points to 0, so as to obtain a new binary mask image.

[0017] Further, expand the binary mask image through the dilation operation in morphology, which is expressed as:

[0018]

[0019] where M interact represents the finally generated interaction space, K represents the dilation convolution kernel, and the dilation convolution kernel is a square all-1 matrix, represents the dilation operation.

[0020] Further, the server determines the trigger behavior by real-time monitoring of the relationship between the human body contour in the user's human motion image and the interaction space, including:

[0021] The server will obtain the human mask image of the current user in real time based on the human motion image, compare it with the already generated interaction space, and calculate the area of the region where the current human body contour in the human mask image exceeds the range of the interaction space; by calculating the ratio of the area of this region to the area of the human mask image and comparing this ratio with a pre-set threshold, when the ratio exceeds the threshold, the server determines it as a valid trigger behavior.

[0022] Further, during the interaction triggering process, the size of the interaction space is dynamically adjusted according to the user's usage time, including:

[0023] In the initial stage of interaction, the preset initial value of the dilation convolution kernel is used to generate a loose interaction space; as time increases, the initial value of the dilation convolution kernel is gradually reduced to the minimum value of the dilation convolution kernel at a preset contraction rate, so that the interaction space gradually shrinks and is closer to the user's body contour; by setting the initial value of the dilation convolution kernel, the minimum value of the dilation convolution kernel, and the contraction rate parameter, the smooth contraction of the interaction space is achieved.

[0024] Further, for the multi-task triggering scenario, three different spatial region division methods are designed:

[0025] First is the dual-region task allocation method, which divides the user's interaction space into two large regions on the left and right. Each region corresponds to a different task set. The user triggers a certain region to enter the corresponding first-level menu, and then can further select a specific task in the second-level menu. Through this hierarchical progressive method, the task selection is finally completed;

[0026] Second is the four-region task division method. This method further divides the two large regions on the left and right into four regions on the basis of the two large regions. Each region corresponds to a specific task set, maintaining the same multi-level menu triggering logic as the dual-region;

[0027] Finally is the eight-region single-trigger method, which divides the interaction space into eight independent regions. Each region directly corresponds to a specific task. The user only needs to trigger once to select and execute the corresponding task.

[0028] A behavior interaction triggering system based on the human body contour includes:

[0029] An input device, which is used to collect the human motion images during user interaction in real time, and then transmit the human motion images in sequence;

[0030] A server, which is used to receive the human motion images, use a semantic segmentation algorithm to segment the human contour part in the human motion images to generate corresponding human mask images, and dynamically store them in the interaction space generation queue;

[0031] After the interaction space generation queue is updated, first perform a superposition operation on multiple human mask images stored in the interaction space generation queue to obtain an accumulated mask image; then, perform binarization processing on the accumulated mask image to obtain a new binary mask image; finally, perform an expansion process on the binary mask image through the dilation operation in morphology, and use a preset dilation convolution kernel to dilate the mask, thereby generating the final interaction space;

[0032] The triggering behavior is determined by real-time monitoring of the relationship between the human body contour in the user's body movement image and the interaction space, so as to use the generated interaction space to achieve the interaction trigger of the user's single-task or multi-task trigger scenario; when the triggering behavior is effective, the corresponding task is executed; during the interaction trigger process, the size of the interaction space is dynamically adjusted according to the user's usage time;

[0033] A display device for displaying the interaction trigger process and the task execution result.

[0034] A terminal device includes a processor, a memory, and a computer program stored in the memory; when the processor executes the computer program, the method for behavior interaction trigger based on human body contour is implemented.

[0035] A computer-readable storage medium stores a computer program; when the computer program is executed by a processor, the method for behavior interaction trigger based on human body contour is implemented.

[0036] Compared with the prior art, the present invention has the following technical features:

[0037] 1. The method of the present invention has significant advantages and practical application values in many aspects:

[0038] In terms of non-interruption, the method realizes a truly non-interruptive interaction experience. Users do not need to remember specific triggering gestures and can maintain a natural and smooth interaction process. In terms of technical implementation, the innovative double-queue mechanism ensures the smooth update of the interaction space, can respond to the user's action changes in real time, and effectively guarantees the continuity of the interaction. In terms of health promotion, the present invention encourages users to maintain a healthy interaction posture through an adaptive adjustment strategy, so that users who maintain the correct posture for a long time can more easily trigger the interaction, thus effectively preventing the occurrence of occupational diseases. In terms of interaction efficiency, the multi-task control strategy can flexibly adapt to the needs of different scenarios, and significantly improves the task processing efficiency through diverse space division methods. In terms of reliability, the trigger determination mechanism based on the area ratio has high reliability, effectively reduces the probability of false triggering, and improves the accuracy of the interaction. In terms of practicality, the present invention does not require special hardware support, is easy to integrate and deploy, and has a wide range of applications.

[0039] 2. Through the organic combination of an innovative human body contour interaction space generation method, an adaptive adjustment strategy, and a multi-task control mechanism, the present invention provides a brand-new solution for non-interruptive human-computer interaction; this solution not only addresses the limitations of traditional interaction methods in the context of healthy office scenarios but also makes important contributions in aspects such as improving interaction efficiency and protecting user health, having important practical application value and promotion significance. Through the implementation of the present invention, the office experience of users can be effectively improved, the occurrence of occupational diseases can be prevented, and at the same time, work efficiency can be ensured, providing new technical support for healthy office work. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the interaction space generation of the present invention;

[0041] Figure 2 It is a schematic flowchart of the method of the present invention;

[0042] Figure 3 It is a schematic flowchart of the task trigger process in an embodiment of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] The present invention proposes a method for triggering behavior interaction based on human body contours. This method uses a human body mask image to generate an interaction space based on human body contours, thereby completing the interaction trigger task without interrupting the current movement of the user. To meet the requirement of encouraging the user to maintain, an interaction space contraction strategy based on time adaptation is used, and the interaction space gradually fits the human body contour as time extends, so that the user can complete the task trigger more simply; three different spatial region division methods under multi-task control are adopted to achieve multi-task interaction, thus meeting the multi-task interaction requirements in the office scenario.

[0044] A method for triggering behavior interaction based on human body contours provided by the present invention includes the following steps:

[0045] Step 1, the input device continuously collects the human motion images during user interaction and then sequentially transmits the human motion images to the server.

[0046] Step 2, after receiving the human motion images, the server uses a semantic segmentation algorithm to segment the human body contour part in the human motion images to generate corresponding human body mask images and dynamically stores them in the interaction space generation queue.

[0047] An interaction space generation queue and a mask update cache queue are set in the server; the interaction space generation queue stores the generated human mask images, and the mask update cache queue stores the latest images to be put into the interaction space generation queue; the interaction space generation queue and the mask update cache queue are updated in a first-in-first-out access mode. The newly generated human mask image first enters the tail of the mask update cache queue, then the human mask image at the head of the mask update cache queue is taken out and put into the tail of the interaction space generation queue, and then the human mask image at the head of the interaction space generation queue is removed, so as to update the interaction space generation queue and the mask update cache queue with the newly generated human mask image.

[0048] Step 3, after the interaction space generation queue is updated, the server first performs a superposition operation on multiple human mask images stored in the interaction space generation queue to obtain an accumulated mask image; then, the accumulated mask image is binarized. The specific method is to set the pixel points greater than zero in the accumulated mask image to 1 and other pixel points to 0, so as to obtain a new binary mask image; finally, the binary mask image is expanded by using the dilation operation in morphology, and the preset dilation convolution kernel is used to dilate the mask, so as to generate the final interaction space.

[0049] Among them, the accumulated mask image is expressed as follows:

[0050]

[0051] Where M sum represents the accumulated mask image, n represents the queue length, and M i represents the i-th human mask image in the interaction space generation queue.

[0052] The binarization process of the accumulated mask image is expressed as:

[0053]

[0054] Where M sum (x, y) represents the image pixel coordinates at the (x, y) position in the accumulated mask image, and M new (x, y) represents the image pixel coordinates at the (x, y) position in the binary mask image.

[0055] The binary mask image is expanded by using the dilation operation in morphology, which is expressed as:

[0056]

[0057] Where M interact represents the finally generated interaction space, K represents the dilation convolution kernel, and the dilation convolution kernel is a square all-1 matrix. Denotes the dilation operation, which is a common morphological operation in image processing and is mainly used for binary images or grayscale images. The definition of the dilation operation is as follows:

[0058]

[0059] where A is the input binary mask image, and B is the dilation convolution kernel, whose shape is square. Denotes the dilation operator, (B) z Denotes the position after the convolution kernel B is translated at position z. ∩ denotes the AND operation, and ≠ Φ means the intersection is not empty. That is, the convolution kernel matrix is used to perform the AND operation with the binary image elements. If all are 0, then the target pixel point is 0, otherwise it is 1.

[0060] This method effectively constructs a spatial region suitable for user interaction through the comprehensive processing of historical human mask images.

[0061] Step 4: The server determines the trigger behavior by real-time monitoring the relationship between the human body contour and the interaction space in the user's human motion image, so as to use the generated interaction space to achieve the interaction trigger of the user's single-task or multi-task trigger scenario; when the trigger behavior is valid, the corresponding task is executed; during the interaction trigger process, the server dynamically adjusts the size of the interaction space according to the user's usage time.

[0062] Specifically, the server will obtain the current user's human mask image in real-time based on the human motion image, compare it with the generated interaction space, and calculate the size of the area where the current human body contour in the human mask image exceeds the range of the interaction space; by calculating the ratio of the area of this region to the area of the human mask image and comparing this ratio with a pre-set threshold, when the ratio exceeds the threshold, the server determines it as a valid trigger behavior; when the trigger scenario is a single task, the corresponding task is executed at this time.

[0063] This trigger determination method based on the area ratio has strong robustness, can effectively avoid false triggers caused by the user's subtle movements, and at the same time ensures the naturalness and comfort of the trigger action; this trigger mechanism does not require the user to remember specific trigger gestures, but can complete the interaction through natural body movements, truly realizing a non-interruptive human-computer interaction experience.

[0064] In this step, the server adaptively adjusts the dilation convolution kernel size used to generate the interaction space according to the length of time the user stays in the interaction space. In the initial stage of interaction, the server uses a relatively large initial value of the dilation convolution kernel to generate a loose interaction space, providing the user with a large range of movement. As the user's staying time in the interaction space increases, the server gradually reduces the initial value of the dilation convolution kernel to the minimum value of the dilation convolution kernel at a preset contraction rate, causing the interaction space to gradually shrink and fit closer to the user's body contour. This dynamic contraction strategy not only reduces the triggering difficulty for the user but also encourages the user to maintain a correct posture during healthy interaction. By setting parameters such as the initial value of the dilation convolution kernel, the minimum value of the dilation convolution kernel, and the contraction rate, the server can achieve smooth contraction of the interaction space and provide the user with a natural and efficient interaction experience. The dynamic adjustment process of the dilation convolution kernel is expressed as:

[0065] K t = max(k init - αt, K min )

[0066] where K t represents the dilation convolution kernel at time t, K init represents the initial value of the dilation convolution kernel, K min represents the minimum value of the dilation convolution kernel, and α represents the contraction rate.

[0067] For the multi-task triggering scenario, three different methods of spatial region division are designed to meet the requirements of the multi-task healthy interaction scenario. First is the dual-region task allocation method, which divides the user's body contour interaction space into two large regions on the left and right. Each region corresponds to a different task set. The user triggers a certain region to enter the corresponding first-level menu and then can further select a specific task in the second-level menu. Through this hierarchical progressive method, the task selection is finally completed. Second is the four-region task division method. This method further divides the interaction space into four regions on the basis of the two large regions on the left and right. Each region corresponds to a specific type of task set, maintaining the same multi-level menu triggering logic as the dual-region method. Finally is the eight-region single-trigger method, which divides the interaction space into eight independent regions. Each region directly corresponds to a specific task. The user only needs to trigger once to select and execute the corresponding task, simplifying the operation process and improving the interaction efficiency. The three division methods can be flexibly selected and used according to the requirements of different scenarios.

[0068] Step 5: Use the display device to display the interaction triggering process and the task execution result.

[0069] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included within the protection scope of the present application.

Claims

1. A behavior interaction triggering method based on human body contour, characterized in that: include: Collect human motion images during user interaction in real time, and then transmit the human motion images in sequence; Receive the human motion image, use a semantic segmentation algorithm to segment the human body contour part in the human motion image to generate a corresponding human body mask map, and dynamically store it in the interactive space generation queue; After the interactive space generation queue is updated, the multiple human mask images stored in the interactive space generation queue are firstly superimposed to obtain an accumulated mask image; then, the accumulated mask image is binarized to obtain a new binary mask image; finally, the binary mask image is expanded by the dilation operation in morphology, and the mask is expanded using the preset dilation convolution kernel to generate the final interactive space; The triggering behavior is determined by real-time monitoring of the relationship between the human body contour and the interactive space in the user's human body motion image, so as to use the generated interactive space to realize the interactive triggering of the user's single-task or multi-task triggering scenario; when the triggering behavior is valid, the corresponding task is executed; During the interaction triggering process, the size of the interaction space is dynamically adjusted according to the user's usage time; The interaction triggering process and the task execution result are displayed.

2. The method for triggering a behavior interaction based on a human body contour according to claim 1, characterized in that: The server is provided with an interactive space generation queue and a mask update cache queue; wherein the interactive space generation queue stores the generated human body mask image, and the mask update cache queue stores the latest image to be placed in the interactive space generation queue; a first-in-first-out access method is adopted to update the interactive space generation queue and the mask update cache queue, wherein the newly generated human body mask image first enters the tail of the mask update cache queue, then the human body mask image at the head of the mask update cache queue is taken out and placed at the tail of the interactive space generation queue, and then the human body mask image at the head of the interactive space generation queue is removed, thereby using the newly generated human body mask image to update the interactive space generation queue and the mask update cache queue.

3. The method for triggering behavior interaction based on human body contour according to claim 1, characterized in that: The binarization process is performed on the accumulated mask image, specifically, setting the pixel points greater than zero in the accumulated mask image to 1, and setting the other pixel points to 0, so as to obtain a new binary mask image.

4. The method for triggering a behavior interaction based on a human body contour according to claim 1, characterized in that: The binary mask image is expanded by the dilation operation in morphology, which is expressed as: Among them, M interact represents the final generated interaction space, K represents the dilated convolution kernel, which is a square matrix of all 1s. Represents an expansion operation.

5. The method for triggering behavior interaction based on human body contour according to claim 1, characterized in that: The server determines the triggering behavior by monitoring the relationship between the human body contour and the interactive space in the user's human body motion image in real time, including: The server will obtain the human mask image of the current user in real time based on the human motion image, compare it with the generated interactive space, and calculate the size of the area in the human mask image where the current human contour exceeds the interactive space range; by calculating the ratio of the area of ​​this area to the area of ​​the human mask image, and comparing this ratio with a pre-set threshold, when the ratio exceeds the threshold, the server determines it as a valid trigger behavior.

6. The method for triggering behavior interaction based on human body contour according to claim 1, characterized in that: In the interaction triggering process, the size of the interaction space is dynamically adjusted according to the user's use time, including: In the initial stage of interaction, the preset initial value of the dilated convolution kernel is used to generate a loose interactive space; as time goes by, the initial value of the dilated convolution kernel is gradually reduced to the minimum value of the dilated convolution kernel according to the preset contraction rate, so that the interactive space gradually shrinks and becomes closer to the user's body contour; by setting the initial value of the dilated convolution kernel, the minimum value of the dilated convolution kernel and the contraction rate parameters, the smooth contraction of the interactive space is achieved.

7. The method for triggering behavior interaction based on human body contour according to claim 1, characterized in that: For multi-task triggering scenarios, three different spatial area division methods are designed: The first is the dual-area task allocation method, which divides the user's interactive space into two large areas on the left and right. Each area corresponds to a different set of tasks. The user enters the corresponding first-level menu by triggering a certain area, and then can further select specific tasks in the second-level menu. In this hierarchical progressive way, the task selection is finally completed; The second is the four-area task division method. This method further divides the left and right areas into four areas based on the two large areas. Each area corresponds to a specific set of tasks, maintaining the same multi-level menu trigger logic as the dual-area method. Finally, there is the eight-area single trigger method, which divides the interactive space into eight independent areas. Each area directly corresponds to a specific task. Users only need one trigger to select and execute the corresponding task.

8. A behavioral interaction triggering system based on human body contour, characterized in that: include: An input device for collecting human motion images during user interaction in real time and then transmitting the human motion images in sequence; The server is used to receive the human motion image, segment the human body contour part in the human body motion image using a semantic segmentation algorithm to generate a corresponding human body mask map, and dynamically store it in an interactive space generation queue; After the interactive space generation queue is updated, the multiple human mask images stored in the interactive space generation queue are firstly superimposed to obtain an accumulated mask image; then, the accumulated mask image is binarized to obtain a new binary mask image; finally, the binary mask image is expanded by the dilation operation in morphology, and the mask is expanded using the preset dilation convolution kernel to generate the final interactive space; The triggering behavior is determined by real-time monitoring of the relationship between the human body contour and the interactive space in the user's human body motion image, so as to use the generated interactive space to realize the interactive triggering of the user's single-task or multi-task triggering scenario; when the triggering behavior is valid, the corresponding task is executed; During the interaction triggering process, the size of the interaction space is dynamically adjusted according to the user's usage time; The display device is used to display the interaction triggering process and the task execution result.

9. A terminal device comprising a processor, a memory and a computer program stored in the memory; characterized in that: When the processor executes the computer program, it implements the behavior interaction triggering method based on human body contour according to any one of claims 1-7.

10. A computer-readable storage medium, wherein a computer program is stored in the medium; characterized in that: When the computer program is executed by a processor, the behavior interaction triggering method based on human body contour according to any one of claims 1-7 is implemented.

Images