Multi-person interaction control method and system based on camera positioning and skeleton posture recognition

Abstract

The invention discloses a multi-person interaction control method and system based on camera positioning and skeleton posture recognition, and belongs to the technical field of somatosensory equipment. An image acquisition device is controlled to obtain a user binding image and first motion image data, user skeleton feature data is generated according to the user binding image, and the user binding image is an image of an interactive device for identifying the identity of a target user; determining skeleton data according to a skeleton recognition model and the first motion image data; determining a user motion action of the target user according to the user skeleton feature data and the skeleton data; and generating corresponding interaction data according to the user motion, thereby realizing the beneficial effects of improving the accuracy of interaction between the equipment and the user and further improving the interaction experience of the user.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of somatosensory devices, in particular to a multi-person interaction control method and system based on camera positioning and skeletal pose recognition. BACKGROUND

[0002] With the pursuit of people's quality of life, various types of entertainment interactive devices are generally provided in entertainment venues. Commonly, these devices are generally provided with a screen and a somatosensory device, such as a somatosensory trampoline. In the current technology, images are generally collected by an image collection device, and the actions performed by the user are determined by recognizing the skeletal joints, and the corresponding interaction content is displayed on the screen according to the user's actions. However, in the actual use process, there are often other people near the entertainment interactive device during the process of collecting images by the image collection device. In this process, multiple skeletal joints are recognized by the image, thereby affecting the action recognition of the user using the entertainment interactive device, and thus the accuracy of device interaction is reduced, thereby affecting the user's interaction experience.

[0003] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide a multi-person interaction control method and system based on camera positioning and skeletal pose recognition, aiming to improve the accuracy of device and user interaction, and thus improve the user's interaction experience. To achieve the above purpose, the present application provides a multi-person interaction control method based on camera positioning and skeletal pose recognition, applied to an interactive device, which comprises the following steps: controlling an image collection device to obtain a user binding image and first motion image data, and generating user skeletal feature data according to the user binding image, the user binding image being an image for identifying the identity of a target user by the interactive device; determining skeletal data according to a skeletal recognition model and the first motion image data; determining a user motion action of the target user according to the user skeletal feature data and the skeletal data; generating corresponding interaction data according to the user motion action.

[0005] Optionally, the step of determining a user motion action of the user according to the user skeletal feature data and the skeletal data comprises: grouping the skeletal data according to the user skeletal feature data to obtain first-type skeletal data and second-type skeletal data, the first-type skeletal data being skeletal data associated with the target user, and the second-type skeletal data being data other than the first-type skeletal data in the skeletal data; determining a matching result according to the first type of skeleton data and the action database; determining the user motion action according to the matching result.

[0006] Optionally, the user skeleton feature data comprises: a number of user skeletons of the target user, and sub-feature data of each user skeleton of the target user, the sub-feature data comprising: a skeleton length, a first ratio of the skeleton length to a body length, a second ratio of the skeleton length to a limb girth, and a number of skeleton connection points, the skeleton data comprising: skeleton feature information of each recognized unlabeled skeleton, and the step of grouping the skeleton data according to the user skeleton feature data comprises: calculating a similarity according to the skeleton feature information and the sub-feature data of each skeleton, to obtain a plurality of similarities; determining an unlabeled skeleton corresponding to each user skeleton according to the plurality of similarities, and generating corresponding labeling information; grouping the skeleton data according to the plurality of labeling information and the number of user skeletons.

[0007] Optionally, before the step of grouping the skeleton data according to the plurality of labeling information and the number of user skeletons, the method further comprises: adjusting the number of user skeletons according to a reference user motion action before a current time, a time difference between the reference user motion action and the first motion image data being less than a preset time.

[0008] Optionally, the interactive device further comprises a pressure sensor, and before the step of determining the user motion action of the target user according to the user skeleton feature data and the skeleton data, the method further comprises: obtaining detection data of the pressure sensor; when the detection data of the pressure sensor is greater than a pressure threshold, not adjusting the skeleton data; when the detection data of the pressure sensor is less than or equal to the pressure threshold, deleting data of a skeleton at a position corresponding to the pressure sensor in the skeleton data.

[0009] Optionally, the number of pressure sensors is more than one, and before the step of determining the skeleton data according to the skeleton recognition model and the first motion image data, the method further comprises: determining position information of the user on a base of the interactive device according to a plurality of detection data; adjusting an image range of the first motion image data according to the position information; The step of determining the user motion action of the target user according to the user skeleton feature data and the skeleton data further comprises the following steps before the step of determining the user motion action of the target user according to the user skeleton feature data and the skeleton data: Adjusting the user skeleton feature data according to the position information.

[0010] Optionally, the interaction data comprises video image data, and the step of generating corresponding interaction data according to the user motion action comprises the following steps: Controlling virtual action data of a virtual character according to the user motion action and a control mapping relationship; Interacting with a virtual game scene according to the virtual action, and generating corresponding video image data.

[0011] In addition, to achieve the above object, the present application further provides a multi-person interaction control system based on camera positioning and skeleton posture recognition, which comprises: A collection module is configured to control an image collection device to acquire user binding image and first motion image data, and generate user skeleton feature data according to the user binding image, wherein the user binding image is an image for identifying the identity of a target user by an interactive device. An identification module is configured to determine skeleton data according to a skeleton recognition model and the first motion image data. An analysis module is configured to determine the user motion action of the target user according to the user skeleton feature data and the skeleton data. An interaction module is configured to generate corresponding interaction data according to the user motion action.

[0012] In addition, to achieve the above object, the present application further provides an interactive device, which comprises a memory, a processor, and a multi-person interaction control program based on camera positioning and skeleton posture recognition stored in the memory and executable on the processor, wherein the multi-person interaction control program based on camera positioning and skeleton posture recognition is configured to implement the steps of the multi-person interaction control method based on camera positioning and skeleton posture recognition according to any one of the above embodiments.

[0013] In addition, to achieve the above object, the present application further provides a storage medium, wherein the storage medium stores a multi-person interaction control program based on camera positioning and skeleton posture recognition, and the multi-person interaction control program based on camera positioning and skeleton posture recognition implements the steps of the multi-person interaction control method based on camera positioning and skeleton posture recognition according to any one of the above embodiments when executed by a processor.

[0014] The application provides a multi-person interaction control method based on camera positioning and skeleton posture recognition, which acquires user binding images and first motion image data through a control image acquisition device, generates user skeleton feature data according to the user binding images, determines skeleton data according to a skeleton recognition model and the first motion image data, effectively screens out user skeleton information when other persons are present near an entertainment interactive device in the prior art, determines user motion actions of the target user according to the user skeleton feature data and the skeleton data, improves the accuracy of user recognition actions, generates corresponding interaction data according to the user motion actions, improves the accuracy of device interaction, and further improves the playing experience of users. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a structural schematic diagram of an interactive device related to a hardware running environment of an embodiment of the application. Figure 2 is a model diagram of a somatosensory trampoline all-in-one machine as an interactive device of the application. Figure 3 is a running state diagram of a somatosensory trampoline all-in-one machine as an interactive device of the application. Figure 4 is a flowchart of a first embodiment of the multi-person interaction control method based on camera positioning and skeleton posture recognition of the application. Figure 5 is a flowchart of a second embodiment of the multi-person interaction control method based on camera positioning and skeleton posture recognition of the application.

[0016] The implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0017] It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

[0018] Reference Figure 1 , Figure 1 is a structural schematic diagram of an interactive device related to a hardware running environment of an embodiment of the application.

[0019] As Figure 1As shown, the interactive device can include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, an interactive device 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection communication between the components. The interactive device 1003 can include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional interactive device 1003 can also be connected with the communication bus through a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 can be a high-speed random access memory (RAM) memory, or a stable non-volatile memory (NVM), such as a disk memory. The memory 1005 can also be a storage device independent of the aforementioned processor 1001.

[0020] Those skilled in the art can understand that Figure 1 The structure shown in the figure does not constitute a limitation on the interactive device, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0021] As Figure 1 As shown, the memory 1005 as a storage medium can include an operating system, a data storage module, a network communication module, a user interface module, and a multi-person interaction control program based on camera positioning and skeletal posture recognition.

[0022] In Figure 1 In the interactive device shown, the network interface 1004 is mainly used for data communication with other devices; the interactive device 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the interactive device of the application can be arranged in the interactive device, and the interactive device calls the multi-person interaction control program based on camera positioning and skeletal posture recognition stored in the memory 1005 through the processor 1001, and executes the multi-person interaction control method based on camera positioning and skeletal posture recognition provided by the embodiment of the application.

[0023] Further, the interactive device here can be a somatosensory trampoline all-in-one machine, the display screen displays a game rendering image, and further includes a trampoline, a pressure sensor, etc. For details, please refer to Figure 2 , Figure 2 The model diagram of the somatosensory trampoline all-in-one machine; for details, the physical device diagram of the device can refer to Figure 3 , Figure 3This is a diagram showing the operating status of the motion-sensing trampoline machine.

[0024] This invention provides a multi-user interactive control method based on camera positioning and skeletal pose recognition, referring to... Figure 4 , Figure 4 This is a flowchart illustrating the first embodiment of a multi-user interactive control method based on camera positioning and skeletal pose recognition according to the present invention.

[0025] In this embodiment, the multi-user interactive control method based on camera positioning and skeletal pose recognition includes: Step S1: Control the image acquisition device to acquire the user-bound image and the first motion image data, and generate user skeletal feature data based on the user-bound image. The user-bound image is the image used by the interactive device to identify the target user. In this embodiment, the interactive device is a motion-sensing trampoline machine. The user receives information on the screen of the motion-sensing trampoline machine, such as initiating interaction, playing games, or authentication. The interactive device is equipped with an image acquisition device, which captures the user-bound image during the user's login process. It should be noted that when it is recognized that the user already has corresponding user image information saved at the current moment, the user-bound image can be used to verify the user's identity, or the accuracy of user data collection can be improved by filtering to include only the user's image. Preferably, the first motion image data consists of multiple images of the user interacting with the interactive device. For example, the user performing actions such as jumping on the motion-sensing trampoline machine. The user's skeletal feature data is obtained through a skeletal recognition model. It should be noted that, preferably, the skeletal recognition model processing the user-bound image and the first motion image data should be the same model.

[0026] Step S2: Determine bone data based on the bone recognition model and the first motion image data; In this embodiment, the skeleton recognition model can be a High-Resolution Network (HRNet) or a MediaPipe Pose estimation model. This embodiment does not limit the algorithm used for skeleton recognition. After skeleton recognition, the user's skeletal feature data is determined by the pixel values ​​occupied by the skeleton. This is because the distance between the user and the device's camera is relatively fixed when interacting with the machine, for example, when stepping on the trampoline of a motion-sensing trampoline machine.

[0027] Step S3: Determine the user's motion actions based on the user skeletal feature data and the skeletal data; The identified skeletal data is classified based on the user's skeletal feature data to determine the bones corresponding to the user and those not corresponding to the user. The corresponding posture is then identified based on the user's corresponding bones, thereby obtaining the user's movement actions.

[0028] Step S4: Generate corresponding interaction data based on the user's movement actions.

[0029] Optionally, corresponding interactive instructions are generated based on the user's movement actions, and the operation of the preset character of the motion-sensing trampoline machine is controlled according to the interactive instructions, such as the character's behavior and movement control.

[0030] In this embodiment, the user-bound image and first motion image data are acquired by controlling the image acquisition device, and user skeletal feature data is generated based on the user-bound image. The skeletal data is determined based on the skeletal recognition model and the first motion image data. Compared with the current solution, when other people are often present near the entertainment interactive device, the skeletal information of the user can be effectively filtered out. The user's movement action of the target user can be determined based on the user skeletal feature data and the skeletal data, improving the accuracy of user action recognition. Then, corresponding interactive data is generated based on the user's movement action, improving the accuracy of device interaction and thus improving the user's gaming experience.

[0031] Furthermore, based on the first embodiment, a second embodiment of the multi-user interactive control method based on camera positioning and skeletal pose recognition of the present invention is proposed. In this embodiment, referring to... Figure 5 The step of determining the user's motion actions based on the user's skeletal feature data and the skeletal data includes: Step S31: Group the bone data according to the user bone feature data to obtain a first type of bone data and a second type of bone data. The first type of bone data is the bone data associated with the target user, and the second type of bone data is the data other than the first type of bone data in the bone data. It should be noted that the first type of skeletal data obtained here refers to the user's body skeleton constantly moving and appearing in different positions in the image due to changes during interaction. The second type of skeletal data generally refers to the skeletons of other people nearby. Filtering based on skeletal features can effectively identify different types of skeletal data. In other embodiments, some interactive devices can be used by more than one user. In this case, the user skeletal feature data can include data corresponding to different users. For example, if there are more than one user, the data can include skeletal feature information corresponding to user 1, user 2, and user 3, etc. In this case, the first type of skeletal data can also be labeled with the corresponding user number.

[0032] Step S32: Match the first type of skeletal data with the motion database to determine the matching result; The first type of skeletal data is matched with the action database. Generally, the first type of skeletal data includes: time sequence number and coordinate data of nodes at both ends of the bone. In addition, the coordinates of bone connection points, i.e., joints, can be saved to effectively reduce the amount of data. It should be noted that for a bone without connection points, a nearest neighbor search is used to connect the first type of skeletal data, for example, connecting or associating the nearest bones. The connection point is determined as the joint. Feature matching is then performed based on the first type of skeletal data and the action database. The specific matching feature is the angle between the bones connected by the joint; optionally, the matching feature is the distance between the bone and the standard position. For example, the joint angles corresponding to standing and squatting actions are different, and the distance between the torso and the standard position is also different.

[0033] Step S33: Determine the user's movement action based on the matching result.

[0034] In this embodiment, the action with the highest similarity in the action database is taken as the user's movement action. It should be noted that the design of this action database is related to the function of the interactive device. For example, for a motion-sensing trampoline machine, the actions in this action database have a high correlation with jumping.

[0035] In this embodiment, the skeletal data is grouped using the user's skeletal feature data to obtain a first type of skeletal data and a second type of skeletal data. The first type of skeletal data is then matched with an action database to determine the matching result. Based on the matching result, the user's movement action is determined, thereby effectively identifying the user's actions and avoiding misidentification of non-user actions.

[0036] Furthermore, the user skeletal feature data includes: the number of user bones of the target user, and sub-feature data of each user bone of the target user. The sub-feature data includes: bone length, a first ratio of bone length to body length, a second ratio of bone length to limb circumference, and the number of bone connection points. The skeletal data includes: skeletal feature information of each identified unlabeled bone. The step of grouping the skeletal data according to the user skeletal feature data includes: The similarity is calculated based on the skeletal feature information and the sub-feature data of each bone to obtain multiple similarity scores; In this embodiment, the user skeletal feature data is not a single data point, but a dataset. Specifically, since the user skeletal feature data includes information corresponding to more than one user bone, sub-feature data for each user bone is recorded. After the bone length is identified by the algorithm to the corresponding pixel in the image, the bone length also needs to be calculated using a conversion coefficient. Here, the torso length is the length between the two ends of the spine. Limb circumference specifically refers to the circumference of the human body part corresponding to the user's bone, such as waist circumference, thigh circumference, etc. The limb circumference is determined by extracting the number of pixels corresponding to that part through image recognition. The bone length, the first ratio of the bone length to the torso length, the second ratio of the bone length to the limb circumference, and the number of bone connection points are generally set by default and can be adjusted adaptively for specific individuals. In some embodiments, the sub-feature data may not include the number of bone connection points.

[0037] Based on the multiple similarities, the unlabeled bones corresponding to each user's skeleton are determined, and the corresponding labeling information is generated; It should be noted that since the similarity of each skeletal feature is calculated separately with its sub-feature data, and since users' bones may be similar (e.g., the bones of the left and right arms are similar), duplicate labeling needs to be avoided. Therefore, before labeling, it is necessary to determine whether an unlabeled bone has already been labeled. After labeling, it indicates that the unlabeled bone corresponds to the user's bone.

[0038] The bone data is grouped based on the multiple marker information and the number of user bones.

[0039] Optionally, it is determined whether the number of bones with labeled information in the first motion image data is the same as the number of user bones, and the bone data, i.e., the multiple bones identified in the first motion image data, are grouped according to the labeled information to obtain a first type of bone data and a second type of bone data. Furthermore, in some interactive devices, when multiple users exist, one type of bone data is further grouped according to different users.

[0040] In this embodiment, similarity is calculated using the skeletal feature information and the sub-feature data of each bone to obtain multiple similarities. Based on the multiple similarities, the unlabeled bones corresponding to each user bone are determined, and corresponding labeling information is generated. The skeletal data is grouped according to the multiple labeling information and the number of user bones. The identified bones are grouped using the user's limb features, which can avoid errors in recognizing user actions in the future.

[0041] Furthermore, based on the first or second embodiment, a third embodiment of the multi-user interactive control method based on camera positioning and skeletal pose recognition of the present invention is proposed. In this embodiment, before the step of grouping the skeletal data according to the multiple marker information and the number of user skeletons, the method further includes: The number of user skeletons is adjusted based on the reference user's motion action prior to the current moment, wherein the difference between the time of the reference user's motion action and the time of the first motion image data is less than a preset time.

[0042] In this embodiment, it should be noted that the user's actions are not singular. For continuous user actions, the number of user skeletons needs to be predicted based on the logic of referencing user movement and interaction. Preferably, the number of user skeletons can be dynamically adjusted. The purpose of this embodiment is to avoid anomalies caused by partial occlusion of skeletons, which would result in the data collected in the first motion image data not including some of the user's joints. For example, during interactive control, when entering different control modes, such as when the user turns to control the movement of the virtual character on the motion-sensing trampoline machine.

[0043] In this embodiment, the accuracy of data matching can be improved by adjusting the number of user skeletons based on the reference user's movement actions before the current moment.

[0044] Furthermore, based on any of the above embodiments, a fourth embodiment of the multi-user interactive control method based on camera positioning and skeletal pose recognition of the present invention is proposed. In this embodiment, the interactive device further includes: a pressure sensor. Before the step of determining the user movement action of the target user based on the user skeletal feature data and the skeletal data, the method further includes: Acquire the detection data from the pressure sensor; When the pressure sensor detects data greater than the pressure threshold, the bone data is not adjusted. When the detection data of the pressure sensor is less than or equal to the pressure threshold, the data of the bone at the corresponding position in the pressure sensor is deleted from the bone data.

[0045] It should be noted that motion-sensing trampoline machines typically include a trampoline and corresponding sensors to determine the force exerted by the user on the device. These sensors can be pressure sensors, specifically located on the inner surface of the trampoline. When the pressure sensor reading exceeds a pressure threshold, it indicates that the user is in contact with the trampoline. When the pressure sensor reading is less than or equal to the pressure threshold, it indicates that the user is not in contact with the trampoline. Therefore, bones whose lower ends are below a preset height can be excluded. For example, the bones of a spectator on a motion-sensing trampoline machine are in contact with the ground, thus their height is below the preset height. This is the method used in this application to distinguish between the user's and non-user's bones in the motion-sensing trampoline machine, effectively reducing the amount of bone data. It is particularly adaptable to various complex scenarios. The pressure threshold can be set to zero.

[0046] Furthermore, the number of pressure sensors is more than one, and before the step of determining the skeletal data based on the skeletal recognition model and the first motion image data, the method further includes: The user's position information on the base of the interactive device is determined based on multiple detection data. Adjust the image range of the first motion image data according to the location information; Before the step of determining the target user's movement based on the user skeletal feature data and the skeletal data, the method further includes: The user's skeletal feature data is adjusted based on the location information.

[0047] In this embodiment, it should be noted that the motion-sensing trampoline machine can include more than one user playing simultaneously. Multiple sensors can be used, positioned at different locations and typically marked with different indicators to allow users to interact based on these indicators. The user's position on the base of the interactive device is identified based on multiple detection data points. This defines the position of the user's skeleton within the first motion image data. The image range of the first motion image data is then adjusted by cropping. Furthermore, since the number of pixels occupied in the image varies depending on the distance from the image acquisition device, the user's skeletal feature data is adjusted based on the position information. The closer the position information is to the camera, the more pixels the corresponding skeleton occupies; therefore, the conversion coefficient in Embodiment 2 needs to be adjusted.

[0048] This avoids the situation where user skeletal feature data is abnormal when multiple users are in multiple available locations.

[0049] In this embodiment, for a multi-user interactive motion-sensing trampoline machine, the user's position information on the base of the interactive device can be determined through multiple detection data. The image range of the first motion image data is adjusted according to the position information, thereby reducing the amount of bones that need to be identified. The user's skeletal feature data is also adjusted according to the position information to improve the accuracy of the user's skeletal feature data.

[0050] Furthermore, based on any of the above embodiments, a fifth embodiment of the multi-user interactive control method based on camera positioning and skeletal pose recognition of the present invention is proposed. In this embodiment, the interactive data includes: video image data, and the step of generating corresponding interactive data based on the user's movement includes: The virtual character's virtual motion data is controlled based on the user's motion actions and the control mapping relationship; The virtual actions interact with the virtual game scene, generating corresponding video and image data.

[0051] In this embodiment, the virtual motion data of the virtual character is controlled through the user's motion actions and control mapping relationship, thereby improving the accuracy of the actions.

[0052] Furthermore, this invention also proposes a multi-user interactive control system based on camera positioning and skeletal pose recognition, wherein the multi-user interactive control system based on camera positioning and skeletal pose recognition includes: The acquisition module is used to control the image acquisition device to acquire user-bound image and first motion image data, and to generate user skeletal feature data based on the user-bound image, wherein the user-bound image is an image used by the interactive device to identify the target user. The recognition module is used to determine bone data based on the bone recognition model and the first motion image data; The analysis module is used to determine the user's motion actions based on the user's skeletal feature data and the skeletal data; The interaction module is used to generate corresponding interaction data based on the user's movement actions.

[0053] Furthermore, this embodiment of the invention also proposes an interactive device, the interactive device comprising: a memory, a processor, and a multi-user interactive control program based on camera positioning and skeletal posture recognition stored in the memory and executable on the processor, the multi-user interactive control program based on camera positioning and skeletal posture recognition being configured to implement the steps of the multi-user interactive control method based on camera positioning and skeletal posture recognition described above.

[0054] Furthermore, this embodiment of the invention also proposes a storage medium storing a multi-user interactive control program based on camera positioning and skeletal posture recognition. When the multi-user interactive control program based on camera positioning and skeletal posture recognition is executed by a processor, it implements the steps of the multi-user interactive control method based on camera positioning and skeletal posture recognition described above.

[0055] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0056] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0057] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0058] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A multi-user interactive control method based on camera positioning and skeletal pose recognition, characterized in that, Applied to interactive devices, the multi-user interactive control method based on camera positioning and skeletal pose recognition includes the following steps: The control image acquisition device acquires user-bound images and first motion image data, and generates user skeletal feature data based on the user-bound images. The user-bound images are images used by the interactive device to identify the target user's identity. Skeletal data is determined based on the skeleton recognition model and the first motion image data; The user's motion actions are determined based on the user's skeletal feature data and the skeletal data. Generate corresponding interactive data based on the user's movement actions.

2. The multi-user interactive control method based on camera positioning and skeletal pose recognition as described in claim 1, characterized in that, The step of determining the user's movement based on the user's skeletal feature data and the skeletal data includes: The skeletal data is grouped according to the user skeletal feature data to obtain a first type of skeletal data and a second type of skeletal data. The first type of skeletal data is the skeletal data associated with the target user, and the second type of skeletal data is the data other than the first type of skeletal data. The matching result is determined by matching the first type of skeletal data with the motion database; The user's movement action is determined based on the matching results.

3. The multi-user interactive control method based on camera positioning and skeletal pose recognition as described in claim 2, characterized in that, The user skeletal feature data includes: the number of user bones of the target user, and sub-feature data of each user bone of the target user. The sub-feature data includes: bone length, a first ratio of bone length to body length, a second ratio of bone length to limb circumference, and the number of bone connection points. The skeletal data includes: skeletal feature information of each identified unlabeled bone. The step of grouping the skeletal data according to the user skeletal feature data includes: The similarity is calculated based on the skeletal feature information and the sub-feature data of each bone to obtain multiple similarity scores; Based on the multiple similarities, the unlabeled bones corresponding to each user's skeleton are determined, and the corresponding labeling information is generated; The bone data is grouped based on the multiple marker information and the number of user bones.

4. The multi-user interactive control method based on camera positioning and skeletal pose recognition as described in claim 3, characterized in that, Before the step of grouping the bone data according to the multiple marker information and the number of user bones, the method further includes: The number of user skeletons is adjusted based on the reference user's motion action prior to the current moment, wherein the difference between the time of the reference user's motion action and the time of the first motion image data is less than a preset time.

5. The multi-user interactive control method based on camera positioning and skeletal pose recognition as described in claim 1, characterized in that, The interactive device further includes: a pressure sensor, and prior to the step of determining the target user's movement based on the user's skeletal feature data and the skeletal data, it further includes: Acquire the detection data from the pressure sensor; When the pressure sensor detects data greater than the pressure threshold, the bone data is not adjusted. When the detection data of the pressure sensor is less than or equal to the pressure threshold, the data of the bone at the corresponding position in the pressure sensor is deleted from the bone data.

6. The multi-user interactive control method based on camera positioning and skeletal pose recognition as described in claim 5, characterized in that, The number of pressure sensors is more than one, and before the step of determining the skeletal data based on the skeletal recognition model and the first motion image data, the method further includes: The user's position information on the base of the interactive device is determined based on multiple detection data. Adjust the image range of the first motion image data according to the location information; Before the step of determining the target user's movement based on the user skeletal feature data and the skeletal data, the method further includes: The user's skeletal feature data is adjusted based on the location information.

7. The multi-user interactive control method based on camera positioning and skeletal pose recognition as described in any one of claims 1 to 6, characterized in that, The interactive data includes: video image data, and the step of generating corresponding interactive data based on the user's movement includes: The virtual character's virtual motion data is controlled based on the user's motion actions and the control mapping relationship; The virtual actions interact with the virtual game scene, generating corresponding video and image data.

8. A multi-user interactive control system based on camera positioning and skeletal pose recognition, characterized in that, The multi-user interactive control system based on camera positioning and skeletal pose recognition includes: The acquisition module is used to control the image acquisition device to acquire user-bound image and first motion image data, and to generate user skeletal feature data based on the user-bound image, wherein the user-bound image is an image used by the interactive device to identify the target user. The recognition module is used to determine bone data based on the bone recognition model and the first motion image data; The analysis module is used to determine the user's motion actions based on the user's skeletal feature data and the skeletal data; The interaction module is used to generate corresponding interaction data based on the user's movement actions.

9. An interactive device, characterized in that, The interactive device includes: a memory, a processor, and a multi-user interactive control program based on camera positioning and skeletal posture recognition stored in the memory and executable on the processor, wherein the multi-user interactive control program based on camera positioning and skeletal posture recognition is configured to implement the steps of the multi-user interactive control method based on camera positioning and skeletal posture recognition as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a multi-user interactive control program based on camera positioning and skeletal posture recognition. When the multi-user interactive control program based on camera positioning and skeletal posture recognition is executed by the processor, it implements the steps of the multi-user interactive control method based on camera positioning and skeletal posture recognition as described in any one of claims 1 to 7.

Images