Multi-user interactive control method and system based on camera positioning and skeletal pose recognition

Abstract

This invention discloses a multi-user interactive control method and system based on camera positioning and skeletal posture recognition, belonging to the field of motion sensing device technology. The invention acquires a user-bound image and first motion image data by controlling an image acquisition device, and generates user skeletal feature data based on the user-bound image, where the user-bound image is used by the interactive device to identify the target user. Skeletal data is determined based on a skeletal recognition model and the first motion image data. The user's motion actions are determined based on the user skeletal feature data and the skeletal data. Corresponding interactive data is generated based on the user's motion actions, thereby improving the accuracy of device-user interaction and enhancing the user's interactive experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of somatosensory devices, in particular to a multi-person interaction control method and system based on camera positioning and skeletal posture recognition. BACKGROUND

[0002] With the pursuit of people's quality of life, various types of entertainment interactive devices are generally provided in entertainment venues. Commonly, these devices are generally provided with a screen and a somatosensory device, such as a somatosensory trampoline. In the current technology, images are generally collected by an image collection device, and the actions performed by the user are determined by recognizing the skeletal joints. The corresponding interaction content is displayed on the screen according to the user's actions. However, in the actual use process, there are often other people near the entertainment interactive device during the image collection process of the image collection device. In this process, multiple skeletal joints are recognized by the image, which affects the action recognition of the user using the entertainment interactive device, thereby reducing the accuracy of device interaction and affecting the user's interaction experience.

[0003] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide a multi-person interaction control method and system based on camera positioning and skeletal posture recognition, aiming to improve the accuracy of device and user interaction, and thereby improve the user's interaction experience. To achieve the above purpose, the present application provides a multi-person interaction control method based on camera positioning and skeletal posture recognition, applied to an interactive device. The multi-person interaction control method based on camera positioning and skeletal posture recognition comprises the following steps:

[0005] The image collection device acquires user binding images and first motion image data, and generates user skeletal feature data according to the user binding images. The user binding images are images for identifying the identity of the target user of the interactive device;

[0006] Determine the skeletal data according to the skeletal recognition model and the first motion image data;

[0007] Determine the user motion action of the target user according to the user skeletal feature data and the skeletal data;

[0008] Generate corresponding interaction data according to the user motion action.

[0009] Optionally, the step of determining the user motion action of the user according to the user skeletal feature data and the skeletal data comprises:

[0010] group the skeleton data according to the user skeleton feature data, to obtain first type skeleton data and second type skeleton data, the first type skeleton data is the skeleton data associated with the target user, and the second type skeleton data is the data other than the first type skeleton data in the skeleton data;

[0011] match the first type skeleton data with a motion database to determine a matching result;

[0012] determine the user motion action according to the matching result.

[0013] Optionally, the user skeleton feature data includes a user skeleton number of the target user and sub-feature data of each user skeleton of the target user, and the sub-feature data includes a skeleton length, a first ratio of the skeleton length to a body length, a second ratio of the skeleton length to a limb girth, and a skeleton connection point number, the skeleton data includes skeleton feature information of each recognized unlabeled skeleton, and the step of grouping the skeleton data according to the user skeleton feature data includes:

[0014] calculate a plurality of similarities according to the skeleton feature information and the sub-feature data of each skeleton;

[0015] determine an unlabeled skeleton corresponding to each user skeleton according to the plurality of similarities, and generate corresponding labeled information;

[0016] group the skeleton data according to the plurality of labeled information and the user skeleton number.

[0017] Optionally, before the step of grouping the skeleton data according to the plurality of labeled information and the user skeleton number, the method further includes:

[0018] adjust the user skeleton number according to a reference user motion action before a current time, and a time difference between the reference user motion action and the first motion image data is less than a preset time.

[0019] Optionally, the interactive device further includes a pressure sensor, and before the step of determining the user motion action of the target user according to the user skeleton feature data and the skeleton data, the method further includes:

[0020] obtain detection data of the pressure sensor;

[0021] when the detection data of the pressure sensor is greater than a pressure threshold, do not adjust the skeleton data;

[0022] when the detection data of the pressure sensor is less than or equal to the pressure threshold, delete the data of the skeleton at a position corresponding to the pressure sensor in the skeleton data.

[0023] Optionally, the number of the pressure sensors is more than one, and before the step of determining the skeleton data according to the skeleton recognition model and the first motion image data, the method further comprises:

[0024] determining position information of the user on a base of the interactive device according to the detection data;

[0025] adjusting an image range of the first motion image data according to the position information;

[0026] Before the step of determining the user motion action of the target user according to the user skeleton feature data and the skeleton data, the method further comprises:

[0027] adjusting the user skeleton feature data according to the position information.

[0028] Optionally, the interaction data comprises video image data, and the step of generating corresponding interaction data according to the user motion action comprises:

[0029] controlling virtual action data of a virtual character according to the user motion action and a control mapping relationship;

[0030] interacting with a virtual game scene according to the virtual action, and generating corresponding video image data.

[0031] In addition, to achieve the above object, the application further provides a multi-person interaction control system based on camera positioning and skeleton posture recognition, which comprises:

[0032] a collection module, configured to control an image collection device to acquire user binding image and first motion image data, and generate user skeleton feature data according to the user binding image, wherein the user binding image is an image for the interactive device to identify the identity of a target user;

[0033] a recognition module, configured to determine skeleton data according to a skeleton recognition model and the first motion image data;

[0034] an analysis module, configured to determine a user motion action of the target user according to the user skeleton feature data and the skeleton data;

[0035] an interaction module, configured to generate corresponding interaction data according to the user motion action.

[0036] In addition, to achieve the above object, the application further provides an interactive device, comprising a memory, a processor, and a multi-person interactive control program based on camera positioning and skeleton posture recognition stored on the memory and executable on the processor, which is configured to implement the steps of any of the multi-person interactive control methods based on camera positioning and skeleton posture recognition.

[0037] In addition, to achieve the above object, the application further provides a storage medium, which stores a multi-person interactive control program based on camera positioning and skeleton posture recognition, which, when executed by a processor, implements the steps of any of the multi-person interactive control methods based on camera positioning and skeleton posture recognition.

[0038] The application provides a multi-person interactive control method based on camera positioning and skeleton posture recognition, which acquires user binding images and first motion image data by controlling an image acquisition device, generates user skeleton feature data according to the user binding images, determines skeleton data according to a skeleton recognition model and the first motion image data, effectively filters out the skeleton information of a user when other people are present near an entertainment interactive device in the prior art, determines the user motion action of the target user according to the user skeleton feature data and the skeleton data, improves the accuracy of user recognition action, generates corresponding interactive data according to the user motion action, improves the accuracy of device interaction, and further improves the playing experience of the user. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 is a structural schematic diagram of an interactive device of a hardware running environment related to the embodiment scheme of the application;

[0040] Figure 2 is a model diagram of a somatosensory trampoline all-in-one machine as an interactive device of the application;

[0041] Figure 3 is a running state diagram of a somatosensory trampoline all-in-one machine as an interactive device of the application;

[0042] Figure 4 is a flowchart of a first embodiment of the multi-person interactive control method based on camera positioning and skeleton posture recognition of the application;

[0043] Figure 5 is a flowchart of a second embodiment of the multi-person interactive control method based on camera positioning and skeleton posture recognition of the application.

[0044] The implementation of the object, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0045] It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

[0046] Referring to Figure 1 , Figure 1 The hardware environment of the interactive device involved in the embodiment of the present application is shown in the structural diagram.

[0047] As Figure 1 shown, the interactive device can include a processor 1001, such as a central processing unit (CPU), a communication bus 1002, an interactive device 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between these components. The interactive device 1003 can include a display screen (Display), an input unit such as a keyboard (Keyboard), and an optional interactive device 1003 can also be connected with the communication bus through a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 can be a high-speed random access memory (RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a magnetic disk memory. The memory 1005 can also be an independent storage device from the aforementioned processor 1001.

[0048] Those skilled in the art can understand Figure 1 that the structure shown in the foregoing is not a limitation on the interactive device, and can include more or fewer components than the diagram, or combine certain components, or different component arrangements.

[0049] As Figure 1 shown, the memory 1005 as a storage medium can include an operating system, a data storage module, a network communication module, a user interface module, and a multi-person interaction control program based on camera positioning and bone posture recognition.

[0050] In Figure 1The network interface 1004 is mainly used for data communication with other devices; the interactive device 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the interactive device of the application can be arranged in the interactive device, the interactive device calls the multi-person interaction control program based on camera positioning and skeleton posture recognition stored in the memory 1005 through the processor 1001, and executes the multi-person interaction control method based on camera positioning and skeleton posture recognition provided by the embodiment of the application.

[0051] Further, the interactive device herein can be a somatosensory trampoline all-in-one machine, the display screen displays a game rendering image, and further comprises a trampoline, a pressure sensor and the like. For details, refer to Figure 2 , Figure 2 a model diagram of the somatosensory trampoline all-in-one machine; for details, refer to the physical device diagram of the device Figure 3 , Figure 3 a running state diagram of the somatosensory trampoline all-in-one machine.

[0052] The embodiment of the application provides a multi-person interaction control method based on camera positioning and skeleton posture recognition, which is described with reference to Figure 4 , Figure 4 a flowchart of a first embodiment of the multi-person interaction control method based on camera positioning and skeleton posture recognition.

[0053] In the embodiment, the multi-person interaction control method based on camera positioning and skeleton posture recognition comprises:

[0054] Step S1, control the image acquisition device to acquire user binding image and first motion image data, and generate user skeleton feature data according to the user binding image, the user binding image being an image for identifying the identity of a target user by the interactive device;

[0055] In the embodiment, the interactive device is a somatosensory trampoline all-in-one machine. In the embodiment, the user receives information on the screen of the somatosensory trampoline all-in-one machine, such as starting interaction, playing a game, identity authentication, and the like. The interactive device is provided with an image acquisition device. In the process of user login and the like, the image acquisition device acquires the user binding image. It should be noted that when it is identified that the user has saved corresponding user image information at the current time, the user binding image can be used to verify the user identity, or the accuracy of data acquisition of the user is improved by screening only images of one individual user. Preferably, the first motion image data herein is a plurality of images of the user in interaction with the interactive device. For example, the user performs a jumping action on the somatosensory trampoline all-in-one machine. The skeletal feature data of the user is obtained through a skeletal recognition model. It should be noted that preferably, the skeletal recognition model for processing the user binding image and the first motion image data should be the same model.

[0056] Step S2, determining skeletal data according to the skeletal recognition model and the first motion image data;

[0057] In the embodiment, the skeletal recognition model can be a high-resolution network (HRNet) or a media pipe pose estimation (MediaPipe Pose). The algorithm for recognizing the skeleton is not limited in the embodiment. After the skeleton is recognized, the user skeletal feature data of the user is determined by the pixel value occupied by the skeleton. This is because the distance between the user and the camera of the device is relatively fixed when the user interacts with the machine, for example, the user steps on the trampoline of the somatosensory trampoline all-in-one machine.

[0058] Step S3, determining the user motion action of the target user according to the user skeletal feature data and the skeletal data;

[0059] According to the user skeletal feature data, the recognized skeletal data is classified to determine the user corresponding skeleton and the non-user corresponding skeleton, and the corresponding posture is recognized according to the user corresponding skeleton, so that the user motion action can be obtained.

[0060] Step S4, generating corresponding interaction data according to the user motion action.

[0061] Optionally, corresponding interaction instructions are generated according to the user motion action, and the operation of a preset role of the somatosensory trampoline all-in-one machine is controlled according to the interaction instructions, for example, behavior and movement control of the role.

[0062] In the embodiment, the user binding image and the first motion image data are acquired by controlling the image acquisition device, the user skeleton feature data is generated according to the user binding image, the skeleton data is determined according to the skeleton recognition model and the first motion image data, compared with the current scheme, when other personnel are often present near the entertainment interactive device, the skeleton information of the user can be effectively screened out, the user motion action of the target user is determined according to the user skeleton feature data and the skeleton data, the accuracy of the user recognition action is improved, and then the corresponding interaction data is generated according to the user motion action, the accuracy of the device interaction is improved, and the playing experience of the user is improved.

[0063] Further, based on the first embodiment, a second embodiment of the multi-person interaction control method based on camera positioning and skeleton posture recognition is proposed. Figure 5 The step of determining the user motion action of the user according to the user skeleton feature data and the skeleton data includes:

[0064] In step S31, the skeleton data is grouped according to the user skeleton feature data, to obtain first-type skeleton data and second-type skeleton data, the first-type skeleton data is the skeleton data associated with the target user, and the second-type skeleton data is data other than the first-type skeleton data in the skeleton data.

[0065] It should be noted that the first-type skeleton data here refers to the movement of the user's body skeleton in the image due to changes in the interactive process, which appears in different positions. The second-type skeleton data here is generally the skeleton corresponding to other personnel near the user. Screening based on skeleton features can effectively distinguish different types of skeleton data. In some other embodiments, some interactive devices can be used by more than one user, at this time, the user skeleton feature data here can include corresponding data of different users, for example: the user can be more than one, and the data can include user skeleton feature information corresponding to user 1, user skeleton feature information corresponding to user 2, and user skeleton feature information corresponding to user 3, etc. At this time, the first-type skeleton data here can also mark the corresponding user label.

[0066] In step S32, the first-type skeleton data and the action database are matched to determine a matching result.

[0067] The first type of skeleton data is matched with the action database. In general, the first type of skeleton data includes time sequence number and coordinate data of nodes at both ends of a bone, and the coordinate data of a joint connecting the bone can be optionally saved, so as to effectively reduce the data volume. It should be noted that, for a bone without a joint, the first type of skeleton data is connected by means of proximity search, for example, the nearest bones are connected or associated. The position of the joint is determined. The first type of skeleton data is matched with the action database by means of feature matching. The matching feature is the angle between the bones connected by the joint, and the distance between the bone and a standard position is optionally used as the matching feature. For example, the joint angle of a standing action is different from that of a squatting action, and the distance between the trunk and a standard position is also different.

[0068] In step S33, the user motion action is determined according to the matching result.

[0069] In this embodiment, the action with the highest similarity in the action database is determined as the user motion action. It should be noted that the design of the action database is related to the function of the interactive device. For example, for a somatosensory trampoline integrated machine, the action database has a higher correlation with jumping actions.

[0070] In this embodiment, the skeleton data is grouped by means of the user skeleton feature data to obtain first type of skeleton data and second type of skeleton data, the first type of skeleton data is matched with the action database to determine a matching result, and the user motion action is determined according to the matching result, so that the action of the user can be effectively recognized, and the action of a non-user can be avoided from being misrecognized.

[0071] Further, the user skeleton feature data includes the number of user skeletons of the target user and sub-feature data of each user skeleton of the target user, the sub-feature data includes the length of a bone, a first ratio of the length of the bone to the length of a body, a second ratio of the length of the bone to the girth of a limb, and the number of joint connecting points, the skeleton data includes skeleton feature information of each recognized unlabeled bone, and the step of grouping the skeleton data according to the user skeleton feature data includes:

[0072] The similarity is calculated according to the skeleton feature information and the sub-feature data of each bone to obtain a plurality of similarities.

[0073] In the embodiment, the user bone feature data is not a single data but a data set. Specifically, since the user bone feature data includes information of more than one user bone, the sub-feature data of each user bone is recorded. After the bone length is recognized by an algorithm to the corresponding pixel point in the image, the bone length needs to be calculated by a conversion coefficient. The body length is the length between the two ends of the spine. The limb girth refers to the girth of the part of the human body corresponding to the user bone, such as the waistline and the thigh girth. The limb girth is determined by the number of pixels corresponding to the part extracted by image recognition. The bone length, the first ratio of the bone length and the body length, the second ratio of the bone length and the limb girth, and the number of bone connection points are generally default settings, which can be adaptively adjusted for specific personnel. In some embodiments, the sub-feature data can not include the number of bone connection points.

[0074] According to the plurality of similarities, an unmarked bone corresponding to each user bone is determined, and corresponding marking information is generated;

[0075] It should be noted that since the bone feature information of each user is calculated with the sub-feature data, and the bones of the user themselves can be similar, for example, the bones of the left arm and the right arm are similar. Therefore, it is necessary to avoid repeated marking. Therefore, it is necessary to determine whether the unmarked bone has been marked before marking. After marking, the unmarked bone represents the bone corresponding to the user bone.

[0076] According to the plurality of marking information and the number of user bones, the bone data is grouped.

[0077] Optionally, it is determined whether the number of bones with marking information in the first motion image data is the same as the number of user bones, and the plurality of bones identified in the bone data, i.e., the first motion image data, are grouped according to the marking information to obtain first-type bone data and second-type bone data. In addition, in some interactive devices, when there are multiple users, one type of bone data is further grouped according to different users.

[0078] In the embodiment, the similarity is calculated by the bone feature information and the sub-feature data of each bone to obtain a plurality of similarities. According to the plurality of similarities, an unmarked bone corresponding to each user bone is determined, and corresponding marking information is generated. According to the plurality of marking information and the number of user bones, the bone data is grouped. The recognized bones are grouped by the user's limb features, and subsequent errors in recognizing user actions can be avoided.

[0079] Further, based on the first embodiment or the second embodiment, the third embodiment of the multi-person interaction control method based on camera positioning and skeleton posture recognition is proposed, in which the step of grouping the skeleton data according to the plurality of marker information and the user skeleton quantity further comprises the following steps before the grouping step:

[0080] Adjusting the user skeleton quantity according to a reference user motion action before the current time, wherein the time difference between the reference user motion action and the first motion image data is less than a preset time.

[0081] In this embodiment, it should be noted that the user's action is not single, and for the user's continuous action, the user skeleton quantity needs to be predicted according to the reference user motion action and the interaction logic, and preferably, the user skeleton quantity can be dynamically adjusted. The purpose of this embodiment is to avoid abnormality caused by mutual occlusion of part of the motion skeleton, which will cause the first motion image data to not include part of the user's joints. For example, in the interactive control process, different control modes are entered, such as the user turning to control the motion of the virtual character of the somatosensory trampoline integrated machine.

[0082] In this embodiment, the user skeleton quantity is adjusted according to the reference user motion action before the current time, so as to improve the accuracy of the data matching.

[0083] Further, based on any of the above embodiments, the fourth embodiment of the multi-person interaction control method based on camera positioning and skeleton posture recognition is proposed, in which the interactive device further comprises a pressure sensor, and the step of determining the user motion action of the target user according to the user skeleton feature data and the skeleton data further comprises the following steps before the determining step:

[0084] Obtaining detection data of the pressure sensor;

[0085] When the detection data of the pressure sensor is greater than the pressure threshold, the skeleton data is not adjusted;

[0086] When the detection data of the pressure sensor is less than or equal to the pressure threshold, the data of the skeleton at the position corresponding to the pressure sensor in the skeleton data is deleted.

[0087] It should be noted that the somatosensory trampoline integrated machine generally sets the trampoline and the corresponding sensing sensor for judging the force of the user on the device. The sensing sensor can be a pressure sensor, which is specifically arranged on the inner surface of the trampoline. When the detection data of the pressure sensor is greater than the pressure threshold, it indicates that the user contacts the trampoline. When the detection data of the pressure sensor is less than or equal to the pressure threshold, it indicates that the user does not contact the trampoline, and therefore, the skeleton with the lower end height lower than the preset height can be excluded, for example, the skeleton of the audience of the somatosensory trampoline integrated machine is in contact with the ground, and therefore, its height is lower than the preset height. This is a way of distinguishing the user's skeleton and the non-user's skeleton in the somatosensory trampoline integrated machine in the present application, which can effectively reduce the amount of skeleton data. Especially suitable for use in various complex scenes. The pressure threshold value can be set to zero.

[0088] Further, the number of pressure sensors is more than one, and before the step of determining the skeleton data according to the skeleton recognition model and the first motion image data, the method further comprises:

[0089] determining the position information of the user on the base of the interactive device according to the plurality of detection data;

[0090] adjusting the image range of the first motion image data according to the position information;

[0091] Before the step of determining the user motion action of the target user according to the user skeleton feature data and the skeleton data, the method further comprises:

[0092] adjusting the user skeleton feature data according to the position information.

[0093] In the present embodiment, it should be noted that the somatosensory trampoline integrated machine can include more than one user playing at the same time. The number of sensing sensors can be multiple and arranged at different positions, which generally indicate different indicators to enable the user to interact according to the indicators. The position information of the user on the base of the interactive device is determined according to the plurality of detection data. Thus, the position of the user's skeleton in the image of the first motion image data can be limited. Further, the image range of the first motion image data is adjusted by cropping. Further, since the distance from the image acquisition device corresponds to the pixel amount occupied in the image is not the same. According to the adjustment of the user skeleton feature data according to the position information, the closer the position information to the camera, the more pixels the corresponding skeleton occupies, and therefore, the conversion coefficient in Embodiment 2 needs to be adjusted.

[0094] Thus, the abnormal situation of the user skeleton feature data can be avoided in the case of multiple available positions of multiple users.

[0095] In the embodiment, for the somatosensory trampoline all-in-one machine for multi-person interaction, the position information of the user on the base of the interactive device can be determined through multiple detection data, the image range of the first motion image data is adjusted according to the position information, so as to reduce the amount of bones to be recognized, and the user bone feature data is adjusted according to the position information, so as to improve the accuracy of the user bone feature data.

[0096] Further, based on any of the above embodiments, the fifth embodiment of the multi-person interaction control method based on camera positioning and skeleton posture recognition is proposed. In the embodiment, the interaction data includes video image data, and the step of generating corresponding interaction data according to the user motion action includes:

[0097] controlling virtual action data of a virtual character according to the user motion action and a control mapping relationship;

[0098] interacting with a virtual game scene according to the virtual action, and generating corresponding video image data.

[0099] In the embodiment, the virtual action data of the virtual character is controlled according to the user motion action and the control mapping relationship, so as to improve the accuracy of the action.

[0100] In addition, the embodiment of the present application also proposes a multi-person interaction control system based on camera positioning and skeleton posture recognition, which comprises:

[0101] The acquisition module is configured to control an image acquisition device to acquire user binding image and first motion image data, and generate user bone feature data according to the user binding image, wherein the user binding image is an image for the interactive device to identify the identity of a target user.

[0102] The identification module is configured to determine bone data according to a skeleton recognition model and the first motion image data.

[0103] The analysis module is configured to determine a user motion action of the target user according to the user bone feature data and the bone data.

[0104] The interaction module is configured to generate corresponding interaction data according to the user motion action.

[0105] In addition, the embodiment of the present application also proposes an interactive device, which comprises a memory, a processor, and a multi-person interaction control program based on camera positioning and skeleton posture recognition stored in the memory and executable on the processor. The multi-person interaction control program based on camera positioning and skeleton posture recognition is configured to implement the steps of the multi-person interaction control method based on camera positioning and skeleton posture recognition according to any of the above embodiments.

[0106] In addition, the embodiment of the present application further provides a storage medium, wherein the storage medium stores a multi-person interaction control program based on camera positioning and skeleton posture recognition, and the multi-person interaction control program based on camera positioning and skeleton posture recognition realizes the steps of the multi-person interaction control method based on camera positioning and skeleton posture recognition when executed by a processor.

[0107] It should be noted that, in this document, the terms "comprising", "containing" or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article or system that includes a list of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent in such a process, method, article or system. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of other identical elements in the process, method, article or system that includes the element.

[0108] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0109] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, of course, they can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, an optical disk) as described above, and includes a plurality of instructions for making a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present application.

[0110] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A multi-person interaction control method based on camera positioning and bone posture recognition, characterized in that, The application is applied to interactive equipment, and the multi-person interaction control method based on camera positioning and skeleton posture recognition comprises the following steps. An image acquisition device is used to acquire user binding images and first motion image data, and user skeleton feature data is generated according to the user binding images, wherein the user binding images are images used by the interactive equipment to identify the identity of a target user. Skeleton data is determined according to a skeleton recognition model and the first motion image data. User motion actions of the target user are determined according to the user skeleton feature data and the skeleton data. Corresponding interaction data is generated according to the user motion actions. The step of determining user motion actions of the target user according to the user skeleton feature data and the skeleton data comprises the following steps. The skeleton data is grouped according to the user skeleton feature data to obtain first skeleton data and second skeleton data, wherein the first skeleton data is skeleton data associated with the target user, and the second skeleton data is data other than the first skeleton data in the skeleton data. The first skeleton data is matched with an action database to determine a matching result. The user motion actions are determined according to the matching result. The user skeleton feature data comprises the number of user skeletons of the target user and sub-feature data of each user skeleton of the target user, wherein the sub-feature data comprises the length of a skeleton, a first ratio of the length of the skeleton to the length of a body, a second ratio of the length of the skeleton to the girth of a limb, and the number of skeleton connection points, and the skeleton data comprises skeleton feature information of each recognized unlabeled skeleton. The skeleton data is grouped according to the skeleton feature information and the sub-feature data of each skeleton to obtain a plurality of similarities. Each user skeleton corresponding to an unlabeled skeleton is determined according to the plurality of similarities, and corresponding labeling information is generated. The skeleton data is grouped according to the plurality of labeling information and the number of user skeletons. 2.The multi-user interaction control method based on camera positioning and bone posture recognition of claim 1, wherein, Before the step of grouping the skeleton data according to the plurality of labeling information and the number of user skeletons, the following step is further included. The number of user skeletons is adjusted according to reference user motion actions before the current time, wherein the time difference between the reference user motion actions and the time of the first motion image data is less than a preset time. 3.The multi-user interaction control method based on camera positioning and bone posture recognition of claim 1, wherein, The interactive equipment further comprises a pressure sensor, and before the step of determining user motion actions of the target user according to the user skeleton feature data and the skeleton data, the following steps are further included. Detection data of the pressure sensor is acquired. When the detection data of the pressure sensor is greater than a pressure threshold, the skeleton data is not adjusted. When the detection data of the pressure sensor is less than or equal to the pressure threshold, the data of the skeleton at the corresponding position in the pressure sensor in the skeleton data is deleted. 4.The multi-user interaction control method based on camera positioning and bone posture recognition of claim 3, wherein, The number of pressure sensors is more than one, and before the step of determining skeleton data according to a skeleton recognition model and the first motion image data, the following step is further included. The position information of the user on the base of the interactive equipment is determined according to a plurality of detection data. adjust an image range of the first motion image data according to the position information; the step of determining the user motion action of the target user according to the user skeleton feature data and the skeleton data further comprises: adjusting the user skeleton feature data according to the position information.

5. The multi-person interaction control method based on video positioning and bone posture recognition according to any one of claims 1 to 4, characterized in that, the interaction data comprises video image data, and the step of generating corresponding interaction data according to the user motion action comprises: controlling virtual action data of a virtual character according to the user motion action and a control mapping relationship; interacting with a virtual game scene according to the virtual action, and generating corresponding video image data.

6. A multi-person interaction control system based on camera positioning and bone pose recognition, characterized in that, the multi-person interaction control system based on camera positioning and skeleton posture recognition comprises: a collection module configured to control an image collection device to acquire user binding image and first motion image data, and generate user skeleton feature data according to the user binding image, the user binding image being an image for identifying a target user identity by an interactive device; an identification module configured to determine skeleton data according to a skeleton recognition model and the first motion image data; an analysis module configured to determine a user motion action of the target user according to the user skeleton feature data and the skeleton data; the step of determining the user motion action of the target user according to the user skeleton feature data and the skeleton data comprises: grouping the skeleton data according to the user skeleton feature data to obtain first skeleton data and second skeleton data, the first skeleton data being skeleton data associated with the target user, and the second skeleton data being data other than the first skeleton data in the skeleton data; matching the first skeleton data and an action database to determine a matching result; determining the user motion action according to the matching result; the user skeleton feature data comprises a number of user skeletons of the target user and sub-feature data of each user skeleton of the target user, the sub-feature data comprising a skeleton length, a first ratio of the skeleton length to a body length, a second ratio of the skeleton length to a limb girth, and a number of skeleton connection points, and the skeleton data comprises skeleton feature information of each recognized unlabeled skeleton, the step of grouping the skeleton data according to the user skeleton feature data comprises: calculating a plurality of similarities according to the skeleton feature information and the sub-feature data of each skeleton; determining an unlabeled skeleton corresponding to each user skeleton according to the plurality of similarities, and generating corresponding labeling information; grouping the skeleton data according to the plurality of labeling information and the number of user skeletons; an interaction module configured to generate corresponding interaction data according to the user motion action.

7. An interactive device, characterized by the interactive device comprises a memory, a processor, and a multi-person interaction control program based on camera positioning and skeleton posture recognition stored on the memory and executable on the processor, the multi-person interaction control program based on camera positioning and skeleton posture recognition being configured to implement the steps of the multi-person interaction control method based on camera positioning and skeleton posture recognition of any one of claims 1 to 5.

8. A storage medium, characterized by The storage medium stores a multi-person interaction control program based on camera positioning and bone posture recognition. When the multi-person interaction control program based on camera positioning and bone posture recognition is executed by the processor, the steps of the multi-person interaction control method based on camera positioning and bone posture recognition according to any one of claims 1 to 5 are implemented.

Images