Human-computer interaction methods and systems for online games

By acquiring and analyzing the edge features of the hand area in interactive videos in real time, the hand interaction area is filtered out and the hand gesture is recognized, which solves the problem of low gesture recognition accuracy in online games and achieves efficient and accurate gesture recognition and interactive experience.

CN120748038BActive Publication Date: 2026-01-30SHANGRAO XINXIN INFORMATION TECHNOLOGY CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510817180.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-06-18
Publication Date
2026-01-30
Estimated Expiration
2045-06-18

AI Technical Summary

Technical Problem

In existing technologies, the low accuracy of gesture recognition during player interaction with online games results in low response accuracy and affects the user experience.

Method used

By acquiring interactive video in real time, the edge contours of suspected hand areas are extracted, the tangent changes and distance features of edge pixels are analyzed, evaluation values ​​are calculated, hand interaction areas are screened out, and the relative position information of finger joints is identified to achieve accurate gesture recognition.

Benefits of technology

It improves the accuracy of gesture recognition, thereby enhancing the responsiveness of online games and the player's interactive experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120748038B_ABST
    Figure CN120748038B_ABST
Patent Text Reader

Abstract

This application relates to the field of gesture interaction technology, specifically to a human-computer interaction method and system for online games. The method includes: acquiring interactive video in real-time during player interaction with the online game; extracting each frame of the interactive video to obtain suspected hand regions; determining a first evaluation value and a second evaluation value for each suspected hand region in each frame of the interactive video; determining a discrimination coefficient for each suspected hand region in each frame of the interactive video; filtering all suspected hand regions in each frame of the interactive video to obtain each hand interaction region; extracting the relative position information of finger joints within the hand interaction region; recognizing the player's gestures; and the online game device issuing corresponding operation commands based on the recognized gestures. This application can accurately segment the player's hand region, improving the accuracy of player gesture recognition and thus enhancing the player's human-computer interaction experience with the online game.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of gesture interaction technology, specifically to human-computer interaction methods and systems for online games. Background Technology

[0002] With the development of smart technology and the popularization of electronic technology, online games equipped with motion-sensing devices have become increasingly popular. These games allow players to control and operate the devices in various ways, with the devices providing corresponding feedback based on the received instructions. Therefore, the interactivity between players and online games has become a crucial indicator for achieving intelligent functionality and judging the ease of use of devices, leading to the emergence of various advanced human-computer interaction technologies.

[0003] Among these methods, human-computer interaction through player gestures is highly efficient and convenient. Accurate recognition of player gestures is the most crucial aspect of this interaction. However, factors such as the diversity of hand postures, background complexity, and lighting variations can lead to inaccurate segmentation of the player's hand area, affecting the precision of gesture recognition. This results in the inability to accurately capture the player's hand posture information, leading to lower response accuracy in online games and a poor user experience. Summary of the Invention

[0004] To address the aforementioned technical issues, a human-computer interaction method and system for online games are provided to resolve existing problems.

[0005] The solution to the technical problem in this application is to provide a human-computer interaction method and system for online games, including the following steps:

[0006] In a first aspect, embodiments of this application provide a human-computer interaction method for online games, the method comprising the following steps:

[0007] The system acquires interactive video in real time during player interaction with the online game; extracts each frame of interactive image from the interactive video to obtain each suspected hand region, and extracts each edge contour within the suspected hand region.

[0008] The first evaluation value of each suspected hand region in each frame of interactive image is determined by the changing trend of the tangents of the edge pixels on different edge contours in each suspected hand region in each frame of interactive image, and the average length of all edge contours.

[0009] Analyze the distance features between any suspected hand region and different suspected hand regions in the other frames of the interactive image to obtain the matching hand region; determine the second evaluation value of any suspected hand region by the distance relationship between the any suspected hand region and the matching hand region in the other frames of the interactive image, as well as the length difference features of the edge contour.

[0010] Based on the first evaluation value and the second evaluation value, the discrimination coefficient of each suspected hand region in each frame of interactive image is determined. Based on the discrimination coefficient, all suspected hand regions in each frame of interactive image are filtered to obtain each hand interaction region.

[0011] The relative position information of the finger joints within the hand interaction area is extracted to recognize the player's gestures. The online game device then issues corresponding operation commands based on the recognized gestures.

[0012] Preferably, the step of obtaining each suspected hand region includes: extracting the ROI region in each frame of the interactive image using the Holistic algorithm in the Media Pipe framework, and recording it as each suspected hand region.

[0013] Preferably, determining the first evaluation value for each suspected hand region in each frame of the interactive image includes:

[0014] Analyze the consistency of the tangent slope of the edge pixels on different edge contours in each suspected hand region in each frame of interactive image, and calculate the edge parallelism of each suspected hand region;

[0015] Calculate the mean length of all edge contours within each suspected hand region, and denot it as the average length;

[0016] The first evaluation value is the ratio of the edge parallelism to the average length.

[0017] Preferably, the process for determining the edge parallelism is as follows:

[0018] Calculate the average slope of the tangent lines at all edge pixels on each edge contour;

[0019] The edge parallelism is the maximum number of all edge contours with the same average value in each suspected hand area.

[0020] Preferably, the process of obtaining the matching hand region is as follows:

[0021] Calculate the distance between the centroid of any suspected hand region in each frame of interactive image and the centroids of all suspected hand regions in adjacent frames of interactive image. The suspected hand region with the smallest distance in adjacent frames of interactive image is recorded as the matched hand region.

[0022] Preferably, determining the second assessment value for any suspected hand region includes:

[0023] The sum of the distances between any suspected hand region and the matching hand regions in all other frame interaction images is taken as the displacement change of any suspected hand region.

[0024] The sum of the differences in the average length between any suspected hand region and the matching hand regions in all other frame interaction images is taken as the gesture change amount of any suspected hand region.

[0025] The second evaluation value is the sum of the displacement change and the gesture change.

[0026] Preferably, the discrimination coefficient is the product of the first evaluation value and the second evaluation value.

[0027] Preferably, the process of obtaining each hand interaction area is as follows:

[0028] Obtain the segmentation threshold of the discrimination coefficients of all suspected hand regions in all frame interactive images;

[0029] In each frame of interactive image, suspected hand regions with a discrimination coefficient greater than or equal to the segmentation threshold are filtered out; according to the discrimination coefficient, all the filtered suspected hand regions are sorted in descending order, and a preset number of suspected hand regions at the top of the sort are selected and recorded as hand interaction regions.

[0030] Preferably, the relative position information of the finger joints within the hand interaction area is extracted to recognize the gestures. The online game then issues corresponding operation commands based on the recognized gestures, including:

[0031] The Hand algorithm in the Media pipe framework is used to obtain the coordinates of all finger key points in each hand interaction area within each frame of interactive image, and the player's hand gestures are identified by the relative positional relationship of the coordinates of all finger key points in each hand interaction area.

[0032] The online gaming device matches the received gestures with a predefined gesture library. If the match is successful, the online gaming device executes the corresponding operation command.

[0033] Secondly, embodiments of this application also provide a human-computer interaction system for online games, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described human-computer interaction methods for online games.

[0034] This application has at least the following beneficial effects:

[0035] This application extracts suspected hand regions from interaction images during player-game interaction. It analyzes the parallelism and length variations of the edge contours within each suspected hand region to calculate a first evaluation value for each suspected hand region in each frame of the interaction image. The beneficial effect is that it considers the characteristics of wrinkles and stripes generated during gesture changes within the hand region, thus initially assessing the likelihood that the suspected hand region corresponds to a hand, thereby eliminating interference areas in the background and improving the accuracy of subsequent hand region segmentation. Secondly, it calculates a second evaluation value for each suspected hand region. The beneficial effect of this is that it considers the motion and edge contour length differences of each suspected hand region in different frames of the interaction image, further assessing the likelihood that the suspected hand region corresponds to the player's hand, eliminating interference areas in the background. The process involves removing other players' hands to further improve the accuracy of subsequent hand region segmentation; determining the discrimination coefficient of each suspected hand region in each frame of the interactive image, and filtering all suspected hand regions in each frame of the interactive image to obtain each hand interaction region. The beneficial effect is that it can segment the player's hand region, thereby improving the accuracy of player hand gesture recognition; extracting the relative position information of the finger joints within the hand interaction region to recognize the player's gestures, and the online game device makes corresponding operation commands based on the recognized gestures. The beneficial effect is that by recognizing the player's gestures, the online game device can make corresponding operation commands, enabling the online game to quickly, efficiently, and accurately capture the player's gesture information, improving the accuracy of player gesture recognition, and enhancing the player's human-computer interaction experience with the online game. Attached Figure Description

[0036] The human-computer interaction method for online games of this application will be further described in detail below with reference to the accompanying drawings.

[0037] Figure 1 A flowchart illustrating the steps of a human-computer interaction method for online games provided in this application embodiment;

[0038] Figure 2 A flowchart illustrating the steps of a method for obtaining the first evaluation value of each suspected hand region in each frame of an interactive image provided in this application embodiment;

[0039] Figure 3 A flowchart illustrating the steps of a method for obtaining a second evaluation value for any suspected hand region provided in an embodiment of this application. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description of the human-computer interaction method and system for online games, in conjunction with the accompanying drawings and implementation examples, is provided. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0042] Please see Figure 1 The diagram illustrates a flowchart of a human-computer interaction method for online games according to an embodiment of this application. The method includes the following steps:

[0043] Step 1: Acquire interactive video in real time during the interaction between the player and the online game; extract each frame of interactive image in the interactive video to obtain each suspected hand area, and extract each edge contour within the suspected hand area.

[0044] Human-computer interaction (HCI) in online games refers to the communication between players and game devices through various input devices. The game devices then provide corresponding feedback based on the received instructions, thus achieving interaction between the player and the game. Among these interactions, accurately recognizing player gestures is the most crucial aspect. However, factors such as the diversity of hand postures, background complexity, and changes in lighting can affect the accuracy of gesture recognition.

[0045] To improve the accuracy of player gesture recognition, the game console is equipped with a high-definition camera to capture the player's interaction video in real time when the player interacts with the game through hand gestures, and the noise is removed from all frames of the interaction video.

[0046] In this embodiment, the time interval for acquisition is 0.2s. As other implementation methods, the implementer can set it according to the actual situation. Secondly, the non-local means (NLM) filtering algorithm is used to denoise all the acquired frame interactive images. The NLM algorithm is a well-known technology and will not be described in detail here. As other implementation methods, the implementer can use other methods of the prior art, such as median filtering algorithm, etc. This embodiment does not impose any special restrictions on this.

[0047] Secondly, Media Pipe technology is an open-source machine learning framework that can accurately detect and track hand keypoints in real time. Therefore, it extracts the Region of Interest (ROI) in each frame of the interactive image that may contain the hand and performs edge detection on it, specifically:

[0048] The Holistic algorithm in the Media pipe framework is used to detect the ROI regions in each frame of interactive images and record them as each suspected hand region.

[0049] It should be noted that the Holistic algorithm in the Media Pipe framework is a well-known technique and will not be elaborated upon here.

[0050] Edge detection is performed on each suspected hand region in each frame of interactive image, and the edge contours within the suspected hand regions are extracted;

[0051] In this embodiment, the Canny edge detection algorithm is used for edge detection. The Canny edge detection algorithm is a well-known technology and will not be described in detail here. As other implementation methods, implementers can use other methods of existing technology, such as the Sobel operator, etc. This embodiment does not impose any special restrictions on this.

[0052] At this point, the edge contours of each suspected hand area in each frame of the interactive image are obtained.

[0053] Step 2: Determine the first evaluation value of each suspected hand region in each frame of interactive image by the changing trend of the tangents of the edge pixels on different edge contours in each suspected hand region in each frame of interactive image, and the average length of all edge contours.

[0054] The flowchart of the method for obtaining the first evaluation value of each suspected hand region in each frame of interactive image provided in this application embodiment is as follows: Figure 2 As shown.

[0055] Furthermore, during human-computer interaction, players will make different hand gestures, which will result in different wrinkles at the hand joints. The wrinkles formed when the hand joints are bent are usually highly parallel to each other, resulting in many highly parallel edge contours in the image of the hand area.

[0056] Based on the above analysis, the parallelism of the edges is calculated by analyzing the tangent change direction of edge pixels on different edge contours, in order to evaluate the parallelism of the edge contours within the suspected hand region. Specifically:

[0057] Calculate the average slope of the tangent lines at all edge pixels on each edge contour;

[0058] The maximum value of the number of all edge contours with the same average value in each suspected hand region is used as the edge parallelism of each suspected hand region in each frame of interactive image;

[0059] It should be noted that the greater the edge parallelism, the higher the degree of parallelism between different edge contours within the suspected hand area, and the more likely the suspected hand area is the area where a human hand is located.

[0060] Secondly, each frame of the interactive image may also contain the edges of other objects, which may also have parallel edges. Therefore, it is necessary to reduce the interference of such edges. Generally, the wrinkles and stripes generated by the player's hand are limited by the size and curvature of the user's palm, and the length of the stripes is relatively small. Therefore, by analyzing the length variation of each edge contour and combining it with the edge parallelism, the first evaluation value is determined, specifically:

[0061] Calculate the mean length of all edge contours within each suspected hand region, and denot it as the average length;

[0062] The ratio of the edge parallelism to the average length is used as the first evaluation value for each suspected hand region in each frame of interactive image.

[0063] It should be noted that the smaller the average length, the smaller the length of the edge contour within the suspected hand area, and the larger the obtained first evaluation value, the more likely the suspected hand area is the area corresponding to a human hand.

[0064] At this point, the first evaluation value of each suspected hand region in each frame of the interactive image is obtained.

[0065] Step 3: Analyze the distance features between any suspected hand region and different suspected hand regions in the other frames of the interactive image to obtain the matching hand region; determine the second evaluation value of any suspected hand region by the distance relationship between the suspected hand region and the matching hand region in the other frames of the interactive image, as well as the length difference features of the edge contour.

[0066] During actual player interaction, the objects in the background are not static; there may be hand areas belonging to other people besides the player. Therefore, it is necessary to distinguish the suspected hand areas of other people in each frame of the interactive image to prevent them from being mistakenly identified as the player's hand areas.

[0067] Secondly, players actually interacting with the game need to remain within the capture range of the game device's high-definition camera. Therefore, the player's hand area is usually present in the interactive image for a long time, with little chance of being obscured. The hand position and gestures change significantly, which is reflected in the interactive image as obvious stripe changes within the hand area, and a large range of movement across multiple frames. For other people not interacting, they make small movements within the camera's range or their position remains largely unchanged, and the changes in their hand area are relatively minimal.

[0068] Based on the above analysis, the motion changes and gesture changes of each suspected hand region within multiple frames of interactive images are analyzed to determine a second evaluation value. The flowchart of the method for obtaining the second evaluation value of any suspected hand region provided in this application embodiment is as follows: Figure 3 As shown, it specifically includes:

[0069] Calculate the distance between the centroid of any suspected hand region in each frame of interactive image and the centroids of all suspected hand regions in adjacent frame interactive images. The suspected hand region with the smallest distance in adjacent frame interactive images is recorded as the matched hand region.

[0070] In this embodiment, the distance is measured by calculating the Euclidean distance between the centroid of any suspected hand region in each frame of the interactive image and the centroids of all suspected hand regions in the next frame of the interactive image. Euclidean distance is a well-known technique and will not be described in detail here. As other implementation methods, implementers may use other methods of the prior art, such as Manhattan distance, etc. This embodiment does not impose any special restrictions on this.

[0071] It should be noted that, for ease of understanding, the distance between the centroid of the i-th suspected hand region in the n-th interactive image and the centroids of all suspected hand regions in the (n+1)-th interactive image is calculated. Assuming the k-th suspected hand region with the smallest distance in the (n+1)-th interactive image is taken as the matching hand region of the i-th suspected hand region, then the same method is used to calculate the distance between the centroid of the k-th suspected hand region in the (n+1)-th interactive image and the centroids of all suspected hand regions in the (n+2)-th interactive image. The r-th suspected hand region with the smallest distance in the (n+2)-th interactive image is taken as the matching hand region of the k-th suspected hand region. Therefore, the positions of the matching hand regions corresponding to the suspected hand regions in all frames of the interactive image can be tracked.

[0072] The sum of the distances between any suspected hand region and the matching hand regions in all of its frame interaction images is taken as the displacement change of any suspected hand region.

[0073] The sum of the differences in the average length between any suspected hand region and the matching hand regions in all other frame interaction images is taken as the gesture change amount of any suspected hand region.

[0074] The sum of the displacement change and the gesture change is used as the second evaluation value for any suspected hand region;

[0075] It should be noted that the greater the displacement change, the greater the positional fluctuation of the suspected hand area; the greater the gesture change, the greater the degree of gesture change in the suspected hand area; and the greater the second evaluation value, the more likely the suspected hand area is the location of the player's hand during game interaction.

[0076] Thus, a second evaluation value is obtained for any of the suspected hand regions.

[0077] Step 4: Based on the first evaluation value and the second evaluation value, determine the discrimination coefficient of each suspected hand region in each frame of interactive image. Based on the discrimination coefficient, filter all suspected hand regions in each frame of interactive image to obtain each hand interaction region. Extract the relative position information of the finger joints in the hand interaction region to recognize the player's gestures. The online game device makes corresponding operation commands based on the recognized gestures.

[0078] Furthermore, based on the first evaluation value and the second evaluation value, a discrimination coefficient is determined to assess the probability that the corresponding suspected hand area belongs to the player, specifically:

[0079] The product of the first evaluation value and the second evaluation value is used as the discrimination coefficient for each suspected hand region in each frame of interactive image;

[0080] Obtain the segmentation threshold of the discrimination coefficient of all suspected hand regions in all frame interactive images, and filter the suspected hand regions in each frame interactive image whose discrimination coefficient is greater than or equal to the segmentation threshold; according to the discrimination coefficient, sort all the filtered suspected hand regions in descending order, and select a preset number of suspected hand regions at the top of the sort, which are recorded as hand interaction regions;

[0081] In this embodiment, a cross-validation method is used to obtain the segmentation threshold. The cross-validation method is a well-known technology and will not be described in detail here. As other implementation methods, implementers can use other methods of the prior art, such as the Otsu threshold segmentation algorithm, etc. This embodiment does not impose any special restrictions on this. Secondly, the two suspected hand areas that appear to be in the top positions are selected as hand interaction areas. Since the player's hands will make gesture changes during human-computer interaction, the two selected hand interaction areas correspond to the areas where the player's hands are located.

[0082] It should be noted that if the number of suspected hand regions with a discrimination coefficient greater than or equal to the segmentation threshold in each frame of interactive image is less than 2, it is determined that the player is not interacting with the computer at this time.

[0083] Furthermore, gesture recognition is performed on all hand interaction areas within each frame of the interactive image to enable human-computer interaction with the game, specifically as follows:

[0084] The Hand algorithm in the Media pipe framework is used to obtain the coordinates of all finger key points in each hand interaction area within each frame of interactive image, and the player's hand gestures are identified by the relative positional relationship of the coordinates of all finger key points in each hand interaction area.

[0085] It should be noted that the Hand algorithm in the Media pipe framework is a well-known technology and will not be elaborated here; secondly, the Hand algorithm in the Media pipe framework obtains the position coordinates of 21 finger key points to describe the hand posture.

[0086] The online gaming device matches the received gestures with a predefined gesture library. If the match is successful, the online gaming device executes the corresponding operation command. If the match fails, the online gaming device identifies and matches the player's gestures in the next frame of the interactive image to complete the human-computer interaction.

[0087] It should be noted that the predefined gesture library is set in advance by the online game designers. Different gestures represent different operation commands. For example, if the player's gesture is to clench their fist, the corresponding operation command is for the game character to jump, accompanied by the sound effect of jumping.

[0088] Based on the same inventive concept as the above methods, this application also provides a human-computer interaction system for online games, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described human-computer interaction methods for online games.

[0089] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0090] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of this application, without departing from the content of the technical solution of this application, shall fall within the protection scope of the technical solution of this application.

Claims

1. A human-machine interaction method for network games, characterized in that, The method comprises the following steps: Real-time acquisition of interactive video during interaction of the player with the network game; extraction of each interactive image in the interactive video, acquisition of each suspected hand region, and extraction of each edge contour in the suspected hand region; Determination of a first evaluation value of each suspected hand region in each interactive image through the change trend of the tangent of each edge pixel point on different edge contours in each suspected hand region in each interactive image and the average level of the length of all edge contours; Analysis of the distance characteristics of any suspected hand region and different suspected hand regions in the remaining interactive images to obtain a matching hand region; determination of a second evaluation value of the any suspected hand region through the distance relationship between the any suspected hand region and the matching hand region in the remaining interactive images and the length difference characteristics of the edge contours; Determination of a discrimination coefficient of each suspected hand region in each interactive image based on the first evaluation value and the second evaluation value, screening of all suspected hand regions in each interactive image based on the discrimination coefficient, and acquisition of each hand interactive region; Extraction of the relative position information of the finger joint points in the hand interactive region, identification of the gesture action of the player, and generation of a corresponding operation instruction by the network game device according to the identified gesture action.

2. The network game-oriented human-machine interaction method according to claim 1, wherein, The acquisition of each suspected hand region comprises: extraction of an ROI region in each interactive image through a Holistic algorithm in a Media pipe framework, and recording of each suspected hand region.

3. The network game-oriented human-machine interaction method according to claim 1, wherein, The determination of the first evaluation value of each suspected hand region in each interactive image comprises: Analysis of the consistency of the tangent slope of each edge pixel point on different edge contours in each suspected hand region in each interactive image, calculation of the edge parallelism of each suspected hand region, calculation of the average value of the length of all edge contours in each suspected hand region, and recording of the average length; The first evaluation value is the ratio of the edge parallelism to the average length. The determination process of the edge parallelism comprises:

4. The network game-oriented human-machine interaction method according to claim 3, wherein, Calculation of the average value of the slope of the tangent of all edge pixel points on each edge contour; The edge parallelism is the maximum value of the number of all edge contours with the same average value in each suspected hand region. The acquisition process of the matching hand region comprises:

5. The network game-oriented human-machine interaction method according to claim 3, wherein, Calculation of the distance between the centroid of the any suspected hand region in each interactive image and the centroid of all suspected hand regions in the adjacent interactive image of the any suspected hand region, and recording of the suspected hand region corresponding to the minimum distance in the adjacent interactive image as the matching hand region. The determination of the second evaluation value of the any suspected hand region comprises:

6. The network game-oriented human-machine interaction method according to claim 5, wherein, Taking the sum of the distances between the any suspected hand region and the matching hand region in all the remaining interactive images as the displacement change amount of the any suspected hand region; Taking the sum of the differences between the average lengths of the any suspected hand region and the matching hand region in all the remaining interactive images as the gesture change amount of the any suspected hand region; The second evaluation value is the sum of the displacement change amount and the gesture change amount. The discrimination coefficient is the product of the first evaluation value and the second evaluation value.

7. The network game-oriented human-machine interaction method according to claim 1, wherein, ​ 8. The network game-oriented human-machine interaction method according to claim 1, wherein, The acquisition process of each hand interaction area is as follows: acquiring a segmentation threshold of a discrimination coefficient of all suspected hand areas in all frame interaction images; screening suspected hand areas in each frame interaction image whose discrimination coefficient is greater than or equal to the segmentation threshold; arranging all screened suspected hand areas in descending order according to the discrimination coefficient, selecting a preset number of suspected hand areas at the front of the arrangement, and marking the suspected hand areas as hand interaction areas.

9. The network game-oriented human-machine interaction method according to claim 1, wherein, extracting relative position information of finger joint points in the hand interaction areas, recognizing hand gesture actions, and making corresponding operation instructions of the network game according to the recognized hand gesture actions, including: acquiring all finger key point coordinates in each hand interaction area in each frame interaction image through a Hand algorithm in a Media pipe framework, and recognizing a player's hand gesture action through relative position relationships of all finger key point coordinates in each hand interaction area; matching the received hand gesture action with a predefined gesture library, and making corresponding operation instructions of the network game device if the matching is successful.

10. A human-machine interaction system for network games, comprising a memory, a processor and a computer program stored in the memory and running on the processor, characterized in that, The processor implements the steps of the human-machine interaction method for network games according to any one of claims 1-9 when executing the computer program.

Citation Information

Patent Citations

  • Manipulation input device and method, program, and recording medium

    CN105612473A

  • Depth image-based method for carrying out hand region segmentation by utilizing MeanShift algorithm

    CN107341811A