Method and related device for moving multiple discrete targets based on gesture recognition

Abstract

The present application relates to a method and related device for moving multiple discrete targets based on gesture recognition, the method comprising the following steps: S1. Determine the terminal position and establish a reference system, load and display the target point; S2. Generate a first reference plane surrounding the target point and display it on the terminal screen, wherein the first reference plane is cylindrical and the axis is vertically arranged; S3. Generate two visible normals on the first reference plane; S4. Move the two visible normals according to gesture recognition to construct a second reference plane and display it on the terminal screen, copy the second reference plane according to gesture recognition or determine the discrete material position on the second reference plane. The present application has the advantage of providing a more intuitive, convenient and layered material operation method within a limited screen space.

Description

Technical Field

[0001] The present application relates to the field of web games, and in particular to a method and related device for moving multiple discrete targets based on gesture recognition. Background Art

[0002] Mobile web games are a form of online games that run on mobile browsers. There is no need to download and install independent clients. Games can be loaded and operated through the browser. Compared with traditional client games, mobile web games are ready to use, do not require a lot of storage space, and are easy to access across platforms. Therefore, they have advantages in portability and ease of operation. However, due to the small size of mobile phone screens and the simplicity of the operating interface, mobile web games face some unique challenges in terms of operating experience.

[0003] At present, the operation of image materials in mobile web games mainly relies on directly pasting on the mobile phone page, and moving or rotating the materials by sliding the finger. Although this direct control method simplifies the operation process to a certain extent, there are many inconveniences in actual use. First of all, although the use of the finger control panel can avoid direct occlusion of the material, its operation experience is relatively unintuitive. Users need to perform additional operations to control the rotation or movement of the material, which is not conducive to improving the smoothness of the game. On the other hand, if the user directly uses his fingers to operate the map, the finger can easily block the map material, affecting the visual effect, resulting in a poor operation experience, especially when precise positioning is required, the occlusion problem is more obvious.

[0004] In addition, current mobile web games usually lack a clear sense of hierarchy when processing multiple image materials, and the relationship between multiple materials is difficult to clearly express. In this case, when users process multiple layers of materials, it is easy for the materials to overlap or become chaotic, which not only increases the difficulty of operation, but also may affect the aesthetics and user experience of the game. Therefore, how to provide a more intuitive, convenient and hierarchical material operation method within a limited screen space has become an urgent problem to be solved in the field of mobile web games. Summary of the invention

[0005] In order to provide a more intuitive, convenient and layered material operation method within a limited screen space, the present application provides a method and related device for moving multiple discrete targets based on gesture recognition.

[0006] In a first aspect, the present application provides a method for moving multiple discrete targets based on gesture recognition, which adopts the following technical solution:

[0007] A method for moving multiple discrete targets based on gesture recognition comprises the following steps:

[0008] S1. Determine the terminal position and establish a reference system, load and display the target point;

[0009] S2. Generate a first reference surface surrounding the target point and display it on the terminal screen, wherein the first reference surface is cylindrical and the axis is vertically arranged;

[0010] S3. generating two visible normal lines on the first reference surface;

[0011] S4. According to the gesture recognition, two visible normals are moved to construct a second reference plane and displayed on the terminal screen, and according to the gesture recognition, the second reference plane is copied or a discrete material position is determined on the second reference plane.

[0012] Optionally, the S1 includes the following steps:

[0013] S101. Obtain terminal location information based on satellite positioning;

[0014] S102. Acquire gyroscope information of the terminal, and establish a first spatial coordinate system based on the terminal position information and the gyroscope information, wherein the z-axis of the first spatial coordinate system is vertical;

[0015] S103. Load the world coordinate information of the target point, and convert the world coordinate information of the target point into coordinates in the first space coordinate system;

[0016] S104. Based on the coordinates of the target point in the first space coordinate system and the gyroscope information, display the target point on the terminal screen.

[0017] Optionally, S2 includes the following steps:

[0018] S201. Based on the coordinates of the target point in the first spatial coordinate system, generate a vertical axis passing through the target point in the first spatial coordinate system;

[0019] S202. Generate a cylindrical surface based on the axis as a first reference surface, wherein the radius of the first reference surface is a preset radius;

[0020] S203. Based on the gyroscope information, display the first reference plane on the terminal screen.

[0021] Optionally, S3 includes the following steps:

[0022] S301. Generate a first normal line at a position of the first reference plane close to the xz reference plane;

[0023] S302. Generate a second normal line at a position of the first reference plane close to the yz reference plane;

[0024] S303. Based on the gyroscope information, display the first normal and the second normal on the terminal screen as visible normals; wherein the transparency of the first normal and the second normal is lower than the transparency of the first reference surface.

[0025] Optionally, S4 includes the following steps:

[0026] S401. Acquire image information, identify the left hand or the right hand in the image, select a first normal based on the left hand, and select a second normal based on the right hand;

[0027] S402. Acquire image information, and identify the left fan hand gesture and the right fan hand gesture in the image information, and horizontally move the visible normal based on the left fan hand gesture and the right fan hand gesture; wherein the visible normal is standardized to move upward in the ring of the first reference plane;

[0028] S403. Acquire image information, identify a fist-clenching gesture in the image information, and fix the position of the visible normal on the first reference plane based on the fist-clenching gesture;

[0029] S404. Generate a visible second reference surface based on two visible normal lines, wherein the second reference surface is horizontal as a first reference direction and vertical as a second reference direction, and the transparency of the second reference surface is higher than the visible normal line and lower than the first reference surface;

[0030] S405. Obtain target material selection information, and display the target material at a default position on the second reference surface;

[0031] S406. Acquire image information, identify left-hand fanning gesture and right-hand fanning gesture in the image information, and move the target material to a position on the second reference surface along the first reference direction based on the left-hand fanning gesture and the right-hand fanning gesture;

[0032] S407. Identify the upper fan hand gesture and the lower fan hand gesture in the image information, and move the target material to a position on the second reference surface along the second reference direction based on the upper fan hand gesture and the lower fan hand gesture;

[0033] S408. Identify the fist-clenching gesture in the image information, and fix the position of the target material on the second reference surface based on the fist-clenching gesture.

[0034] Optionally, the S4 further includes the following steps:

[0035] S411. Identify the push-pull gesture of the palm in the image information, copy the second reference surface and move the copied reference surface forward and backward along the normal direction of the second reference surface;

[0036] S412. Identify a fist-clenching gesture in the image information to fix the position of the second reference plane.

[0037] Optionally, the following steps are also included:

[0038] Based on the gyroscope information, a projection plane passing through the coordinate origin is generated, and the materials on each second reference plane are projected toward the projection plane and displayed, wherein the pixel points closer to the coordinates of the projection plane are preferentially displayed on the projection plane;

[0039] Based on the gyroscope information, the projection surface image is superimposed on the display image of the terminal screen.

[0040] In a second aspect, the present application provides a mobile operation device for multiple discrete targets based on gesture recognition, which adopts the following technical solution:

[0041] A mobile operation device for multiple discrete targets based on gesture recognition, comprising:

[0042] Reference determination module, used to determine the terminal position and establish a reference system, load and display the target point;

[0043] A first reference plane generating module, used to generate a first reference plane surrounding a target point and display it on a terminal screen, wherein the first reference plane is cylindrical and its axis is vertically arranged;

[0044] A visible normal line generation module, used for generating two visible normal lines on the first reference surface;

[0045] The second reference plane generation and material positioning module is used to move two visible normals according to gesture recognition to construct a second reference plane and display it on the terminal screen, and to copy the second reference plane according to gesture recognition or determine discrete material positions on the second reference plane.

[0046] In a third aspect, the present application provides a computer device, which adopts the following technical solution:

[0047] A computer device comprising:

[0048] one or more processors;

[0049] Memory;

[0050] One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method for moving multiple discrete targets based on gesture recognition.

[0051] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

[0052] A computer-readable storage medium stores a computer program that can be loaded by a processor and execute the above method.

[0053] The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the above-mentioned method for moving multiple discrete targets based on gesture recognition.

[0054] In summary, the present application includes at least one of the following beneficial technical effects:

[0055] 1. Users can use simple gestures (such as left hand slap, right hand slap, up hand slap, down hand slap, fist clenching, push and pull, etc.) to accurately move, rotate, copy and fix multiple materials in three-dimensional space on a two-dimensional screen. This gesture operation requires low computing power, responds quickly, and is intuitive to operate, which reduces the threshold for users to learn and operate.

[0056] 2. By establishing multiple reference surfaces and normals, users can dynamically generate and adjust reference surfaces according to gestures and perform material operations on the reference surfaces, providing precise control over the movement and placement of materials, especially in complex 3D scenes. By copying and moving reference surfaces multiple times, users can achieve more complex layouts. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of an application environment of a method for moving multiple discrete objects based on gesture recognition according to an embodiment of the present invention is shown.

[0058] Figure 2 A flowchart of a method for moving multiple discrete objects based on gesture recognition according to an embodiment of the present invention is shown.

[0059] Figure 3 A flowchart of sub-step S1 in one embodiment of the present invention is shown.

[0060] Figure 4 A flowchart of sub-step S2 in one embodiment of the present invention is shown.

[0061] Figure 5 A flowchart of sub-step S3 in one embodiment of the present invention is shown.

[0062] Figure 6 A flowchart of sub-step S4 in one embodiment of the present invention is shown.

[0063] Figure 7 A schematic diagram of a computer device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0064] The present application is further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] In the following description, for the purpose of explanation, many specific details are set forth in order to provide a thorough understanding of the inventive concept. As part of this specification, some of the drawings in the drawings of the present disclosure represent structures and devices in the form of block diagrams to avoid making the disclosed principles complicated and difficult to understand. For the sake of clarity, not all features of the actual implementation are necessary to be described. In addition, the language used in this disclosure has been selected primarily for readability and instructional purposes, and may not be selected to delineate or limit the subject matter of the present invention, thereby resorting to the necessary claims to determine such inventive subject matter. References to "a specific implementation" or "specific implementations" in this disclosure mean that the specific features, structures or characteristics described in conjunction with the specific implementation are included in at least one specific implementation, and multiple references to "a specific implementation" or "specific implementations" should not be understood as necessarily all referring to the same specific implementation.

[0066] Unless expressly limited, the terms "a", "an", and "the" are not intended to refer to a singular entity, but rather to include general categories of which specific examples may be used for illustration. Thus, the use of the terms "a" or "an" may mean any number of at least one, including "one", "one or more", "at least one", and "one or more than one". The term "or" means any of the alternatives and any combination of the alternatives, including all alternatives, unless the alternatives are expressly indicated to be mutually exclusive. The phrase "at least one of" when combined with a list of items refers to a single item in the list or any combination of the items in the list. The phrase does not require all of the listed items unless expressly limited to that.

[0067] The present application embodiment discloses a method for moving multiple discrete targets based on gesture recognition. The method for moving multiple discrete targets based on gesture recognition provided by the embodiment of the present invention can be applied in the following aspects: Figure 1In an application environment, the mobile operation method of multiple discrete targets based on gesture recognition is applied in a mobile operating system of multiple discrete targets based on gesture recognition, and the mobile operating system of multiple discrete targets based on gesture recognition includes a client and a server, wherein the client communicates with the server through a network. The client, also known as the user end, refers to a program corresponding to the server that provides local services to the client. Furthermore, the client is a computer-side program, an APP program for a smart device, or a third-party applet embedded in other APPs. The client can be installed on, but not limited to, various computer devices such as personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server can be implemented with an independent server or a server cluster consisting of multiple servers.

[0068] Reference Figure 2 The method for moving multiple discrete targets based on gesture recognition includes the following steps S1-S4.

[0069] S1. Determine the terminal position and establish the reference system, load and display the target point.

[0070] A terminal refers to a device that can perform user input, output, and computing. The terminal must at least have satellite positioning function, camera function, screen, and processor capable of graphics processing. The satellite positioning function is used to obtain the terminal's geographic location information, thereby providing a basis for the positioning of the target point and the establishment of the reference system in the game. The camera function captures user gestures through the camera for image recognition to achieve the operation of target points and materials in the game. The screen is used to display game content and interactive interface, and provide users with real-time operational feedback for adjusting reference planes, normals, and target points. The graphics processor is used to handle complex image recognition, rendering, and coordinate conversion tasks to ensure the smoothness and real-time operation of the game.

[0071] The terminal location is usually located by obtaining geographic location information through GPS and establishing the terminal's spatial coordinate system in combination with gyroscope information. In certain scenarios, auxiliary technologies such as Wi-Fi positioning can be used in indoor environments or areas with weak GPS signals. By combining different positioning methods, more accurate terminal location information can be provided, thereby ensuring accurate display and operation of the target point.

[0072] The purpose of establishing a reference system is to provide a unified spatial coordinate system for the game, ensuring that the target points and materials can be accurately located in three-dimensional space and that the user's gesture operations can be responded to in real time. The reference system is usually built based on the terminal's positioning information and direction sensor. Common reference system types include world coordinate system, local coordinate system, and screen coordinate system. The selection of each reference system is based on specific operational requirements to ensure that the operation process of the target points and materials in the game is intuitive and smooth.

[0073] In mobile games, a target point refers to a point with a specific meaning, and users usually need to interact around this point. For example, the target point may be a key element or task trigger point in the game scene, and users need to move or adjust materials around this point through gestures. Accurately locating the target point is a crucial part of the game experience, which determines whether users can successfully complete related operations and have a good game experience.

[0074] As an example, refer to Figure 3 In one embodiment, S1 includes the following steps S101-S104.

[0075] S101. Obtain terminal location information based on satellite positioning.

[0076] Web games are a type of online game that runs through a web browser. It does not require users to download and install a client, but relies on the transmission of data in the network environment to load and interact with game content. Compared with traditional client games, web games are highly instant, easy to operate, and do not require installation, so they can be accessed more conveniently on various devices. However, the calculation of web games mainly depends on the processing power of the server, so they occupy relatively less local resources and computing power. In addition, the running effect of web games is closely related to the user's network speed. The speed of the network directly affects the loading time of the game, the smoothness of the picture, and the speed of interactive response.

[0077] Existing mobile games usually need to be installed and run through the client. One of the more typical implementation methods is to use point cloud recognition technology to achieve positioning and interaction in the game. Point cloud recognition is a technology that relies on high-precision coordinate point data for image, object or scene detection. It can accurately identify the position of objects operated by users, thereby avoiding image drift problems. During operation, users adjust the position of objects by swiping their fingers on the screen to achieve more precise control. However, point cloud recognition technology has high requirements on the computing power of the device, and usually requires a large amount of computing resources and storage space, which increases the operating burden of the device, especially for portable devices such as mobile phones, which may cause obvious performance bottlenecks.

[0078] If point cloud recognition technology is applied to web games, some problems will arise. First, the amount of point cloud data is large. If it is transmitted over the network, it will take a lot of time to download, which will affect the loading speed of the game. Secondly, the calculation process of point cloud recognition is complicated. Web games mainly rely on server-side calculations, but if a large amount of point cloud data needs to be processed, it may cause problems such as excessive server load or response delays, thereby reducing the overall game experience. Therefore, although the use of point cloud recognition technology in web games can effectively prevent image drift, it will have an adverse effect on network speed, computing resources and user experience.

[0079] In this solution, satellite positioning is adopted. The advantages of satellite positioning are its wide coverage and high positioning accuracy, especially in open outdoor environments, and it can be positioned through multiple satellites such as Global Navigation Satellite Systems (GNSS) such as GPS, GLONASS, Beidou, etc. These systems determine the geographic location of the device by triangulating by receiving multiple satellite signals. Multi-satellite positioning can improve positioning accuracy. By using multiple satellite signals, the impact of single satellite signal transmission error on positioning accuracy can be reduced. Under normal circumstances, the error range of GNSS positioning can be reduced to within a few meters, especially in the case of good signal reception conditions, the positioning error can be further reduced.

[0080] In this technical solution, satellite positioning is used to determine the geographical location of the terminal. When the terminal is close to the target positioning area, for example, within 50 meters from the target point, the system starts to measure based on the current position of the terminal and the world coordinates of the target point, and preliminarily calculates the theoretical position of the target point. During multiple measurements, the terminal will make a comprehensive estimate based on the position data of different time nodes to eliminate the error caused by positioning drift. This estimation method based on multiple measurements can effectively improve the final positioning accuracy, eliminate temporary interference and drift in satellite signal transmission, and ensure that the final position of the target point is relatively accurate. In this mode, the final position of the terminal will not affect the user's operating experience due to accidental drift, and can provide users with a stable positioning reference point, thereby optimizing subsequent gesture operations and material adjustments.

[0081] S102. Acquire gyroscope information of the terminal, and establish a first space coordinate system based on the terminal position information and the gyroscope information, wherein the z-axis of the first space coordinate system is vertical.

[0082] The purpose of obtaining the terminal's gyroscope information and establishing the first spatial coordinate system based on the terminal's position information and gyroscope information is to ensure that the system can accurately describe and process the terminal's direction and position in three-dimensional space. The gyroscope is a common sensor that can detect the device's angular velocity and attitude changes in real time, helping the device to sense its own rotation and tilt state in three-dimensional space. In game operations or gesture recognition scenarios, the gyroscope's information can be used to determine the terminal's orientation, pitch angle, roll angle, etc., to ensure that the reference system is established in accordance with the terminal's current attitude.

[0083] In this step, by combining the terminal's satellite positioning information (obtaining the absolute position of the device) and gyroscope information (obtaining the device's attitude), a three-dimensional spatial coordinate system that matches the physical position and orientation of the terminal can be constructed. Specifically, the z-axis of the first spatial coordinate system is usually set to the vertical direction. This is to ensure that the coordinate system can adapt to the vertical direction of the terminal in the real world, thereby providing a unified reference benchmark for subsequent three-dimensional object operations. For example, when a user uses gestures to move or rotate a target point in a game, the system can update the relative position of the target point relative to the terminal in real time based on the gyroscope data to ensure the intuitiveness and accuracy of the operation.

[0084] In this way, the terminal can not only determine its absolute coordinates in geographical location (provided by satellite positioning), but also provide users with more accurate operation feedback and control experience based on the current posture of the device. The introduction of gyroscope information greatly improves the accuracy of spatial operations in games, especially in situations where the target point needs to be moved or rotated, ensuring that the user's operation is more in line with spatial logic and avoiding operation deviations caused by tilting or rotating the device.

[0085] S103. Load the world coordinate information of the target point, and convert the world coordinate information of the target point into coordinates in the first space coordinate system.

[0086] In this step, the world coordinate information of the target point is loaded and converted into coordinates in the first space coordinate system in order to accurately display the position of the target point in the game in the three-dimensional space reference system of the terminal. The target point here is usually a key element for interaction with the user, and the world coordinate information represents the absolute position of the target point in the world coordinate system. By loading the world coordinates of the target point into the system, the device can obtain the initial position of the target point in the absolute space.

[0087] Converting the world coordinate information of the target point into coordinates on the first space coordinate system is actually a coordinate system conversion process. In this conversion process, the world coordinates of the target point need to adapt to the first space coordinate system constructed by the terminal position and posture. The first space coordinate system is established based on the current actual position of the terminal (obtained through satellite positioning) and the device posture (obtained through gyroscope information). Therefore, the purpose of the world coordinate conversion is to map the absolute, global coordinate information to the relative coordinate system where the terminal device is currently located, so that the position of the target point relative to the position and direction of the terminal becomes more intuitive and easy to operate.

[0088] Through this conversion, the position of the target point is associated with the current posture of the terminal, allowing the user to accurately see the position of the target point on the terminal screen and perform interactive operations. When the user moves the terminal or adjusts the viewing angle, the display of the target point on the screen can be updated in real time and always maintains the corresponding relationship with the terminal position and direction, which can ensure the smoothness and consistency of operation.

[0089] S104. Based on the coordinates of the target point in the first space coordinate system and the gyroscope information, display the target point on the terminal screen.

[0090] In the previous step, the world coordinates of the target point have been converted to the first space coordinate system where the terminal is located. This step uses this information and combines the terminal's posture to project the three-dimensional coordinates of the target point onto the two-dimensional plane of the screen, ensuring that the user can accurately see the target point at the current viewing angle and perform interactive operations.

[0091] Specifically, this step converts the target point in the three-dimensional space into a two-dimensional coordinate on the screen and renders it on the screen. The gyroscope information is used to adjust the display position of the target point to ensure that it matches the posture of the terminal. For example, when the terminal rotates or tilts, the position of the target point on the screen will be adjusted as the terminal moves to keep the display consistent with the real space position. This is a typical mapping process from three-dimensional space to two-dimensional space, which usually relies on the graphics processor to complete.

[0092] For example, in a mobile game, assuming that the user's terminal device is a smart phone, the game scene is a virtual world, and the user needs to find and interact with a target point in the virtual world. The world coordinates of the target point are (100, 50, 200). In step S103, this world coordinate is converted into the coordinates in the first space coordinate system of the terminal, such as (10, 5, 20).

[0093] In S104, the system uses the gyroscope data of the terminal to determine the current orientation and angle of the device. Assume that the mobile phone is currently held vertically and the user views the virtual world through the screen. At this time, the system converts the position of the target point (10,5,20) in the first spatial coordinate system into a two-dimensional coordinate on the screen. The conversion process is based on the current pitch angle, yaw angle and roll angle of the terminal (provided by the gyroscope). For example, assuming that according to the gyroscope information, the terminal has a certain left and right tilt, then the display position of the target point will move left and right on the screen according to this tilt.

[0094] In the embodiment, assuming that the screen resolution of the mobile phone is 1080x1920, the system calculates the two-dimensional projection coordinates of the target point as (540,960), which is the center point of the screen. The user can see the target point accurately displayed in the center of the screen from this perspective. When the user moves the terminal or changes the tilt angle of the mobile phone, the gyroscope information will be fed back to the system, and the system will update the display position of the target point in real time. For example, when the user tilts the phone to the left, the display of the target point on the screen will move to the right to keep consistent with the spatial position in the virtual world.

[0095] The system can also adjust the display size of the target point based on the device's distance sensor or accelerometer information. For example, when the user approaches the target point, the display of the target point on the screen will become larger, and vice versa, to simulate the visual experience in reality.

[0096] S2. Generate a first reference plane surrounding the target point and display it on the terminal screen, wherein the first reference plane is cylindrical and the axis is vertically arranged.

[0097] As an example, refer to Figure 4 In one embodiment, S2 includes the following steps:

[0098] S201. Based on the coordinates of the target point in the first space coordinate system, generate a vertical axis passing through the target point in the first space coordinate system.

[0099] In this step, a clear spatial reference axis is created to facilitate the generation of reference surfaces and user operations in subsequent steps. The generation of this axis is based on the target point and ensures that its direction is aligned with the vertical axis (usually the z-axis) of the first spatial coordinate system, thereby providing a spatial reference for subsequent operations.

[0100] The significance of generating a vertical axis passing through the target point is that it provides a fixed spatial reference for the operation of the target point in the game. By creating this axis in the first spatial coordinate system, the system can establish a clear direction and spatial perception for the user's operation, so that the movement, rotation or other interactive operations of the target point have a clear reference basis.

[0101] In addition, since the position of the target point in three-dimensional space has been determined, the vertical axis passes through the point, so that the system can generate other geometric structures (such as reference planes, normals, etc.) with the axis as the center. These geometric structures can help users intuitively control and operate the target point. This vertical axis is unique and fixed in space, ensuring that the user will not lose the reference of the operation due to changes in spatial position during the operation.

[0102] S202. Generate a cylindrical surface based on the axis as a first reference surface, wherein the radius of the first reference surface is a preset radius.

[0103] The main purpose of generating a cylindrical surface is to provide users with a visual operating boundary, allowing users to move, adjust and manipulate elements in an intuitive three-dimensional space. The role of a cylindrical surface is similar to an operating range or control area. The set radius provides a clear range limit for the reference surface to ensure that the user's operation is carried out within a reasonable area and does not get out of control or deviate from the target. This preset radius can be adjusted according to specific application requirements. For example, in some application scenarios, a smaller radius can be set for precise operation, while a larger radius can be set in scenarios that require a larger range of control.

[0104] The cylindrical surface is centered on the vertical axis, so it can always maintain spatial consistency with the target point and the vertical direction. This consistency ensures that no matter how the terminal's posture changes, the reference surface can provide stable visual feedback when the user operates the target point, thereby avoiding confusion and uncertainty in the operation. Since the reference surface is a regular geometric body (cylindrical surface), the user can use gestures to manipulate the normal or target point to move and rotate smoothly along the cylindrical surface.

[0105] S203. Based on the gyroscope information, display the first reference plane on the terminal screen.

[0106] In this step, the gyroscope can sense the posture changes of the terminal device (such as pitch angle, roll angle and yaw angle). When the terminal device rotates or tilts, the display position and angle of the reference surface will also be adjusted accordingly to keep in sync with the device posture. In this way, when the user operates at different angles, the reference surface always keeps consistent with the device direction, avoiding operation deviation or visual confusion caused by changes in the device posture.

[0107] Specifically, the system first reads the gyroscope information through the terminal device's sensor to determine the device's current posture. The gyroscope provides real-time data about the device's rotation angle and direction, including angle changes in three dimensions: pitch, roll, and yaw.

[0108] Based on the data fed back by the gyroscope, the system adjusts the generated cylindrical reference surface. The adjustment process includes three-dimensional rotation and translation of the reference surface to ensure that the reference surface matches the posture of the device. For example, if the device tilts a certain amount, the reference surface will also tilt to ensure that the user always sees the reference surface at the correct angle.

[0109] The system projects the adjusted 3D reference surface coordinates into the 2D coordinate system of the terminal screen. Through geometric transformation and projection algorithms, the 3D cylindrical surface can be correctly displayed on the flat screen while maintaining the proportion and perspective effect. After the projection is completed, the system renders the cylindrical reference surface on the terminal screen through the graphics processor. As the user continues to move the terminal or change the device angle, the system reads the new gyroscope data in real time and recalculates, adjusts, and renders the reference surface according to the above steps.

[0110] S3. Generate two visible normals on the first reference surface.

[0111] As an example, refer to Figure 5 In one embodiment, S3 includes the following steps S301-S303.

[0112] S301. Generate a first normal line at a position of the first reference plane close to an xz reference plane.

[0113] The xz reference plane usually represents a plane in a three-dimensional coordinate system, where the x-axis represents the horizontal direction and the z-axis represents the vertical direction.

[0114] S302. Generate a second normal line at a position of the first reference plane close to the yz reference plane.

[0115] Similar to S301 , the yz reference plane represents another plane in the three-dimensional coordinate system, wherein the y-axis generally represents another horizontal direction (perpendicular to the x-axis), and the z-axis still represents the vertical direction.

[0116] S303. Based on the gyroscope information, display the first normal and the second normal on the terminal screen as visible normals; wherein the transparency of the first normal and the second normal is lower than the transparency of the first reference surface.

[0117] When the device is tilted, the normal line displayed on the screen automatically adjusts its angle to reflect the actual posture of the terminal in three-dimensional space. In this way, no matter how the terminal is moved or rotated, the user can see the consistency of the normal line with the reference surface.

[0118] The purpose of setting a lower transparency is to make the first and second normals the main visual references for more detailed adjustments to the normals. This difference in transparency can help users more easily distinguish different operating elements and improve the efficiency and intuitiveness of operations.

[0119] S4. According to the gesture recognition, two visible normals are moved to construct a second reference plane and displayed on the terminal screen, and according to the gesture recognition, the second reference plane is copied or a discrete material position is determined on the second reference plane.

[0120] In the prior art, image recognition requires a large amount of local computing resources. This is because a general neural network model trained with a large number of samples will have thousands of layers, which will take up a lot of computing power when performing operations. However, in this solution, only a few gestures are involved, such as slapping, clenching fists, pushing and pulling, and other basic actions. The gesture recognition process does not need to process complex shapes or features, but only needs to recognize the direction of the action and the basic posture of the hand. Therefore, this simple gesture recognition task has low computing power requirements and does not take up a lot of processor resources. This is an important advantage of gesture recognition, especially when implemented on the web page of a mobile terminal. The low computing power requirement ensures that the terminal can run smoothly without causing performance bottlenecks due to image processing.

[0121] Image recognition of gestures mainly relies on the camera to capture the user's hand movements, and then identifies the gesture type through simple image processing algorithms. For example, the system can identify basic gestures such as slapping and clenching fists by simply analyzing the features of the palm or fist in the image, and determine the direction of these gestures (such as left and right, up and down).

[0122] As an example, refer to Figure 6 In one embodiment, S4 includes the following steps S401-S408.

[0123] S401. Acquire image information, and identify the left hand or the right hand in the image, select a first normal based on the left hand, and select a second normal based on the right hand.

[0124] The system needs to capture the image of the user's hand through the camera, and use the image processing algorithm to determine whether it is a left or right hand gesture, and then decide which normal to operate. The left hand usually corresponds to the selection and operation of the first normal, and the right hand corresponds to the selection and operation of the second normal.

[0125] The system captures the user's hand image in real time to obtain the hand posture and position. Image acquisition here does not require high resolution or complex processing, because gesture recognition does not involve detailed shape analysis, but is based on simple hand features, such as the distinction between left and right hands and basic action recognition. In other words, this design idea is conducive to implementation in web games.

[0126] Specifically, the system first starts the terminal's camera and starts capturing the user's hand image in real time. The camera needs to continuously capture images of the hand movements to ensure smooth gesture recognition. The system processes the captured images through an image processing algorithm to identify the user's hand gesture. Based on features such as the direction of the palm and the arrangement of the fingers, the system can determine whether the currently displayed hand is the left hand or the right hand.

[0127] Once the system recognizes the left or right hand, it automatically selects the corresponding normal for operation. If the left hand is detected, the system will select the first normal, allowing the user to control the movement or adjustment of the normal through subsequent gesture operations. If the right hand is detected, the system will select the second normal, also allowing the user to adjust it.

[0128] The system provides visual feedback to the user to confirm the currently selected normal. For example, the selected normal can be highlighted, or the color can change to let the user know clearly which normal is currently being operated. When the user changes the gesture or moves the hand, the system recaptures the image and updates the gesture recognition and normal selection in real time. For example, if the user switches from the left hand to the right hand, the system will automatically switch to the control of the second normal. This dynamic update mechanism ensures that the user can switch between the two normals freely and smoothly.

[0129] S402. Acquire image information, and identify left-hand slapping gesture and right-hand slapping gesture in the image information, and horizontally move the visible normal based on the left-hand slapping gesture and the right-hand slapping gesture; wherein the visible normal is standardized to move in a circular direction of the first reference plane.

[0130] In this step, the user's "left slap hand gesture" and "right slap hand gesture" are identified by capturing and processing image information. The left slap hand gesture usually refers to the user making a fan-shaped swing with the left hand in a specific direction, while the right slap hand gesture is to complete a similar action with the right hand. By identifying these gestures, the system can determine the direction in which the user wants to control the normal movement on the horizontal plane.

[0131] On the first reference plane, the normal moves in a circumferential direction. This means that the movement of the normal is done on a cylindrical surface, following a path around the axis of the first reference plane. The system determines the direction and speed of the normal movement based on the direction of the recognized gesture. For example, a left slap may mean that the normal moves in a counterclockwise direction, while a right slap may mean that it moves in a clockwise direction.

[0132] Furthermore, gesture recognition is performed in real time, and the system continuously updates the position of the normal line according to the user's gesture. The faster and larger the user's gesture, the greater the distance the normal line moves.

[0133] S403. Acquire image information, and identify a fist-clenching gesture in the image information, and fix the position of the visible normal on the first reference plane based on the fist-clenching gesture.

[0134] In this step, the key is to identify the "fist action gesture", which is a typical closed gesture. The system uses image processing algorithms to detect the user's hand changing from an open state to a fist. This gesture is usually clearer and easier to identify than other dynamic gestures (such as the slap gesture) because fisting involves obvious changes in hand shape, so the system can more easily and accurately determine the user's intention.

[0135] After recognizing the fist action, the system's task is to fix the visible normal on the first reference plane. Fixing the normal means that the user has completed the adjustment or movement of the normal, and the current position is the target position. The fist action gesture serves as a signal to end the operation, which can notify the system to stop the subsequent movement of the normal and keep it in the current position. This design helps to avoid misoperation and ensures that the user can lock the normal after accurately adjusting it.

[0136] Furthermore, when the normal is fixed on the reference surface, the system may inform the user that the normal has been successfully fixed through some form of visual feedback (such as a change in the color of the normal, a flashing of the normal, etc.).

[0137] S404. Generate a visible second reference plane based on two visible normal lines, wherein the second reference plane has a horizontal first reference direction and a vertical second reference direction, and a transparency of the second reference plane is higher than the visible normal line and lower than the first reference plane.

[0138] In the previous step, the user moved two visible normals through gestures, which were located near the xz and yz planes on the first reference plane. By controlling the position of these normals, the user is actually defining the planar characteristics of the second reference plane. The two normals are equivalent to providing a reference for the direction and position of the second reference plane. In this step, a new plane is constructed using these two normals, which will serve as the basis for further user operations in three-dimensional space.

[0139] The generated second reference surface has a higher transparency than the visible normal, but lower transparency than the first reference surface. The transparency setting here plays a role in balancing visual feedback. By setting the transparency, the system can ensure that the second reference surface is clearly visible on the screen, but does not completely block other operating elements. Through this design, users can maintain perception and control of each element in a complex three-dimensional space, especially in multi-level operations, the distinction in transparency can help users better understand the hierarchical relationship of each reference surface.

[0140] S405. Obtain target material selection information, and display the target material at a default position of the second reference surface.

[0141] The system identifies the target material that the user wants to operate through gestures or other input methods. For example, in a game or application, if the user selects a specific object or material, the system determines the target material through the user's operation, such as clicking the screen.

[0142] After selecting the target material, the system will place the material on the second reference plane and display it on the screen. The second reference plane is a three-dimensional plane generated in the previous step, which serves as the basis for operation. The display of the material must not only maintain its association with the reference plane, but also ensure that its position on the screen is consistent with the user's operation logic. The advantage of this is that the user can intuitively see the relationship between the material and the reference plane in the visual interface, so as to make further moves, rotations or adjustments.

[0143] It should be noted that the default position refers to the initial position preset by the system when placing the material on the second reference surface. As an example, the default position may be at the center of the second reference surface or a position slightly above the center.

[0144] S406. Acquire image information, and identify left-hand slapping gesture and right-hand slapping gesture in the image information, and move the position of the target material on the second reference surface along the first reference direction based on the left-hand slapping gesture and the right-hand slapping gesture.

[0145] In this step, the system captures the user's hand movements through the camera and identifies whether the user is swiping with the left or right hand. These gestures are mapped to the horizontal movement direction of the material. For example, a left swiping gesture means moving the material to the left, and a right swiping gesture means moving the material to the right. The larger the gesture amplitude, the farther the material moves, ensuring that the user can accurately control the position of the material.

[0146] As the user's gesture continues, the system updates the position of the material in real time to ensure that the user's gesture effect is reflected instantly on the screen. This real-time feedback makes the operation more intuitive and controllable, and the user can adjust the horizontal position of the material through continuous gestures.

[0147] S407. Identify the upward fanning hand gesture and the downward fanning hand gesture in the image information, and move the position of the target material on the second reference plane along the second reference direction based on the upward fanning hand gesture and the downward fanning hand gesture.

[0148] The system uses the camera to capture the user's up and down hand gestures and recognizes that the user wants to adjust the position of the material in the vertical direction. The up hand gesture corresponds to moving the material upward, while the down hand gesture corresponds to moving it downward. This vertical movement gesture control allows users to make more precise height adjustments to the material.

[0149] Similar to S406, S407 also provides real-time feedback mechanism. The system dynamically adjusts the vertical position of the material on the reference surface according to the user's gesture. The user can achieve smooth movement of the material by continuously swiping up or down until it reaches the desired vertical position.

[0150] S408. Identify the fist-clenching gesture in the image information, and fix the position of the target material on the second reference surface based on the fist-clenching gesture.

[0151] After recognizing the user's fist gesture, the system will stop moving the target material and fix it on the current second reference surface. The fist gesture serves as a clear stop signal, notifying the system to no longer respond to movement instructions for the material, thereby preventing unintentional movement operations.

[0152] After the material is fixed, the system will provide corresponding visual feedback (such as highlighting, color change, etc.) to confirm the completion of the operation. This feedback mechanism can help users confirm that the material has been locked, thereby avoiding unnecessary adjustments or operations.

[0153] Optionally, in a certain embodiment, in order to achieve the duplication of the second reference surface to facilitate the rapid arrangement of materials, S4 may also include the following steps S411-S412.

[0154] S411. Identify the palm push and pull action gesture in the image information, copy the second reference surface and move the copied reference surface forward and backward along the normal direction of the second reference surface.

[0155] The system captures the user's hand movements through the camera and recognizes the "palm push and pull gesture". This gesture usually means that the user wants to operate the reference surface by pushing and pulling forward or backward. When the system recognizes this gesture, it automatically copies the current second reference surface to generate a new copy of the reference surface. This allows users to handle multiple reference surfaces in the same scene, providing more flexibility in complex operation scenarios.

[0156] The copied reference surface is not static. Users can continue to use the "palm push and pull gesture" to control the copied reference surface to move forward and backward along the normal direction (usually the z-axis direction). Pushing the hand means that the reference surface moves forward, and pulling the hand moves the reference surface backward. Movement in the normal direction means that the reference surface can move freely in the direction perpendicular to its surface, allowing users to flexibly adjust the relative depth or distance of the reference surface in three-dimensional space.

[0157] S412. Identify a fist-clenching gesture in the image information to fix the position of the copied second reference surface.

[0158] The system captures and recognizes the user's fist gesture through the camera, which is used here as a signal to terminate the operation. Once the system recognizes the fist gesture, it will immediately fix the reference surface in the current position and stop any further movement of the reference surface. This is a clear sign of the end of the operation, ensuring that the user's operation can proceed as expected and that the reference surface will not undergo any unexpected displacement.

[0159] The system informs the user that the reference surface has been fixed through visual feedback (such as changing color, flashing, etc.). This feedback mechanism allows the user to clearly know that the operation has been completed and avoid confusion.

[0160] Optionally, in order to realize the display of the material, in a certain embodiment, the following steps are included:

[0161] Based on the gyroscope information, a projection plane passing through the coordinate origin is generated, and the materials on each second reference plane are projected toward the projection plane and displayed, wherein the pixel points closer to the coordinates of the projection plane are preferentially displayed on the projection plane;

[0162] Based on the gyroscope information, the projection surface image is superimposed on the display image of the terminal screen.

[0163] For example, first, the gyroscope of the mobile phone will provide information about the current device's attitude, including the phone's pitch angle, roll angle, and yaw angle. These data can be used to determine the spatial position and angle of the phone relative to the object. Suppose the phone currently has a certain tilt angle, such as 30 degrees relative to the horizontal plane, and the user rotates the phone 45 degrees to the left.

[0164] Based on the gyroscope information, the system generates a projection surface in three-dimensional space that passes through the coordinate origin and is consistent with the viewing angle of the phone. This projection surface is usually a plane that is perpendicular to the user's current line of sight. For example, if the user's phone is tilted forward 30 degrees, the projection surface will adjust accordingly to keep the same angle as the phone screen.

[0165] There may be multiple materials on each second reference surface, such as multiple virtual furniture distributed on different reference surfaces. The system projects these materials toward the generated projection surface based on their three-dimensional coordinates. The projection process converts the three-dimensional coordinates of each object into two-dimensional coordinates on the projection surface. For example, suppose a virtual sofa is located 1 meter away from the projection surface. After projection, the position of the sofa is represented as a two-dimensional coordinate point (300,500) on the projection surface.

[0166] During the projection process, the system calculates the distance between each material and the projection surface. The closer the material is to the projection surface, the higher the priority of displaying its pixels. This approach simulates the occlusion relationship in reality. For example, a virtual coffee table is located in front of the sofa, only 0.5 meters away from the projection surface, so the pixels of the coffee table will cover the sofa and be displayed on the screen first. In this way, users can see that the objects in front block the objects behind, enhancing the sense of three-dimensional space and depth effect.

[0167] The system projects all the material on the projection surface as a two-dimensional image layer and superimposes it on the actual screen display of the terminal. Assuming that the screen is displaying a virtual room scene, the user sees the real floor and walls through the mobile phone camera, and the virtual furniture on the projection surface will be seamlessly integrated with the real environment. For example, the user will see a virtual coffee table and sofa superimposed on the real floor on the screen, and the coffee table will partially cover the sofa, enhancing the sense of reality.

[0168] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0169] In one embodiment, a mobile operation device for multiple discrete targets based on gesture recognition is provided, and the mobile operation device for multiple discrete targets based on gesture recognition corresponds to the mobile operation method for multiple discrete targets based on gesture recognition in the above embodiment. The mobile operation device for multiple discrete targets based on gesture recognition includes a reference determination module, a first reference surface generation module, a visual normal generation module, and a second reference surface generation and material positioning module. The functional modules are described in detail as follows:

[0170] Reference determination module, used to determine the terminal position and establish a reference system, load and display the target point;

[0171] A first reference plane generating module, used to generate a first reference plane surrounding a target point and display it on a terminal screen, wherein the first reference plane is cylindrical and its axis is vertically arranged;

[0172] A visible normal line generation module, used for generating two visible normal lines on the first reference surface;

[0173] The second reference plane generation and material positioning module is used to move two visible normals according to gesture recognition to construct a second reference plane and display it on the terminal screen, and to copy the second reference plane according to gesture recognition or determine discrete material positions on the second reference plane.

[0174] The specific definition of the mobile operation device for multiple discrete targets based on gesture recognition can be found in the definition of the mobile operation method for multiple discrete targets based on gesture recognition above, which will not be repeated here. Each module in the mobile operation device for multiple discrete targets based on gesture recognition can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0175] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used for data related to a method for moving multiple discrete targets based on gesture recognition. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for moving multiple discrete targets based on gesture recognition is implemented.

[0176] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for moving multiple discrete targets based on gesture recognition in the above embodiment is implemented, for example Figure 2 Alternatively, when the processor executes the computer program, the functions of each module / unit of the mobile operation device for multiple discrete targets based on gesture recognition in the above embodiment are realized. To avoid repetition, they are not described here.

[0177] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for moving multiple discrete targets based on gesture recognition in the above embodiment is implemented, for example Figure 2 Alternatively, when the computer program is executed by the processor, the functions of each module / unit in the mobile operation device for multiple discrete targets based on gesture recognition in the above device embodiment are realized. To avoid repetition, it will not be described here.

[0178] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments of the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0179] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0180] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A method for moving multiple discrete targets based on gesture recognition, characterized in that: The following steps are involved: S1. Determine the terminal position and establish a reference system, load and display the target point; S2. Generate a first reference surface surrounding the target point and display it on the terminal screen, wherein the first reference surface is cylindrical and the axis is vertically arranged; S3. generating two visible normal lines on the first reference surface; S4. According to the gesture recognition, two visible normals are moved to construct a second reference plane and displayed on the terminal screen, and according to the gesture recognition, the second reference plane is copied or a discrete material position is determined on the second reference plane.

2. The method for moving multiple discrete targets based on gesture recognition according to claim 1, characterized in that: The S1 comprises the following steps: S101. Obtain terminal location information based on satellite positioning; S102. Acquire gyroscope information of the terminal, and establish a first spatial coordinate system based on the terminal position information and the gyroscope information, wherein the z-axis of the first spatial coordinate system is vertical; S103. Load the world coordinate information of the target point, and convert the world coordinate information of the target point into coordinates in the first space coordinate system; S104. Based on the coordinates of the target point in the first space coordinate system and the gyroscope information, display the target point on the terminal screen.

3. The method for moving multiple discrete targets based on gesture recognition according to claim 2, characterized in that: The S2 comprises the following steps: S201. Based on the coordinates of the target point in the first spatial coordinate system, generate a vertical axis passing through the target point in the first spatial coordinate system; S202. Generate a cylindrical surface based on the axis as a first reference surface, wherein the radius of the first reference surface is a preset radius; S203. Based on the gyroscope information, display the first reference plane on the terminal screen.

4. The method for moving multiple discrete targets based on gesture recognition according to claim 3, characterized in that: The S3 comprises the following steps: S301. Generate a first normal line at a position of the first reference plane close to the xz reference plane; S302. Generate a second normal line at a position of the first reference plane close to the yz reference plane; S303. Based on the gyroscope information, display the first normal and the second normal on the terminal screen as visible normals; wherein the transparency of the first normal and the second normal is lower than the transparency of the first reference surface.

5. The method for moving multiple discrete targets based on gesture recognition according to claim 4, characterized in that: The S4 comprises the following steps: S401. Acquire image information, identify the left hand or the right hand in the image, select a first normal based on the left hand, and select a second normal based on the right hand; S402. Acquire image information, and identify a left slapping hand gesture and a right slapping hand gesture in the image information, and horizontally move a visible normal based on the left slapping hand gesture and the right slapping hand gesture; wherein the visible normal is standardized to move upward in a ring of the first reference plane; the left slapping hand gesture refers to a user swinging his hand with his left hand in a predetermined direction, and the right slapping hand gesture refers to a user swinging his hand with his right hand in a predetermined direction; S403. Acquire image information, identify a fist-clenching gesture in the image information, and fix the position of the visible normal on the first reference plane based on the fist-clenching gesture; S404. Generate a visible second reference surface based on two visible normal lines, wherein the second reference surface is horizontal as a first reference direction and vertical as a second reference direction, and the transparency of the second reference surface is higher than the visible normal line and lower than the first reference surface; S405. Obtain target material selection information, and display the target material at a default position on the second reference surface; S406. Acquire image information, identify left-hand fanning gesture and right-hand fanning gesture in the image information, and move the target material to a position on the second reference surface along the first reference direction based on the left-hand fanning gesture and the right-hand fanning gesture; S407. Identify the upper fan hand gesture and the lower fan hand gesture in the image information, and move the target material to a position on the second reference surface along the second reference direction based on the upper fan hand gesture and the lower fan hand gesture; S408. Identify the fist-clenching gesture in the image information, and fix the position of the target material on the second reference surface based on the fist-clenching gesture.

6. The method for moving multiple discrete targets based on gesture recognition according to claim 5, characterized in that: The S4 further comprises the following steps: S411. Identify the push-pull gesture of the palm in the image information, copy the second reference surface and move the copied reference surface forward and backward along the normal direction of the second reference surface; S412. Identify a fist-clenching gesture in the image information to fix the position of the second reference plane.

7. The method for moving multiple discrete targets based on gesture recognition according to claim 6, characterized in that: The following steps are also included: Based on the gyroscope information, a projection plane passing through the coordinate origin is generated, and the materials on each second reference plane are projected toward the projection plane and displayed, wherein the pixel points closer to the coordinates of the projection plane are preferentially displayed on the projection plane; Based on the gyroscope information, the projection surface image is superimposed on the display image of the terminal screen.

8. A mobile operation device for multiple discrete targets based on gesture recognition, characterized in that: include: Reference determination module, used to determine the terminal position and establish a reference system, load and display the target point; A first reference plane generating module, used to generate a first reference plane surrounding a target point and display it on a terminal screen, wherein the first reference plane is cylindrical and its axis is vertically arranged; A visible normal line generation module, used for generating two visible normal lines on the first reference surface; The second reference plane generation and material positioning module is used to move two visible normals according to gesture recognition to construct a second reference plane and display it on the terminal screen, and to copy the second reference plane according to gesture recognition or determine discrete material positions on the second reference plane.

9. A computer device, characterized in that: It includes: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs are configured to: execute the method for moving multiple discrete targets based on gesture recognition according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement: the mobile operation method of multiple discrete targets based on gesture recognition as described in any one of claims 1 to 7.

Images