Device and method for carrying out billiard battle training according to batting behavior simulation of opponent

Abstract

The invention relates to a device and a method for performing billiard battle training according to opponent batting behavior simulation, which are characterized in that the motion state of a billiard and batting behaviors of a plurality of players are obtained by analyzing historical batting images, and machine learning models corresponding to the players are trained according to the motion state and the batting behaviors of the players; a virtual player and a virtual billiard are displayed using a reality technology, when the virtual player hits a ball, a machine learning model of a player corresponding to the virtual player is used to generate a virtual ball hitting behavior of the virtual player and generate a motion state of the virtual billiard according to the virtual ball hitting behavior, and when the user hits the ball, the machine learning model of the player corresponding to the virtual player is used to generate the motion state of the virtual billiard according to the virtual ball hitting behavior. According to the technical means that the machine learning model corresponding to the user is used for generating the motion state of the virtual billiard according to the target ball hitting behavior of the user, and finally the motion state of the virtual billiard is displayed through the reality technology, the ball hitting behavior of a player can be responded in time; and the technical effect that the player and the virtual opponent can carry out playing training is achieved.

Description

Technical Field

[0001] A billiards battle training device and method, in particular, a device and method for billiards battle training based on opponent's hitting behavior simulation. Background Art

[0002] Table tennis is a racket sport in which two players use table tennis rackets to play against each other across a table with a net. The two players hit the ball in turn until one player is unable to return the ball or the ball hit by one player does not land on the table, and the other player can score.

[0003] In table tennis, the player's batting skills, footwork, and tactical strategies have a direct impact on the outcome of the game. Therefore, when training in table tennis, the main focus is on the player's batting skills, footwork, and tactical strategies. In the past, there were basic training aids for players' batting skills and footwork, such as teaching videos, while players' tactical strategies usually needed to rely on the guidance of coaches. With the increase in processing efficiency and the advancement of technology, there are also auxiliary training systems that can analyze the images of the game for technical guidance.

[0004] However, most of the current sparring assistance training systems analyze players' game videos to evaluate their batting performance, or provide training suggestions based on players' past game records through big data analysis. In other words, the current sparring assistance training systems generate evaluations or suggestions based on the playing records between real players, but cannot respond to the players' batting behaviors in real time. Therefore, they cannot provide players with sparring practice with virtual opponents.

[0005] In summary, it can be seen that the existing technology has long had the problem of being unable to respond to the player's hitting behavior in real time, so it is necessary to propose improved technical means to solve this problem. Summary of the invention

[0006] In view of the problem that the prior art cannot respond to the player's hitting behavior in real time, the present invention discloses a device and method for performing table tennis battle training based on the opponent's hitting behavior simulation, wherein:

[0007] The device disclosed in the present invention for performing table tennis match training based on opponent's batting behavior simulation at least comprises: an image acquisition module for acquiring historical match records, wherein the historical match records include multiple historical match images of table tennis activities; an image analysis module for analyzing each historical match image to acquire the motion state of the table tennis and the batting actions and batting strategies of multiple players, wherein the players include the users, the batting actions include moving steps, batting postures, and batting positions, and the batting strategies include the opponent's postures, the opponent's positions, the batting methods, the batting angles, and the batting strengths; a model training module for training each machine learning model corresponding to each player based on the motion state and each player's batting actions and batting strategies; a reality interaction module for generating virtual players and virtual table tennis balls, and using reality technology to display the virtual players and virtual table tennis balls. Virtual billiards, virtual players correspond to players; an interactive simulation module, for generating a virtual batting action and a virtual batting strategy of the virtual player based on the motion state of the virtual billiard ball, the current position of the virtual player, and the current position of the user using a machine learning model of the player corresponding to the virtual player when it is the virtual player's turn to bat, and for regenerating the motion state of the virtual billiard ball based on the virtual batting action and the virtual batting strategy, and for displaying the virtual batting action of the virtual player and the motion state of the virtual billiard ball by the reality interaction module using reality technology, and for regenerating the motion state of the virtual billiard ball based on the motion state of the virtual billiard ball and the user's target batting behavior when it is the user's turn to bat using a machine learning model corresponding to the user, and for displaying the motion state of the virtual billiard ball by the reality interaction module using reality technology.

[0008] The method disclosed in the present invention for performing table tennis match training based on opponent's batting behavior simulation includes at least the following steps: obtaining historical match records, the historical match records including multiple historical match images of table tennis activities; analyzing each historical match image to obtain the motion state of the table tennis and the batting actions and batting strategies of multiple players, the players including the user, the batting actions including moving steps, batting posture, batting position, the batting strategy including the opponent's posture, the opponent's position, the batting method, the batting angle, and the batting strength; training each machine learning model corresponding to each player based on the motion state and each player's batting action and batting strategy; generating virtual players and virtual table tennis balls, and displaying the virtual players and virtual table tennis balls using real-life technology, The virtual player corresponds to the player; when it is the virtual player's turn to bat, the virtual batting action and virtual batting strategy of the virtual player are generated based on the movement state of the virtual table tennis ball, the current position of the virtual player, and the current position of the user using the machine learning model of the player corresponding to the virtual player, and the movement state of the virtual table tennis ball is regenerated based on the virtual batting action and the virtual batting strategy, and the virtual batting action of the virtual player and the movement state of the virtual table tennis ball are displayed using reality technology; when it is the user's turn to bat, the movement state of the virtual table tennis ball and the user's target batting behavior are regenerated based on the machine learning model corresponding to the user, and the movement state of the virtual table tennis ball is displayed using reality technology.

[0009] The device and method disclosed in the present invention are as described above. The difference between the present invention and the prior art is that the present invention obtains the movement state of the billiard ball and the batting behaviors of multiple players by analyzing historical playing images, and after training each machine learning model corresponding to each player based on the movement state and the batting behavior of the player, uses reality technology to display the virtual player and the virtual billiard ball. When it is the virtual player's turn to bat, the machine learning model of the player corresponding to the virtual player is used to generate the virtual batting behavior of the virtual player and the movement state of the virtual billiard ball is generated based on the virtual batting behavior. When it is the user's turn to bat, the machine learning model corresponding to the user is used to generate the movement state of the virtual billiard ball based on the user's target batting behavior. Finally, the movement state of the virtual billiard ball is displayed using reality technology, thereby solving the problems existing in the prior art and achieving the technical effect of providing players with virtual opponents for playing training. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a schematic diagram of components of a device for performing billiards battle training based on opponent's hitting behavior simulation provided by the present invention.

[0011] Figure 2 This is a module diagram of the processor provided by the present invention.

[0012] Figure 3 This is a flow chart of a method for table tennis battle training based on opponent's hitting behavior simulation provided by the present invention.

[0013] The following are the descriptions of the reference numerals:

[0014] 100: Installation

[0015] 110: Memory

[0016] 120: Camera

[0017] 130: Communication interface

[0018] 140: Storage Media

[0019] 150: Input element

[0020] 170: Processor

[0021] 190: Bus

[0022] 210: Image acquisition module

[0023] 230: Image analysis module

[0024] 250:Model training module

[0025] 270: Reality Interaction Module

[0026] 290:Interactive Simulation Module

[0027] 400: Reality Device

[0028] Step 310: Obtain multiple historical billiard game images

[0029] Step 320: Analyze the historical playing images to obtain the motion state of the billiard ball and the player's hitting behavior

[0030] Step 330: Train the machine learning models corresponding to the players based on the motion state and the hitting behavior of the players.

[0031] Step 340: Generate virtual players and virtual billiard balls, and display the virtual players and virtual billiard balls using real-world technology

[0032] Step 350: When it is the virtual player's turn to hit the ball, the virtual player's virtual hitting behavior is generated based on the motion state of the virtual billiard ball, the current position of the virtual player, and the current position of the user using the machine learning model of the player corresponding to the virtual player, and the motion state of the virtual billiard ball is regenerated based on the virtual hitting behavior, and the virtual hitting action of the virtual player and the motion state of the virtual billiard ball are displayed using real-world technology.

[0033] Step 370: When it is the user's turn to hit the ball, the motion state of the virtual billiard ball is regenerated using a machine learning model corresponding to the user according to the motion state of the virtual billiard ball and the user's target hitting action, and the motion state of the virtual billiard ball is displayed using real-world technology. DETAILED DESCRIPTION

[0034] The features and implementation methods of the present invention will be described in detail below with reference to the accompanying drawings and embodiments. The content is sufficient to enable any person skilled in the art to easily and fully understand the technical means used by the present invention to solve the technical problems and implement them accordingly, thereby achieving the effects achievable by the present invention.

[0035] The present invention can analyze the opponent's hitting behavior of the table tennis to train the machine learning model, and can use the trained machine learning model to simulate and generate virtual players, and provide users with table tennis battle training with virtual players through real-life technology. The hitting behavior mentioned in the present invention includes hitting movements and hitting strategies, and the real-life technology mentioned in the present invention includes but is not limited to augmented reality (AR), virtual reality (VR), and mixed reality (MR).

[0036] The apparatus for implementing the present invention may be a computing device. The computing device mentioned in the present invention includes but is not limited to hardware components such as one or more processing modules, one or more memory modules, and a bus connecting different hardware components (including memory modules and processing modules). Through the multiple hardware components included, the computing device can load and execute an operating system so that the operating system runs on the computing device, and can also execute software or programs. The computing device also includes a housing, and the above-mentioned hardware components are arranged in the housing.

[0037] The bus of the computing device mentioned in the present invention may include one or more types, such as a data bus, an address bus, a control bus, an expansion bus, and / or a local bus. The bus of the computing device includes, but is not limited to, an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, a Video Electronics Standards Association (VESA) local bus, a serial Universal Serial Bus (USB), a Peripheral Component Interconnect Express (PCI Express, PCI-E / PCIe) bus, and the like.

[0038] The processing module of the computing device mentioned in the present invention is coupled to a bus. The processing module includes a register group or register space, which can be completely set on the processing chip of the processing module, or all or part of it can be set outside the processing chip and coupled to the processing chip via a dedicated electrical connection and / or via a bus. The processing module can be a central processing unit, a microprocessor or any suitable processing element. If the computing device is a multi-processor device, that is, the computing device includes multiple processing modules, the processing modules included in the computing device are the same or similar, and are coupled and communicated through a bus. In some embodiments, the processing module can interpret a computer instruction or a series of multiple computer instructions to perform specific operations or operations, such as mathematical operations, logical operations, data comparison, copying / moving data, etc., so as to drive other hardware components in the computing device or run an operating system or execute various programs and / or modules. The computer instructions may be assembly language instructions, instruction set architecture instructions, machine instructions, machine-dependent instructions, microinstructions, firmware instructions, or source code or object code (Object Code) written in any combination of one or more programming languages, and the computer instructions may be executed completely on a single computing device, partially on a single computing device, partially on one computing device and partially on another connected computing device. The above-mentioned programming languages ​​include object-oriented programming languages, such as Common Lisp, Python, C++, Objective-C, Smalltalk, Delphi, Java, Swift, C#, Perl, Ruby, etc., and conventional procedural programming languages, such as C language or other similar programming languages.

[0039] The computing device also usually includes one or more chipsets. The processing module of the computing device can be coupled to the chipset or electrically connected to the chipset through a bus. The chipset is composed of one or more integrated circuits (ICs), including a memory controller and a peripheral input / output (I / O) controller, etc. That is, the memory controller and the peripheral input / output (I / O) controller can be included in one integrated circuit, or can be implemented using two or more integrated circuits. The chipset usually provides input / output and memory management functions, as well as multiple general and / or special registers, timers, etc., wherein the above-mentioned general and / or special registers and timers can be accessed or used by one or more processing modules coupled or electrically connected to the chipset. In some embodiments, the chipset may also be part of the processing module.

[0040] The processing module of the computing device can also access the data in the memory module and the large-capacity storage area installed on the computing device through the memory controller. The above-mentioned memory module includes any type of volatile memory and / or non-volatile memory (NVRAM) memory, such as static random access memory (SRAM), dynamic random access memory (DRAM), read-only memory (ROM), flash memory, etc. The above-mentioned large-capacity storage area can include any type of storage device or storage medium, such as a hard disk drive, an optical disc, a flash drive, a memory card, a solid-state disk (SSD), or any other storage device. In other words, the memory controller can access the data in the static random access memory, the dynamic random access memory, the flash memory, the hard disk drive, and the solid-state disk.

[0041] The processing module of the computing device can also be connected and communicated with peripheral devices or interfaces such as peripheral output devices, peripheral input devices, communication interfaces, various data or signal receiving devices, etc. through the peripheral input and output controller via the peripheral input and output bus. The peripheral input device can be any type of input device, such as a keyboard, a mouse, a trackball, a touchpad, a joystick, etc. The peripheral output device can be any type of output device, such as a display, a printer, etc. The peripheral input device and the peripheral output device can also be the same device, such as a touch screen, etc. The communication interface can include a wireless communication interface and / or a wired communication interface. The wireless communication interface can include an interface supporting a wireless local area network (such as Wi-Fi, Zigbee, etc.), Bluetooth, infrared, near-field communication (NFC), 3G / 4G / 5G and other mobile communication networks (cellular networks) or other wireless data transmission protocols. The wired communication interface can be an Ethernet device, a DSL modem, a cable modem, an asynchronous transfer mode (ATM) device, or an optical fiber communication interface and / or component, etc. The data or signal receiving device may include a GPS receiver or a physiological signal receiver, and the physiological signals received by the physiological signal receiver include but are not limited to heartbeat, blood oxygen, etc. The processing module may periodically poll various peripheral devices and interfaces, so that the computing device can input and output data through various peripheral devices and interfaces, and can also communicate with another computing device having the hardware elements described above.

[0042] The following first Figure 1 The schematic diagram of the components of the device for table tennis training based on the opponent's hitting behavior simulation provided by the present invention is used to illustrate the device for implementing the present invention. Figure 1 As shown, the device 100 of the present invention includes a memory 110, a camera 120, a communication interface 130, a storage medium 140, an input element 150, a processor 170, and a bus 190. The processor 170 is connected to the memory 110, the camera 120, the communication interface 130, the storage medium 140, and the input element 150 through the bus 190.

[0043] Memory 110 may store one or more sets of computer instructions.

[0044] The camera 120 may include a circuit board, a lens assembly, and an image sensing element (all not shown in the figure), and the lens assembly and the image sensing element are connected through the circuit board. The camera 120 can obtain images through the lens assembly and the image sensing element. In the present invention, the device 100 is not limited to including one camera 120, and may also include multiple cameras 120 that synchronously obtain images.

[0045] It should be noted that in most embodiments, the camera 120 is usually not disposed inside the housing of the device 100, but is disposed outside the housing of the device 100, and is connected to the device 100 through a physical line or a wired or wireless network. In this case, the camera 120 can be connected to the communication interface 130, and indirectly connected to the bus 190 and the processor 170 through the communication interface 130.

[0046] The communication interface 130 can be connected to an external network storage device or a network device such as a server, and request and download data from the connected network device.

[0047] The storage medium 140 may store data or signals downloaded by the communication interface 130 , may store data or signals provided to the processor 170 or required by the processor 170 when operating, and may store data or signals generated by the processor 170 .

[0048] The input element 150 may provide input data through a peripheral input device of the device 100. For example, the input element 150 may input data through a keyboard, a mouse, a touch pad, or a touch screen.

[0049] The processor 170 may be configured as follows: Figure 2 As shown in the module diagram provided by the present invention, it includes modules such as image acquisition module 210, image analysis module 230, model training module 250, reality interaction module 270, and interactive simulation module 290. In some embodiments, the processor 170 can execute the computer instructions stored in the memory 110, and can generate after executing the computer instructions. Figure 2 In each module; in another part of the embodiment, Figure 2 Each module in the above-mentioned can be generated by one or more circuits and / or complete or partial chips and other hardware components, that is, the processor 170 includes components Figure 2 That is, the modules included in the processor 170 may be software modules or hardware modules, and the present invention has no particular limitation.

[0050] The image acquisition module 210 is responsible for acquiring the historical match records of the billiard activity. The historical match records acquired by the image acquisition module 210 include one or more historical match images. Each historical match image includes two players playing against each other, and the two players playing against each other included in each historical match image are not necessarily the same. For example, the first historical match image may include the first player and the second player, and the second historical match image may include the first player and the third player, etc., but the present invention is not limited to this. Among them, one of the two players included in all or part of the historical match images is a user. In other words, all the historical match images acquired by the image acquisition module 210 may include two or more than two players, and among all the players included in the historical match images, one of the players is a user.

[0051] Generally speaking, the image acquisition module 210 can read the historical fighting images from the storage medium 140 , or can connect to an external network device through the communication module 130 to obtain the historical fighting images, and the present invention has no particular limitation.

[0052] The image analysis module 230 is responsible for analyzing each historical match image acquired by the image acquisition module 210 to acquire the motion state of the table tennis ball in each historical match image and the hitting actions and hitting strategies of the two players. The hitting actions analyzed by the image analysis module 230 include the player's moving steps, hitting posture, and hitting position. The hitting strategy analyzed by the image analysis module 230 includes the opponent's posture and position (also represented by the opponent's posture and opponent's position in the present invention), and the player's hitting method, hitting angle, hitting force, etc., but the data analyzed by the image analysis module 230 is not limited to the above.

[0053] The image analysis module 230 can identify the position of the billiard ball in the historical match image through an object detection algorithm, and after the position of the billiard ball is identified, use an object tracking algorithm to track the movement path of the billiard ball (i.e., the motion state mentioned in the present invention) in multiple consecutive frames. The above-mentioned object detection algorithm includes but is not limited to YOLO, SSD or Faster R-CNN, etc., and the above-mentioned object tracking algorithm includes Kalman Filter or SORT, etc., but the present invention is not limited thereto.

[0054] The image analysis module 230 can also use human posture estimation algorithms such as OpenPose and MediaPipe to detect key points of the body of each player in the historical match images (such as the top of the head, shoulders, elbows, hips, knees, ankles, etc.) to generate skeleton images of different players, and track and capture the footwork and movement trajectory of each player based on the skeleton image of each player.

[0055] The image analysis module 230 can also use the features of the racket (such as shape, color) in the historical playing images for object detection. For example, an object detection algorithm or other deep learning model can be used to identify the boundaries and positions of the racket, and an object tracking algorithm can be used to track the motion path of the racket, and the moment when the racket hits the table tennis ball can be determined based on the tracked motion paths of the racket and the table tennis ball, and the angle at which the racket hits the table tennis ball and the angular velocity of the table tennis ball after being hit can be calculated based on the angle change of the racket at the moment of hitting the ball, thereby calculating the motion state of the table tennis ball. Among them, the image analysis module 230 can use deep learning technology (such as convolutional neural network CNN, or long short-term memory model LSTM, etc.) to analyze the player's batting action in the image, and can segment and classify the batting action, distinguish different types of batting actions (such as forehand, backhand, slice, drive, smash, etc.), and can capture the player's posture and batting angle when hitting the ball.

[0056] The image analysis module 230 can also calculate the speed and / or acceleration change of the billiard ball at the moment it is hit by the racket based on the movement state of the billiard ball in continuous frames, and can use the principle of conservation of momentum according to the change in speed (the increase in the speed of the billiard ball after being hit is proportional to the force of the hit) to estimate the hitting force, or use a trained deep learning model to infer the hitting force based on the speed and / or acceleration.

[0057] In some embodiments, if the historical playing images acquired by the image acquisition module 210 are acquired by a high-speed camera, the image analysis module 230 can also analyze the rotation state and the corresponding bouncing state of the table tennis ball when analyzing the motion state of the table tennis ball in the historical playing images. For example, an algorithm such as an optical flow method or feature point matching is used to track the movement state of the pattern or marking point on the surface of the table tennis ball to calculate the rotation speed (angular velocity) and rotation direction of the table tennis ball, so as to determine the rotation state of the table tennis ball.

[0058] In some embodiments, the image analysis module 230 may also pre-process the historical fighting images, such as noise filtering, image stabilization, etc., to increase the accuracy of subsequent analysis, processing, and judgment.

[0059] The image analysis module 230 may also perform face recognition on the players included in the historical match images to determine whether a player included in different historical match images is the same player.

[0060] The model training module 250 is responsible for training the machine learning model corresponding to each player based on the motion state of the billiard ball and the hitting behavior of each player generated by the image analysis module 230. In some embodiments, the model training module 250 can also train the machine learning model corresponding to each player based on the position and posture of another player (opponent) when the player generates the hitting behavior when training the machine learning model corresponding to each player. The posture and position of the opponent can be obtained by the image analysis module 230 as described above.

[0061] The model training module 250 can use a convolutional neural network (CNN) to analyze the player's actions based on the batting methods (forehand, backhand, drive, slice, smash, etc.) in the batting strategy included in the batting behavior, the movement state of the table tennis ball, the batting results (the opponent cannot return the ball to score, the opponent returns the ball incorrectly to score, the player makes a return error, etc.), the opponent's posture, and the opponent's position. Models such as recurrent neural network (RNN) or long short-term memory model (LSTM) are used to process sequence data to capture the temporal relationship of the batting action, and deep reinforcement learning (DRL) can be used to simulate the player's behavior and decision-making, and the opponent's reaction can be simulated to train the above-mentioned model to learn to choose the batting strategy with the greatest chance of scoring.

[0062] The model training module 250 can also adjust the parameters of the movement state of the virtual table tennis ball generated by the machine learning model corresponding to the user when the reality interaction module 270 determines that the user makes a predetermined voice or performs a predetermined action. For example, the movement state of the virtual table tennis ball can be re-determined based on the hitting angle, hitting position, and hitting force.

[0063] The reality interaction module 270 is responsible for generating virtual players and virtual billiard balls, and using reality technology to display the virtual players and virtual billiard balls. The reality interaction module 270 can also generate a playing field, and use the same reality technology to display the playing field.

[0064] The reality interaction module 270 is also responsible for displaying the motion state of the virtual billiard ball and the hitting behavior of the virtual player regenerated by the interactive simulation module 290 using the same reality technology as that used to display the virtual player. If the reality technology is virtual reality, the reality interaction module 270 can also display a virtual character representing the user and simulate the motion state of the virtual billiard ball based on the motion state data of the virtual billiard ball generated by the interactive simulation module 290 and simulate the hitting behavior of the virtual character based on the target hitting behavior of the user when hitting the ball.

[0065] The reality interaction module 270 may also analyze whether the user makes a predetermined voice or performs a predetermined action through voice analysis or a posture estimation model.

[0066] The interactive simulation module 290 is responsible for generating the virtual player's virtual batting behavior (including virtual batting movements and virtual batting strategies) based on the motion state of the virtual billiard ball, the current position of the virtual player, and the current position of the user when it is the virtual player's turn to bat, and regenerating the motion state of the virtual billiard ball based on the virtual batting movements and virtual batting strategies. In more detail, the interactive simulation module 290 can first determine whether the virtual player can successfully hit the billiard ball based on the motion trajectory of the virtual billiard ball and the current position of the virtual player. When the interactive simulation module 290 determines that the virtual player can hit the billiard ball, it can generate the virtual player's footwork (movement trajectory) and virtual batting behavior based on the motion state of the virtual billiard ball and the current position of the user or the virtual character representing the user using the machine learning model of the player corresponding to the virtual player, and regenerate the motion state of the virtual billiard ball based on the virtual batting behavior of the virtual player.

[0067] In some embodiments, if there are multiple players corresponding to the virtual player, the interactive simulation module 290 may also use the machine learning model of each player corresponding to the virtual player in sequence to generate simulated batting behavior (including simulated batting actions and simulated batting strategies), and generate the motion state of the virtual table tennis according to the simulated batting behavior, and select the simulated batting action and simulated batting strategy included in the simulated batting behavior with the highest scoring possibility as the virtual batting action and virtual batting strategy in the virtual batting behavior according to the motion state of the virtual table tennis and the current position of the user or the virtual character representing the user.

[0068] The interactive simulation module 290 is also responsible for obtaining the user's batting behavior. For example, the interactive simulation module 290 can receive the sensing signals generated by the sensors or controllers such as the hands, wrists and feet (or even the racket) worn or worn by the user through the communication interface 130, and can determine the user's batting behavior based on the obtained sensing signals. More specifically, the interactive simulation module 290 can determine the angle of the user's arm and wrist, the swing time, the swing speed, the swing angle, and the swing strength based on the sensing signals of the hands and wrists (and the racket), and can also determine the user's moving speed and step distance based on the sensing signals of the feet; the interactive simulation module 290 can also receive the image signal obtained by the motion capture camera through the communication interface 130, and can analyze the user's batting behavior through the image analysis module 230. However, the way in which the interactive simulation module 290 obtains the user's batting behavior is not limited to the above.

[0069] The interactive simulation module 290 is also responsible for regenerating the motion state of the virtual billiard ball based on the motion state of the virtual billiard ball and the user's target hitting behavior using a machine learning model corresponding to the user when it is the user's turn to hit the ball. More specifically, the interactive simulation module 290 may first determine whether the user's target hitting behavior can successfully hit the billiard ball based on the motion state of the virtual billiard ball, and when it is determined that the target hitting behavior can successfully hit the billiard ball, the interactive simulation module 290 may regenerate the motion state of the virtual billiard ball based on the user's target hitting behavior using a machine learning model corresponding to the user.

[0070] Next, an embodiment is used to explain the system operation and method of the present invention, and please refer to Figure 3 Flow chart of the method for performing table tennis battle training based on opponent's hitting behavior simulation provided by the present invention. In this embodiment, it is assumed that the device 100 is a server providing mixed reality, but the present invention is not limited thereto.

[0071] First, the image acquisition module 210 of the device 100 can acquire the historical fighting images (step 310). In this embodiment, it is assumed that the administrator of the server can store a large number of historical fighting images in the storage medium 140 of the device 100 in advance, and the image acquisition module 210 can read the historical fighting images from the storage medium 140. The administrator can also input the network address or domain name of the network storage device storing the historical fighting records and the storage path of the historical fighting images through the input element 150 of the device 100. The image acquisition module 210 can connect to the network storage device through the communication interface 130 of the device 100 according to the network address or domain name and the storage path to read the historical fighting images contained in the historical fighting records.

[0072] After the image acquisition module 210 of the device 100 acquires the historical playing images (step 310), the image analysis module of the device 100 can analyze the historical playing images to generate the motion state of the billiard ball and the player's hitting behavior (step 320). In this embodiment, assuming that the historical playing images are taken by a high-speed camera, the image analysis module 230 can analyze the player's moving steps, hitting posture, hitting position and other hitting behaviors, and the player's hitting method, hitting angle, hitting force and other hitting strategies when analyzing the historical playing images, and can also analyze the rotation state and bounce state of the billiard ball.

[0073] After the image analysis module 230 of the device 100 analyzes and generates the motion state of the billiard ball and the hitting behavior of the player, the model training module 250 of the device 100 can train the machine learning model corresponding to each different player according to the historical playing images obtained by the image acquisition module 210 and the motion state of the billiard ball and the hitting behavior of the player generated by the image analysis module 230 of the device 100 (step 330). In this embodiment, the model training module 250 can generate a machine learning model corresponding to the user and a machine learning model corresponding to other players other than the user.

[0074] After the model training module 250 of the device 100 completes the training of the machine learning model, the communication interface 130 of the device 100 can provide the user with a connection to the real device 400, and the real interaction module 270 can generate virtual players and virtual billiard balls after the real device 400 is connected, and can transmit them to the real device 400 through the communication interface 130, so that the real device 400 displays the virtual players and virtual billiard balls (step 340). In this embodiment, since the device 100 can provide a mixed reality, the real interaction module 270 can generate a display model of a virtual table set at a specific position in the real scene, and can also generate display models of virtual players and virtual billiard balls, and can transmit the generated display models of the virtual table, virtual players and virtual billiard balls to the real device 400, so that the real device 400 worn by the user can display the virtual table, virtual players and virtual billiard balls in the real scene according to the received display models.

[0075] After the reality interaction module 270 of the device 100 generates the display model of the virtual player and the virtual billiard, the interactive simulation module 290 of the device 100 can determine whether it is the virtual player's turn to hit the ball or the user's turn to hit the ball according to the billiard serving rules and the rules of both players taking turns to hit the ball.

[0076] When it is the virtual player's turn to bat, the interactive simulation module 290 of the device 100 can use the machine learning model of the player corresponding to the virtual player to generate the virtual player's virtual batting behavior based on the movement state of the virtual table tennis ball, the current position of the virtual player, and the current position of the user, and regenerate the movement state of the virtual table tennis ball based on the virtual batting action and the virtual batting strategy. The reality interaction module 270 of the device 100 can transmit the movement state of the virtual table tennis ball to the reality device 400 through the communication interface 130 of the device 100, so that the reality device 400 displays the movement state of the virtual table tennis ball after the virtual player bats the ball in the real scene (step 370). In this embodiment, it is assumed that the interactive simulation module 290 can first determine whether the virtual player can successfully hit the billiard ball based on the movement trajectory of the virtual billiard ball and the current position of the virtual player. When the interactive simulation module 290 determines that the virtual player can hit the virtual billiard ball, the virtual player's footwork (movement trajectory) and virtual hitting behavior can be generated based on the movement state of the virtual billiard ball and the current position of the user using the machine learning model of the player corresponding to the virtual player, and the movement state of the virtual billiard ball can be regenerated based on the virtual player's virtual hitting behavior. Then, after the real interaction module 270 transmits the display signal of the virtual player's virtual hitting behavior and the movement state of the virtual billiard ball generated by the interactive simulation module 290 to the real device 400, the real device 400 can display the movement and hitting action of the virtual player based on the received display signal of the virtual hitting behavior and the movement state of the virtual billiard ball, and can display the movement state of the virtual billiard ball at a predetermined time (i.e., the moment when the virtual player's racket hits the virtual billiard ball determined by the interactive simulation module 290), so as to simulate the virtual player hitting the virtual billiard ball so that the virtual billiard ball changes to move toward the user.

[0077] When it is the user's turn to bat, the interactive simulation module 290 can first obtain the user's target batting behavior, and can use the machine learning model corresponding to the user to regenerate the motion state of the virtual billiard ball based on the motion state of the virtual billiard ball and the user's target batting behavior. The reality interaction module 270 can transmit the motion state of the virtual billiard ball to the reality device 400 through the communication interface 130 of the device 100, so that the reality device 400 displays the motion state of the virtual billiard ball after the user bats the ball (step 370). In this embodiment, it is assumed that the real device 400 can obtain the sensing signal generated by the sensor worn by the user and can transmit the obtained sensing signal back to the device 100. The interactive simulation module 290 can obtain the sensing signal after the communication interface 130 of the device 100 receives the sensing signal transmitted by the real device 400, and can judge the user's target hitting behavior based on the obtained sensing signal. Then, it can judge whether the user's target hitting behavior can successfully hit the virtual billiard ball based on the movement state of the virtual billiard ball. When it is judged that the target hitting behavior can successfully hit the billiard ball, the movement state of the virtual billiard ball can be regenerated based on the user's target hitting behavior using a machine learning model corresponding to the user, and the generated movement state of the virtual billiard ball can be transmitted to the real device 400. Afterwards, the real device 400 can display the received movement state of the virtual billiard ball at the moment when the user's racket hits the virtual billiard ball during the interactive simulation module 290 judging the user's target hitting behavior, so as to simulate the user hitting the virtual billiard ball so that the virtual billiard ball moves towards the virtual opponent instead.

[0078] Thus, through the present invention, a billiard player can be simulated to generate a virtual player and the user can be provided with a game training with the virtual player through the reality technology.

[0079] In the above embodiment, when the interactive simulation module 290 of the device 100 can use the machine learning model of the player corresponding to the virtual player to generate the virtual batting behavior of the virtual player according to the movement state of the virtual table tennis, the current position of the virtual player, and the current position of the user, and regenerate the movement state of the virtual table tennis according to the virtual batting action and the virtual batting strategy (step 350), the interactive simulation module 290 of the device 100 can use the machine learning model of each player corresponding to the virtual player in sequence to generate the simulated batting behavior, and generate the movement state of the virtual table tennis according to the simulated batting behavior, and select the simulated batting behavior with the highest scoring rate according to the movement state of the virtual table tennis and the current position of the user, and use the simulated batting action and the simulated batting strategy included in the selected simulated batting behavior as the virtual batting action and the virtual batting strategy.

[0080] In addition, after the reality interaction module 270 of the device 100 transmits the motion state of the virtual billiard ball generated by the interactive simulation module 290 of the device 100 to the reality device 400 through the communication interface 130 of the device 100, so that the reality device 400 displays the motion state of the virtual billiard ball (step 370), the reality interaction module 270 can also determine whether the user makes a predetermined voice or performs a predetermined action. For example, when the user believes that the motion state of the virtual billiard ball hit is different from the expected, if the user makes a predetermined voice or performs a predetermined action, the model training module 250 of the device 100 can adjust the parameters of the motion state of the virtual billiard ball generated by the machine learning model corresponding to the user.

[0081] In summary, it can be seen that the difference between the present invention and the prior art lies in the technical means of analyzing historical playing images to obtain the movement state of the table tennis ball and the batting behaviors of multiple players, and training each machine learning model corresponding to each player based on the movement state and the batting behavior of the player, and then using reality technology to display the virtual player and the virtual table tennis ball. When it is the virtual player's turn to bat, the machine learning model of the player corresponding to the virtual player is used to generate the virtual batting behavior of the virtual player and the movement state of the virtual table tennis ball is generated based on the virtual batting behavior. When it is the user's turn to bat, the machine learning model corresponding to the user is used to generate the movement state of the virtual table tennis ball based on the user's target batting behavior. Finally, reality technology is used to display the movement state of the virtual table tennis ball. This technical means can solve the problem of the prior art that it is impossible to respond to the player's batting behavior in real time, thereby achieving the technical effect of providing players with virtual opponents for playing training.

[0082] Furthermore, the method of the present invention for performing table tennis battle training based on opponent's hitting behavior simulation can be implemented in hardware, software or a combination of hardware and software, and can also be implemented in a centralized manner in a computer system or in a decentralized manner with different components distributed in several interconnected computer systems.

[0083] Although the embodiments disclosed in the present invention are as above, the contents described are not intended to directly limit the scope of patent protection of the present invention. Any ordinary technician in the technical field to which the present invention belongs, without departing from the spirit and scope disclosed in the present invention, makes some modifications and modifications to the form and details of the implementation of the present invention, which belong to the scope of patent protection of the present invention. The scope of patent protection of the present invention shall still be based on the scope defined by the attached claims.

Claims

1. A method for table tennis training based on opponent's hitting behavior simulation, applied to a device or system, characterized in that: The method comprises at least the following steps: Obtaining a historical match record, wherein the historical match record includes a plurality of historical match images of billiard activities; Analyze each of the historical match images to obtain the motion state of the billiard ball and the hitting actions and hitting strategies of multiple players, wherein the multiple players include the user, the multiple hitting actions include moving steps, hitting postures, and hitting positions, and the hitting strategies include opponent postures, opponent positions, hitting methods, hitting angles, and hitting strengths; Training each machine learning model corresponding to each of the players according to the motion state and each of the players' batting movements and batting strategies; Generate a virtual player and a virtual billiard ball, and display the virtual player and the virtual billiard ball using real-world technology, wherein the virtual player corresponds to at least one of the players; When it is the virtual player's turn to hit the ball, a virtual hitting action and a virtual hitting strategy of the virtual player are generated using a machine learning model of at least one of the players corresponding to the virtual player according to the motion state of the virtual billiard ball, the current position of the virtual player, and the current position of the user, and the motion state of the virtual billiard ball is regenerated according to the virtual hitting action and the virtual hitting strategy, and the virtual hitting action of the virtual player and the motion state of the virtual billiard ball are displayed using the real-world technology; and When it is the user's turn to hit the ball, the motion state of the virtual billiard ball is regenerated using the machine learning model corresponding to the user based on the motion state of the virtual billiard ball and the user's target hitting behavior, and the motion state of the virtual billiard ball is displayed using the real-world technology.

2. The method for performing table tennis battle training based on the simulation of the opponent's hitting behavior as claimed in claim 1, characterized in that: The step of analyzing each of the historical playing images to obtain the motion state of the billiard ball further includes the step of analyzing and obtaining the rotation state and the corresponding bounce state of the billiard ball.

3. The method for table tennis training based on opponent's hitting behavior simulation as claimed in claim 1, characterized in that: The step of using a machine learning model of at least one of the players corresponding to the virtual player to generate the virtual batting action and the virtual batting strategy of the virtual player according to the movement state of the virtual table tennis ball and the current position of the virtual player, and regenerating the movement state of the virtual table tennis ball according to the virtual batting action and the virtual batting strategy also includes sequentially using the machine learning model of each of the players corresponding to the virtual player to generate simulated batting behavior, and generating the movement state of the virtual table tennis ball according to the simulated batting action and the simulated batting strategy included in the simulated batting behavior, and selecting the simulated batting action and the simulated batting strategy included in the simulated batting behavior as the virtual batting action and the virtual batting strategy according to the movement state of the virtual table tennis ball and the current position of the user.

4. The method for performing table tennis battle training based on opponent's hitting behavior simulation as claimed in claim 1, characterized in that: The step of generating the virtual player and the virtual billiard ball and displaying the virtual player and the virtual billiard ball using the real-world technology further includes the step of generating a playing field and displaying the playing field using the real-world technology.

5. The method for performing table tennis battle training based on opponent's hitting behavior simulation as claimed in claim 1, characterized in that: The method further includes the step of adjusting the parameters of the motion state of the virtual billiard ball generated by the machine learning model corresponding to the user when determining that the user makes a predetermined voice or performs a predetermined action, after regenerating the motion state of the virtual billiard ball using the machine learning model corresponding to the user based on the motion state of the virtual billiard ball and the target hitting behavior of the user, and displaying the motion state of the virtual billiard ball using the real-life technology.

6. A device for performing table tennis combat training based on the opponent's hitting behavior simulation, characterized in that: The device at least comprises: An image acquisition module is used to acquire historical match records, wherein the historical match records include multiple historical match images of billiard activities; An image analysis module, for analyzing each of the historical playing images to obtain the motion state of the billiard ball and the hitting actions and hitting strategies of multiple players, wherein the multiple players include the user, the multiple hitting actions include moving steps, hitting postures, and hitting positions, and the hitting strategies include opponent postures, opponent positions, hitting methods, hitting angles, and hitting strengths; A model training module, for training each machine learning model corresponding to each of the players according to the motion state and the hitting action and hitting strategy of each of the players; A reality interaction module, used for generating a virtual player and a virtual billiard ball, and displaying the virtual player and the virtual billiard ball using reality technology, wherein the virtual player corresponds to at least one of the players; and The interactive simulation module is used to generate a virtual batting action and a virtual batting strategy of the virtual player based on the motion state of the virtual billiard ball, the current position of the virtual player, and the current position of the user using a machine learning model of at least one of the players corresponding to the virtual player when it is the virtual player's turn to bat, and to regenerate the motion state of the virtual billiard ball based on the virtual batting action and the virtual batting strategy, and to display the virtual batting action of the virtual player and the motion state of the virtual billiard ball by the reality interaction module using the reality technology, and to regenerate the motion state of the virtual billiard ball based on the motion state of the virtual billiard ball and the user's target batting behavior by using the machine learning model corresponding to the user when it is the user's turn to bat, and to display the motion state of the virtual billiard ball by the reality interaction module using the reality technology.

7. The device for performing table tennis battle training based on the simulation of the opponent's hitting behavior as claimed in claim 6, characterized in that: The image analysis module is also used to analyze and obtain the rotation state and the corresponding bounce state of the billiard ball.

8. The device for performing table tennis battle training based on the simulation of the opponent's hitting behavior as claimed in claim 6, characterized in that: The interactive simulation module sequentially uses the machine learning model of each player corresponding to the virtual player to generate simulated batting behavior, and generates the motion state of the virtual table tennis according to the simulated batting action and simulated batting strategy included in the simulated batting behavior, and selects the simulated batting action and simulated batting strategy included in the simulated batting behavior as the virtual batting action and the virtual batting strategy according to the motion state of the virtual table tennis and the current position of the user.

9. The device for performing table tennis battle training based on the simulation of the opponent's hitting behavior as claimed in claim 6, characterized in that: The reality interaction module is also used to generate a fighting venue and display the fighting venue using the reality technology.

10. The device for performing table tennis battle training based on the simulation of the opponent's hitting behavior as claimed in claim 6, characterized in that: The model training module is also used to adjust the parameters of the movement state of the virtual billiard ball generated by the machine learning model corresponding to the user when the reality interaction module determines that the user makes a predetermined voice or performs a predetermined action.

Images