Holographic billiard auxiliary learning system based on head-mounted augmented reality equipment
Through the holographic billiards assisted learning system based on head-mounted augmented reality devices, three-dimensional batting guidance is provided using positionable cameras and machine learning models, which solves the problems of difficulty in correcting errors and the dependence of existing systems and visual singleness, and achieves the problem of difficulty in correcting errors in traditional teaching modes and difficulty in correcting site dependence and visual singleness in the existing system. Quickly improve players' aiming and cue control capabilities.
Patent Information
- Application Number
- CN202411166011.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-29
AI Technical Summary
The traditional billiards teaching mode is difficult to effectively guide players to correct hitting errors, and the existing projection-type augmented reality assisted learning system has high requirements for the venue, a single mode, and only provides two-dimensional visual guidance, which cannot meet the players' three-dimensional perception needs.
Using a holographic billiards assisted learning system based on head-mounted augmented reality devices, the positionable camera, image processing module, object positioning module and machine learning model can calculate the coordinates and hitting instructions of the ball in real time, providing three-dimensional visual guidance to help players improve. Aim and cue ball control ability.
The system allows players to quickly master the aiming ability in unsupervised training and form robust cue ball control skills in a short period of time, overcoming the dependence of traditional teaching methods and the two-dimensional limitations of visual guidance, and improving the training effect.
Smart Images

Figure CN119045664A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of billiards technology, and in particular to a holographic billiards auxiliary learning system based on a head-mounted augmented reality device. Background Art
[0002] Billiards is a sport that is difficult to get started with. According to a survey, the average player needs to train under the guidance of a professional coach for more than three months to reach the entry level.
[0003] Billiards involves two key techniques: aiming and cue ball control. Aiming focuses on how to score a goal. Cue ball control focuses on how to use the appropriate hitting point and force so that the cue ball position still supports continued hitting after scoring a goal.
[0004] The traditional billiards teaching model generally includes three stages. The first stage: learn the correct batting posture and process under the guidance of a professional coach to ensure that the aiming direction is consistent with the hitting direction, and then train the brain to correctly perceive the batting angle. The second stage: learn the impact of different hitting points on the separation trajectory of the cue ball after collision, and establish the brain's correct perception of the separation trajectory through repeated practice. The third stage: learn how to control the cue ball trajectory while scoring a goal so that the cue ball position still supports continued hitting.
[0005] Billiards practice is a process of correcting the brain's perception of aiming direction, hitting point and force. In traditional teaching methods, players cannot get any other effective guidance except the coach's guidance on the hitting action. Players have no way of knowing whether the error in hitting the ball comes from the error in hitting the ball or the aiming action, so they cannot effectively correct the hitting link based on the results.
[0006] People have proposed some solutions to this problem. Among them, the projection-based augmented reality billiards assisted learning system is a very promising implementation strategy, such as Pool Live Aid developed by Portugal's Forge IT company and the light and shadow intelligent billiards robot developed by Jingdian Hi-Tech in China. This type of product uses a camera to track the position of the ball under a standard environment, calculates the trajectory of the goal and the separation trajectory of the cue ball, and then projects it onto the table by a projection device to provide visual guidance. It can provide players with a simple means of correcting perception errors, and the implementation process can also maintain good stability. However, it has shortcomings: high requirements for the venue, requiring the special laying of relevant equipment; the mode is single, and instructions can only be provided in simple scenarios, such as single-target ball practice; it can only provide two-dimensional visual guidance, rather than more easily perceived three-dimensional visual guidance. Therefore, a holographic billiards assisted learning system based on a head-mounted augmented reality device is invented. Summary of the invention
[0007] In view of the above-mentioned and / or existing problems in the existing holographic billiards assisted learning system based on head-mounted augmented reality device, the present invention is proposed.
[0008] Therefore, the purpose of the present invention is to provide a holographic billiards assisted learning system based on a head-mounted augmented reality device, which can solve the above-mentioned existing problems.
[0009] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions:
[0010] The holographic billiards assisted learning system based on head-mounted augmented reality device includes:
[0011] Augmented reality framework;
[0012] Augmented reality equipment for photographing and displaying billiard balls;
[0013] A spatial modeling module, used to locate the position of the billiard ball based on the image captured by the augmented reality device;
[0014] A technical and tactical analysis module, used to obtain hitting instructions based on the data of the spatial modeling module;
[0015] The augmented reality framework is provided with an augmented reality framework, an augmented reality device, a space modeling module and a technical and tactical analysis module. The augmented reality device is connected to the space modeling module, the space modeling module is connected to the technical and tactical analysis module, and the technical and tactical analysis module is connected to the augmented reality device.
[0016] As a preferred solution of the holographic billiards assisted learning system based on a head-mounted augmented reality device according to the present invention, the augmented reality device comprises:
[0017] Positionable camera for photographing billiard balls;
[0018] A display, used to display the hitting instructions obtained by the technical and tactical analysis module;
[0019] The locatable camera is connected to the space modeling module, and the display is connected to the technical and tactical analysis module.
[0020] As a preferred solution of the holographic billiards assisted learning system based on a head-mounted augmented reality device according to the present invention, the space modeling module includes:
[0021] An image processing module, used for processing images taken by a locatable camera;
[0022] An object positioning module is used to realize object positioning according to the data processed by the image processing module to obtain ball positioning information;
[0023] The image processing module is connected to the locatable camera, the image processing module is connected to the object positioning module, and the object positioning module is connected to the technical and tactical analysis module.
[0024] As a preferred solution of the holographic billiards assisted learning system based on a head-mounted augmented reality device described in the present invention, the image processing module is composed of a deep learning detection model and a visual information screening algorithm.
[0025] As a preferred solution of the holographic billiards assisted learning system based on a head-mounted augmented reality device described in the present invention, the object positioning module is composed of spatial prior information and a spatial analysis algorithm.
[0026] As a preferred solution of the holographic billiards assisted learning system based on a head-mounted augmented reality device according to the present invention, the technical and tactical analysis module includes:
[0027] A machine learning model for learning from ball positioning information;
[0028] A physical simulation module for performing simulations based on data learned by the machine learning model;
[0029] The object positioning module is connected to the machine learning model, the machine learning model is connected to the physical simulation module, the machine learning model is connected to the display, and learnable parameters are provided in the machine learning model.
[0030] As a preferred solution of the holographic billiards assisted learning system based on a head-mounted augmented reality device described in the present invention, it also includes a system process, and the specific steps are as follows:
[0031] S1: The player wears a head-mounted augmented reality device and walks around the stage. During this process, the camera position V is calculated in real time. i =(x i ,y i , z i ), i=1,...,n; camera posture P i =(α i , β i , γ i ), i = 1, ..., n, represents the rotation angle around the x', y', z' axis of the coordinate system O'x'y'z'; the line of sight direction d i =(x i ,y i , z i), i = 1, ..., n; at the same time, the locatable camera function of the head-mounted augmented reality device will be called to automatically select the time to shoot the image according to the camera position and camera posture; its selection mechanism is as follows: i images have been selected at the current moment, with camera position V', camera posture P', sight direction d', image I'; the score at the current moment
[0032]
[0033] Where A is the straight line V'+d't1 and the plane y=H table The intersection point, t2, t3∈[-0.5, 0.5], if ε=1, I i+1 =I',V i+1 =V',P i+1 =P',d i+1 = d', the shooting ends when ζ = 1;
[0034]
[0035] Based on the above data, solve the coordinates v of all the balls on the table. k and Category C k ;
[0036] O={(v k ,C k )|k=1,...,q}
[0037] Among them, q is the total number of balls on the current table;
[0038] S2: i = 1, 2, ..., n, loop to perform the following operations:
[0039] S21: Take I i , P i , making I i Rotate γ around (W / 2, H / 2) i ', get I i ';
[0040] S22: Detection using pre-trained object detection model I i 'The billiard balls visible in the image, get the ball projection set;
[0041] p={((x j ,y j ),C j )|j=1,...,m}
[0042] Where (x j ,y j ) is the position of the ball in the image, C j For categories;
[0043] S23: j = 1, ..., m, the following operations are performed cyclically:
[0044] S231: Calculate the horizontal angle ω of the camera's sight line in the direction of the sphere center during shooting based on the perspective relationship and camera parameters h The tangent value and vertical angle ω v The tangent value of
[0045] tanω h =x j tan(FOV h / 2) / (W / 2)
[0046] tanω v =y j tan(FOV v / 2) / (H / 2)
[0047] S232: By V i d i ,ω h ,ω v The point-to-point linear parametric equation l can be solved j , where l j The components of are x=x0+x1t, y=y0+y1t, z=z0+z1t, from l j With plane y = H table +r can be solved to get the parameter t of the straight line equation, and substitute it into l j The center of the sphere (x', y', z') can be obtained, and the sphere o can be obtained. j ;
[0048] S24: Execute according to S23 to obtain balls o1, ..., o j , merged into a ball set O i ;
[0049] S3: Execute according to S2 to obtain the ball set O1, ..., O n , we need to solve O1, ..., O n The union of is O in S1; if the ball is closer to the camera, not blocked, and in the center of the field of view, it will be estimated to have a higher confidence. At this time, the same ball will be imaged from different directions, and the highest confidence estimation result will be selected for each ball to construct a confidence score function:
[0050] S=as1+bs2+cs3
[0051] Where a, b, c are adjustable weights, satisfying the constraint condition a+b+c=1, and the preset values are a=0.2, b=0.3, c=0.5;
[0052]
[0053] Where V is the shooting position and v is the estimated center of the ball;
[0054]
[0055] Where (x, y) is the center of the projection of the ball;
[0056]
[0057] Where p1, p2, …, p m are all the ball projections in the current image, r p is the radius of the estimated projection p, and the function f is used to calculate the overlapping area of the two ball projections;
[0058] S4: Solve for O. For each estimated ball object o
[0059]
[0060] Thus, the distribution of the balls on the table at this moment is obtained;
[0061] O = {(v k , C k ) | k = 1, ..., q}
[0062] S5: Let o be the target ball, θ h ∈ [0, 2π) be the angle between the projection of the cue in the xz plane and the x-axis; θ v ∈ (0, 2π) be the angle formed by the cue and the horizontal plane; (ρ, θ) be the polar coordinates of the hitting point, ρ < R, θ ∈ [0, 2π), where R is the radius of the effective hitting area, R = 0.75r; τ is the cue speed; θ h is derived from the elastic collision phenomenon. When a linearly moving ball hits another stationary ball of equal mass and undergoes an elastic collision, the component velocity of the moving ball in the collision direction will be transferred to the originally stationary ball;
[0063]
[0064] Where v o is the coordinate of the target ball, v h is the coordinate of the pocket, v c is the coordinate of the cue ball; Use the fully connected neural network G to calculate θ v 、ρ、θ、τ at this moment; The input shape of G is 16×2, representing 16 balls, each ball having a two-dimensional coordinate; The output shape is 4, respectively representing θ v, ρ, θ, τ, and the middle layer contains a flattening layer, a fully connected layer 1, a fully connected layer 2, a fully connected layer 3, and an output layer; the flattening layer flattens the 16×2 input into 32×1; the fully connected layers 1, 2, and 3 contain 128, 64, and 32 neurons respectively, all using the ReLU activation function; the output layer contains 4 neurons and uses a linear activation function;
[0065] G performs self-supervised learning in a simulated billiard program environment, where the program can simulate the ball sliding, rolling, rotating, and colliding. The loss function guiding G’s training is:
[0066] L=δ1E1+δ2E2+δ3E3+δ4E1E4+δ5(1-E1)E5
[0067] Among them, δ1, δ2, δ3, δ4, and δ5 satisfy the constraints;
[0068] δ1+δ2+δ3+δ4=1, δ2+δ3+δ5=C
[0069] By default, δ1=0.3, δ2=0.3, δ3=0.3, δ4=0.3, δ5=0.3, C=0.7, C is the defense limit coefficient; E1 indicates whether a certain ball is pocketed, if it is pocketed, E1=1, otherwise it is 0; E2 is used to suppress the selection of abnormal speed:
[0070]
[0071] Where S is [1, 6]; E3 is used to suppress the selection of abnormal hitting areas;
[0072]
[0073] E4 indicates the feasibility of continuous hitting;
[0074] E4=1-E5
[0075] The difficulty of ball o is:
[0076]
[0077] Its φ1=0.4,φ2=0.2,φ3=0.4; the difficulty of obtaining e balls of one's own color is sorted from low to high:
[0078] (D1,...,D e )
[0079] E5 indicates the difficulty of scoring a goal;
[0080]
[0081] Where σ∈(0,1) is the attenuation coefficient, which makes the loss function pay more attention to the easiest ball to hit. The smaller the value, the more attention is paid to the easiest ball. The default value is 0.33;
[0082] S6: Positioning the holographic image; calculating the cue ball collision position where the target ball falls into the pocket;
[0083]
[0084] In v t Rendering a holographic ball of equal size and an ideal hitting direction l at the same location can provide players with hitting guidance. The straight line l is expressed as:
[0085] v t +(tcosθ h ,ttanθ v ,tsinθ h )+(cosθ h ,0,sinθ h )+ρcosθ(-sinθ h ,0,cosθ h )+ρsinθ(0,1,0)
[0086] Where t is the parameter of the straight line equation;
[0087] Display the straight line l in the virtual space, so that the user can have an intuitive perception of the direction of the club. In addition, the schematic diagram of the hitting point and the club speed are displayed in the field of vision in the form of a two-dimensional user interface to ensure that all key information can be captured by the user.
[0088] S7: The user hits the ball according to the instruction information, thus completing one hitting process and waiting for the next hitting.
[0089] Compared with existing technologies:
[0090] 1. Compared with the projection-type augmented reality assisted learning system, the present invention adopts a head-mounted AR device to overcome the problems of high dependence on the venue and single processing mode, and provides more easily perceived three-dimensional visual guidance, so that players can obtain better training effects.
[0091] 2. The present invention enables players to master aiming skills through 2 months of unsupervised training. For players who have received basic stroke training, they can develop a robust aiming ability within 1 week. For players who have reached the entry level, the present invention enables them to master basic cue ball control skills within 1 month and acquire the ability to clear the table with a simple ball within 3 months. BRIEF DESCRIPTION OF THE DRAWINGS
[0092] Figure 1 A schematic diagram for establishing the observation and coordinate system of the present invention;
[0093] Figure 2 is a flow chart of the method of the present invention;
[0094] Figure 3 It is a system structure diagram of the present invention;
[0095] Figure 4 This is a neural network structure diagram of the present invention;
[0096] Figure 5 is the elastic collision diagram of the present invention;
[0097] Figure 6 This is a diagram showing the effect of the present invention. DETAILED DESCRIPTION
[0098] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
[0099] The present invention provides a holographic billiards assisted learning system based on a head-mounted augmented reality device, please refer to Figure 1-Figure 6 , including: an augmented reality framework; an augmented reality device for photographing and displaying billiard balls; a spatial modeling module for locating the position of billiard balls according to the images photographed by the augmented reality device; a technical and tactical analysis module for obtaining hitting instructions according to data from the spatial modeling module;
[0100] The augmented reality framework is provided with an augmented reality framework, an augmented reality device, a space modeling module and a technical and tactical analysis module. The augmented reality device is connected to the space modeling module, the space modeling module is connected to the technical and tactical analysis module, and the technical and tactical analysis module is connected to the augmented reality device.
[0101] The augmented reality device includes: a positionable camera for photographing billiard balls; a display for displaying the hitting instructions obtained by the technical and tactical analysis module;
[0102] The locatable camera is connected to the space modeling module, and the display is connected to the technical and tactical analysis module.
[0103] The spatial modeling module includes: an image processing module for processing images taken by a positionable camera; an object positioning module for realizing object positioning according to data processed by the image processing module to obtain ball positioning information;
[0104] The image processing module is connected to the locatable camera, the image processing module is connected to the object positioning module, and the object positioning module is connected to the technical and tactical analysis module;
[0105] The image processing module is composed of a deep learning detection model and a visual information screening algorithm; the object positioning module is composed of spatial prior information and a spatial parsing algorithm.
[0106] The technical and tactical analysis module includes: a machine learning model for learning according to ball positioning information; a physical simulation module for simulating according to the data learned by the machine learning model;
[0107] The object positioning module is connected to the machine learning model, the machine learning model is connected to the physical simulation module, the machine learning model is connected to the display, and learnable parameters are provided in the machine learning model.
[0108] According to the real space, a three-dimensional rectangular coordinate system Oxyz is established as the world coordinate system, where the y-axis direction corresponds to the vertical direction of the real world, and the z-axis direction corresponds to the direction the player is facing. At the same time, a three-dimensional rectangular coordinate system O'x'y'z' is established to describe the head posture, where the z-axis direction corresponds to the line of sight and the y-axis direction points to the top of the player's head. The image captured by the camera is located in the two-dimensional rectangular coordinate system ouv, and the lower left corner of the image corresponds to the origin of the coordinate axis. Observation and coordinate system establishment are shown in the attached Figure 1 .
[0109] The known parameters are: camera horizontal field of view FOV h , vertical field of view FOV v , the camera captures the image with a width W and a height H, as well as parameters related to billiards: the radius r of the ball, the height H of the table table , table length L table , table width W table , table center V table .
[0110] When using the present invention, the player can position the camera to obtain the table image while wearing the head-mounted augmented reality device and observing around the table. The system detects the balls in the image and positions all the balls according to the spatial relationship. In order to obtain all the information of the table, excluding factors such as occlusion, too far imaging distance, and incomplete image coverage, the positionable camera will shoot the table from several positions. Then calculate the aiming position, hitting point and hitting speed of the ball. Finally, the corresponding holographic image is rendered to the player's field of view to provide instructions. See the attached system flow. Figure 2 , the system structure is attached Figure 3 .
[0111] A system flow is also included, and the specific steps are as follows:
[0112] S1: The player wears a head-mounted augmented reality device and walks around the stage. During this process, the camera position V is calculated in real time. i =(x i ,y i , zi ), i=1,...,n; camera posture P i =(α i , β i , γ i ), i = 1, ..., n, represents the rotation angle around the x', y', z' axis of the coordinate system O'x'y'z'; the line of sight direction d i =(x i ,y i , z i ), i=1,...,n; at the same time, the locatable camera function of the head-mounted augmented reality device will be called to automatically select the time to shoot the image according to the camera position and camera posture; because a single image may cause balls to block each other, the imaging distance is too far, and the entire table surface cannot be captured. Acquiring several images from multiple angles can avoid these situations. Therefore, the system needs to automatically select several positions around the table surface for shooting, and its selection mechanism is as follows: i images have been selected at the current moment, with camera position V', camera posture P', line of sight direction d', image I'; the score at the current moment
[0113]
[0114] Where A is the straight line V'+d't1 and the plane y=H table The intersection point, t2, t3∈[-0.5, 0.5], if ε=1, I i+1 =I',V i+1 =V',P i+1 =P',d i+1 = d', the shooting ends when ζ = 1;
[0115]
[0116] Based on the above data, solve the coordinates v of all the balls on the table. k and Category C k (full color, fancy color, cue ball, black ball);
[0117] O={(v k ,C k )|k=1,...,q}
[0118] Among them, q is the total number of balls on the current table;
[0119] S2: i = 1, 2, ..., n, loop to perform the following operations:
[0120] S21: Take I i , P i , making I i Rotate γ around (W / 2, H / 2) i ', get I i';
[0121] S22: Detection using pre-trained object detection model I i 'The billiard balls visible in the image, get the ball projection set;
[0122] p={((x j ,y j ),C j )|j=1,...,m}
[0123] Where (x j ,y j ) is the position of the ball in the image, C j For categories, the detection model is optional. The present invention uses YOLOV8, and the model version can be determined according to the performance of the device;
[0124] S23: j = 1, ..., m, the following operations are performed cyclically:
[0125] S231: Calculate the horizontal angle ω of the camera's sight line in the direction of the sphere center during shooting based on the perspective relationship and camera parameters h The tangent value and vertical angle ω v The tangent value of
[0126] tanω h =x j tan(FOV h / 2) / (W / 2)
[0127] tanω v =y j tan(FOV v / 2) / (H / 2)
[0128] S232: By V i d i ,ω h ,ω v The point-to-point linear parametric equation l can be solved j , where l j The components of are x=x0+x1t, y=y0+y1t, z=z0+z1t, from l j With plane y = H table +r can be solved to get the parameter t of the straight line equation, and substitute it into l j The center of the sphere (x', y', z') can be obtained, and the sphere o can be obtained. j ;
[0129] S24: Execute according to S23 to obtain balls o1, ..., o j , merged into a ball set O i ;
[0130] S3: Execute according to S2 to obtain the ball set O1, ..., On , it is necessary to solve for O1, ..., O n The union of, that is, O in S1; if the ball is close to the camera, not blocked, and at the center of the field of view, a higher confidence will be estimated. At this time, the same ball will be imaged from different directions, and the estimation result with the highest confidence will be selected for each ball to construct a confidence score function:
[0131] S = as1 + bs2 + cs3
[0132] where a, b, c are adjustable weights that satisfy the constraint a + b + c = 1, and are preset as a = 0.2, b = 0.3, c = 0.5;
[0133]
[0134] where V is the shooting position and v is the estimated center of the ball;
[0135]
[0136] where (x, y) is the projected center of the circle of the ball;
[0137]
[0138] where p1, p2, …, p m are all ball projections in the current image, and r p is the radius of the projected p to be estimated, and the function f is used to calculate the overlapping area of the two ball projections;
[0139] S4: Solve for O. For each estimated ball object o
[0140]
[0141] At this point, the distribution of the balls on the table at this moment is obtained
[0142] O = {(v k , C k )|k = 1, ..., q}
[0143] S5: Let o be the target ball, and θ h ∈[0, 2π) be the angle between the projection of the cue on the xz plane and the x-axis; θ v ∈(0, 2π) be the angle formed by the cue and the horizontal plane; (ρ, θ) are the polar coordinates of the hitting point, ρ < R, θ ∈ [0, 2π), where R is the radius of the effective hitting area, R = 0.75r; τ is the cue speed; θ h is derived from the phenomenon of elastic collision. When a linearly moving ball hits another stationary ball of equal mass and undergoes an elastic collision, the component velocity of the moving ball in the collision direction will be transferred to the originally stationary ball. See Appendix Figure 5;
[0144]
[0145] where v o is the target sphere coordinate, v h is the bag opening coordinate, v c is the coordinate of the mother ball; use the fully connected neural network G to calculate θ at this moment v , ρ, θ, τ; the input shape of G is 16×2, representing 16 balls, each with a two-dimensional coordinate; the output shape is 4, representing θ v , ρ, θ, τ, and the middle layer includes a flattening layer, a fully connected layer 1, a fully connected layer 2, a fully connected layer 3, and an output layer; the flattening layer flattens the 16×2 input into 32×1; the fully connected layers 1, 2, and 3 contain 128, 64, and 32 neurons respectively, all using the ReLU activation function; the output layer contains 4 neurons and uses a linear activation function. The structure is shown in the attached Figure 4 ;
[0146] G performs self-supervised learning in a simulated billiard program environment, where the program can simulate the ball sliding, rolling, rotating, and colliding. The loss function guiding G’s training is:
[0147] L=δ1E1+δ2E2+δ3E3+δ4E1E4+δ5(1-E1)E5
[0148] Among them, δ1, δ2, δ3, δ4, and δ5 satisfy the constraints;
[0149] δ1+δ2+δ3+δ4=1, δ2+δ3+δ5=C
[0150] By default, δ1=0.3, δ2=0.3, δ3=0.3, δ4=0.3, δ5=0.3, C=0.7, C is the defense limit coefficient; E1 indicates whether a certain ball is pocketed, if it is pocketed, E1=1, otherwise it is 0; E2 is used to suppress the selection of abnormal speed:
[0151]
[0152] Where S is [1, 6]; E3 is used to suppress the selection of abnormal hitting areas;
[0153]
[0154] E4 indicates the feasibility of continuous hitting;
[0155] E4=1-E5
[0156] The difficulty of ball o is:
[0157]
[0158] Its φ1=0.4,φ2=0.2,φ3=0.4; the difficulty of obtaining e balls of one's own color is sorted from low to high:
[0159] (D1,...,D e )
[0160] E5 indicates the difficulty of scoring a goal;
[0161]
[0162] Where σ∈(0,1) is the attenuation coefficient, which makes the loss function pay more attention to the easiest ball to hit. The smaller the value, the more attention is paid to the easiest ball. The default value is 0.33;
[0163] S6: Positioning the holographic image; calculating the cue ball collision position where the target ball falls into the pocket;
[0164]
[0165] In v t Rendering a holographic ball of equal size and an ideal hitting direction l at the same location can provide players with hitting guidance. The straight line l is expressed as:
[0166] v t +(tcosθ h ,ttanθ v ,tsinθ h )+(cosθ h ,0,sinθ h )+ρcosθ(-sinθ h ,0,cosθ h )+ρsinθ(0,1,0)
[0167] Where t is the parameter of the straight line equation;
[0168] The straight line l is displayed in the virtual space, so that the user can have an intuitive perception of the direction of the club. In addition, the schematic diagram of the hitting point and the club speed are displayed in the field of vision in the form of a two-dimensional user interface to ensure that all key information can be captured by the user. The display effect is shown in the attached Figure 6 ;
[0169] S7: The user hits the ball according to the instruction information, thus completing one hitting process and waiting for the next hitting.
[0170] The present invention is applied to the Chinese eight-ball game, and the game equipment consists of a table, balls, and a cue. The balls include a mother ball, a black ball, seven full-color balls, and seven colored balls. In the game, two players compete with each other, and the goal of the game is to use the cue to hit all the balls of their own color into the pocket. After the opening, the colors will be divided according to the first legal goal, and the player who finally hits the black ball legally wins.
[0171] Shooting rules: In a shot, a player hits the cue ball. If the first ball that the cue ball hits is of his own suit, and at least one ball on the table bounces off the edge of the table or falls into a pocket after the collision, and the cue ball does not fall into a pocket in the end, then this shot is a legal shot; otherwise, it is a foul shot. If a ball of his own suit falls into a pocket after a legal shot, it is a legal goal, and he gets a connecting shot, and can continue to shoot. If no ball of his own suit falls into a pocket after a legal shot, it is a normal shot, and another player takes over the shot. If it is a foul shot, the other player gets a free ball and can place the cue ball at any position and shoot.
[0172] Color: The first legal goal scored by any player after the kick-off will determine the color. If a full-color ball is scored, the full-color ball belongs to that player in this game, and the other player automatically gets the suit.
[0173] Winning and losing: Players can hit the black ball only after clearing all the balls of their own color. If they hit the black ball with a legal shot, they win. If they make the black ball fall into the pocket with an illegal shot, they lose. If they make the black ball fall into the pocket when there are still balls of their own color on the table, they lose.
[0174] Every time you hit the ball, the player needs to complete the operations of selecting the target ball, aiming, determining the hitting point and hitting speed, and hitting the ball.
[0175] Select the target ball: Select the target ball to hit.
[0176] Aiming: Determine where to hit the cue ball so that the object ball can be pocketed.
[0177] Determine the hitting point and hitting speed: Different hitting points and hitting speeds will affect the speed and rotation state of the cue ball after the collision, causing the cue ball to stop in different positions.
[0178] Shot: Execute the shot according to the content determined in the above steps.
[0179] Although the present invention has been described above with reference to the embodiments, various modifications may be made thereto and parts thereof may be replaced by equivalents without departing from the scope of the present invention. In particular, as long as there is no structural conflict, the various features in the embodiments disclosed in the present invention may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.
Claims
1. A holographic billiards assisted learning system based on a head-mounted augmented reality device, characterized in that: include: Augmented reality framework; Augmented reality equipment for photographing and displaying billiard balls; A spatial modeling module, used to locate the position of the billiard ball based on the image captured by the augmented reality device; A technical and tactical analysis module, used to obtain hitting instructions based on the data of the spatial modeling module; The augmented reality framework is provided with an augmented reality framework, an augmented reality device, a space modeling module and a technical and tactical analysis module. The augmented reality device is connected to the space modeling module, the space modeling module is connected to the technical and tactical analysis module, and the technical and tactical analysis module is connected to the augmented reality device.
2. The holographic billiards assisted learning system based on a head-mounted augmented reality device according to claim 1, characterized in that: The augmented reality device comprises: Positionable camera for photographing billiard balls; A display, used to display the hitting instructions obtained by the technical and tactical analysis module; The locatable camera is connected to the space modeling module, and the display is connected to the technical and tactical analysis module.
3. The holographic billiards assisted learning system based on a head-mounted augmented reality device according to claim 2, characterized in that: The spatial modeling module includes: An image processing module, used for processing images taken by a locatable camera; An object positioning module is used to realize object positioning according to the data processed by the image processing module to obtain ball positioning information; The image processing module is connected to the locatable camera, the image processing module is connected to the object positioning module, and the object positioning module is connected to the technical and tactical analysis module.
4. The holographic billiards assisted learning system based on a head-mounted augmented reality device according to claim 3, characterized in that: The image processing module is composed of a deep learning detection model and a visual information screening algorithm.
5. The holographic billiards assisted learning system based on a head-mounted augmented reality device according to claim 3, characterized in that: The object positioning module is composed of spatial prior information and spatial parsing algorithm.
6. The holographic billiards assisted learning system based on a head-mounted augmented reality device according to claim 3, characterized in that: The technical and tactical analysis module includes: A machine learning model for learning from ball positioning information; A physical simulation module for performing simulations based on data learned by the machine learning model; The object positioning module is connected to the machine learning model, the machine learning model is connected to the physical simulation module, the machine learning model is connected to the display, and learnable parameters are provided in the machine learning model.
7. The holographic billiards assisted learning system based on a head-mounted augmented reality device according to claim 1, characterized in that: A system flow is also included, and the specific steps are as follows: S1: The player wears a head-mounted augmented reality device and walks around the stage. During this process, the camera position V is calculated in real time. i =(x i ,y i , z i ), i=1,...,n; camera posture P i =(α i , β i , γ i ), i = 1, ..., n, represents the rotation angle around the x', y', z' axis of the coordinate system O'x'y'z'; the line of sight direction d i =(x i ,y i , z i ), i = 1, ..., n; at the same time, the locatable camera function of the head-mounted augmented reality device will be called to automatically select the time to shoot the image according to the camera position and camera posture; its selection mechanism is as follows: i images have been selected at the current moment, with camera position V', camera posture P', sight direction d', image I'; the score at the current moment Where A is the straight line V'+d't1 and the plane y=H table The intersection point, t2, t3∈[-0.5, 0.5], if ε=1, I i+1 =I',V i+1 =V',P i+1 =P',d i+1 = d', the shooting ends when ζ = 1; Based on the above data, solve the coordinates v of all the balls on the table. k and Category C k ; O={(v k ,C k )|k=1,...,q} Among them, q is the total number of balls on the current table; S2: i = 1, 2, ..., n, loop to perform the following operations: S21: Take I i , P i , making I i Rotate γ around (W / 2, H / 2) i ', get I i '; S22: Detection using pre-trained object detection model I i 'The billiard balls visible in the image, get the ball projection set; p={((x j ,y j ),C j )|j=1,...,m} Where (x j ,y j ) is the position of the ball in the image, C j For categories; S23: j = 1, ..., m, the following operations are performed cyclically: S231: Calculate the horizontal angle ω of the camera's sight line in the direction of the sphere center during shooting based on the perspective relationship and camera parameters h The tangent value and vertical angle ω v The tangent value of andω h =x j and(FOV h / 2) / (W / 2) tanω v =and j tan(FOV v / 2) / (H / 2) S232: By V i d i ,ω h ,ω v The point-to-point linear parametric equation l can be solved j , where l j The components of are x=x0+x1t, y=y0+y1t, z=z0+z1t, from l j With plane y = H table +r can be solved to get the parameter t of the straight line equation, and substitute it into l j The center of the sphere (x', y', z') can be obtained, and the sphere o can be obtained. j ; S24: Execute according to S23 to obtain balls o1, ..., o j , merged into a ball set O i ; S3: Execute according to S2 to obtain the ball set O1, ..., O n , we need to solve O1, ..., O n The union of is O in S1; if the ball is closer to the camera, not blocked, and in the center of the field of view, it will be estimated to have a higher confidence. At this time, the same ball will be imaged from different directions, and the highest confidence estimation result will be selected for each ball to construct a confidence score function: S=as1+bs2+cs3 Where a, b, c are adjustable weights, satisfying the constraint condition a+b+c=1, and the preset values are a=0.2, b=0.3, c=0.5; Where V is the shooting position and v is the estimated center of the sphere; Where (x, y) is the center of the projection of the sphere; where p1, p2, …, p m For all spherical projections in the current image, r p is the radius of the estimated projection p, and the function f is used to calculate the overlapping area of the two sphere projections; S4: Solve O, for each estimated ball object o So far, we have obtained the distribution of balls on the table at this moment; O={(v k ,C k )|k=1,...,q} S5: Let o be the target ball, and θ h ∈ [0, 2π) be the angle between the projection of the cue on the xz plane and the x-axis; θ v ∈ (0, 2π) be the angle formed by the cue and the horizontal plane; (ρ, θ) are the polar coordinates of the hitting point, ρ < R, θ ∈ [0, 2π), where R is the radius of the effective hitting area, R = 0.75r; τ is the cue speed; θ h Derived from the elastic collision phenomenon, when a linearly moving ball hits another stationary ball of equal mass and undergoes an elastic collision, the component velocity of the moving ball in the collision direction will be transferred to the originally stationary ball; where v o is the target sphere coordinate, v h is the bag opening coordinate, v c is the coordinate of the mother ball; use the fully connected neural network G to calculate θ at this moment v , ρ, θ, τ; the input shape of G is 16×2, representing 16 balls, each with a two-dimensional coordinate; the output shape is 4, representing θ v , ρ, θ, τ, and the middle layer contains a flattening layer, a fully connected layer 1, a fully connected layer 2, a fully connected layer 3, and an output layer; the flattening layer flattens the 16×2 input into 32×1; the fully connected layers 1, 2, and 3 contain 128, 64, and 32 neurons respectively, all using the ReLU activation function; the output layer contains 4 neurons and uses a linear activation function; G performs self-supervised learning in a simulated billiard program environment, where the program can simulate the ball sliding, rolling, rotating, and colliding. The loss function guiding G’s training is: L=δ1E1+δ2E2+δ3E3+δ4E1E4+δ5(1-E1)E5 Among them, δ1, δ2, δ3, δ4, and δ5 satisfy the constraints; δ1+δ2+δ3+δ4=1, δ2+δ3+δ5=C By default, δ1=0.3, δ2=0.3, δ3=0.3, δ4=0.3, δ5=0.3, C=0.7, C is the defense limit coefficient; E1 indicates whether a certain ball is pocketed, if it is pocketed, E1=1, otherwise it is 0; E2 is used to suppress the selection of abnormal speed: Where S is [1, 6]; E3 is used to suppress the selection of abnormal hitting areas; E4 indicates the feasibility of continuous hitting; E4=1-E5 The difficulty of ball o is: Its φ1=0.4,φ2=0.2,φ3=0.4; the difficulty of obtaining e balls of one's own color is sorted from low to high: (D1,...,D e ) E5 indicates the difficulty of scoring a goal; Where σ∈(0,1) is the attenuation coefficient, which makes the loss function pay more attention to the easiest ball to hit. The smaller the value, the more attention is paid to the easiest ball. The default value is 0.33; S6: Positioning the holographic image; calculating the cue ball collision position where the target ball falls into the pocket; In v t Rendering a holographic ball of equal size and an ideal hitting direction l at the same location can provide players with hitting guidance. The straight line l is expressed as: v t +(tcosθ h ,ttanθ v ,tsinθ h )+(cosθ h ,0,sinθ h )+ρcosθ(-sinθ h ,0,cosθ h )+ρsinθ(0,1,0) Where t is the parameter of the straight line equation; Display the straight line l in the virtual space, so that the user can have an intuitive perception of the direction of the club. In addition, the schematic diagram of the hitting point and the club speed are displayed in the field of vision in the form of a two-dimensional user interface to ensure that all key information can be captured by the user. S7: The user hits the ball according to the instruction information, thus completing one hitting process and waiting for the next hitting.
