A billiards interaction method and device based on AR and real-time trajectory analysis
By collecting data through cameras and LiDAR, and combining it with physical models and LSTM neural networks, AR guide lines are generated, which solves the problems of low trajectory prediction accuracy and untimely feedback in billiards. It achieves high-precision trajectory prediction and multimodal interactive feedback, enhancing the immersive and interactive experience of billiards.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGDONG QIANYING INTELLIGENT LIGHTING CO LTD
- Filing Date
- 2025-12-25
- Publication Date
- 2026-06-26
AI Technical Summary
In traditional billiards, beginners often struggle to master shot-hitting techniques. Existing simulation systems fail to seamlessly blend virtual and reality, exhibiting low trajectory prediction accuracy and untimely feedback, thus failing to meet the personalized needs of different skill levels.
Real-time data collection from the pool table is achieved using cameras and LiDAR. The resulting trajectory is predicted using a physical model and an LSTM neural network, generating an AR guide line. The projection is then calibrated using LiDAR, providing multimodal interactive feedback to suit different skill levels.
It achieves high-precision trajectory prediction and multimodal interactive feedback, improving the beginner's experience and providing strategic advice for experts, thus enhancing immersion and interactive fun.
Smart Images
Figure CN121458928B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a billiards interaction method and device based on AR and real-time trajectory analysis. Background Technology
[0002] In traditional billiards, novice players often lack professional guidance and struggle to master the skills of judging the angle, power, and trajectory of the shot, resulting in a long learning curve and a rapid decline in interest. Existing billiards simulation systems mostly rely on separate screens to display virtual scenes, failing to naturally integrate virtual prompts with the real billiard table environment. Players need to switch their attention between virtual and reality, resulting in a poor sense of immersion.
[0003] Even with attempts to incorporate visual assistance, some problems remain, including low trajectory prediction accuracy (relying solely on simple physics formulas), untimely feedback (failing to provide simultaneous effect feedback at the moment of impact), and limited interaction that fails to cater to the personalized needs of players with varying skill levels. Therefore, a billiards interaction method capable of achieving high-precision trajectory prediction and multimodal interactive feedback is urgently needed. Summary of the Invention
[0004] Therefore, it is necessary to provide a billiards interaction method and device based on AR and real-time trajectory analysis that can achieve high-precision trajectory prediction and multimodal interactive feedback, addressing the aforementioned technical problems.
[0005] In a first aspect, this application provides a billiards interaction method based on AR and real-time trajectory analysis, the method comprising:
[0006] The system uses cameras and LiDAR to collect real-time image data of the entire table and spatial position data of the table surface, and performs noise reduction, format conversion and time axis alignment on the real-time collected data.
[0007] Based on the processed real-time data, combined with a preset physical model and LSTM neural network, the trajectory of the cue ball, the collision path of the target ball, and the probability of entering the hole are predicted, and the optimal shot route adapted to the player's skill level is generated.
[0008] Based on the optimal shot trajectory, an AR guide line is generated, which is then projected onto the surface of the real table tennis table. Spatial positioning calibration is performed using a lidar to adjust the projection coordinates of the AR guide line.
[0009] Based on the player's selected mode, real-time ball-hitting behavior, and ball movement results, multimodal interactive feedback is output, including novice mode, expert mode, and training mode.
[0010] In one embodiment, the step of predicting the cue ball's trajectory, the target ball's collision path, and the probability of entering the hole based on processed real-time data, combined with a preset physical model and an LSTM neural network, and generating an optimal shot route suitable for the player's skill level includes:
[0011] A physical model is established by introducing physical parameters of billiards motion, including the ball mass, table friction coefficient, and collision recovery coefficient.
[0012] Based on the processed real-time data, the trajectory of the billiard ball is simulated using the physical model.
[0013] Acquire historical shot data and environmental parameters, and input the historical shot data and environmental parameters into an LSTM neural network to obtain the deviation correction value;
[0014] Based on the mode selected by the player, and combining the deviation correction value with the simulated billiard ball trajectory, an optimal shot route suitable for the player's skill level is generated.
[0015] In one embodiment, the real-time acquisition of full-area image data of the pool table and spatial position data of the table surface via camera and LiDAR, and the processing of noise reduction, format conversion and timeline alignment of the real-time acquired data, includes:
[0016] The camera collects real-time image data of the entire table, including the three-dimensional outline of the ball, the cue stick's striking action, the initial ball position coordinates, and the outline information of the hole.
[0017] The spatial position data of the tabletop surface is collected in real time by lidar, including the coordinates of the tabletop edge, the center coordinates of the hole, and the three-dimensional coordinates of any point on the tabletop.
[0018] The data collected in real time is processed for noise reduction, format conversion, and timeline alignment.
[0019] In one embodiment, the noise reduction, format conversion, and timeline alignment processing of the real-time acquired data includes:
[0020] The Gaussian filtering algorithm and morphological operations are used to denoise the global image data of the pool table to filter out ambient light reflections, shadows and image noise.
[0021] Convert the spatial location data into Cartesian coordinate system data;
[0022] By using timestamp alignment technology, a unified timestamp is added to the global image data of the pool table and the spatial location data to achieve timeline alignment of multi-source data.
[0023] In one embodiment, generating an AR guide line based on the optimal shot trajectory, projecting the AR guide line onto the real table surface, and adjusting the projection coordinates of the AR guide line using LiDAR for spatial positioning calibration includes:
[0024] Based on the optimal shot trajectory, an AR guide line is generated, which is then transmitted to multiple projection devices. The AR guide line is then projected onto the surface of a real table tennis table by stitching together the images from the multiple projection devices.
[0025] Control the lidar to scan the surface of the pool table and collect the real spatial coordinates of a preset number of calibration points;
[0026] Calculate the deviation between the AR guide line projection coordinates and the real space coordinates, and adjust the AR guide line projection coordinates according to the deviation.
[0027] In one embodiment, the billiards interaction method based on AR and real-time trajectory analysis further includes:
[0028] Acquire the ambient light intensity and determine whether the ambient light intensity is greater than a preset light intensity threshold;
[0029] If so, increase the display brightness of the AR guide line;
[0030] If not, reduce the display brightness of the AR guide line.
[0031] In one embodiment, the step of outputting multimodal interactive feedback based on the player's selected mode, real-time ball-hitting behavior, and ball motion results includes:
[0032] Based on the mode selected by the player and the ball's movement results, corresponding differentiated visual content is generated. The ball's movement results include the cue ball not hitting the target ball, hitting the target ball but not going into the hole, and the target ball going into the hole. The visual content includes trajectory end markers, collision point light effects, and 3D score text at the hole.
[0033] By combining real-time ball-striking motions and impact forces, different types and volumes of sound effects are output. The real-time ball-striking motions include swinging the club, the target ball hitting the edge of the table, and sinking the ball.
[0034] Secondly, this application also provides a billiards interactive device based on AR and real-time trajectory analysis. The device includes:
[0035] The data processing module is used to collect real-time image data of the entire table and spatial position data of the table surface through cameras and LiDAR, and to perform noise reduction, format conversion and time axis alignment on the real-time collected data.
[0036] The shot path generation module is used to predict the cue ball's trajectory, the target ball's collision path, and the probability of going into the hole based on processed real-time data, combined with a preset physical model and LSTM neural network, and generate the optimal shot path that suits the player's skill level.
[0037] The guide line projection module is used to generate an AR guide line based on the optimal ball-hitting path, project the AR guide line onto the real table surface, and perform spatial positioning calibration using a lidar to adjust the projection coordinates of the AR guide line.
[0038] The interactive feedback output module is used to output multimodal interactive feedback based on the player's selected mode, real-time ball-hitting operation behavior, and ball movement results. The modes include novice mode, expert mode, and training mode.
[0039] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:
[0040] The system uses cameras and LiDAR to collect real-time image data of the entire table and spatial position data of the table surface, and performs noise reduction, format conversion and time axis alignment on the real-time collected data.
[0041] Based on the processed real-time data, combined with a preset physical model and LSTM neural network, the trajectory of the cue ball, the collision path of the target ball, and the probability of entering the hole are predicted, and the optimal shot route adapted to the player's skill level is generated.
[0042] Based on the optimal shot trajectory, an AR guide line is generated, which is then projected onto the surface of the real table tennis table. Spatial positioning calibration is performed using a lidar to adjust the projection coordinates of the AR guide line.
[0043] Based on the player's selected mode, real-time ball-hitting behavior, and ball movement results, multimodal interactive feedback is output, including novice mode, expert mode, and training mode.
[0044] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:
[0045] The system uses cameras and LiDAR to collect real-time image data of the entire table and spatial position data of the table surface, and performs noise reduction, format conversion and time axis alignment on the real-time collected data.
[0046] Based on the processed real-time data, combined with a preset physical model and LSTM neural network, the trajectory of the cue ball, the collision path of the target ball, and the probability of entering the hole are predicted, and the optimal shot route adapted to the player's skill level is generated.
[0047] Based on the optimal shot trajectory, an AR guide line is generated, which is then projected onto the surface of the real table tennis table. Spatial positioning calibration is performed using a lidar to adjust the projection coordinates of the AR guide line.
[0048] Based on the player's selected mode, real-time ball-hitting behavior, and ball movement results, multimodal interactive feedback is output, including novice mode, expert mode, and training mode.
[0049] In summary, this application includes the following beneficial technical effects:
[0050] Real-time data acquisition via cameras and LiDAR, followed by noise reduction, format conversion, and timeline alignment, eliminates environmental interference and data asynchrony issues, providing high-quality data support for subsequent functions. Combining a pre-set physical model and LSTM neural network, it generates optimal shot paths tailored to player skill levels, offering easy-to-implement introductory guidance for beginners and advanced strategy suggestions for experienced players, meeting diverse needs. Based on these optimal shot paths, an AR guide line is generated and spatially calibrated using LiDAR, achieving precise integration between the AR guide line and the real table, enhancing guidance effectiveness and user experience. Multimodal interactive feedback is implemented based on the player's selected mode, real-time shot actions, and ball movement results, enhancing the fun and immersion of the interaction. Attached Figure Description
[0051] Figure 1 This is a flowchart illustrating a billiards interaction method based on AR and real-time trajectory analysis in one embodiment.
[0052] Figure 2 This is a flowchart illustrating a billiards interaction method based on AR and real-time trajectory analysis in another embodiment.
[0053] Figure 3 This is a structural block diagram of a billiards interactive device based on AR and real-time trajectory analysis in one embodiment. Detailed Implementation
[0054] This invention provides a billiards interaction method and device based on AR and real-time trajectory analysis.
[0055] The embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the accompanying drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.
[0056] In the description of the embodiments disclosed in this invention, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.
[0057] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The billiards interaction method based on AR and real-time trajectory analysis in this embodiment of the invention includes:
[0058] The S100 uses a camera and LiDAR to collect real-time image data of the entire table and spatial position data of the table surface, and performs noise reduction, format conversion and time axis alignment on the real-time collected data.
[0059] Specifically, a high-definition camera covers the entire table area, acquiring real-time image data of the entire table. This image data includes the three-dimensional contour information of the balls (covering the diameter parameters, surface color characteristics, and dynamic real-time position of each ball), the club striking action (including swing trajectory, impact point, and clubhead speed), the hole contour information (including the edge shape and opening size of the hole), and the initial ball position coordinates (the placement of all balls before the tee shot). A LiDAR scanner is used to scan the table surface, acquiring spatial position data such as the table edge coordinates (the three-dimensional coordinates of the four sides of the table, determining the movement boundaries), the hole center coordinates, and the three-dimensional coordinates of any point on the table. Then, the real-time acquired image data of the entire table and the spatial position data of the table surface undergo noise reduction, format conversion, and timeline alignment to ensure data accuracy and synchronization.
[0060] In this embodiment, noise reduction, format conversion, and time axis alignment are performed on the collected data, which can effectively improve data quality, eliminate errors caused by environmental interference and data format differences, ensure the time synchronization of multi-source data, and lay a reliable foundation for subsequent trajectory prediction and AR projection.
[0061] The S200, based on processed real-time data, combines a preset physical model with an LSTM neural network to predict the trajectory of the cue ball, the collision path of the target ball, and the probability of it going into the hole, generating the optimal shot route that suits the player's skill level.
[0062] Specifically, using the processed data, combined with a billiards physics model (simulating the forces, collisions, and rebounds of a real shot) and an LSTM neural network (correcting prediction biases caused by environmental interference), the system accurately calculates the cue ball's trajectory, the target ball's path after collision, and the probability of the target ball going into the hole. Then, based on the player's chosen beginner, expert, or training mode, it generates shot routes of corresponding difficulty. Specifically, the beginner mode routes are simple and easy to operate, prioritizing the success rate of going into the hole; the expert mode routes increase strategic complexity, such as multi-ball collisions and table-edge bounces, to meet advanced needs.
[0063] In this embodiment, the integration of a billiards motion physics model and an LSTM neural network not only solves the problem of insufficient accuracy in traditional single physics calculations, but also meets the personalized needs of different players, serving as a key bridge connecting data acquisition and AR guidance.
[0064] The S300 generates an AR guide line based on the optimal shot trajectory, projects the AR guide line onto the real table surface, and uses LiDAR for spatial positioning calibration to adjust the projection coordinates of the AR guide line.
[0065] Specifically, based on the generated optimal shot path adapted to the player's skill level, the abstract trajectory data (such as the coordinates connecting the cue ball's starting point → collision point → object ball's starting point → pocket) is first transformed into a visual AR guide line. The basic form of the AR guide line is a continuous line, with prominent markers added to key nodes (such as the cue ball's striking point, the object ball's collision point, and the pocket position). Before the shot, the guide line remains statically displayed, and a trailing dynamic effect is triggered at the moment of the shot (such as the line gradually disappearing as the ball moves), enhancing immersion and avoiding visual overlap between the guide line and the moving ball. The AR guide line is projected onto the surface of the real pool table using multiple projection devices to ensure no blind spots. The coordinates of the calibration point on the real pool table are collected by LiDAR as a comparison benchmark. The projected coordinates of the AR guide line at the calibration point are compared with the real coordinates collected by LiDAR in real time, and the deviation is calculated and dynamically corrected.
[0066] In this embodiment, projection technology is used to physically integrate the AR guide line with the real table tennis table, and LiDAR is used to calibrate and eliminate deviations, ensuring that the guide line can accurately indicate the shot path, so that players do not need to switch their attention between virtual and reality and can directly complete the shot according to the guidance.
[0067] The S400 outputs multimodal interactive feedback based on the player's selected mode, real-time ball-hitting actions, and ball movement results.
[0068] Specifically, based on the player's selected mode (beginner, expert, or training mode), the current swing operation (such as swing force, whether the ball was hit), and the final movement result of the ball (such as missing the target ball, hitting but not going into the hole, or making the shot), multi-dimensional feedback is provided, combining visual (such as light effects, score text) and auditory (such as different sound effects). For example, a beginner will receive obvious light effects, cheerful sound effects, and score prompts when making a shot, while an expert will receive simpler feedback when making a shot. The training mode will also provide additional suggestions for adjusting the operation.
[0069] In this embodiment, by outputting multimodal interactive feedback, players can intuitively perceive the effect of hitting the ball, while adapting to the usage needs of different modes, avoiding the problems of single feedback and disconnection from the scene.
[0070] In one embodiment, such as Figure 2 As shown, S200 includes:
[0071] S210 introduces physical parameters of billiards motion and establishes a physical model;
[0072] S220 simulates the trajectory of a billiard ball using a physical model based on processed real-time data;
[0073] S230: Acquire historical shot data and environmental parameters, and input the historical shot data and environmental parameters into the LSTM neural network to obtain the deviation correction value;
[0074] S240 generates the optimal shot path that suits the player's skill level by combining the deviation correction value with the simulated billiard ball trajectory based on the mode selected by the player.
[0075] Specifically, this study introduces physical parameters of billiards, including key indicators such as ball mass, table friction coefficient, and collision recovery coefficient. A physical model is constructed based on classical mechanics principles to simulate the ball's rolling, collision, and rebound trajectory on the table, providing a basic computational framework for trajectory prediction. Processed real-time data is input into the constructed physical model, and mechanical equations are used to calculate the cue ball's trajectory, the collision angle with the target ball, and the target ball's subsequent path. Historical shot data (including successful shots and deviations) and environmental parameters (such as table flatness and ambient temperature) are collected and input into a trained LSTM neural network. The LSTM network learns from the deviation patterns in historical data and outputs a trajectory deviation correction value for the current scenario, compensating for errors in the physical model caused by environmental interference or parameter approximation. Based on the player's selected mode, the optimal shot route is generated according to the deviation correction value and the trajectory simulated by the physical model, adapting to the player's skill level. In beginner mode, simple routes with a high probability of pocketing and few collisions are prioritized; in advanced mode, complex routes such as multi-ball continuous collisions and high-difficulty rebounds into the pocket are available; and in training mode, targeted routes are generated based on training objectives (such as accurate shots and rebound techniques).
[0076] In one embodiment, real-time acquisition of full-area image data of the pool table and spatial position data of the table surface is performed using a camera and LiDAR, and the real-time acquired data is processed for noise reduction, format conversion, and timeline alignment, including:
[0077] Real-time image data of the entire table is acquired via camera, including the three-dimensional outline of the ball, the cue's striking action, the initial ball position coordinates, and the outline of the hole. Real-time spatial position data of the table surface is acquired via LiDAR, including the coordinates of the table edge, the center coordinates of the hole, and the three-dimensional coordinates of any point on the table. The real-time acquired data is then processed for noise reduction, format conversion, and time axis alignment.
[0078] Specifically, high-definition wide-angle cameras are used, installed at appropriate locations above or around the table to ensure complete coverage of the entire table surface without blind spots. These cameras capture real-time dynamic and static image information of the table surface, including the 3D outline of the ball, the cue stroke, the initial ball position coordinates, and the pocket outline. A LiDAR device is used to perform a comprehensive scan of the table surface, focusing on acquiring the coordinates of the table edges, the center coordinates of the pockets, and the 3D coordinates of any point on the table, ensuring that the acquired spatial location data can completely reconstruct the physical structure of the table. The acquired image and spatial location data undergo noise reduction, format conversion, and timeline alignment processing to provide high-quality data for subsequent trajectory prediction and AR calibration.
[0079] In one embodiment, noise reduction, format conversion, and timeline alignment of real-time acquired data include:
[0080] The Gaussian filtering algorithm and morphological operations are used to denoise the global image data of the pool table, filtering out ambient light reflections, shadows and image noise; the spatial location data is converted into Cartesian coordinate system data; and a unified timestamp is added to the global image data and spatial location data of the pool table through timestamp alignment technology to achieve time axis alignment of multi-source data.
[0081] Specifically, for the full-area image data of the pool table captured by the camera, a Gaussian filtering algorithm is used to filter out random noise in the image. Morphological operations (dilation and erosion) are used to eliminate image interference caused by ambient light reflection and table shadows. The processed image can clearly present the outline of the ball, the cue action, and the position of the hole, ensuring the accuracy of image recognition. The spatial position data (mostly polar or spherical coordinates) collected by the LiDAR is converted into Cartesian coordinate system data to be consistent with the preset physical coordinate system of the pool table. The converted coordinate data can be directly used to calculate the ball's trajectory and the AR guide line projection position, avoiding calculation errors caused by format incompatibility. Through timestamp alignment technology, a unified timestamp is added to each set of image data and spatial position data. Based on the timestamp, image data and spatial position data collected at the same time are associated and matched to ensure that the synchronous data of the same time dimension is used in subsequent calculations, avoiding problems such as trajectory prediction misalignment and AR guidance deviation caused by data acquisition delay.
[0082] In this embodiment, noise reduction of the entire image data of the pool table can improve image clarity; converting spatial location data into rectangular coordinates can unify the coordinate format and improve data retrieval efficiency; and timestamp alignment technology ensures the synchronization of multi-source data and avoids prediction errors caused by data problems.
[0083] In one embodiment, an AR guide line is generated based on the optimal shot trajectory, projected onto the real table surface, and spatial positioning calibration is performed using LiDAR. Adjusting the projection coordinates of the AR guide line includes:
[0084] Based on the optimal shot trajectory, an AR guide line is generated and transmitted to multiple projection devices. The AR guide line is then projected onto the surface of the real table tennis table by stitching together the images from the multiple projection devices. The LiDAR scanner is controlled to scan the surface of the table tennis table and collect the real spatial coordinates of a preset number of calibration points. The deviation between the projected coordinates of the AR guide line and the real spatial coordinates is calculated, and the projected coordinates of the AR guide line are adjusted according to the deviation.
[0085] Specifically, by combining a pre-set physical model with an LSTM neural network, an optimal shot path tailored to the player's skill level is generated. A visual AR guide line is then created using image rendering technology. The guide line style can be adjusted according to the game mode; for example, a beginner mode uses thick lines with arrow prompts, while an advanced mode uses thin lines with trajectory node markers. The AR guide line data is transmitted to multiple distributed projection devices. Through projection image stitching technology, the guide line is fully projected across the entire billiard table surface, ensuring complete coverage of the shot path without blind spots. A laser radar is controlled to perform high-frequency scanning of the table surface, selecting a pre-set number of evenly distributed calibration points, including the area around the pockets, the center of the table, and the edge areas. The real-time spatial coordinate data of these calibration points is collected, and the deviation between the projected coordinates of the AR guide line at each calibration point and the real-time spatial coordinates is calculated. Based on the deviation, the projection parameters of the projection devices are dynamically adjusted to correct the projected coordinates of the AR guide line, ensuring that the guide line perfectly matches the actual path on the real table surface.
[0086] In this embodiment, the AR guide line achieves full coverage with no projection blind spots through image stitching from multiple projection devices; the real-time calibration mechanism of the LiDAR ensures precise alignment between the guide line and the actual tabletop, allowing players to directly perform ball-hitting operations based on the guide line. The intuitiveness and accuracy of AR guidance effectively reduce the learning curve for beginners and enhance the overall interactive experience.
[0087] In one embodiment, the billiards interaction method based on AR and real-time trajectory analysis further includes:
[0088] The system acquires the ambient light intensity and determines whether it exceeds a preset light intensity threshold. If it does, the system increases the display brightness of the AR guide line; otherwise, the system decreases the display brightness of the AR guide line.
[0089] Specifically, light sensors are installed around the pool table to collect ambient light intensity data in real time at a frequency of once per second, ensuring timely detection of light changes (such as light switching on / off, changes in external light, etc.). A light intensity threshold (e.g., 800 lux) is set, based on the human visual comfort range and the characteristics of AR projection display. The real-time collected light intensity data is compared with the preset threshold to determine whether the current ambient light is in a strong light state (above the light intensity threshold) or a weak light state (below the light intensity threshold). If the ambient light intensity is greater than the light intensity threshold, the display brightness of the AR guide line is automatically increased to ensure that the guide line remains clearly visible even in strong light; if the ambient light intensity is less than the light intensity threshold, the display brightness of the AR guide line is reduced to prevent the guide line from being too glaring in dim light, protecting the player's eyesight and maintaining visual comfort.
[0090] In this embodiment, the AR guide line display brightness adaptive adjustment mechanism solves the problem of blurred guide lines under strong light and avoids visual interference under low light.
[0091] In one embodiment, the multimodal interactive feedback output, based on the player's selected mode, real-time ball-hitting behavior, and ball motion, includes:
[0092] Based on the player's selected mode and the ball's movement results, differentiated visual content is generated. The ball's movement results include the cue ball not hitting the target ball, hitting the target ball but not going into the hole, and the target ball going into the hole. The visual content includes the trajectory end mark, the collision point light effect, and the 3D score text at the hole. Combined with the real-time hitting action and the hitting force, different types and volumes of sound effects are output. The real-time hitting action includes the swing hitting the ball, the target ball hitting the edge of the table, and going into the hole.
[0093] Specifically, the system monitors the ball's movement in real time, including three core scenarios: the cue ball misses the target ball, the target ball is hit but not pocketed, and the target ball is pocketed. Based on the player's selected mode, corresponding visual content is generated. Specifically, in beginner mode, when the cue ball misses the target ball, a red cross is displayed at the end of the trajectory; when it hits but not pocketed, a blue light effect is triggered at the point of impact; when the target ball is pocketed, 3D dynamic score text and a green flashing light effect are displayed at the hole. In advanced mode, the visual feedback is simpler, with a gold light effect and a minimalist score indicator only displayed at key points (such as high-difficulty pockets). In training mode, additional trajectory deviation prompts can be displayed (such as a red dotted line indicating the difference between the actual trajectory and the suggested trajectory). The system captures real-time ball-hitting actions using sound sensors or ball-hitting motion recognition modules, including swinging the club, the target ball hitting the edge of the table, and sinking the ball. Specifically, when swinging the club, it outputs a "bang" sound of varying loudness depending on the force of the swing, with louder sounds indicating greater force. When the target ball hits the edge of the table, it outputs a crisp "clang" sound to enhance the realism of the collision. When sinking the ball, it outputs a cheerful "swish" sound and a score notification tone.
[0094] In this embodiment, by combining visual and auditory feedback with player mode, operation behavior, and ball movement results, a differentiated and real-time feedback experience is provided, enhancing the sense of immersion.
[0095] In one embodiment, such as Figure 3 As shown, an AR-based billiards interactive device with real-time trajectory analysis is provided, including: a data processing module 10, a shot trajectory generation module 20, a guide line projection module 30, and an interactive feedback output module 40, wherein:
[0096] The data processing module 10 is used to collect real-time image data of the entire table and spatial position data of the table surface through the camera and LiDAR, and to perform noise reduction, format conversion and time axis alignment processing on the real-time collected data.
[0097] The shot path generation module 20 is used to predict the trajectory of the cue ball, the collision path of the target ball, and the probability of entering the hole based on the processed real-time data, combined with a preset physical model and LSTM neural network, and generate the optimal shot path that suits the player's skill level.
[0098] The guide line projection module 30 is used to generate an AR guide line based on the optimal shot path, project the AR guide line onto the real table surface, and perform spatial positioning calibration using a lidar to adjust the projection coordinates of the AR guide line.
[0099] The interactive feedback output module 40 is used to output multimodal interactive feedback based on the player's selected mode, real-time ball-hitting operation behavior, and ball movement results. The modes include novice mode, expert mode, and training mode.
[0100] In one embodiment, the ball trajectory generation module 20 is further used to introduce billiards motion physical parameters and establish a physical model. The billiards motion physical parameters include the ball mass, table friction coefficient, and collision recovery coefficient. Based on the processed real-time data, the physical model is used to simulate the billiards trajectory. Historical ball hitting data and environmental parameters are acquired and input into an LSTM neural network to obtain a deviation correction value. According to the mode selected by the player, the deviation correction value is combined with the simulated billiards trajectory to generate an optimal ball hitting route that suits the player's skill level.
[0101] In one embodiment, the data processing module 10 is also used to acquire real-time global image data of the pool table via a camera, including the three-dimensional outline of the ball, the cue stick's striking action, the initial ball position coordinates, and the hole outline information; to acquire real-time spatial position data of the pool table surface via a lidar, including the table edge coordinates, the hole center coordinates, and the three-dimensional coordinates of any point on the table; and to perform noise reduction, format conversion, and time axis alignment processing on the real-time acquired data.
[0102] In one embodiment, the data processing module 10 is further used to perform noise reduction processing on the global image data of the pool table using a Gaussian filtering algorithm and morphological operations to filter ambient light reflections, shadows and image noise; convert the spatial location data into Cartesian coordinate system data; and add a unified timestamp to the global image data and spatial location data of the pool table through timestamp alignment technology to achieve time axis alignment of multi-source data.
[0103] In one embodiment, the guide line projection module 30 is further configured to generate an AR guide line based on the optimal shot trajectory, transmit the AR guide line to multiple projection devices, and project the AR guide line onto the real table surface by splicing the images of the multiple projection devices; control the lidar to scan the table surface and collect the real spatial coordinates of a preset number of calibration points; calculate the deviation between the AR guide line projection coordinates and the real spatial coordinates, and adjust the AR guide line projection coordinates according to the deviation.
[0104] In one embodiment, the billiards interactive device based on AR and real-time trajectory analysis further includes a light intensity judgment module, which is used to obtain the ambient light intensity and determine whether the ambient light intensity is greater than a preset light intensity threshold; if so, the display brightness of the AR guide line is increased; if not, the display brightness of the AR guide line is decreased.
[0105] In one embodiment, the interactive feedback output module 40 is also used to generate differentiated visual content according to the mode selected by the player and the ball movement results, wherein the ball movement results include the cue ball not hitting the target ball, hitting the target ball but not going into the hole, and the target ball going into the hole, and the visual content includes trajectory end markers, collision point light effects, and 3D score text at the hole; combined with real-time hitting actions and hitting force, it outputs sound effects of different types and volumes, wherein the real-time hitting actions include swinging the club to hit the ball, the target ball hitting the edge of the table, and going into the hole.
[0106] In one embodiment, this application discloses a computer device including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor loads the computer program, it executes a billiards interaction method based on AR and real-time trajectory analysis as described in the above embodiment.
[0107] In one embodiment, this application discloses a computer-readable storage medium storing a computer program, wherein when the computer program is loaded by a processor, it executes a billiards interaction method based on AR and real-time trajectory analysis as described above.
[0108] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.
Claims
1. A billiards interaction method based on AR and real-time trajectory analysis, characterized in that, include: The system uses cameras and LiDAR to collect real-time image data of the entire table and spatial position data of the table surface, and performs noise reduction, format conversion and time axis alignment on the real-time collected data. Based on the processed real-time data, combined with a preset physical model and LSTM neural network, the trajectory of the cue ball, the collision path of the target ball, and the probability of entering the hole are predicted, and the optimal shot route adapted to the player's skill level is generated. Based on the optimal shot trajectory, an AR guide line is generated, which is then projected onto the surface of the real table tennis table. Spatial positioning calibration is performed using a lidar to adjust the projection coordinates of the AR guide line. Based on the player's selected mode, real-time ball-hitting behavior, and ball movement results, multimodal interactive feedback is output, including novice mode, expert mode, and training mode; Based on the processed real-time data, combined with a preset physical model and LSTM neural network, the system predicts the trajectory of the cue ball, the collision path of the target ball, and the probability of it going into the hole, generating an optimal shot route that suits the player's skill level, including: A physical model is established by introducing physical parameters of billiards motion, including the ball mass, table friction coefficient, and collision recovery coefficient. Based on the processed real-time data, the trajectory of the billiard ball is simulated using the physical model. Acquire historical shot data and environmental parameters, and input the historical shot data and environmental parameters into an LSTM neural network to obtain the deviation correction value; Based on the mode selected by the player, and combining the deviation correction value with the simulated billiard ball trajectory, an optimal shot route suitable for the player's skill level is generated. The process of generating an AR guide line based on the optimal shot trajectory, projecting the AR guide line onto the real table surface, and adjusting the projection coordinates of the AR guide line using LiDAR for spatial positioning calibration includes: Based on the optimal shot trajectory, an AR guide line is generated, which is then transmitted to multiple projection devices. The AR guide line is then projected onto the surface of a real table tennis table by stitching together the images from the multiple projection devices. Control the lidar to scan the surface of the pool table and collect the real spatial coordinates of a preset number of calibration points; Calculate the deviation between the AR guide line projection coordinates and the real space coordinates, and adjust the AR guide line projection coordinates according to the deviation.
2. The billiards interaction method based on AR and real-time trajectory analysis according to claim 1, characterized in that, The process of acquiring real-time image data of the entire table and spatial position data of the table surface through cameras and LiDAR, and performing noise reduction, format conversion, and timeline alignment on the real-time acquired data includes: The camera collects real-time image data of the entire table, including the three-dimensional outline of the ball, the cue stick's striking action, the initial ball position coordinates, and the outline information of the hole. The spatial position data of the tabletop surface is collected in real time by lidar, including the coordinates of the tabletop edge, the center coordinates of the hole, and the three-dimensional coordinates of any point on the tabletop. The data collected in real time is processed for noise reduction, format conversion, and timeline alignment.
3. The billiards interaction method based on AR and real-time trajectory analysis according to claim 2, characterized in that, The noise reduction, format conversion, and timeline alignment processing of the real-time acquired data includes: The Gaussian filtering algorithm and morphological operations are used to denoise the global image data of the pool table to filter out ambient light reflections, shadows and image noise. Convert the spatial location data into Cartesian coordinate system data; By using timestamp alignment technology, a unified timestamp is added to the global image data of the pool table and the spatial location data to achieve timeline alignment of multi-source data.
4. The billiards interaction method based on AR and real-time trajectory analysis according to claim 1, characterized in that, Also includes: Acquire the ambient light intensity and determine whether the ambient light intensity is greater than a preset light intensity threshold; If so, increase the display brightness of the AR guide line; If not, reduce the display brightness of the AR guide line.
5. The billiards interaction method based on AR and real-time trajectory analysis according to claim 1, characterized in that, The multimodal interactive feedback output based on the player's selected mode, real-time ball-hitting behavior, and ball motion results includes: Based on the mode selected by the player and the ball's movement results, corresponding differentiated visual content is generated. The ball's movement results include the cue ball not hitting the target ball, hitting the target ball but not going into the hole, and the target ball going into the hole. The visual content includes trajectory end markers, collision point light effects, and 3D score text at the hole. By combining real-time ball-striking motions and impact forces, different types and volumes of sound effects are output. The real-time ball-striking motions include swinging the club, the target ball hitting the edge of the table, and sinking the ball.
6. A billiards interactive device based on AR and real-time trajectory analysis, characterized in that, include: The data processing module is used to collect real-time image data of the entire table and spatial position data of the table surface through cameras and LiDAR, and to perform noise reduction, format conversion and time axis alignment on the real-time collected data. The shot path generation module is used to predict the cue ball's trajectory, the target ball's collision path, and the probability of going into the hole based on processed real-time data, combined with a preset physical model and LSTM neural network, and generate the optimal shot path that suits the player's skill level. The guide line projection module is used to generate an AR guide line based on the optimal ball-hitting path, project the AR guide line onto the real table surface, and perform spatial positioning calibration using a lidar to adjust the projection coordinates of the AR guide line. The interactive feedback output module is used to output multimodal interactive feedback based on the player's selected mode, real-time ball-hitting operation behavior, and ball movement results. The modes include novice mode, expert mode, and training mode. The ball trajectory generation module is also used to introduce billiards motion physical parameters and establish a physical model, including ball mass, table friction coefficient, and collision recovery coefficient; based on processed real-time data, the physical model is used to simulate the billiards trajectory; historical ball hitting data and environmental parameters are acquired and input into an LSTM neural network to obtain a deviation correction value; according to the mode selected by the player, the deviation correction value is combined with the simulated billiards trajectory to generate an optimal ball hitting route that suits the player's skill level; The guide line projection module is also used to generate an AR guide line based on the optimal shot trajectory, transmit the AR guide line to multiple projection devices, and project the AR guide line onto the real table surface by stitching together the images from the multiple projection devices; control the lidar to scan the table surface and collect the real spatial coordinates of a preset number of calibration points; calculate the deviation between the AR guide line projection coordinates and the real spatial coordinates, and adjust the AR guide line projection coordinates according to the deviation.
7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.