A projection billiards teaching and auxiliary system, method and electronic device

Abstract

The application relates to a projection type billiards teaching and auxiliary system, method and electronic equipment. The system comprises a data acquisition module, a processing module and a projection module. The data acquisition module comprises a camera and an inertial measurement unit, and is used for acquiring visual data of a table and motion sensing data of a billiards cue respectively. The processing module is used for image preprocessing and target detection processing of the visual data, and identifies each ball body on the billiards table. A multi-target tracking algorithm based on a Kalman filter is used to create and maintain a tracking object for each identified ball body, and a data association algorithm is used to update the state of the tracking object. Based on the state of the tracking object corresponding to each ball body, a aiming point of a white ball hitting a target ball to a specified ball pocket is calculated through geometric analysis, and the feasibility of a hitting path is evaluated through a collision detection algorithm to generate a hitting strategy. The projection module is used for projecting and displaying the hitting strategy in a graphic form on the billiards table. The system improves the billiards teaching and auxiliary effect.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a projection-based billiards teaching and assistance system, method, and electronic device. Background Technology

[0002] Billiards, as a precise and skill-based sport, requires precise spatial awareness, geometric analysis, power control, and a stable cueing motion. However, learning and improving billiards skills typically relies on long-term, repetitive practice to develop muscle memory.

[0003] Currently, billiards teaching and assistance mainly exist in the following modes: (1) Traditional manual teaching mode: relying on on-site guidance from professional assistants. Although this method can provide personalized feedback, it suffers from problems such as a scarcity of high-quality coaching resources, high costs, and strong time and geographical limitations, making it difficult to popularize to a wide range of billiards enthusiasts. (2) Video teaching mode: billiards enthusiasts learn by watching teaching videos. Although this method is low-cost and easy to access, it lacks interactivity and real-time feedback. Learners cannot know the specific deviation between their own hitting action and the ideal posture, resulting in limited teaching effectiveness.

[0004] Currently, no effective solution has been proposed to address the significant limitations of existing billiards teaching and assistance solutions, which result in poor teaching and assistance effects, low levels of intelligence, and narrow applicability. Summary of the Invention

[0005] The present invention provides a projection-based billiards teaching and assistance system, method, and electronic device, which at least solves the problems of poor billiards teaching and assistance effects, low level of intelligence, and narrow applicable scenarios in related technologies.

[0006] To address the aforementioned problems, one aspect of this invention provides a projection-based billiards teaching and assistance system, comprising: The data acquisition module includes a camera mounted above the billiard table and an inertial measurement unit mounted on the billiard cue, which are used to acquire visual data of the billiard table and motion sensing data of the billiard cue, respectively. The processing module, communicatively connected to the data acquisition module, is used for: performing image preprocessing and target detection processing on the visual data to identify each ball on the billiard table; using a Kalman filter-based multi-target tracking algorithm to create and maintain a tracking object for each identified ball, and updating the state of the tracking object through a data association algorithm, the state including position, velocity, and motion state managed by a state machine; based on the state of the tracking object corresponding to each ball, calculating the aiming point of the cue ball hitting the target ball to the designated pocket through geometric analysis, and evaluating the feasibility of the hitting path through a collision detection algorithm to generate a hitting strategy; A projection module, communicatively connected to the processing module, is used to project the ball-striking strategy in graphical form onto the billiard table; wherein the graphical form includes aiming lines representing the path of the cue ball to the aiming point.

[0007] In some embodiments, the system further includes a user terminal; wherein, The processing module is further configured to: calculate the user's cueing action parameters based on the motion sensing data; compare the cueing action parameters with the ball-striking strategy, generate action correction feedback information, and send the action correction feedback information to the user terminal for display, so as to provide ball-striking action correction guidance.

[0008] In some embodiments, the processing module is further configured to: perform image preprocessing on the visual data to generate a standard top view of the billiard table; wherein the image preprocessing includes lens distortion correction processing and perspective transformation correction processing.

[0009] In some embodiments, the processing module is further configured to: employ a YOLO deep learning model with a lightweight network structure to perform target detection processing on the standard top view generated after image preprocessing, so as to output the category, position, and confidence information of the ball, bag, and club.

[0010] In some of these embodiments, the YOLO deep learning model is deployed on a neural network inference engine.

[0011] In some embodiments, the processing module is further configured to: group the spheres according to their semantic categories when executing the data association algorithm, and use a nearest neighbor matching strategy to associate the target detection results of the current frame with the existing tracked objects to update the state of the tracked objects; wherein the state of the tracked objects switches between a stationary state, a moving state, and a lost state based on the association results and preset conditions.

[0012] In some embodiments, the processing module is further configured to: when generating the hitting strategy, predict and filter one or more feasible hitting paths from the cue ball to the aiming point without any other ball obstructing the path based on the collision detection algorithm; wherein the collision detection algorithm determines whether a ball collision will occur on the path based on the prediction of the cue ball's trajectory.

[0013] In some embodiments, the data acquisition module, the processing module, and the projection module are all integrated into a local edge computing device, enabling the system to operate independently in an offline environment.

[0014] To address the aforementioned problems, one aspect of this invention provides a projection-based billiards teaching and assistance method, applied to any of the projection-based billiards teaching and assistance systems described above, the method comprising: Visual data of the billiard table is collected by a camera, and motion sensing data of the billiard cue is collected by an inertial measurement unit. The visual data undergoes image preprocessing and target detection to identify the balls on the billiard table. A multi-target tracking algorithm based on a Kalman filter is used to create and maintain a tracking object for each identified ball. The state of the tracking object is updated using a data association algorithm, and the state includes position, velocity, and motion state managed by a state machine. Based on the state of the tracking object corresponding to each ball, geometric analysis is used to calculate the aiming point for the cue ball to strike the target ball to the designated pocket. A collision detection algorithm is then used to evaluate the feasibility of the striking path to generate a striking strategy. The ball-striking strategy is projected onto the billiard table in a graphical form, wherein the graphical form includes aiming lines representing the path of the cue ball to the aiming point.

[0015] To address the aforementioned problems, one aspect of this invention provides an electronic device, comprising: a processor and a memory storing a program, characterized in that the program includes instructions that, when executed by the processor, cause the processor to perform the aforementioned projection-based billiards teaching and assistance method.

[0016] The beneficial effects of the embodiments of the present invention are as follows: The projection-type billiards teaching and assistance system provided in this application integrates a data acquisition module, a processing module, and a projection module. The data acquisition module utilizes visual data provided by a camera to ensure the objectivity of the situation analysis and uses an inertial measurement unit to accurately capture and calculate the parameters of the cueing action. This allows the system to quantitatively compare the user's actual operation with the ideal strategy, thereby generating specific action correction feedback. This fundamentally changes the limitations of video teaching ("only watching, no practice") and traditional assistance systems ("only guiding aiming, no correcting action"), extending the teaching effect to the practical correction level, improving the intelligence level of the billiards teaching and assistance scheme, and significantly enhancing the billiards teaching and assistance effect. Finally, the projection module directly projects the graphical hitting strategy onto the real billiards table, creating a seamless, immersive assistance experience and expanding the system's applicable scenarios.

[0017] Details of one or more embodiments of the present invention are set forth in the following drawings and description, so that other features, objects and advantages of the invention will be more readily understood. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the framework of a projection-type billiards teaching and auxiliary system according to one embodiment of the present invention; Figure 2 This is a flowchart illustrating a projection-based billiards teaching and assistance method according to one embodiment of the present invention. Figure 3 This is a schematic diagram of the electronic device of the present invention. Detailed Implementation

[0020] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While some embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the invention. It should be understood that the accompanying drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the invention.

[0021] The rapid development of computer vision, artificial intelligence, and projection display technologies has made it possible to apply these advanced computer technologies to billiards teaching and training assistance, but significant technical obstacles still exist.

[0022] Deep learning-based object detection technology has made significant progress in recent years, with the YOLO (You Only Look Once) series of algorithms demonstrating excellent performance in real-time object detection. Specifically in the field of sports analysis, research has applied the YOLO algorithm to player tracking and tactical analysis in sports such as football and basketball. However, these technologies often employ traditional image processing algorithms or basic object detection models, focusing on large-scale motion scenes. When faced with challenges such as low accuracy in detecting small objects, sensitivity to lighting changes, and easy confusion between similar balls in a billiards scenario, the detection results can become unstable, leading to missed detections, false detections, and detection box jitter, severely impacting the accuracy of subsequent analysis.

[0023] Multi-target tracking algorithms are mainly classified into two categories: detection-based tracking and filter-based tracking. The Kalman filter, as a classic state estimation algorithm, is widely used for predicting target position and velocity. However, in billiards teaching scenarios, when the ball collides or is briefly obscured, existing tracking algorithms are prone to frequent switching of target identifiers or breakage of tracking trajectories, failing to continuously and stably track the motion state of each ball and struggling to support complex situation analysis and decision-making.

[0024] Furthermore, it is understandable that even if the aforementioned technical obstacles are overcome and the computer-related technologies are applied to billiards teaching and assistance scenarios, it will only be possible to indirectly infer the striking action by analyzing visual changes on the table. It will be impossible to directly and accurately capture key parameters during the user's stroke, such as the straightness, angle, speed, and stability of the stroke. Therefore, while improving the intelligence level of billiards teaching and assistance solutions, it is also necessary to focus on how to quantitatively evaluate and professionally correct the striking posture and action quality of billiards enthusiasts to improve the effectiveness of billiards teaching and assistance.

[0025] To address the aforementioned problems, embodiments of the present invention provide a projection-based billiards teaching and auxiliary system, such as... Figure 1 As shown, the projection-based billiards teaching and auxiliary system 100 mainly includes: The data acquisition module 110 includes a camera mounted above the billiard table and an inertial measurement unit mounted on the billiard cue, which are used to collect visual data of the billiard table and motion sensing data of the billiard cue, respectively. The processing module 120, which is communicatively connected to the data acquisition module, is used for: performing image preprocessing and target detection processing on visual data to identify each ball on the pool table; using a multi-target tracking algorithm based on a Kalman filter to create and maintain a tracking object for each identified ball, and updating the state of the tracking object through a data association algorithm, including position, velocity, and motion state managed by a state machine; based on the state of the tracking object corresponding to each ball, calculating the aiming point of the cue ball hitting the target ball to the designated pocket through geometric analysis, and evaluating the feasibility of the hitting path through a collision detection algorithm to generate a hitting strategy; The projection module 130 is communicatively connected to the processing module and is used to project the ball-striking strategy in a graphical form onto the billiard table; wherein the graphical form includes aiming lines representing the path of the cue ball to the aiming point.

[0026] Based on the above setup, the data acquisition module provides reliable data input, the processing module realizes intelligent transformation from data to decision-making, and finally the projection module seamlessly integrates the decision results into the user's operating environment. This fundamentally solves the limitations of traditional billiards assistance solutions that "only guide aiming but do not correct movements" and video tutorials that "lack real-time feedback," improving the intelligence, teaching effectiveness, and user experience of billiards teaching and assistance systems, while also expanding the system's applicable scenarios.

[0027] Specifically, the data acquisition module 110 works in conjunction with a camera and an inertial measurement unit (IMU). The camera, mounted above the billiard table, continuously acquires images of the table surface at high resolution (e.g., 3840×2160). The visual data provides global and objective information about the table situation, ensuring objective identification of the ball's position and type, and avoiding errors from subjective human judgment. The IMU is directly integrated into the cue stick, acquiring motion sensor data such as acceleration and angular velocity at high frequency (e.g., 100Hz). This enables high-precision capture of the cue stick's motion parameters (such as angle, speed, and stability), accurately capturing subtle changes in the user's movements during the cueing process—something difficult to achieve with purely visual methods. Through the collaborative work of both, multimodal data acquisition is achieved, significantly improving the system's ability to perceive the billiard table situation and the user's striking motion.

[0028] The processing module 120 employs a closed-loop processing flow encompassing image preprocessing, target detection, multi-target tracking, and decision reasoning. Specifically, image preprocessing (such as distortion correction and perspective transformation) converts the original image into a standard top-down view, eliminating errors caused by camera perspective and lens distortion, thus providing geometric consistency for subsequent detection. Target detection utilizes a lightweight YOLO model optimized for small targets on billiard balls, accelerated by an RKNN inference engine, ensuring both accuracy and real-time performance. Multi-target tracking, based on Kalman filters and state machines (such as STATIC, MOVING, and LOST), continuously updates the ball's state through data association (such as nearest neighbor matching), effectively addressing complex scenarios like occlusion and collisions, and preventing switching of tracked object identifiers and tracking breaks. Geometric analysis and collision detection, based on billiard ball physics principles (such as calculating the angle of entry into the pocket and predicting collision time), combined with path scoring functions and constraints, dynamically select the optimal striking strategy, ensuring the feasibility and scientific validity of the recommendations.

[0029] According to a specific embodiment of this application, the processing module 120 is further configured to: after identifying and tracking all balls, introduce a situation analysis algorithm to construct a global situation map of the current table, the situation analysis algorithm marking all other balls except the current target ball as potential next target balls; and calculate the corresponding basic shot difficulty value for each potential next target ball; after determining a feasible shot path (cue ball → aiming point → target ball into the pocket) through geometric analysis and collision detection, simulate the trajectory of the cue ball after colliding with the target ball based on the billiards physics model to obtain the position prediction result of the next target ball; and use the situation analysis algorithm to fuse the cue ball position area prediction result with the position prediction result corresponding to the next target ball to determine one or more recommended shot strategies and corresponding shot position path predictions. Based on the above settings, the system not only provides a solution for the current shot into the pocket, but also makes a tactical prediction step, guiding the user to consciously control the cue ball to a position conducive to continuous attacks, thereby greatly improving the professionalism and teaching depth of the auxiliary system.

[0030] The aforementioned situation analysis algorithm can employ a layered hybrid algorithm layout strategy to balance performance and real-time performance. This can include an implementation layer (aimed at rapid response), which can use a heuristic evaluation function (based on geometry and rules) combined with search algorithms (such as greedy algorithms and dynamic programming algorithms) to quickly calculate the hitting path of different target balls; and an analysis layer (deep analysis), which can use algorithms such as Monte Carlo tree search or call a lightweight neural network model pre-trained on a server to evaluate the situation, assess the appropriate position of the cue ball after hitting the ball, and generate recommended movement suggestions.

[0031] Specifically, in the process of calculating the basic shot difficulty value for the next target ball, the situation analysis algorithm needs to comprehensively consider the geometric difficulty of the shot (calculating the range of available pocket angles from the target ball to each pocket; the wider the range, the lower the difficulty), obstacles (checking whether there are other balls blocking the path from the target ball to each pocket), and positional value (according to billiards rules (such as Chinese eight-ball), certain balls (such as the more difficult black ball or key colored ball) may be assigned different weights). Taking all these factors into account, the situation analysis algorithm uses balls with a shot difficulty below a set threshold to select as the recommended set of next target balls.

[0032] The steps for determining the predicted cue ball position area include: after determining a feasible shot path (cue ball → aiming point → target ball pocket) through geometric analysis and collision detection, a situation analysis algorithm is used based on a billiards physics model to simulate the trajectory of the cue ball after colliding with the target ball. Key parameters in the simulation include: the point of impact (the position where the cue ball strikes the target ball (e.g., high shot, low shot, spin)), collision dynamics (calculating the magnitude and direction of the cue ball's velocity after the collision based on momentum conservation and energy loss models), and table friction and buffering effect (simulating the rolling friction of the cue ball on the table and its rebound after colliding with the table edge). According to a specific implementation, through Monte Carlo simulation or parametric sampling, the situation analysis algorithm generates a probability distribution area of ​​the possible stopping positions of the cue ball after the collision, i.e., the predicted cue ball position area.

[0033] The steps for generating the position prediction result corresponding to the next target ball, i.e. the position suggestion range, specifically include: using a situation analysis algorithm to fuse and analyze the cue ball position area prediction result with the position prediction result corresponding to the next target ball. The core logic is: in the predicted cue ball possible stopping area, find those positions that can provide better or at least feasible next shot conditions for hitting the balls in the aforementioned determined recommended set of next target balls. The rules for generating the suggested positioning range may include: for each recommended next target ball, calculating an "ideal positioning area" around it, which is usually located in a position where the angle between the cue ball, the target ball, and the target pocket is moderate, the distance is relatively short, and the path is unobstructed when the next shot is made; comparing the predicted cue ball stopping probability distribution area with each "ideal positioning area"; selecting areas where the two overlap, and scoring them according to the degree of overlap and the expected difficulty of the next shot; finally, projecting the one or more overlapping areas with the highest scores onto the billiard table surface on the projection module 130 in the form of semi-transparent color blocks, contour lines, or specific boundary graphics, as a visual output of the suggested positioning range; at the same time, the corresponding recommended next target ball number can also be displayed.

[0034] The projection module 130 projects the ball-striking strategy generated by the processing module 120 directly onto the billiard table in graphical form (such as aiming lines). Specifically, the projection display can accurately superimpose virtual aiming lines onto the real billiard table surface through high-precision coordinate transformation (the billiard table coordinate system is transformed into the projector coordinate system), forming an augmented reality effect. At the same time, the graphical display (including dashed aiming lines, target ball markings, etc.) is more in line with the laws of human visual cognition, lowers the user's understanding threshold, and enables them to quickly focus on key ball-striking elements.

[0035] According to a specific implementation of the embodiments of this application, a specific module design scheme for a projection-type billiards teaching and auxiliary system is provided. By constructing an intelligent auxiliary system integrating visual detection, motion sensing, state tracking and decision projection, it mainly includes the following core components: (1) Multimodal data acquisition layer: including high-resolution camera module and IMU motion sensor; (2) Visual perception processing layer: including YOLO target detection module, image preprocessing module and coordinate transformation module; (3) State tracking and fusion layer: including Kalman filter module, multi-target tracking module and state machine management module; (4) Decision reasoning layer: including geometric algorithm module, collision detection module and optimal path calculation module; (5) Projection feedback layer: real-time projection display module and user interaction interface.

[0036] Furthermore, to facilitate the promotion and application of the system, the projection-type billiards teaching and auxiliary system in this specific embodiment adopts an architecture design in which a single terminal device performs all computing tasks, satisfying the following settings: an edge computing device equipped with an ARM processor; all algorithm modules are executed locally on the terminal to ensure real-time performance and offline availability; the system operates completely autonomously and does not rely on the cloud or remote servers.

[0037] According to another specific embodiment of this application, a visual data acquisition example is provided: using a 3840×2160 high-resolution USB camera, continuous acquisition of table image data (BGR color image, 8-bit depth) is performed at 30fps, and then the image data is transmitted via a data transmission path of camera → USB interface → memory buffer. A motion sensing data acquisition example is also provided: using a Bluetooth IMU sensor, the acquisition parameters are set to three-axis data from accelerometer, gyroscope, and magnetometer, and motion sensing data of a billiard cue is acquired at a high frequency of 100Hz, and then the motion sensing data is transmitted via a data transmission path of IMU sensor → Bluetooth → asynchronous data queue.

[0038] In some embodiments, the system further includes a user terminal; wherein the processing module is further configured to: calculate the user's cueing action parameters based on motion sensing data; compare the cueing action parameters with the hitting strategy, generate action correction feedback information, and send the action correction feedback information to the user terminal for display, so as to provide hitting action correction guidance.

[0039] The user terminal can be a standalone app running on a smartphone or other mobile device, or it can be a mini-program.

[0040] Based on the above setup, the Inertial Measurement Unit (IMU) is used as a direct motion measurement tool. An intelligent comparison mechanism between motion parameters and hitting strategies is established, and a user terminal serves as a dedicated feedback interface. Users can instantly adjust their hitting posture and movements based on the correction information displayed on the terminal, thus forming an effective learning cycle and accelerating the formation of muscle memory and correct movements. This successfully extends the system's capabilities to provide professional-level motion coaching services. It overcomes the shortcomings of traditional video teaching systems in providing motion correction, achieving faster and more effective improvement of users' hitting skills through quantified data and closed-loop feedback.

[0041] According to an embodiment of this application, an inertial measurement unit (IMU) is directly mounted on the billiard cue, enabling direct measurement of the cue's linear acceleration and angular velocity in three-dimensional space at high frequencies (e.g., 100Hz). By processing this raw data using sensor fusion algorithms (e.g., complementary filtering, Kalman filtering), key motion parameters such as the cue's straightness, pitch angle, yaw angle, speed, and stability can be calculated with high precision. This solution cannot be achieved solely through observation using a top-mounted camera, as visual methods are susceptible to occlusion and struggle to quantify minute three-dimensional motion deviations. Therefore, the IMU provides a direct and quantitative motion data source, forming the basis for achieving precise motion correction.

[0042] Meanwhile, the aforementioned processing module does not analyze motion data in isolation during execution; instead, it intelligently compares it with the currently generated hitting strategy. For example, if the hitting strategy requires hitting the cue ball at a specific point with a specific angle and speed, the processing module compares the actual cueing direction calculated by the IMU with the ideal cueing direction required by the hitting strategy. If there is a deviation, it generates feedback such as the cueing deviating 3 degrees to the left. Simultaneously, it compares the actual cueing speed with the ideal force suggested by the strategy, generating feedback such as the hitting force being too strong or insufficient. This target-based comparison mechanism ensures that the generated feedback information is specific and clearly directional, thereby guaranteeing the effectiveness of the correction guidance.

[0043] Furthermore, the processing module can directly send motion correction information to independent user terminals (such as mobile phones and tablets) for display, allowing users to focus on aiming and striking the projected ball on the main visual (billiard table), maintaining immersion. After striking the ball, users can view a detailed motion analysis report on the terminal, thus avoiding information overload during the striking process and optimizing the interaction flow. Moreover, the terminal screen provides ample space to display detailed comparative analysis results, historical trends, and improvement suggestions for various motion parameters in the form of text, charts, and even animations, making the feedback information more comprehensive and easier to understand.

[0044] According to a specific embodiment of this application, taking a WeChat mini-program running on a smartphone or other mobile device as an example, this mini-program serves as the core software interface for users to interact with, control, and manage the aforementioned projection-based billiards teaching and auxiliary system. It integrates multiple advanced functions such as device connection, function control, mode settings, and data analysis. Exemplarily, the core functional modules and interaction flow of this mini-program may include: (1) Device connection and intelligent billing management process: When the user uses it for the first time, they can scan the QR code on the edge computing device through the mini program or search for the device's Bluetooth / Wi-Fi name in the mini program to complete the pairing and binding of the device; with the mini program as the control center, the user can turn the projection auxiliary system on or off with one click; in order to achieve commercial operation, the mini program can integrate an intelligent billing module, and the user can choose to bill by time (such as hour) or by number of sessions. The fee is completed through the payment interface built into the mini program. The usage status of the device (start / end) and the billing time are synchronized by the mini program and the edge computing device to ensure accurate billing.

[0045] (2) Free configuration and switching process of projection assistance function: The mini-program provides a clear function control panel. Users can independently turn on or off various auxiliary information projected by the projection module 130 according to their own training needs to achieve personalized training. Further, it may include: ball trajectory recommendation: switch control whether to project the path line of the cue ball to the aiming point; ball force and position recommendation: switch control whether to project the suggested ball force (such as represented by line thickness or color gradient) and the recommended hitting point position indicated on the cue ball; movement range recommendation: switch control whether to project the suggested ideal landing area of ​​the cue ball generated according to the situation analysis (such as a semi-transparent color block). At the same time, the status of all function switches is synchronized to the processing module 120 in real time, and the processing module adjusts the graphic content output to the projection module 130 accordingly.

[0046] (3) Multi-mode game setting and management process: The mini-program provides a rich game mode selection interface. Users can preset or select different game or training modes in real time, such as: Standard game mode: such as "Chinese eight-ball", "American nine-ball", etc. After the user selects, the system will automatically load the initial ball layout of the corresponding rules (projecting the initial ball placement point through the projection module) and perform situation analysis and strategy recommendation according to the rules. Special practice mode: Provides classic training ball layouts such as snake ball and five-point ball, and also supports users to upload custom training ball layouts; after the user selects, the system will automatically project the standard practice ball layout, and may include the specific goal of the practice (such as clearing the table in sequence). Custom ball layout mode: The mini-program can support users to upload custom ball layout images or place the ball on the virtual table by dragging and dropping. The custom ball layout data will be sent to the processing module 120. The processing module can control the projection module 130 to accurately project the ball layout onto the real table for users to conduct targeted practice or tactical design.

[0047] (4) Game replay and analysis function flow: In the mini-program, users can save any game or training session; the saved data may include: the precise position and state of all balls before and after each shot (derived from the historical records of the tracked object), the user's shot parameters, and the shot strategy generated by the system at that time. Users can also select saved games for review through the "history" or "replay" interface of the mini-program. When starting the replay, users can select the "replay" function. At this time, the mini-program will send a replay command to the processing module 120, and the processing module will control the projection module 130 to accurately reproduce the position and state of the billiard balls at the selected historical moment on the real billiard table. Accordingly, users can practice hitting the ball repeatedly in the exact same position to consolidate muscle memory or try different solutions; at the same time, if users are not satisfied with the result of a shot in an actual game or practice, they can restore the state of the game to the moment before the shot through the replay function to realize the undo operation, so as to re-analyze and try different shot options.

[0048] In some embodiments, the processing module is further configured to: perform image preprocessing on the visual data to generate a standard top view of a billiard table; wherein the image preprocessing includes lens distortion correction processing and perspective transformation correction processing.

[0049] Based on the above setup, image preprocessing technology systematically solves two core problems affecting the accuracy of visual measurements through the coordinated operation of lens distortion correction and perspective transformation correction: inherent optical distortion of the hardware and angular distortion caused by the installation position. It transforms the raw, irregular observation data into a geometrically accurate, scale-uniform standard top view. This standard top view provides high precision and high reliability for subsequent functions such as target detection, multi-target tracking, geometric analysis, and precise projection.

[0050] Specifically, on the one hand, camera lenses, especially wide-angle lenses, introduce radial distortion (straight lines at the image edges bend inwards or outwards) and tangential distortion due to optical principles. Without lens distortion correction, a ball on the edge of a billiard table would appear offset in the image due to its distance from the optical center. Lens distortion correction involves calibrating the camera beforehand to obtain its inherent intrinsic parameter matrix and distortion coefficients. Then, using functions like the `undistort` function in OpenCV, these parameters are mathematically transformed to effectively straighten curved lines and restore objects to their proper positions. This eliminates sensor-level system errors and ensures that every pixel coordinate extracted from the image is geometrically accurate.

[0051] On the other hand, when a camera shoots a pool table from an oblique angle, a perspective effect occurs, causing objects to appear larger in the foreground and smaller in the background, and the originally parallel edges of the table no longer appear parallel in the image. Images from this perspective cannot be directly used for precise geometric measurements because the scales of different areas of the image are inconsistent. In perspective transformation correction, by identifying the four corner points of the pool table in the image and defining their target positions in a virtual, top-down standardized coordinate system, a homography transformation matrix can be calculated. Applying this transformation matrix to transform the distortion-corrected image "flattens" the oblique perspective image into a standard top-down view without perspective effects. This unifies the image coordinate system to a fixed, measurable tabletop coordinate system, ensuring that every point in the image corresponds to a specific physical location on the table.

[0052] The above correction process enables a high-precision, linear correspondence between the pixel distance between any two points in the image and the actual physical distance (millimeter level) on the billiard table. This is a prerequisite for achieving millimeter-level position detection and accurate angle calculation for small target objects (billiard table scene).

[0053] Understandably, by generating a standard top view through image preprocessing, the system can ultimately process a standard, head-on top view regardless of the height or angle at which the camera is installed. This greatly simplifies the design and complexity of subsequent target detection, tracking, and geometric analysis algorithms, and improves the robustness and repeatability of the system.

[0054] According to an embodiment of this application, considering that lighting conditions vary greatly in different billiard halls, with some areas being very bright and others relatively dim, the aforementioned processing module is further configured to perform adaptive pre-setting of camera parameters so that the camera parameters are adapted to the lighting environment of the billiard hall.

[0055] Specific steps may include: (1) Generating and placing standard calibration balls: The processing module can control the projection module to project a clear standard calibration ball diagram onto the billiard table surface. The diagram clearly indicates the precise position and order in which each billiard ball (including all colored balls and cue balls) should be placed, so that the user (or staff) can place the physical billiard balls in the corresponding positions according to the projection guidance. Preferably, the standard calibration ball is a specific layout that covers all types of balls and is dispersed in position (e.g., simulated opening ball placement or specific training ball pattern) to ensure that the calibration process can fully test the imaging conditions of each area of ​​the table. (2) Performing camera parameter space scanning and image acquisition: In response to the activation of the camera parameter calibration function, the processing module controls the camera to enter the automatic parameter scanning mode. The system traverses within a preset parameter combination space. The core adjustment parameters may include exposure time, gain, white balance, brightness, contrast, etc. For each set of camera parameter combinations to be tested, the camera acquires a frame (or average of multiple frames) of table image containing the placed standard calibration balls. (3) Perform performance testing and scoring: For each image acquired under specific parameters, the processing module performs a complete image preprocessing (distortion and perspective correction) process, converts it into a standard top view, and uses the YOLO model deployed on the neural network inference engine to perform object detection on the top view; the built-in scoring algorithm is used to quantitatively evaluate the detection results and calculate the parameter fitness score. Specifically, the fitness scores of the above parameters can comprehensively consider the following dimensions: detection recall (the ratio of the number of balls identified by the algorithm to the total number of balls actually placed, with more deductions for missed detections), class accuracy (the proportion of balls correctly identified in terms of class (e.g., white ball, specific color ball), confidence level (the average confidence score of all detection boxes, with higher average confidence usually indicating better image quality for model judgment), detection box stability (the consistency of position and size of detection boxes for the same ball in multiple frames of images (if multiple frames are captured), with parameter combinations with large jitter assigning lower scores), and image quality auxiliary evaluation (a weighted technique combining traditional image quality indicators such as image sharpness (e.g., gradient calculation) and brightness uniformity). (4) Optimal Parameter Selection and Persistence: After scanning and scoring all preset parameter combinations, the camera parameter set with the highest fitness score is selected based on the scoring algorithm. The processing module sets this set of parameters as the optimal parameters for the current operation of the camera and can associate this parameter set with the current environment identifier (such as device ID, geographical location) and persistently store it in the non-volatile memory of the edge computing device. In addition, users can periodically start calibration through a mini-program to cope with seasonal changes in ambient light or site modifications. At the same time, when the system detects a continuous abnormal decline in confidence through real-time detection, it can proactively prompt the user to recalibrate.

[0056] In some embodiments, the processing module is also used to: employ a YOLO deep learning model with a lightweight network structure to perform target detection processing on a standard top view generated after image preprocessing, so as to output the category, position and confidence information of the ball, bag and club.

[0057] YOLO (You Only Look Once) is a single-stage object detection algorithm. Its core idea is to treat object detection as a regression problem, requiring only one forward propagation computation of the neural network on the image to directly output the bounding boxes and class probabilities of all objects. Lightweight network structures (such as YOLOv11n) significantly reduce model complexity and computational overhead by reducing the number of network layers and channels or using efficient convolutional modules, while maintaining accuracy. This allows complex deep learning models to be deployed on resource-constrained terminal devices, ensuring that the system can run independently and smoothly in offline environments without cloud reliance.

[0058] The deep learning model outputs the category, position, and confidence information of the ball, pocket, and cue: Category information helps accurately distinguish between the cue ball, target ball, pocket, and cue, and is a prerequisite for performing correct situation analysis and strategy generation (e.g., "which target ball should be hit by the cue ball into which pocket"). Position information, provided as the center point of the bounding box in pixel coordinates, becomes the data source for calculating the distance, angle, and velocity between balls, as well as for collision detection after coordinate system transformation. Confidence information provides crucial quality information for subsequent multi-target tracking algorithms. These algorithms can use confidence to weight detection results; for example, prioritizing high-confidence detection boxes for association matching, while low-confidence detection results may trigger "not update" or "further confirmation needed" logic, helping to enhance the system's anti-interference capability in complex situations.

[0059] Based on the above settings, by introducing a lightweight YOLO model and outputting complete perception information including category information, location information, and confidence information, the system is successfully endowed with accurate visual recognition capabilities. This helps to solve the technical challenge of achieving real-time, multi-category target detection on terminal devices. More importantly, it provides accurate and quantifiable input data for downstream tracking and decision-making modules, ensuring the start-up and reliable operation of the entire system's intelligent closed loop.

[0060] In some of these embodiments, the YOLO deep learning model is deployed on a neural network inference engine.

[0061] By deploying the YOLO model on top of the neural network inference engine, the real-time performance and stability of the model are ensured, which helps to expand the applicable scenarios of the system.

[0062] Specifically, general-purpose deep learning frameworks, in order to be compatible with various hardware, have relatively universal computational operations. In contrast, neural network inference engines (such as RKNN) are designed specifically for particular chip architectures, which helps reduce computational load and improve model processing efficiency. This is because the optimization of the inference engine is not isolated; the entire process, from model loading, data input, computation execution to result output, has been restructured and accelerated. This allows the entire detection task to achieve optimal performance as a whole, avoiding bottlenecks that might occur in general-purpose frameworks. Consequently, the billiards teaching and assistance system can reduce latency during operation, ensuring a smooth and lag-free experience at all times.

[0063] In some embodiments, the processing module is further configured to: group the spheres according to their semantic categories when executing the data association algorithm, and use a nearest neighbor matching strategy to associate the target detection results of the current frame with existing tracked objects in order to update the state of the tracked objects; wherein the state of the tracked objects switches between a stationary state, a moving state, and a lost state based on the association results and preset conditions.

[0064] Based on the above setup, a robust multi-target tracking core was constructed through a progressive design of grouping isolation, proximity association, and state management. This core maintains a tracking object with identity and state for each sphere. This effectively solves the most challenging problems in multi-target tracking: identity switching and trajectory breakage, providing a temporal data foundation for all subsequent trajectory-based decision analyses (such as collision detection and situation assessment).

[0065] According to the embodiments of this application, in a billiards scenario, balls of different types (such as cue balls and striped balls) cannot be physically converted into each other. If global matching is performed indiscriminately, the algorithm may incorrectly associate a detection box of a cue ball with the tracking trajectory of a striped ball due to their proximity. Based on the above setting, grouping by semantic category first logically eliminates this fundamental error, limiting the association search space to the same category, greatly reducing the risk of mismatch, and effectively ensuring the consistency of the tracked object's identity. Furthermore, within the same category group, nearest neighbor matching is a very effective and computationally efficient choice. Its technical idea is that between two consecutive frames (with a very short time interval), the movement of the same ball is continuous, and its position change is minimal. Therefore, the distance between the detection box of the current frame and the tracker's predicted next position (given by the Kalman filter) is calculated, and the closest one is selected for association. This ensures accuracy while having low computational complexity, meeting the real-time requirements of the system. Meanwhile, the tracking object's state switches based on the association results and preset conditions, further solving the problem of uncertainty management. The state machine manages the lifecycle and state of the tracking object through preset conditions: Static state: The ball is not hit and its position remains basically unchanged; Moving state: After successful association, the ball is detected to be moving; Unseen / Lost state: When a tracking object fails to be successfully associated with any detection result for several consecutive frames, it will not be deleted immediately, but will first enter the "lost" state. This provides fault tolerance for dealing with brief occlusions (such as the club passing by) or momentary detection failures. Only when the number of consecutive lost frames exceeds a preset threshold will the system finally determine that the target has gone out of bounds or permanently disappeared and delete it. This ensures that the multi-target tracking process will not be interrupted by brief perceived noise, greatly improving the tracking continuity in real complex environments.

[0066] In some embodiments, the processing module is further configured to: predict and filter one or more feasible hitting paths from the cue ball to the aiming point without any other ball obstructing the path when generating a hitting strategy; wherein the collision detection algorithm determines whether a ball collision will occur on the path based on the prediction of the cue ball's trajectory.

[0067] Based on the above settings, a collision detection algorithm based on physics trajectory prediction is introduced and used as a mandatory screening condition for generating hitting strategies, giving the system's decision-making capabilities a crucial real-world constraint. By actively predicting and eliminating invalid paths that might be blocked midway, it greatly improves the success rate, practicality, and professionalism of the generated hitting strategies, ensuring the actual value of the auxiliary guidance and the user's trust.

[0068] According to the embodiments of this application, a simple static method to determine whether the cue ball will collide with ball C on its path from point A to point B is to check whether line segment AB intersects with the circular area of ​​ball C. However, this method is inaccurate in billiards because it ignores the physical dimensions of the cue ball itself and cannot handle the complex situation where the cue ball deviates before hitting the target ball. Based on the above characteristics, this application adopts a more scientific and rigorous dynamic prediction method: treating the cue ball as a moving rigid body with a physical radius, and predicting its motion trajectory envelope as it moves from its starting position along a certain direction. The collision detection algorithm checks whether this trajectory with width overlaps in space and time with the motion envelopes of other stationary (or moving) balls on the path. Essentially, it simulates the physical scene at the instant after the ball is hit, thus enabling high-precision prediction of whether a mid-course collision will occur.

[0069] Furthermore, the collision detection algorithm provided in this application is not for post-hoc verification, but rather serves as a core filtering module in the shot strategy generation process. The processing module first calculates multiple candidate aiming points and paths through geometric analysis, and then verifies the feasibility of each candidate path using the collision detection algorithm. Only those shot paths that pass the verification and are determined to be valid are retained and recommended to the user as the final strategy.

[0070] In some of these embodiments, the data acquisition module, processing module, and projection module are all integrated into the local edge computing device, enabling the system to operate independently in an offline environment.

[0071] In traditional cloud computing models, data undergoes a long process: "terminal acquisition, network upload, cloud processing, network distribution, and terminal execution." Network transmission is the primary source of uncertainty and latency. By integrating the data acquisition, processing, and projection modules into a local edge computing device, the data processing flow is simplified to a local closed loop of "acquisition, processing, and projection." This bypasses the network transmission stage, minimizing latency and eliminating the impact of external network quality fluctuations on system performance, thus achieving extremely high reliability.

[0072] Meanwhile, since all calculations are performed locally, sensitive raw data (such as images) and derived data (such as ball-striking strategies and user action parameters) never leave the user's device, achieving a localized closed loop for data privacy. Furthermore, offline operation means that the system's core functions are completely decoupled from internet services, making its deployment no longer dependent on the availability or quality of the network environment, greatly expanding the product's market boundaries and application possibilities. Understandably, this also saves users potential cloud service costs and avoids the risk of system paralysis due to cloud service outages.

[0073] This invention also provides a projection-based billiards teaching and assistance method, applicable to any of the projection-based billiards teaching and assistance systems described above, such as... Figure 2 As shown, this projection-based billiards teaching and support method includes: Step S201: Visual data of the billiard table is collected by a camera, and motion sensing data of the billiard cue is collected by an inertial measurement unit. Step S202: Perform image preprocessing and target detection processing on the visual data to identify each ball on the pool table; use a multi-target tracking algorithm based on Kalman filter to create and maintain a tracking object for each identified ball, and update the state of the tracking object through a data association algorithm. The state includes position, velocity, and motion state managed by a state machine; based on the state of the tracking object corresponding to each ball, calculate the aiming point of the cue ball hitting the target ball to the designated pocket through geometric analysis, and evaluate the feasibility of the hitting path through a collision detection algorithm to generate a hitting strategy; Step S203: The striking strategy is projected onto the billiard table in a graphical form; wherein the graphical form includes aiming lines representing the path of the cue ball to the aiming point.

[0074] The camera module captures the situation on the billiard table before the shot, and the inertial measurement unit acquires the movement sensor data of the cue stick to achieve multimodal perception fusion. Then, based on the TrackedBall state management mechanism, it realizes accurate and continuous tracking of the ball object. Combined with the geometric analysis algorithm and collision detection algorithm based on the physics of billiards, it automatically calculates the shooting strategy including the optimal shooting angle. The motion capture module captures the direction and angle of the cue stick, and the optimal shooting angle image and the current shooting angle image are projected on the billiard table through the projection module.

[0075] According to a specific implementation of the present application, before the method is executed, initialization and calibration operations for the projection-type billiards teaching and auxiliary system also need to be performed: such as device initialization: connecting the camera, projector, and IMU sensor; IMU calibration: performing static calibration on the IMU to obtain the initial attitude reference; camera calibration parameter loading: loading the camera's intrinsic parameter matrix and distortion coefficients.

[0076] According to a specific embodiment of the present application, during the execution of the method, the camera and the inertial measurement unit perform parallel data acquisition, wherein the main thread acquires visual data (30fps) by the camera; and the sub-thread acquires motion sensing data (100Hz) by the IMU.

[0077] In some embodiments, the method further includes: calculating the user's cueing action parameters based on motion sensing data; comparing the cueing action parameters with the hitting strategy to generate action correction feedback information; and sending the action correction feedback information to the user terminal for display to provide hitting action correction guidance.

[0078] Based on this solution, after the ball is struck, the motion capture data is analyzed by a processing module (such as host computer software), the correctness of the action is calculated, and feedback is given to the user through a user terminal (such as a mobile phone), thus realizing intelligent billiards teaching.

[0079] In some embodiments, the above-described image preprocessing step for visual data further includes: performing image preprocessing on the visual data to generate a standard top view of a billiard table; wherein the image preprocessing includes lens distortion correction processing and perspective transformation correction processing.

[0080] Based on the above settings, image preprocessing technology, through the coordinated operation of lens distortion correction and perspective transformation correction, transforms the original, irregular observation data into a geometrically accurate, scale-uniform standard top view. This standard top view provides a high-precision and high-reliability guarantee for subsequent implementations such as target detection, multi-target tracking, geometric analysis, and precise projection.

[0081] According to a specific embodiment of this application, the lens distortion correction process includes: taking the original image I_raw captured by the camera, the camera's intrinsic parameter matrix K, and the distortion coefficients D as inputs; and using OpenCV's undistort function to implement radial and tangential distortion correction in order to output a distortion-free image.

[0082] The perspective transformation correction process includes: taking the coordinates of the four corner points of the billiard table, P_camera = [(620,431), (3589,551), (3283,1784), (863,1609)], as input; taking the standard rectangle, P_standard = [(0,0), (2540,0), (2540,1270), (0,1270)], as the target; using H = findHomography(P_camera, P_standard) as the transformation matrix; and outputting a standard top view.

[0083] In some embodiments, the above-described steps for object detection processing of visual data include: using a YOLO deep learning model with a lightweight network structure to perform object detection processing on a standard top view generated after image preprocessing, so as to output the category, position, and confidence information of the ball, bag, and club.

[0084] Based on the above settings, by introducing a lightweight YOLO model and outputting complete perception information including category information, location information, and confidence information, the system is successfully endowed with accurate visual recognition capabilities. This helps to solve the technical challenge of achieving real-time, multi-category target detection on terminal devices. More importantly, it provides accurate and quantifiable input data for downstream tracking and decision-making modules, ensuring the start-up and reliable operation of the entire system's intelligent closed loop.

[0085] According to a specific implementation of the present application, before the target detection process is executed, the lightweight YOLO model is configured as follows: Model architecture: using the lightweight version of YOLOv11n; Input size: 640×640 pixels; Detection categories: 23 categories (19 types of balls + white ball + 2 types of ball bags + cue sticks); System deployment platform: RKNN inference engine optimization. The target detection process may include: (1) Image preprocessing: scaling the detection object (the aforementioned generated standard top view) to 640×640 and performing letterbox filling, normalization, etc.; (2) Model inference: using RKNN to accelerate inference and output detection boxes, confidence, category information, etc.; (3) Output result processing: such as NMS non-maximum suppression (IoU threshold 0.45); cross-category NMS to avoid duplicate detection; coordinate transformation back to the original image scale.

[0086] Regarding the steps of creating and maintaining a tracking object for each identified sphere using the Kalman filter-based multi-target tracking algorithm described above, this application provides a specific implementation method: overcoming the detection instability problem of the YOLO model by designing a Kalman filter-based multi-target tracking algorithm.

[0087] The Kalman filter configuration example is as follows: State vector: X = [x,y,vx,vy]^T; State transition matrix: F = [[1,0,dt,0], [0,1,0,dt], [0,0,1,0], [0,0,0,1]]; Observation matrix: H = [[1,0,0,0], [0,1,0,0]]; Process noise covariance: Q = dt^4 / 4 * σ_a^2 * identity matrix; Observation noise covariance: R = σ_r^2 * identity matrix.

[0088] The TrackedBall class design example is as follows: State definitions: STATIC (stationary state), MOVING (moving state), UNSEEN (temporarily lost state), LOST (permanently lost state); core attributes include: position, velocity, confidence level, historical trajectory, and Kalman filter.

[0089] In some embodiments, the step of updating the state of the tracked object through the data association algorithm, wherein the state includes position, velocity, and motion state managed by a state machine, includes: when executing the data association algorithm, grouping the objects according to the semantic category of the spheres, and using a nearest neighbor matching strategy to associate the target detection results of the current frame with existing tracked objects to update the state of the tracked object; wherein the state of the tracked object switches between a stationary state, a moving state, and a lost state based on the association results and preset conditions.

[0090] Based on the above setup, a robust multi-target tracking core was constructed through a progressive design of grouping isolation, proximity association, and state management. This core maintains a tracking object with identity and state for each sphere. This effectively solves the most challenging problems in multi-target tracking: identity switching and trajectory breakage, providing a temporal data foundation for all subsequent trajectory-based decision analyses (such as collision detection and situation assessment).

[0091] According to a specific implementation of the present application, the above data association algorithm adopts the association strategy of nearest neighbor matching by grouping by category, specifically including: (1) category pre-grouping: grouping the tracking ball and detection results by class_id; (2) distance calculation: calculating the Euclidean distance between the predicted position and the detection position; (3) nearest neighbor matching: selecting the detection result with the smallest distance for association; (4) new target creation: creating a new TrackedBall object for the unmatched detection result, that is, creating a new tracking object.

[0092] In some embodiments, the processing module is further configured to: predict and filter one or more feasible hitting paths from the cue ball to the aiming point without any other ball obstructing the path when generating a hitting strategy; wherein the collision detection algorithm determines whether a ball collision will occur on the path based on the prediction of the cue ball's trajectory.

[0093] Based on the above settings, a collision detection algorithm based on physics trajectory prediction is introduced and used as a mandatory screening condition for generating hitting strategies, giving the system's decision-making capabilities a crucial real-world constraint. By actively predicting and eliminating invalid paths that might be blocked midway, it greatly improves the success rate, practicality, and professionalism of the generated hitting strategies, ensuring the actual value of the auxiliary guidance and the user's trust.

[0094] In some embodiments, the data acquisition module, processing module, and projection module of the above-mentioned projection-type billiards teaching and assistance system are all integrated into a local edge computing device, enabling the system to operate independently in an offline environment.

[0095] Based on the state of the tracking object corresponding to each ball, geometric analysis is used to calculate the aiming point of the cue ball striking the target ball to the designated pocket. This application provides a specific implementation method: calculating the optimal striking path based on the principles of billiards physics. The main steps involved are as follows: (1) Target classification: Separate the white ball, target ball, and ball pocket from the detection results; (2) Calculation of the angle of entry into the pocket: For each target ball, calculate the range of the angle of entry into each ball pocket; consider the geometry of the ball pocket (e.g., bottom pocket 115.5mm, side pocket 130mm); apply the angle range function: angle_range(ball_positions); (3) Calculation of the aiming point: For the target ball position P_ball and the ball pocket position P_pocket, calculate respectively: ball to pocket direction vector: v_ball_to_pocket=P_pocket - P_ball; unit direction vector: u=v_ball_to_pocket / ||v_ball_to_pocket||; aiming point position: P_aim=P_ball - 2 * R_ball * u; where R_ball is the ball radius (28.575mm).

[0096] To assess the feasibility of a shot path using a collision detection algorithm in order to generate a shot strategy, this application provides a specific implementation method: employing a ball-to-ball collision detection algorithm based on the prediction of the cue ball's trajectory to construct a mathematical model for calculation.

[0097] Meanwhile, for multiple shot strategies calculated, the optimal shot strategy can be determined by the optimal shot strategy selection algorithm, specifically including: (1) designing a scoring function Score = cos(θ) * angle_width / distance; where θ is the angle between the cue ball and the target ball and the pocket; angle_width is the pocket angle width; distance is the distance between the cue ball and the aiming point. (2) designing constraints: (a) the cue ball cannot collide with other balls on the path to the aiming point; (2) the target ball cannot be blocked by other balls on the path to the pocket; (3) the pocket angle is within the allowable range.

[0098] According to a specific implementation of the present application, when projecting the ball-striking strategy onto the billiard table in graphic form, a projection coordinate transformation step is also included: First, the projection coordinate transformation is performed, and the transformation process includes: (1) Determining the billiard table coordinate system: with the lower left corner of the billiard table as the origin, and the unit is millimeters; (2) Determining the projector coordinate system: with the upper left corner of the projected image as the origin, and the unit is pixels; (3) Performing the perspective transformation matrix: calculating the projection transformation matrix based on the four corner points. Then, the aiming line needs to be drawn, and the drawing elements mainly include: (1) White ball marker: solid circle, filled with white; (2) Target ball marker: dashed circle, corresponding color; (3) Ball pocket marker: solid dot, filled with white; (4) Aiming line: from the white ball to the aiming point: draw a dashed line, from the target ball to the ball pocket: draw a dotted line; aiming point: draw a dashed circle.

[0099] Furthermore, according to the specific implementation of the embodiments of this application, the above method also includes providing accurate technical analysis for advanced users, which may include: displaying multiple optional hitting schemes to achieve multi-strategy comparison; calculating the hitting difficulty based on geometric parameters to achieve difficulty assessment of the hitting strategy; and saving the hitting trajectory and success rate statistics for easy user review.

[0100] This invention also provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to an embodiment of this invention.

[0101] This invention also provides a computer program product, including a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform the methods of embodiments of this invention. The computer program product should be understood as a software product that primarily implements the methods described above through a computer program.

[0102] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, which, when executed by the at least one processor, causes the electronic device to perform the method of this invention.

[0103] refer to Figure 3The present invention will now be described in the form of a structural block diagram of an electronic device that can serve as an embodiment of the present invention, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0104] like Figure 3 As shown, the electronic device includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of the electronic device. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0105] Multiple components in the electronic device are connected to I / O interface 305, including: input unit 306, output unit 307, storage unit 308, and communication unit 309. Input unit 306 can be any type of device capable of inputting information into the electronic device. Input unit 306 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device. Output unit 307 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 308 may include, but is not limited to, disks and optical discs. Communication unit 309 allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, and / or wireless communication transceivers, such as Bluetooth devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0106] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, CPUs, graphics processing units (GPUs), various special-purpose artificial intelligence (AI) computing units, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present invention can be implemented as a computer program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device via ROM 302 and / or communication unit 309. In some embodiments, the computing unit 301 can be configured to perform the methods described above by any other suitable means (e.g., by means of firmware).

[0107] Computer programs for implementing the methods of embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0108] In the context of embodiments of the present invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, or infrared systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0109] It should be noted that the term "comprising" and its variations used in the embodiments of the present invention are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present invention are illustrative and not restrictive. Those skilled in the art should understand that, unless explicitly indicated otherwise in the context, they should be understood as "one or more".

[0110] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in the embodiments of the present invention are all information and data that have been permitted by the user or have been fully agreed upon by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to agree or refuse.

[0111] The steps described in the method embodiments provided by this invention can be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of protection of this invention is not limited in this respect.

[0112] The term "embodiment" in this specification refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of the invention. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply independence or alternativeity from other embodiments. The various embodiments in this specification are described in a related manner, with reference to each other for similar or identical parts. In particular, for apparatus, device, and system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, and relevant details are referred to in the description of the method embodiments.

[0113] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A projection-based billiards teaching and auxiliary system, characterized in that, include: The data acquisition module includes a camera mounted above the billiard table and an inertial measurement unit mounted on the billiard cue, which are used to acquire visual data of the billiard table and motion sensing data of the billiard cue, respectively. The processing module, communicatively connected to the data acquisition module, is used for: performing image preprocessing and target detection processing on the visual data to identify each ball on the billiard table; using a Kalman filter-based multi-target tracking algorithm to create and maintain a tracking object for each identified ball, and updating the state of the tracking object through a data association algorithm, the state including position, velocity, and motion state managed by a state machine; based on the state of the tracking object corresponding to each ball, calculating the aiming point of the cue ball hitting the target ball to the designated pocket through geometric analysis, and evaluating the feasibility of the hitting path through a collision detection algorithm to generate a hitting strategy; A projection module, communicatively connected to the processing module, is used to project the ball-striking strategy in graphical form onto the billiard table; wherein the graphical form includes aiming lines representing the path of the cue ball to the aiming point.

2. The system according to claim 1, characterized in that, The system also includes a user terminal; wherein... The processing module is further configured to: calculate the user's cueing action parameters based on the motion sensing data; compare the cueing action parameters with the ball-striking strategy, generate action correction feedback information, and send the action correction feedback information to the user terminal for display, so as to provide ball-striking action correction guidance.

3. The system according to claim 1, characterized in that, The processing module is further configured to: perform image preprocessing on the visual data to generate a standard top view of the billiard table; wherein the image preprocessing includes lens distortion correction processing and perspective transformation correction processing.

4. The system according to claim 3, characterized in that, The processing module is also used to: employ a YOLO deep learning model with a lightweight network structure to perform target detection processing on the standard top view generated after image preprocessing, so as to output the category, position and confidence information of the ball, bag and club.

5. The system according to claim 4, characterized in that, The YOLO deep learning model is deployed on a neural network inference engine.

6. The system according to claim 1, characterized in that, The processing module is further configured to: group the objects according to their semantic categories when executing the data association algorithm, and use a nearest neighbor matching strategy to associate the target detection results of the current frame with the existing tracked objects to update the state of the tracked objects; wherein the state of the tracked objects switches between a stationary state, a moving state, and a lost state based on the association results and preset conditions.

7. The system according to claim 1, characterized in that, The processing module is further configured to: when generating the hitting strategy, predict and filter one or more feasible hitting paths from the cue ball to the aiming point without any other balls obstructing the path based on the collision detection algorithm; wherein the collision detection algorithm determines whether a ball collision will occur on the path based on the prediction of the cue ball's trajectory.

8. The system according to claim 1, characterized in that, The data acquisition module, the processing module, and the projection module are all integrated into the local edge computing device, enabling the system to operate independently in an offline environment.

9. A projection-based billiards teaching and auxiliary method, characterized in that, The method, applied to the projection-type billiards teaching and auxiliary system as described in any one of claims 1-8, comprises: Visual data of the billiard table is collected by a camera, and motion sensing data of the billiard cue is collected by an inertial measurement unit. The visual data undergoes image preprocessing and target detection to identify the balls on the billiard table. A multi-target tracking algorithm based on a Kalman filter is used to create and maintain a tracking object for each identified ball. The state of the tracking object is updated using a data association algorithm, and the state includes position, velocity, and motion state managed by a state machine. Based on the state of the tracking object corresponding to each ball, geometric analysis is used to calculate the aiming point for the cue ball to strike the target ball to the designated pocket. A collision detection algorithm is then used to evaluate the feasibility of the striking path to generate a striking strategy. The ball-striking strategy is projected onto the billiard table in a graphical form, wherein the graphical form includes aiming lines representing the path of the cue ball to the aiming point.

10. An electronic device, comprising: A processor and a memory storing a program, characterized in that the program includes instructions that, when executed by the processor, cause the processor to perform the method according to claim 9.

Images