Binocular high-definition sensing system for indoor AR golf course

Abstract

The invention discloses an indoor AR golf course binocular high-definition sensing system, and the system comprises a dynamic capturing module which comprises a high-speed camera unit, a laser radar unit, and an inertial measurement unit; wherein the high-speed camera unit is used for calculating the position change of a golf ball; the laser radar unit is used for providing three-dimensional space position information of the golf ball; the inertial measurement unit is used for monitoring the rotation rate and the speed change of the golf ball in real time; and the vibration feedback module is used for simulating the impact feeling and the rebound feeling during real ball hitting. According to the invention, the dynamic capture module and the vibration feedback module cooperate with each other, the position, track and motion state of a golf ball in a projection picture are updated in real time, the illumination, texture and shadow effects in a virtual scene are dynamically adjusted, the high-definition and low-delay visual effect is supported, and the visual effect is improved. And the sound effect matched with the ball hitting action and the motion state of the ball is matched, so that the sense of reality of indoor golf playing is provided.

Description

Technical Field

[0001] The present invention belongs to the technical field of indoor golf sensing systems, and particularly relates to a binocular high-definition sensing system for an indoor AR golf course. Background Art

[0002] In the field of indoor golf simulation, the limitations of the prior art significantly affect the user experience and the authenticity of the simulation. The main problem is that traditional sensing systems cannot accurately capture the high-speed movement trajectory of a golf ball. After being hit, the golf ball flies at a very high speed, accompanied by rapid rotation and irregular bouncing, and its trajectory is complex and difficult to predict. Due to technical limitations, existing systems cannot track this high-speed movement in real time and accurately, resulting in a large deviation between the simulated ball path and the actual situation, affecting the training effect and the user experience.

[0003] In addition, existing simulation systems lack an effective vibration feedback mechanism. In real golf strokes, the impact feeling at the moment of hitting the ball, the tactile feeling of the club contacting the ball, and the vibration generated by the rolling and bouncing of the ball are crucial for players to perceive the hitting effect. However, the screens of existing systems cannot simulate these tactile feedbacks, and users cannot obtain a real hitting experience, thus affecting the mastery of hitting skills.

[0004] In terms of visual and auditory effects, existing systems also have deficiencies. In a real golf scenario, the flight trajectory of the ball, the rolling and bouncing after landing, and the interaction with different terrains will produce rich visual and auditory effects. These effects are crucial for players to judge the hitting effect. However, existing systems often cannot provide visual and auditory effects that match the real scenario. The images displayed on the screen lack a sense of reality, and the sound effects cannot simulate the various sounds of real hitting, further reducing the authenticity of the simulation.

[0005] These problems make it difficult for existing indoor golf simulation systems to meet the user's demand for a real experience and limit the development of golf simulation technology. Summary of the Invention

[0006] The purpose of the present invention is to provide a binocular high-definition sensing system for an indoor AR golf course, which is proposed to solve the problem that current indoor golf simulation cannot simulate the overspin and the flying path of the ball, and at the same time cannot give feedback, nor can it provide a real sense of experience in terms of vision and hearing like the outdoor environment.

[0007] To achieve the above purpose, the present invention adopts the following technical solution: A binocular high-definition sensing system for an indoor AR golf course, comprising:

[0008] A dynamic capture module, which includes a high-speed camera unit, a lidar unit, and an inertial measurement unit;

[0009] The high-speed camera unit is used to calculate the position change of the golf ball;

[0010] The lidar unit is used to provide three-dimensional spatial position information of the golf ball;

[0011] The inertial measurement unit is used to monitor the rotation rate and speed change of the golf ball in real time;

[0012] The vibration feedback module is used to simulate the impact and rebound feelings during real hitting.

[0013] As a further description of the above technical solution:

[0014] The high-speed camera unit adopts the image recognition algorithm of OpenCV to perform differential operations on consecutive frames;

[0015] Operation formula: D(x, y, t) = |I(x, y, t) - I(x, y, t - 1)|), to calculate the position change of the golf ball, where I(x, y, t) represents the pixel value of the image at coordinates (x, y) and time t.

[0016] As a further description of the above technical solution:

[0017] The lidar unit uses the triangulation principle to provide three-dimensional spatial position information of the golf ball; Measurement formula: Z = f × B / d, where Z is the distance, f is the lens focal length, B is the baseline length, and d is the parallax.

[0018] As a further description of the above technical solution:

[0019] The inertial measurement unit includes an accelerometer and a gyroscope. The accelerometer and the gyroscope are used to collect acceleration and angular velocity data, and use the Kalman filter algorithm to fuse the data to monitor the rotation rate and speed change of the golf ball in real time;

[0020] Algorithm formula:

[0021] As a further description of the above technical solution:

[0022] The vibration feedback module includes a vibration feedback screen. The vibration feedback screen integrates piezoelectric ceramic materials. According to the preset vibration formula y = Asin(ωt + φ), the vibration frequency and intensity of the piezoelectric ceramic are controlled by an electrical signal to simulate the impact and rebound feelings during hitting.

[0023] As a further description of the above technical solution:

[0024] It further includes a low-power consumption mixed reality enhancement module, and the low-power consumption mixed reality enhancement module includes a low-power consumption LED projection unit and an intelligent sound and light control unit. Among them, the low-power consumption LED projection unit adjusts the projection screen in real time according to the user's hitting action and the movement trajectory of the ball by using a dynamic rendering algorithm, and the intelligent sound and light control unit realizes the sound effects matching the hitting action and the movement state of the ball through a preset sound effect library and a light and shadow algorithm, including hitting sounds, ball rolling sounds, and grass friction sounds; at the same time, it simulates the light change in the real environment.

[0025] As a further description of the above technical solution:

[0026] The dynamic rendering algorithm adopts the OpenGL or DirectX algorithm.

[0027] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are as follows:

[0028] 1. In the present invention, the differential operation is performed on consecutive frames by using the image recognition algorithm based on OpenCV to calculate the position change of the golf ball. Through the formula D(x, y, t) = |I(x, y, t) - I(x, y, t - 1)|, the system can quickly detect the position change of the ball in the image, where I(x, y, t) represents the pixel value of the image at coordinates (x, y) and time t. By emitting and receiving laser beams, the three-dimensional spatial position information of the golf ball is provided according to the principle of triangulation; the lidar can accurately measure the distance between the ball and the sensor, so as to provide the three-dimensional position data of the ball, integrate the accelerometer and gyroscope, and collect the acceleration and angular velocity data. The Kalman filter algorithm is used to fuse the data to monitor the rotation rate and speed change of the golf ball in real time. The acceleration and angular velocity data provided by the inertial measurement unit can help the system understand the dynamic behavior of the ball, including rotation and speed change. The data of the above three sensors are comprehensively processed through a deep learning algorithm (the combination of convolutional neural network CNN and long short-term memory network LSTM) to achieve high-precision tracking and prediction of the high-speed movement trajectory of the golf ball, and the accuracy reaches the millimeter level. The deep learning algorithm can learn patterns from a large amount of historical hitting data to improve the accuracy and stability of tracking and prediction. In addition, the vibration feedback screen: integrates piezoelectric ceramic materials, and controls the vibration frequency and intensity of the piezoelectric ceramic through an electric signal according to a preset vibration formula (sine wave y = Asin(ωt + φ) or square wave) to simulate the impact and rebound feelings when hitting the ball. The system dynamically adjusts the vibration parameters according to the hitting force and speed to provide a personalized feedback experience.

[0029] 2. In the present invention, for the sound effect simulation in the intelligent sound and light control unit: sound effects matching the hitting action and the movement state of the ball are provided through a preset sound effect library (WAV files), including hitting sounds, ball rolling sounds, grass friction sounds, etc.; different sound effects can be preset according to different field types (such as grassland, sand, green) in the sound effect library to enhance the user's auditory experience. For the light and shadow effect simulation: the light and shadow effect in the projection screen is dynamically adjusted according to the movement trajectory and speed of the ball by using a light and shadow algorithm (Phong lighting model) to simulate the light change in the real environment, including the grassland light and shadow under sunlight, the shadow of the ball, etc.; and the dynamic rendering function is implemented by using OpenGL or DirectX graphics programming interfaces to update the position, trajectory and movement state of the golf ball in the projection screen in real time, dynamically adjust the light, texture and shadow effects in the virtual scene, support high-definition and low-latency visual effects, and provide the realism of indoor golf playing. Detailed implementation manners

[0030] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts belong to the scope of protection of the present invention.

[0031] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the embodiments of the present invention. Obviously, the described embodiments are a part of the embodiments of the present invention, rather than all of the embodiments.

[0032] Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts belong to the scope of protection of the present invention.

[0033] In the description of the embodiments of the present invention, it should be noted that the orientation or positional relationship indicated by terms such as "upper", "inner", etc. is the orientation or positional relationship in which the product of the invention is usually placed during use. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0034] In the description of the present invention, it should also be noted that unless otherwise clearly specified and defined, the terms "set", "installed", "connected", and "linked" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0035] The present invention provides a technical solution: an indoor AR golf course binocular high-definition sensing system, including:

[0036] A dynamic capture module, which includes a high-speed camera unit, a lidar unit, and an inertial measurement unit;

[0037] Wherein the high-speed camera unit is used to calculate the position change of the golf ball;

[0038] The lidar unit is used to provide three-dimensional spatial position information of the golf ball;

[0039] The inertial measurement unit is used to monitor the rotation rate and speed change of the golf ball in real time;

[0040] The vibration feedback module is used to simulate the impact and rebound feelings during real hitting.

[0041] The high-speed camera unit adopts the image recognition algorithm of OpenCV to perform differential operations on consecutive frames;

[0042] Operation formula: D(x, y, t) = |I(x, y, t) - I(x, y, t - 1)|), to calculate the position change of the golf ball, where I(x, y, t) represents the pixel value of the image at coordinates (x, y) and time t.

[0043] The lidar unit uses the triangulation principle to provide three-dimensional spatial position information of the golf ball; measurement formula: Z = f × B / d, where Z is the distance, f is the lens focal length, B is the baseline length, and d is the parallax.

[0044] The inertial measurement unit includes an accelerometer and a gyroscope. The accelerometer and the gyroscope are used to collect acceleration and angular velocity data, and use the Kalman filter algorithm to fuse the data to monitor the rotation rate and speed change of the golf ball in real time;

[0045] Algorithm formula:

[0046] The vibration feedback module includes a vibration feedback screen which integrates piezoelectric ceramic materials. According to the preset vibration formula y = Asin(ωt + φ), the vibration frequency and intensity of the piezoelectric ceramic are controlled by an electrical signal to simulate the impact and rebound feelings when hitting a ball.

[0047] It further includes a low - energy consumption mixed reality enhancement module. The low - energy consumption mixed reality enhancement module includes a low - energy consumption LED projection unit and an intelligent sound and light control unit. Among them, the low - energy consumption LED projection unit adjusts the projection screen in real time according to the user's hitting action and the movement trajectory of the ball by using a dynamic rendering algorithm. The intelligent sound and light control unit realizes sound effects matching the hitting action and the movement state of the ball through a preset sound effect library and a light and shadow algorithm, including the sound of hitting the ball, the sound of the ball rolling, and the sound of grass friction; meanwhile, it simulates the change of light in the real environment.

[0048] The dynamic rendering algorithm adopts the OpenGL or DirectX algorithm. According to the user's hitting action and the movement trajectory of the ball, the position, trajectory, and movement state of the golf ball in the projection screen are dynamically updated. Combining with light and shadow algorithms such as the Phong lighting model, the lighting, texture, and shadow effects in the virtual scene are dynamically adjusted to match the user's perspective and hitting action, supporting high - resolution and low - latency visual effects to ensure that users obtain a smooth visual experience. The use of this algorithm can achieve high - efficiency graphics rendering.

[0049] It supports complex light and shadow effects and dynamic scene updates, thus providing an immersive visual experience.

[0050] Working principle: The differential operation is performed on consecutive frames using an image recognition algorithm based on OpenCV to calculate the position change of the golf ball. Through the formula D(x, y, t) = |I(x, y, t) - I(x, y, t - 1)|, the system can quickly detect the position change of the ball in the image, where I(x, y, t) represents the pixel value of the image at coordinates (x, y) and time t. By emitting and receiving laser beams, the three-dimensional spatial position information of the golf ball is provided according to the principle of triangulation. The lidar can accurately measure the distance between the ball and the sensor, thus providing the three-dimensional position data of the ball. An accelerometer and a gyroscope are integrated to collect acceleration and angular velocity data. The Kalman filter algorithm is used to fuse the data to monitor the rotation rate and speed change of the golf ball in real time. The acceleration and angular velocity data provided by the inertial measurement unit can help the system understand the dynamic behavior of the ball, including rotation and speed change. The data of the above three sensors are comprehensively processed through a deep learning algorithm (the combination of convolutional neural network CNN and long short-term memory network LSTM) to achieve high-precision tracking and prediction of the high-speed movement trajectory of the golf ball, with the precision reaching the millimeter level. The deep learning algorithm can learn patterns from a large amount of historical hitting data to improve the accuracy and stability of tracking and prediction. In addition, the vibration feedback screen: piezoelectric ceramic materials are integrated, and the vibration frequency and intensity of the piezoelectric ceramics are controlled by electrical signals according to the preset vibration formula (sine wave y = Asin(ωt + φ) or square wave) to simulate the impact and rebound feelings during hitting. The system dynamically adjusts the vibration parameters according to the hitting force and speed to provide a personalized feedback experience.

[0051] As described above, it is only the preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and all should be covered by the protection scope of the present invention.

Claims

1. An indoor AR golf course binocular high-definition sensing system, characterized in that, Including: A dynamic capture module, which includes a high-speed camera unit, a lidar unit, and an inertial measurement unit; Wherein the high-speed camera unit is used to calculate the position change of the golf ball; The lidar unit is used to provide three-dimensional spatial position information of the golf ball; The inertial measurement unit is used to monitor the rotation rate and speed change of the golf ball in real time; The vibration feedback module is used to simulate the impact and rebound feelings during actual hitting.

2. The binocular high-definition sensing system for an indoor AR golf course according to claim 1, wherein The high-speed camera unit uses the image recognition algorithm of OpenCV to perform differential operations on consecutive frames; Operation formula: D(x, y, t) = |l(x, y, t) - I(x, y, t - 1)|), to calculate the position change of the golf ball, where I(x, y, t) represents the pixel value of the image at coordinates (x, y) and time t.

3. The binocular high-definition sensing system for an indoor AR golf course according to claim 1, characterized in that, The lidar unit uses the triangulation principle to provide three-dimensional spatial position information of the golf ball; measurement formula: Z = f × B / d, where Z is the distance, f is the lens focal length, B is the baseline length, and d is the parallax.

4. An indoor AR golf course binocular high-definition sensing system according to claim 3, characterized in that, The inertial measurement unit includes an accelerometer and a gyroscope. The accelerometer and the gyroscope are used to collect acceleration and angular velocity data, and use the Kalman filter algorithm to fuse the data to monitor the rotation rate and speed change of the golf ball in real time; Algorithm formula:

5. An indoor AR golf course binocular high-definition sensing system according to claim 1, characterized in that, The vibration feedback module includes a vibration feedback screen, and the vibration feedback screen integrates piezoelectric ceramic materials. According to the preset vibration formula y = Asin(ωt + φ), the vibration frequency and intensity of the piezoelectric ceramic are controlled by an electrical signal to simulate the impact and rebound feelings during hitting.

6. The binocular high-definition sensing system for an indoor AR golf course according to claim 1, wherein It also includes a low-power mixed reality enhancement module. The low-power mixed reality enhancement module includes a low-power LED projection unit and an intelligent sound and light control unit. Among them, the low-power LED projection unit adjusts the projection screen in real time according to the user's hitting action and the movement trajectory of the ball using a dynamic rendering algorithm. The intelligent sound and light control unit realizes sounds matching the hitting action and the movement state of the ball, including hitting sounds, ball rolling sounds, and grass friction sounds, through a preset sound effect library and a light and shadow algorithm; At the same time, it simulates the light change in the real environment.

7. The binocular high-definition sensing system for an indoor AR golf course according to claim 6, wherein, The dynamic rendering algorithm uses the OpenGL or DirectX algorithm.