Intelligent golf cart based on GPS and vision fusion

Abstract

The invention relates to the technical field of golf carts, in particular to an intelligent golf cart based on GPS and vision fusion, which is characterized in that an APP is opened, a two-dimensional code on the cart is scanned, and connection is established; a start button on the APP is clicked, and a GPS and visual intelligent following mode is entered; the golf cart receives the GPS position sent by the APP and calculates the distance between the golf cart and the APP according to the GPS position of the golf cart; when the distance is greater than a set threshold value, calculating a point position closest to the kid on the road and a track from the kid to the point position according to a road map, the GPS position of the kid and the GPS position of the kid; extracting a road drivable area; extracting left and right sidelines of a road according to the road travelable area, and calculating a road center line; smoothing the road center line by using a cubic spline curve method to generate a smooth path curve; according to the method, the kid can get on the kid when the kid runs to the position closest to the kid, so that a large amount of useless time is saved, and the working efficiency is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of golf carts, and in particular to an intelligent golf cart based on the fusion of GPS and vision. Background Art

[0002] Currently, golf carts primarily achieve autonomous following by adding positioning tags or using visual tracking. The positioning tag method requires both the installation of a positioning receiver on the cart and the need for a person to carry the positioning tag and transmit positioning data in real time, resulting in a high overall hardware cost. Visual tracking, on the other hand, is suitable for close-range tracking but carries the risk of loss and mistracking, and places high demands on the algorithm. Furthermore, due to the golf course's inherent requirements for the site, carts cannot travel directly on grass and must instead follow roads. This necessitates the pre-establishment of road map information based on positioning tags, increasing development and maintenance costs. Visual tracking, on the other hand, is limited by the tracking distance, and the risk of mistracking increases with distance. Summary of the Invention

[0003] The present invention aims to provide an intelligent golf cart based on the fusion of GPS and vision. This invention primarily addresses the problem of a golf cart intelligently following a caddie on a course while being unable to enter the course. This ensures that the golf cart's position on the road is always closest to the caddie. As the caddie walks on the course, the cart calculates the shortest point on the map from the caddie based on its own position, the caddie's position, and a road map, and then autonomously follows the road. This significantly reduces the time it takes for the caddie to return to the cart, helping the caddie reach the advancing cart in the shortest possible time, thereby effectively improving both personnel efficiency and the operational efficiency of the course.

[0004] The purpose of the present invention can be achieved through the following technical solutions:

[0005] A smart golf cart based on GPS and vision fusion, and a wireless remote control method for the smart golf cart, comprising the following steps:

[0006] S1. Open the app and scan the QR code on the caddy to establish a connection. Click the start button on the app to enter GPS and visual intelligent follow mode. The caddy receives the GPS location sent by the app and calculates the distance between the two based on the caddy's GPS location. When the distance is greater than the set threshold, the caddy calculates the nearest point on the road to the caddy and the caddy's trajectory to that point based on the road map, the caddy's GPS location, and the caddy's GPS location.

[0007] S2, obtain real-time images through the front-view camera and extract the drivable area of ​​the road;

[0008] S3. Extract the left and right side lines of the road based on the drivable area of ​​the road surface and calculate the center line of the road;

[0009] S4. Smoothing the road centerline using a cubic spline curve method to generate a smooth path curve;

[0010] S5. According to the smooth path curve, select sparse points with equal intervals as path planning points.

[0011] As a further technical solution of the present invention: in S2, a real-time image is acquired by a front-view camera, and a drivable area of ​​the road is extracted using computer vision technology, including the following method:

[0012] Adaptive threshold segmentation is performed using road color characteristics, combined with road edge detection methods and ROI masks to extract road areas.

[0013] As a further technical solution of the present invention: in S2, a real-time image is acquired by a front-view camera, and a drivable area of ​​the road is extracted using computer vision technology, further comprising the following method:

[0014] A convolutional neural network model is constructed for road surface semantic segmentation. By collecting a large number of real golf course road surface images, semantically segmenting and annotating the images, training data is constructed to train the segmentation model. Finally, the segmentation model is deployed to edge devices for road surface area extraction.

[0015] As a further technical solution of the present invention: the road surface label is 1, and other backgrounds are 0.

[0016] As a further technical solution of the present invention, the following steps are also included:

[0017] S6. Calculate control parameters based on the path planning points and control the trolley to move forward along the center line of the road.

[0018] As a further technical solution of the present invention: in S6, the trolley is controlled to move forward without receiving a stop instruction, and the system indicator light is set to flash red.

[0019] As a further technical solution of the present invention, the following steps are also included:

[0020] S7. The ultrasonic radar installed in front of the vehicle detects obstacles ahead. When an obstacle is detected, a stop command is sent to the vehicle control system, and the vehicle stops moving forward.

[0021] As a further technical solution of the present invention: in S7, when there is no obstacle, the golf cart is controlled to move forward along the center line of the road, and when no stop instruction is received, the golf cart is controlled to move forward, and the system indicator light is set to flash red.

[0022] As a further technical solution of the present invention, the following steps are also included:

[0023] S8. Determine whether the trolley has reached the set GPS point, and if so, stop.

[0024] Beneficial effects of the present invention:

[0025] The present invention primarily achieves the autonomous following function of a golf cart through the fusion of GPS and vision technology. The cart is equipped with a differential GPS locator, which can sense its own position in real time. Before autonomously following, the cart circles the golf course and records the road map information. To use the cart, the caddy opens the app and scans a code to connect to the cart's Wi-Fi hotspot, establishing a connection with the cart. The app then transmits the caddy's GPS location to the cart's main control unit, allowing the cart to sense the caddy's real-time location. When the distance between the cart and the caddy exceeds a set threshold, the cart calculates the shortest point on the map from the caddy, as well as the trajectory to that point, based on its own position, the caddy's position, and a road map. To ensure more accurate navigation of the cart on the road, the present invention also utilizes a forward-looking camera mounted on the roof of the cart's front shed to detect the road ahead. Computer vision technology is used to extract the drivable area of ​​the road surface, and the road centerline is calculated based on this area. This generates path planning points. Combined with the cart's control system, the cart is controlled to follow the road centerline and autonomously stop when it reaches a set point. Based on this system, the caddy does not need to return to the starting point of the golf cart. Instead, he can get on the cart when the cart reaches the position closest to him, thus saving a lot of useless time and greatly improving work efficiency.

[0026] This invention uses GPS integrated with visual perception technology to enable a golf cart to autonomously follow a caddie on the road. The caddie does not require any operation on the course. The cart can autonomously plan a route based on the caddie's real-time location information, intelligently perceive road information, and intelligently drive along the road to the optimal point on the road from the caddie. This greatly enhances the intelligence level of golf courses. The caddie no longer needs to return to the starting point to drive the cart; they only need to get on the cart midway in the direction of the cart's forward movement. With the support of this intelligent following system, the intelligent golf cart will effectively improve personnel work efficiency and service quality, and will also greatly help improve the operational efficiency of golf courses. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 This is a flow chart of a smart golf cart based on GPS and vision fusion provided by the first embodiment of the present invention;

[0029] Figure 2 This is a hardware system block diagram of an intelligent golf cart based on GPS and vision fusion provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0031] Example 1

[0032] Figure 1 This is a flow chart of a smart golf cart based on GPS and vision fusion provided in Example 1 of the present invention. The smart golf cart mainly uses GPS and vision fusion technology to achieve the function of autonomous golf cart following. The autonomous golf cart following includes the following steps:

[0033] S1. The caddy opens the app and scans the QR code on the caddy to establish a connection. Click the start button on the app to enter GPS and visual intelligent follow mode. The caddy receives the caddy's GPS location from the app and calculates the distance between the two based on the caddy's GPS location. When the distance is greater than the set threshold, the caddy calculates the nearest point on the road to the caddy and the caddy's trajectory to that point based on the road map, the caddy's GPS location, and the caddy's GPS location.

[0034] Exemplarily, the APP establishes communication with the caddy and starts the follow mode;

[0035] Get the GPS coordinates of the caddy and the cart, and use the Haversine formula to calculate the great circle distance between the two points.

[0036] If the distance is less than or equal to the threshold, the caddy continues to wait or operates according to the previous mode;

[0037] If the distance is greater than the threshold, calculate the point on the road closest to the caddy. Specifically, represent the road map as a series of line segments (line segments composed of adjacent road points), traverse each line segment, calculate the shortest distance from the caddy's position to each line segment, and record the projection point corresponding to the shortest distance. Compare the shortest distances of all line segments and take the smallest corresponding projection point as the caddy's closest point P on the road.

[0038] Plan the trajectory of the golf cart to point P: Find the path point Q closest to the golf cart on the road map. Since the road is continuous, the golf cart may not be on the road node at the moment, so it is necessary to first project point Q onto the road and obtain the path from point Q to point P. The road map is ordered (for example, from the starting point to the end point), so the direction needs to be determined. If point Q is after point P (i.e., Q is behind P according to the road order), the golf cart waits in place.

[0039] S2: Use the front-facing camera to obtain real-time images and use computer vision technology to extract the drivable area of ​​the road. There are two options:

[0040] S21, using the road surface color characteristics to perform adaptive threshold segmentation, and combining the road surface edge detection method and ROI mask to extract the road surface area;

[0041] For example, image acquisition and preprocessing: the original RGB image is acquired through the caddy's front-view camera (resolution 1280×720, frame rate 30fps), distortion correction is performed (using the pre-calibrated camera intrinsic parameter matrix and distortion coefficient), converted to the HSV color space, and adaptive histogram equalization (CLAHE) is performed on the V channel to enhance lighting robustness;

[0042] Dynamic color modeling and segmentation: Build a road color sample library (containing over 3,000 samples under different lighting conditions), calculate the V channel mean of the current frame in real time, dynamically adjust the HSV threshold range, generate a color mask Mcolor (a binary image, 255 if the threshold is met, otherwise 0), and perform morphological operations (opening and closing a 7×7 ellipse kernel) to eliminate noise;

[0043] Edge feature extraction and optimization: Convert the original image to grayscale and perform two edge detection methods in parallel: a) standard Canny detection (threshold = 30, 90), b) Canny detection after Gaussian blur (Gaussian kernel 9×9, threshold = 15, 30) Edge fusion: Medge = thinning (E1∪E2) (thinning process reduces edge width) to generate an edge exclusion mask:

[0044] Dynamic ROI generation: Calculate the height ratio of the region of interest based on the vehicle's real-time speed v (unit: m / s): η = 0.7 + 0.1 × (1-min(1, v / 5)), construct a trapezoidal ROI with the bottom edge being the image bottom edge (width, W), the top edge being located at the image height H × η, with a width of 0.4W, and vertex coordinates being (0, H), (W, H), (0.7W, ηH), and (0.3W, ηH); generate the ROI mask Mroi;

[0045] Multi-feature fusion: Preliminary fusion: Mtemp = Mcolor ∩ Medge_inv ∩ Mroi, extract the largest connected domain (eliminate isolated noise), use 8-neighborhood connectivity to detect contours, select the contour with the largest area Cmax, fill the contour to generate the final road surface area Mroad;

[0046] S22. Build a convolutional neural network model for road surface semantic segmentation, with the road surface label set to 1 and the background labeled to 0. By collecting a large number of real golf course road surface images, performing semantic segmentation and annotation on the images, and building training data to train the segmentation model, the segmentation model is finally deployed to edge devices for road surface area extraction.

[0047] For example, the data collection and annotation system uses a camera to capture images of a golf course pavement (resolution ≥ 1920 × 1080). Using annotation tools, the pavement pixels are labeled as 1, and the background (turf, bunkers, obstacles, etc.) are labeled as 0. A dedicated dataset containing 100,000 annotated images is constructed, with the training / validation / test sets divided in an 8:1:1 ratio.

[0048] Lightweight segmentation model design: Adopting an encoder-decoder structure, the encoder uses MobileNetV3 to extract multi-scale features, and the decoder integrates the ASPP module to enhance multi-scale context perception. The output layer uses 1×1 convolution + Sigmoid activation to achieve binary classification, and the loss function is a joint optimization of Dice Loss and BCE.

[0049] Edge deployment optimization method: compress the model to <2MB through channel pruning, implement quantization-aware training to achieve FP16 precision storage, develop a TensorRT inference engine adapter module, and deploy it to Jetson Nano edge devices to achieve real-time inference (>15fps);

[0050] S3. Extract the left and right side lines of the road based on the drivable area of ​​the road surface and calculate the center line of the road;

[0051] For example, the left and right sideline extraction of the road: the drivable area is binarized, and a semantic segmentation model (such as UNet) is used to output a road surface area probability map. A threshold (such as ≥0.8) is set to generate a binary mask: the road surface area is 1 and the background is 0; the left and right sideline point set extraction method: the vertical scanning method: scans the image column by column along the horizontal direction (X-axis), and records the coordinates of the leftmost and rightmost non-zero points in each column; among them, noise filtering: a sliding window (window width ≥5 pixels) is used to eliminate sudden changes and retain continuous trajectories; the sideline smoothing method uses the least squares method to fit a polynomial curve (3rd order is recommended);

[0052] Road centerline calculation: Apply the Zhang-Suen thinning algorithm to the binary mask M to obtain a single-pixel wide skeleton line. Remove short branches less than 10 pixels in length, retain the main skeleton, connect the skeleton points into a polyline based on the adjacency relationship, and use the Savitzky-Golay filter to smooth the node coordinates.

[0053] S4. Smoothing the road centerline using a cubic spline curve method to generate a smooth path curve;

[0054] For example, the segmented processing detects the curvature mutation point of the road centerline (threshold k>0.3m -1 ), the path is divided into 3 segments and then fitted independently C 2 Continuous splicing; increase the control point density for high curvature segments (spacing is compressed to 0.5m), introduce curvature penalty term to optimize the objective function, lateral deviation: <0.12m (42% lower than Bezier curve), curvature continuity: G 2 Continuous (acceleration step <0.5m / s 3 );

[0055] S5. Select equally spaced sparse points as path planning points based on the smooth path curve;

[0056] For example, parameterized curve: If the original path curve is a series of discrete points, we usually first represent the curve in a parameterized way (such as cubic spline interpolation) to calculate arc length and equidistant sampling.

[0057] Calculate the total arc length: Set n points on the path curve, calculate the Euclidean distance between two adjacent points, and accumulate them to get the total arc length;

[0058] Determine the sampling interval d: Determine the sampling interval d according to actual needs. Note: d cannot be greater than the total arc length;

[0059] Equal arc length sampling: Starting from the starting point, we sample a point every arc length d along the curve until the end point. Since the curve is represented by discrete points, we interpolate between two adjacent points (linearly or according to spline interpolation) to accurately find points whose arc length is a multiple of d.

[0060] S6. Calculate control parameters based on the path planning points, control the trolley to move forward along the center line of the road, and control the trolley to move forward if no stop command is received, and set the system indicator light to flash red;

[0061] For example, coordinate transformation: transform the path point from the global coordinate system to the vehicle coordinate system (with the center of the vehicle's rear axle as the origin and the vehicle's front direction as the positive x-axis); let the vehicle's position in the global coordinate system be (xv, yv) and the heading angle be θv (the angle between the vehicle's front direction and the global x-axis). Transform the path point (xi, yi) to the vehicle coordinate system:

[0062] Determine the preview point: In the vehicle coordinate system, find the point closest to the vehicle from the path points, then search forward from this point to find the first point whose distance from the vehicle is greater than or equal to the preview distance L, which is used as the preview point;

[0063] Calculate curvature: In the vehicle coordinate system, the coordinates of the preview point are (gx, gy), then the distance from the vehicle to the preview point is According to the pure tracking algorithm, the curvature κ is calculated as:

[0064] Calculate the steering angle: According to the vehicle kinematic model, the relationship between the front wheel steering angle δ and the curvature is: δ = arctan(κ*wb); where wb is the vehicle's wheelbase;

[0065] Speed ​​control: speed adjustment based on the curvature of the path, v = vmax*e -a*|k| , where vmax is the maximum allowable speed and a is the coefficient of curvature on speed

[0066] S7. The ultrasonic radar installed at the front of the vehicle detects obstacles ahead. When an obstacle is detected, a stop command is sent to the vehicle control system, causing the vehicle to stop moving forward. When there is no obstacle, the vehicle is controlled to move forward along the center line of the road. If no stop command is received, the vehicle is controlled to move forward and the system indicator light is set to flash red.

[0067] S8, determining whether the golf cart has reached the set GPS point, and if so, stopping;

[0068] In the above situation, when the vehicle is in a stopped state, the system indicator light is set to green.

[0069] Example 2

[0070] Figure 2 This is a hardware system block diagram of a smart golf cart based on GPS and vision fusion provided by the second embodiment of the present invention. The smart golf cart system mainly includes:

[0071] Power module: used to power the system;

[0072] On-board main control board module: This module sets up the main control program and perception algorithm to acquire real-time images from the front-facing camera and use computer vision technology to extract the drivable area of ​​the road. Based on the drivable area, it extracts the left and right sidelines of the road and calculates the road centerline. It uses the cubic spline curve method to smooth the road centerline and generate a smooth path curve. Based on the smooth path curve, it selects sparse points at equal intervals as path planning points. Based on the path planning points, it calculates control parameters.

[0073] Perception module: It is equipped with a camera and ultrasonic radar. The front-facing camera acquires real-time images, and the ultrasonic radar in front of the vehicle detects obstacles ahead.

[0074] Control module: golf cart motion control, including steering angle and speed;

[0075] Wireless module: The mobile app and the vehicle's mainboard transmit data directly via Wi-Fi. Open the app, scan the QR code on the vehicle, and establish a connection. Click the start button on the app to enter GPS and visual intelligent follow mode. The caddy receives the GPS location sent by the app and calculates the distance between the two based on the vehicle's GPS location. When the distance is greater than a set threshold, the nearest point on the road to the caddy is calculated based on the road map, the caddy's GPS location, and the caddy's GPS location, as well as the caddy's trajectory to that point.

[0076] Interaction module: used to control the system indicator lights to display different colors and receive emergency stop switch buttons.

[0077] The above is a detailed description of an embodiment of the present invention. However, the content described is only a preferred embodiment of the present invention and should not be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. An intelligent golf cart based on GPS and vision fusion, characterized in that: The wireless remote control method of the intelligent golf cart includes the following steps: S1. Open the app and scan the QR code on the caddy to establish a connection. Click the start button on the app to enter GPS and visual intelligent follow mode. The caddy receives the GPS location sent by the app and calculates the distance between the two based on the caddy's GPS location. When the distance is greater than the set threshold, the caddy calculates the nearest point on the road to the caddy and the caddy's trajectory to that point based on the road map, the caddy's GPS location, and the caddy's GPS location. S2, obtain real-time images through the front-view camera and extract the drivable area of ​​the road; S3. Extract the left and right side lines of the road based on the drivable area of ​​the road surface and calculate the center line of the road; S4. Smoothing the road centerline using a cubic spline curve method to generate a smooth path curve; S5. According to the smooth path curve, select sparse points with equal intervals as path planning points.

2. The smart golf cart based on GPS and vision fusion according to claim 1, characterized in that: In S2, real-time images are acquired through the front-view camera, and computer vision technology is used to extract the drivable area of ​​the road, including the following methods: Adaptive threshold segmentation is performed using road color characteristics, combined with road edge detection method and ROI mask to extract road area.

3. The smart golf cart based on GPS and vision fusion according to claim 1, characterized in that: In S2, real-time images are acquired through the front-view camera, and computer vision technology is used to extract the drivable area of ​​the road. The following methods are also included: A convolutional neural network model is constructed for road surface semantic segmentation. By collecting a large number of real golf course road surface images, semantically segmenting and annotating the images, training data is constructed to train the segmentation model. Finally, the segmentation model is deployed to edge devices for road surface area extraction.

4. The smart golf cart based on GPS and vision fusion according to claim 3, characterized in that: The road surface label is 1, and other background labels are 0.

5. The smart golf cart based on GPS and vision fusion according to claim 1, characterized in that: The following steps are also included: S6. Calculate control parameters based on the path planning points and control the trolley to move forward along the center line of the road.

6. The smart golf cart based on GPS and vision fusion according to claim 5, characterized in that: In S6, control the caddy forward without receiving a stop command and set the system indicator light to flash red.

7. The smart golf cart based on GPS and vision fusion according to claim 1, characterized in that: The following steps are also included: S7. The ultrasonic radar installed in front of the vehicle detects obstacles ahead. When an obstacle is detected, a stop command is sent to the vehicle control system, and the vehicle stops moving forward.

8. The smart golf cart based on GPS and vision fusion according to claim 7, characterized in that: In S7, when there are no obstacles, the golf cart is controlled to move forward along the center line of the road. If no stop command is received, the golf cart is controlled to move forward and the system indicator light is set to flash red.

9. The smart golf cart based on GPS and vision fusion according to claim 1, characterized in that: The following steps are also included: S8. Determine whether the trolley has reached the set GPS point, and if so, stop.

Images