Intelligent golf cart driving system based on visual perception

Abstract

The invention relates to the technical field of golf carts, in particular to an intelligent golf cart driving system based on visual perception, comprising a vehicle-mounted main control board module which is provided with a main control program and a perception algorithm and is used for acquiring a real-time image through a foresight camera and extracting a drivable area on a road by using a computer vision technology; extracting left and right sidelines of a road according to the road travelable area, and calculating a road center line; smoothing the road center line by using a cubic spline curve method to generate a smooth path curve; according to the smooth path curve, uniformly-spaced sparse points are selected as path planning points; calculating control parameters according to the path planning points; the sensing module is used for sending a parking instruction to the vehicle control system according to the parking point marker detected by the foresight camera, and the vehicle stops advancing; according to the invention, the golf cart can autonomously run to the next serving point along the road, so that a ball kid is helped to arrive at the golf cart in the shortest time, and the working efficiency of personnel is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of golf carts, and in particular to an intelligent golf cart driving system based on visual perception. Background Art

[0002] Currently, golf carts primarily achieve autonomous tracking capabilities by adding positioning sensors, such as GPS, UWB, or ultrasonic radar. These methods are easily affected by the surrounding environment, resulting in low positioning accuracy, which in turn affects tracking effectiveness. Furthermore, due to the specific requirements of golf courses, golf carts cannot drive directly on grass and must follow roads. This requires pre-established road maps for positioning sensor-based solutions, increasing development and maintenance costs. Summary of the Invention

[0003] The present invention aims to provide an intelligent golf cart driving system based on visual perception. This system primarily addresses the efficiency issue of a golf cart carrying a player to a designated hitting spot. After the player hits the ball, the caddy must return to the cart's location and then drive the cart along the road to the nearest landing point. By intelligently sensing the drivable area on the road using a forward-looking camera and combining it with the cart's control system, the golf cart can autonomously drive along the road to the next teeing spot, significantly reducing the caddy's return time and helping them reach the cart in the shortest possible time, greatly improving work efficiency.

[0004] The purpose of the present invention can be achieved through the following technical solutions:

[0005] An intelligent golf cart driving system based on visual perception includes the following modules:

[0006] On-board main control board module: This module sets up the main control program and perception algorithm to acquire real-time images from the front-facing camera and use computer vision technology to extract the drivable area of the road. Based on the drivable area, it extracts the left and right sidelines of the road and calculates the road centerline. It uses the cubic spline curve method to smooth the road centerline and generate a smooth path curve. Based on the smooth path curve, it selects sparse points at equal intervals as path planning points. Based on the path planning points, it calculates control parameters.

[0007] Perception module: When the front-view camera detects a parking point marker, it sends a parking command to the vehicle control system, causing the vehicle to stop moving;

[0008] Control module: Golf cart motion control, including steering angle and speed.

[0009] As a further technical solution of the present invention, the process of extracting the drivable area of the road using computer vision technology is as follows:

[0010] Use the pavement color characteristics for adaptive threshold segmentation, and combine the pavement edge detection method and the ROI mask to extract the pavement area.

[0011] As a further technical solution of the present invention: Using computer vision technology to extract the drivable area of the road, the method further includes:

[0012] Construct a convolutional neural network model for pavement semantic segmentation. By collecting a large number of real pavement images of golf courses and performing semantic segmentation annotation on the images, construct training data to train the segmentation model, and finally deploy the segmentation model to edge devices for pavement area extraction.

[0013] As a further technical solution of the present invention: The pavement label is 1, and other backgrounds are 0.

[0014] As a further technical solution of the present invention: Perception module: Set a camera and an ultrasonic radar. Obtain real-time images through the front-view camera, and detect obstacles in front through the ultrasonic radar in front of the vehicle head.

[0015] As a further technical solution of the present invention: It further includes: Power supply module: Used to supply power to the system.

[0016] As a further technical solution of the present invention: It further includes: Start-stop module: The start or stop of the golf cart is triggered by a button on the light control board. When the golf cart is not in the assisted driving state, pressing the button will control the golf cart to start, and when the golf cart has started and entered the assisted driving state, pressing the button again will trigger the golf cart to stop.

[0017] As a further technical solution of the present invention: It further includes: Light control module: Used to control the system indicator light to display different colors.

[0018] As a further technical solution of the present invention: Control the golf cart to move forward when no parking instruction is received, and set the system indicator light to flash red.

[0019] As a further technical solution of the present invention: When the vehicle is in a stopped state, set the system indicator light to green.

[0020] The beneficial effects of the present invention:

[0021] The present invention mainly solves the efficiency problem that when a golf cart carries a person to the designated hitting point, after the player hits the ball, the caddie needs to return to the golf cart point and then drive the golf cart along the road to the nearest ball landing point. Through the front-view camera, the drivable area of the road is intelligently sensed, combined with the golf cart control system, so that the golf cart can autonomously drive along the road to the next tee-off point, thus greatly reducing the time cost of the caddie's return, helping the caddie reach the golf cart in the shortest time, and greatly improving the work efficiency of personnel;

[0022] The present invention adopts camera vision perception technology and combines ultrasonic radar for obstacle avoidance, which can effectively achieve the intelligent driving of a golf cart along a road. Compared with the method based on a positioning sensor, it not only has a lower cost but also higher accuracy, and will not be affected by environmental factors such as trees to affect the positioning accuracy. Since the intelligent driving of the cart is realized, there is no need for a caddie to drive the cart back to the starting point, and the caddie only needs to get on the cart during its forward journey, which greatly improves the work efficiency and service quality of personnel and is also very helpful for improving the operation efficiency of a golf course. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0024] Figure 1 is a flowchart of an intelligent golf cart driving system based on vision perception provided in Embodiment 1 of the present invention;

[0025] Figure 2 is a block diagram of the hardware system of an intelligent golf cart driving system based on vision perception provided in Embodiment 2 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] In order to enable those skilled in the art to better understand the solution of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0027] Embodiment 1

[0028] Figure 1 is a flowchart of an intelligent golf cart driving system based on vision perception provided in Embodiment 1 of the present invention. The control method of the driving system of the golf cart includes the following steps:

[0029] S1. The caddie presses the start button on the cart, and the cart enters the vision intelligent assisted driving function;

[0030] In S1, when the caddie presses the start button on the cart, it is necessary to judge whether the cart starts or stops;

[0031] If the golf cart is in the start state, it enters S2. If the golf cart is in the stop state, the indicator light of the golf cart shows green;

[0032] S2. Obtain the real-time image through the front view camera, and use computer vision technology to extract the drivable area of the road. The following are two optional solutions:

[0033] S21. Use the road surface color characteristics for adaptive threshold segmentation, and combine the road surface edge detection method and the ROI mask to extract the road surface area;

[0034] Exemplarily, image acquisition and preprocessing: Obtain the original RGB image through the front view camera of the golf cart (resolution 1280×720, frame rate 30fps), perform distortion correction (using the pre-calibrated camera internal parameter matrix and distortion coefficients), convert it to the HSV color space, and perform adaptive histogram equalization (CLAHE) on the V channel to enhance the illumination robustness;

[0035] Dynamic color modeling and segmentation: Build a road surface color sample library (including more than 3000 samples under different illumination conditions), calculate the mean value of the current frame V channel in real time, dynamically adjust the HSV threshold range, generate the color mask Mcolor (binary image, with the threshold range satisfied as 255, otherwise 0), and perform morphological operations (7×7 elliptical kernel, first opening operation and then closing operation) to eliminate noise;

[0036] Edge feature extraction and optimization: Convert the original image to a grayscale image, and perform two edge detections in parallel: a) Standard Canny detection (threshold = 30, 90), b) Canny detection after Gaussian blur (Gaussian kernel 9×9, threshold = 15, 30). Edge fusion: Medge = thinning(E1∪E2) (thinning processing to reduce the edge width), generate the edge exclusion mask:

[0037] ROI dynamic generation: Calculate the attention area height ratio according to the real-time vehicle speed v (unit: m / s): η = 0.7 + 0.1×(1 - min(1, v / 5)), construct a trapezoidal ROI, bottom edge: the bottom edge of the image (width, W), top edge: located at the image height H×η, width 0.4W, vertex coordinates: (0, H), (W, H), (0.7W, ηH), (0.3W, ηH); generate the ROI mask Mroi;

[0038] Multi-feature fusion: Preliminary fusion: Mtemp = Mcolor∩Medge_inv∩Mroi, extract the largest connected domain (eliminate isolated noise), use 8-neighborhood connectivity to detect the contour, select the contour Cmax with the largest area, and fill the contour to generate the final road surface area Mroad;

[0039] S22. Build a convolutional neural network model for road surface semantic segmentation, with the road surface label set to 1 and the background labeled to 0. By collecting a large number of real golf course road surface images, performing semantic segmentation and annotation on the images, and building training data to train the segmentation model, the segmentation model is finally deployed to edge devices for road surface area extraction.

[0040] For example, the data acquisition and annotation system uses a camera to capture images of a golf course pavement (resolution ≥ 1920 × 1080). Using an annotation tool, the pavement pixels are labeled as 1, and the background (turf, bunkers, obstacles, etc.) are labeled as 0. A dedicated dataset containing 100,000 annotated images is constructed, with the training / validation / test sets divided in an 8:1:1 ratio.

[0041] Lightweight segmentation model design: Adopting an encoder-decoder structure, the encoder uses MobileNetV3 to extract multi-scale features, and the decoder integrates the ASPP module to enhance multi-scale context perception. The output layer uses 1×1 convolution + Sigmoid activation for binary classification, and the loss function is a joint optimization of Dice Loss and BCE.

[0042] Edge deployment optimization method: compress the model to <2MB through channel pruning, implement quantization-aware training to achieve FP16 precision storage, develop a TensorRT inference engine adapter module, and deploy it to Jetson Nano edge devices to achieve real-time inference (>15fps);

[0043] S3. Extract the left and right side lines of the road based on the drivable area of the road surface and calculate the center line of the road;

[0044] For example, the left and right sideline extraction of the road: the drivable area is binarized, and a semantic segmentation model (such as UNet) is used to output a road surface area probability map. A threshold (such as ≥0.8) is set to generate a binary mask: the road surface area is 1 and the background is 0; the left and right sideline point set extraction method: the vertical scanning method: scans the image column by column along the horizontal direction (X-axis), and records the coordinates of the leftmost and rightmost non-zero points in each column; among them, noise filtering: a sliding window (window width ≥5 pixels) is used to eliminate sudden changes and retain continuous trajectories; the sideline smoothing method uses the least squares method to fit a polynomial curve (3rd order is recommended);

[0045] Road centerline calculation: Apply the Zhang-Suen thinning algorithm to the binary mask M to obtain a single-pixel wide skeleton line. Remove short branches less than 10 pixels in length, retain the main skeleton, connect the skeleton points into a polyline based on the adjacency relationship, and use the Savitzky-Golay filter to smooth the node coordinates.

[0046] S4. Smoothing the road centerline using a cubic spline curve method to generate a smooth path curve;

[0047] Exemplarily, the segmented processing detects the curvature mutation points of the road center line (threshold k > 0.3m -1 ), divides the path into 3 independent segments for fitting and then C 2 continuously splices; increases the control point density for the high-curvature segment (spacing compressed to 0.5m), introduces a curvature penalty term to optimize the objective function, lateral deviation: < 0.12m (42% lower than the Bezier curve), curvature continuity: G 2 continuous (acceleration step < 0.5m / s 3 );

[0048] S5. According to the smooth path curve, select equally spaced sparse points as path planning points;

[0049] Exemplarily, parametric curve: If the original path curve is a series of discrete points, we usually represent the curve in a parametric way first (such as cubic spline interpolation) to calculate the arc length and equidistant sampling.

[0050] Calculate the total arc length: Set n points on the path curve, calculate the Euclidean distance between adjacent points, and accumulate to get the total arc length;

[0051] Determine the sampling interval d: Determine the sampling interval d according to actual needs. Note: d cannot be greater than the total arc length;

[0052] Equi-arc-length sampling: Starting from the starting point along the curve, take a point every arc length d until the end point. Since the curve is represented by discrete points, interpolation (linear interpolation or spline interpolation) is performed between adjacent points to accurately find the points whose arc lengths are multiples of d.

[0053] S6. According to the path planning points, calculate the control parameters, control the ball car to move forward along the road center line, control the ball car to move forward when no parking instruction is received, and set the system indicator light to flash red;

[0054] Exemplarily, coordinate transformation: Transform the path points from the global coordinate system to the vehicle coordinate system (with the center of the rear axle of the vehicle as the origin and the head direction as the positive x-axis direction); assume the position of the vehicle in the global coordinate system is (xv, yv), and the heading angle is θv (the angle between the head direction and the global x-axis). Transform the path point (xi, yi) to the vehicle coordinate system:

[0055] Determine the preview point: In the vehicle coordinate system, find the point on the path that is closest to the vehicle, and then search forward from this point to find the first point whose distance from the vehicle is greater than or equal to the preview distance L as the preview point;

[0056] Calculate the curvature: In the vehicle coordinate system, if the coordinates of the preview point are (gx, gy), then the distance from the vehicle to the preview point is According to the pure pursuit algorithm, the calculation formula for the curvature κ is as follows:

[0057] Calculate the steering angle: According to the vehicle kinematic model, the relationship between the front wheel steering angle δ and the curvature is: δ = arctan(κ * wb); where wb is the wheelbase of the vehicle;

[0058] Speed control: Adjust the speed based on the curvature of the path, v = vmax * e -a*|k| , where vmax is the maximum allowable speed and a is the influence coefficient of the curvature on the speed

[0059] S7. Detect obstacles ahead according to the ultrasonic radar installed in front of the vehicle head. When there are obstacles, send a stop command to the vehicle control system and the vehicle stops moving forward; when there are no obstacles, control the golf cart to move forward along the road center line, control the golf cart to move forward without receiving a stop command, and set the system indicator light to flash red;

[0060] S8. When receiving a stop command triggered by the emergency stop button on the golf cart, control the vehicle to stop moving forward;

[0061] S9. When the front-view camera detects the parking point marker, send a stop command to the vehicle control system and the vehicle stops moving forward;

[0062] Exemplarily, image acquisition and preprocessing: The high-definition camera collects the front road image in real time at a fixed frame rate (such as 30fps), performs Gaussian filtering on the image to eliminate noise, and enhances the contrast through histogram equalization;

[0063] Parking marker detection: Use algorithms such as convolutional neural network (CNN) or YOLO to detect the parking marker in the image, output the bounding box coordinates and confidence, and calculate the distance d = (f * wr) / wp according to the marker pixel width (wp) and the known actual width (wr); where f is the focal length;

[0064] Decision logic: When the marker confidence > 90% and the distance ≤ 5 meters, it is determined as a valid parking point. If the vehicle speed is too high (such as > 10km / h), the system calculates the deceleration curve in advance to ensure a smooth stop;

[0065] Instruction generation and execution: Generate a CAN bus protocol data packet containing the target position and deceleration acceleration (such as -2m / s 2 ); Vehicle control response: Send an instruction to the braking system through the controller area network (CAN), and at the same time turn off the power output;

[0066] S10. In an emergency, the caddy presses the emergency stop button and the vehicle stops moving forward;

[0067] S11. After the golf cart stops at the parking point, the caddy presses the start button again, and the golf cart re-enters the visual intelligent assisted driving function and automatically drives to the next parking point;

[0068] In the above situation, when the vehicle is in a stopped state, the system indicator light is set to green.

[0069] Embodiment 2

[0070] Figure 2 The following is a hardware system block diagram of an intelligent golf cart driving system based on visual perception provided by Embodiment 2 of the present invention. The driving system includes:

[0071] Power supply module: used to supply power to the system;

[0072] Vehicle-mounted main control board module: Set the main control program and perception algorithm, used to obtain real-time images through the front-view camera, and use computer vision technology to extract the drivable area of the road; Extract the left and right sidelines of the road according to the drivable area of the road surface, and calculate the center line of the road; Use the cubic spline curve method to smooth the center line of the road to generate a smooth path curve; According to the smooth path curve, select equally spaced sparse points as path planning points; According to the path planning points, calculate the control parameters;

[0073] Perception module: Set the camera and ultrasonic radar, obtain real-time images through the front-view camera, detect obstacles in front through the ultrasonic radar in front of the vehicle head; And when the parking point marker is detected by the front-view camera, send a parking instruction to the vehicle control system, and the vehicle stops moving forward;

[0074] Control module: Control the movement of the golf cart, including the steering angle and speed;

[0075] Start-stop module: The start or stop of the golf cart is triggered by a button on the light control board. When the golf cart is not in the assisted driving state, pressing the button will control the golf cart to start, and when the golf cart has started and entered the assisted driving state, pressing the button again will trigger the golf cart to stop;

[0076] Light control module: used to control the system indicator light to display different colors.

[0077] The above has described an embodiment of the present invention in detail, but the above content is only a preferred embodiment of the present invention and cannot be considered as limiting the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of the application of the present invention should still fall within the scope covered by the patent of the present invention.

Claims

1. An intelligent golf cart driving system based on visual perception, characterized in that, The driving system includes the following modules: On-vehicle main control board module: Set the main control program and perception algorithm, which are used to obtain real-time images through the front-view camera, and use computer vision technology to extract the drivable area of the road; extract the left and right edges of the road according to the drivable area of the road surface, and calculate the center line of the road; use the cubic spline curve method to smooth the center line of the road to generate a smooth path curve; select equally spaced sparse points as path planning points according to the smooth path curve; calculate control parameters according to the path planning points; Perception module: When the parking point marker is detected by the front-view camera, send a parking instruction to the vehicle control system, and the vehicle stops moving forward; Control module: Control the movement of the golf cart, including the steering angle and speed.

2. The intelligent golf cart driving system based on visual perception according to claim 1, wherein The process of using computer vision technology to extract the drivable area of the road is as follows: Use the pavement color characteristics for adaptive threshold segmentation, and combine the pavement edge detection method and the ROI mask to extract the pavement area.

3. The intelligent golf cart driving system based on visual perception according to claim 1, characterized in that, Using computer vision technology to extract the drivable area of the road also includes the following methods: Construct a convolutional neural network model for pavement semantic segmentation. By collecting a large number of real pavement images of the golf course and performing semantic segmentation annotation on the images, construct training data to train the segmentation model, and finally deploy the segmentation model to the edge device for pavement area extraction.

4. The intelligent golf cart driving system based on visual perception according to claim 3, characterized in that, The pavement label is 1, and other backgrounds are 0.

5. The intelligent golf cart driving system based on visual perception according to claim 1, wherein Perception module: Set the camera and ultrasonic radar. Obtain real-time images through the front-view camera, and detect obstacles ahead through the ultrasonic radar in front of the vehicle head.

6. The intelligent golf cart driving system based on visual perception according to claim 1, characterized in that, It also includes: Power supply module: Used to supply power to the system.

7. An intelligent golf cart driving system based on visual perception according to claim 1, characterized in that It also includes: Start-stop module: The start or stop of the golf cart is triggered by the button on the light control board. When the golf cart is not in the assisted driving state, pressing the button will control the golf cart to start. When the golf cart has started and entered the assisted driving state, pressing the button again will trigger the golf cart to stop.

8. The intelligent golf cart driving system based on visual perception according to claim 7, characterized in that, It also includes : Light control module: This module contains a single-chip microcomputer subsystem. This subsystem establishes a connection with the main control board through the serial port. The main control sends different instructions to the single-chip microcomputer through the serial port, and the single-chip microcomputer controls the indicator light to display different colors.

9. The intelligent golf cart driving system based on visual perception according to claim 8, characterized in that, Control the golf cart to move forward without receiving a parking instruction, and set the system indicator light to flash red.

10. The intelligent golf cart driving system based on visual perception according to claim 8, characterized in that, When the vehicle is in a stopped state, set the system indicator light to green.

Images