Green detection methods, equipment, and storage media based on object feature recognition

Abstract

This application discloses a green detection method, device, and storage medium based on object feature recognition, including the following steps: when a user's club-taking action is detected, image data and point cloud data are acquired; the texture features of the club are acquired based on the image data, and the geometric features of the club are acquired based on the point cloud data; the geometric features and texture features are concatenated to obtain a comprehensive feature vector, and the club type is determined based on the comprehensive feature vector; if the club type matches a preset target club type, it is confirmed that the user has reached the green area, and a stop-follow command is sent to control the golf cart to brake. This application determines the user's hitting area by the type of club the user takes out. When the user is in the green area, the golf cart is controlled to stop following to avoid damaging the green.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a green detection method, device and storage medium based on object feature recognition. Background Technology

[0002] According to the rules of golf, equipment such as golf carts are prohibited from entering the green area to avoid disrupting the smoothness of the green surface. Because green grass is typically meticulously mowed and maintained, it exhibits distinct appearance characteristics compared to other areas, such as shorter grass and a more uniform green color. Greens are usually distinguished from other areas by features such as grass color (green) and texture (uniform grain). However, the height, density, and color of the grass between the green and fairway typically transition gradually rather than forming a clear boundary. For example, green grass may be shorter and greener, while fairway grass may be slightly longer or lighter in color. Traditional algorithms based on color thresholds or texture segmentation struggle to accurately pinpoint these transitional boundaries, especially under uneven lighting or shaded conditions, where color distortion further exacerbates misjudgments, making it impossible to correctly determine whether a user has entered the green area.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a green detection method, device and storage medium based on object feature recognition, which aims to solve the technical problem of how to ensure the security and integrity of the original data when performing data quality checks.

[0005] To address the aforementioned problems, this application provides a green detection method based on object feature recognition, the method comprising: When the user's lever-taking action is detected, image data and point cloud data are acquired; The texture features of the cue stick are obtained from the image data, and the geometric features of the cue stick are obtained from the point cloud data. The geometric features and the texture features are concatenated to obtain a comprehensive feature vector, and the club type is determined based on the comprehensive feature vector. If the club type matches the preset target club type, confirm arrival at the green area and send a stop following command to control the golf bag cart to brake.

[0006] In one embodiment, the method further includes the following step before generating acquisition instructions based on the user's lever action and acquiring depth maps and color images: When the distance between the user and the ball bag in the point cloud data is less than a preset distance, and the human confidence score obtained based on the image data is greater than a preset confidence score threshold, the action detection process is triggered. Determine whether the user has made a hand-raising motion and whether the cabin door of the ball bag is open; If the hand lifting action occurs and the hatch is in the open state, then the stick retrieval action is confirmed to have occurred.

[0007] In one embodiment, the step of determining whether the user has raised their hand includes: Continuous point cloud frames are acquired using a TOF sensor, and hand feature points are extracted from these continuous point cloud frames using a key point extraction algorithm. Obtain the displacement vector of the hand feature points from the current frame to the next frame; If the change in the vertical direction of the displacement vector is greater than a preset change threshold, then the hand lifting action is confirmed to exist.

[0008] In one embodiment, the step of determining that the cabin door of the ball bag vehicle is in an open state includes: The image captured by the RGB camera is input into a pre-trained semantic segmentation model to obtain the pixel-level mask of the hatch; The area of ​​the opening / closing region is determined based on the non-zero pixels in the pixel-level mask. The opening / closing ratio is determined based on the ratio of the area of ​​the opening / closing area to the area of ​​the entire door. If the opening / closing ratio is greater than a preset ratio threshold, the door is determined to be in the open state.

[0009] In one embodiment, the steps of obtaining the texture features of the golf club based on the image data, obtaining the geometric features of the golf club based on the point cloud data, and concatenating the geometric features and the texture features to obtain a comprehensive feature vector include: Texture features are extracted from the image data using a texture analysis algorithm; The shape and length features of the point cloud data are extracted based on the edge detection algorithm; The texture features, shape features, and length features are transformed and concatenated using linear discriminant analysis to obtain the comprehensive feature vector.

[0010] In one embodiment, the step of determining the cue type based on the comprehensive feature vector includes: The category score is determined by the sum of the product of the weight matrix and the eigenvector and the bias vector. The category scores are converted into a probability distribution, and the category with the highest probability is taken as the club type.

[0011] In one embodiment, after the step of concatenating the geometric features and the texture features to obtain a comprehensive feature vector, and determining the cue type based on the comprehensive feature vector, the method further includes: According to the preset target detection algorithm, the club is identified in the point cloud data to obtain the club bounding box, and the target point cloud data of the corresponding area is obtained based on the club bounding box; Obtain the normal vector of each point in the target point cloud data, perform cluster analysis on the normal vector, and identify the cylindrical point cloud region with the same normal vector direction as the grip end; Obtain the local density of the target point cloud data, and identify the high-density area as the pole head; If the distance between the grip and the clubhead is not within the standard length range of a putter, then the preset club type identification model is incrementally trained based on the abnormal data.

[0012] In one embodiment, after the step of acquiring image data and point cloud data upon detecting a user's lever movement, the method further includes: The point cloud data is classified according to a clustering algorithm to obtain multiple clusters; The occluded area is determined based on the density and geometric characteristics of the cluster; The point cloud data of the unoccluded area is fitted according to the preset fitting algorithm to generate a fitting curve; The fitted curve is extended to the occluded area to generate point cloud data of the occluded area.

[0013] In addition, to achieve the above objectives, this application also proposes a green detection device based on object feature recognition, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the green detection method based on object feature recognition as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the green detection method based on object feature recognition as described above.

[0015] This application provides a green detection method based on object feature recognition. It extracts texture features from image data and geometric features from point cloud data, combining the two for club type identification. Image data helps capture visual features such as texture and color of the club, while point cloud data provides three-dimensional information such as the club's geometric shape and size. By combining image and point cloud data, the combined texture and geometric features result in a higher input dimension for the classification model, enabling it to capture more detailed information. If the club type matches a preset target club type, the method confirms arrival at the green area and sends a stop-follow command to control the golf cart to brake. Algorithms using color thresholding or texture segmentation struggle to accurately locate gradient boundaries, especially under uneven lighting or shadow conditions, where color distortion further exacerbates misjudgments, making it impossible to correctly determine whether a user has entered the green area. This method avoids the problem of inaccurately locating gradient boundaries under uneven lighting or shadow conditions when using color thresholding or texture segmentation to determine green entry. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the first process for the green detection method based on object feature recognition in this application; Figure 2 This is a second flowchart illustrating the green detection method based on object feature recognition provided in this application. Figure 3 A third flowchart illustrating the green detection method based on object feature recognition provided in this application; Figure 4 This is a fourth flowchart illustrating the green detection method based on object feature recognition provided in this application. Figure 5 This is a schematic diagram of the hardware operating environment involved in the green detection method based on object feature recognition in the embodiments of this application.

[0019] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] To achieve the above objectives, this application proposes a green detection method based on object feature recognition. The method includes: upon detecting a user's club-taking action, acquiring image data and point cloud data; acquiring the texture features of the club based on the image data, and acquiring the geometric features of the club based on the point cloud data; concatenating the geometric features and the texture features to obtain a comprehensive feature vector, and determining the club type based on the comprehensive feature vector; if the club type matches a preset target club type, confirming arrival at the green area, and sending a stop-follow command to control the golf cart to brake.

[0023] According to the rules of golf, equipment such as golf carts are prohibited from entering the green area to avoid disrupting the smoothness of the green surface. Because green grass is typically meticulously mowed and maintained, it exhibits distinct appearance characteristics compared to other areas, such as shorter grass and a more uniform green color. Greens are usually distinguished from other areas by features such as grass color (green) and texture (uniform grain). However, the height, density, and color of the grass between the green and fairway typically transition gradually rather than forming a clear boundary. For example, green grass may be shorter and greener, while fairway grass may be slightly longer or lighter in color. Traditional algorithms based on color thresholds or texture segmentation struggle to accurately pinpoint these transitional boundaries, especially under uneven lighting or shaded conditions, where color distortion further exacerbates misjudgments, making it impossible to correctly determine whether a user has entered the green area.

[0024] This application provides a green detection method based on object feature recognition. It extracts texture features from image data and geometric features from point cloud data, combining the two for club type identification. Image data helps capture visual features such as texture and color of the club, while point cloud data provides three-dimensional information such as the club's geometric shape and size. By combining image and point cloud data, the combined texture and geometric features result in a higher input dimension for the classification model, enabling it to capture more detailed information. If the club type matches a preset target club type, the method confirms arrival at the green area and sends a stop-follow command to control the golf cart to brake. Algorithms using color thresholding or texture segmentation struggle to accurately locate gradient boundaries, especially under uneven lighting or shadow conditions, where color distortion further exacerbates misjudgments, making it impossible to correctly determine whether a user has entered the green area. This method avoids the problem of inaccurately locating gradient boundaries under uneven lighting or shadow conditions when using color thresholding or texture segmentation to determine green entry.

[0025] It should be noted that the executing entity in this embodiment can be a computing service device with network communication and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or apparatus capable of performing the above functions. The following description uses a green detection device based on object feature recognition as an example to illustrate this embodiment and the subsequent embodiments.

[0026] Based on this, embodiments of this application provide a green detection method based on object feature recognition, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the green detection method based on object feature recognition in this application.

[0027] In this embodiment, the green detection method based on object feature recognition is applied to a green detection device based on object feature recognition, and the method includes steps S10 to S40: Step S10: When the user's lever-taking action is detected, image data and point cloud data are acquired.

[0028] In this implementation, once a user's cue-taking action is confirmed, a synchronous acquisition command is sent to the TOF sensor and the RGB camera to ensure that both capture data within the same time window. A unified timestamp is assigned to the TOF point cloud data and the RGB image to avoid data misalignment due to sensor latency. After acquiring the TOF point cloud data and RGB image data, data preprocessing is performed. The TOF point cloud data preprocessing process includes: converting the raw photon count data of the TOF sensor into a depth map and labeling the distance information of each pixel; removing environmental interference (such as flying insects and light reflections) through median filtering or statistical filtering; and transforming the point cloud data from the sensor coordinate system to the global coordinate system of the golf cart. When preprocessing the RGB image data, a region of interest (ROI) is extracted, and the processing range of the RGB image is narrowed based on the TOF depth map, retaining only the area where the cue is located. The TOF point cloud and the RGB image are spatially registered using calibration parameters (such as camera intrinsic and extrinsic parameters) to ensure that the position of the same object is consistent in both types of data.

[0029] Step S20: Obtain the texture features of the cue stick based on the image data, and obtain the geometric features of the cue stick based on the point cloud data.

[0030] Step S30: The geometric features and the texture features are concatenated to obtain a comprehensive feature vector, and the club type is determined based on the comprehensive feature vector.

[0031] In this embodiment, the YOLOv8 model is used to locate the position of the golf club in the RGB image and output a bounding box. The TOF point cloud is used to filter background interference (such as the ground and other objects). A golf club type detection model, such as ResNet-50, is used to classify the detected golf clubs and output the type (wood, iron, putter).

[0032] In one feasible implementation, step S20 includes the following steps: Texture features are extracted from the image data using a texture analysis algorithm; The shape and length features of the point cloud data are extracted based on the edge detection algorithm; The texture features, shape features, and length features are transformed and concatenated using linear discriminant analysis to obtain the comprehensive feature vector.

[0033] In this embodiment, texture analysis algorithms, such as Gray-Level Co-occurrence Matrix (GLCM) and Local Binary Pattern (LBP), are used for texture feature extraction. The LBP method generates a binary sequence by comparing the gray values ​​of each pixel with its neighboring pixels, and then converts this sequence into a histogram to reflect texture features. The GLCM describes texture by statistically analyzing the gray-level co-occurrence probabilities at different distances and directions in the image. Shape features extracted from point cloud data can be clubhead shape features. Edge detection, contour extraction, and shape analysis are used to identify the contour and shape features of the clubhead. Length features refer to the overall length of the club or the specific dimensions of the clubhead. After identifying the club's contour and feature points through edge detection and contour extraction, the overall length of the club or the dimensions of a specific part are calculated based on the positional relationships of the feature points.

[0034] In this embodiment, LDA is used for feature transformation. Feature transformation maps features from different information sources to the same space or a lower-dimensional space, thereby achieving feature fusion. The optimal projection direction is found by maximizing inter-class divergence and minimizing intra-class divergence. Matrix multiplication is used to multiply the original feature vector with the projection direction matrix to obtain the projected feature vector. After feature transformation, features from image data and point cloud data are concatenated, with the feature vectors joined end-to-end to form a composite feature vector. Feature transformation reduces the dimensionality of the features and removes redundant information, making the concatenated feature vector more compact and effective. The concatenated feature vector combines texture features from image data and shape and length features from point cloud data, enabling more accurate capture of the characteristics of the golf club.

[0035] In this embodiment, a TOF sensor provides depth information to determine the three-dimensional structure of the golf club (such as the short grip and wide head features of a putter), while an RGB camera provides visual features for recognizing the club (such as the unique shape and brand logo of a putter). Combining depth and visual data improves the reliability of club type identification.

[0036] In one feasible implementation, step S30 includes the following steps: The category score is determined by the sum of the product of the weight matrix and the eigenvector and the bias vector. The category scores are converted into a probability distribution, and the category with the highest probability is taken as the club type.

[0037] In this implementation, the input composite feature vector is used to obtain the unnormalized category scores (logits) through matrix multiplication and addition operations. The composite feature vector a = [a1, a2, ..., a...]. D ], where D is the dimension of the feature vector. A fully connected layer has a weight matrix W∈R D×C and a bias vector b∈R CWhere C is the total number of categories. For each category i, the weight matrix is ​​calculated by multiplying the i-th row of the weight matrix with the feature vector, plus the bias b. i Calculate its score z i Let z = Wa + b. The class scores are converted into probabilities using the Softmax function, and all calculated class probabilities are organized into a probability distribution. Based on this distribution, the class with the highest probability is identified as the most likely club type.

[0038] Step S40: If the club type matches the preset target club type, confirm arrival at the green area and send a stop following command to control the golf bag cart to perform braking.

[0039] In this embodiment, if the classification result is a push rod, a stop following command is sent to the ball bag car control system. After a certain time interval after sending the stop following command, it is verified whether the motor has stopped. The real-time speed is calculated by capturing the encoder pulse frequency. If the speed is 0 RPM for three consecutive samplings, it is determined that the motor has stopped.

[0040] In this embodiment, by using TOF and RGB camera fusion perception technology, combined with a pre-trained model to identify the type of golf club, the system can automatically stop following the player when they take out the putter, thereby effectively preventing the golf cart from entering the green area.

[0041] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Steps S01 to S03 may be included before step S10: Step S01: When the distance between the user and the ball bag in the point cloud data is less than a preset distance, and the human confidence score obtained based on the image data is greater than a preset confidence score threshold, the action detection process is triggered.

[0042] In this embodiment, distance monitoring by a Time-of-Flight (TOF) sensor and visual recognition by an RGB camera are combined to detect whether a user is approaching the golf bag cart. When the TOF sensor detects that the distance has shortened to a preset distance, and the RGB camera detects that a human body has entered a preset area, it is confirmed that the player has approached the golf bag cart and the motion detection process is triggered to determine whether the user has made a club-grabbing motion.

[0043] When using a Time-of-Flight (TOF) sensor for distance monitoring, a specific area in any direction of the globe carrier is designated as the Region of Interest (ROI). For example, a sector with a radius of 2 meters and a horizontal angle of 120° can be defined as the ROI. The real-time distance to objects within this area is measured using a TOF sensor. When the distance to an object within the ROI is less than a preset distance threshold, it is preliminarily determined that a target is approaching. Optionally, to distinguish between user approach and interference from static objects, further judgment is made based on the distance change rate. If the distance between the target and the globe carrier decreases by more than a preset distance change amount within a preset time period, it is considered that a moving target is approaching.

[0044] When performing visual recognition based on an RGB camera, a pre-trained human detection model is used to detect humans in images, outputting human bounding boxes and confidence scores. Optionally, lightweight models such as YOLOv8n or MobileNet-SSD are employed to achieve real-time human bounding box detection. Video streams or single-frame images are captured in real-time using an RGB camera or other image acquisition devices. Preprocessed images (e.g., scaling, normalization) are fed into the human detection model, which extracts image features through a series of convolutional and pooling layers. At each grid cell of the feature map, the human detection model predicts multiple bounding boxes with different scales and aspect ratios. Each bounding box has a confidence score, representing the probability of an object being present within that bounding box. Since bounding boxes may overlap, the Non-Maximum Score (NMS) algorithm is further used to remove redundant bounding boxes. The optimal bounding box is retained based on its confidence score and overlap, while those with high overlap and low confidence scores are removed. Optionally, the Intersection over Union (IoU) ratio is used to measure the degree of overlap between two bounding boxes. Sort all bounding boxes by confidence score from highest to lowest, and iterate through each bounding box sequentially. For the currently visited bounding box, if its IoU with previously retained bounding boxes is greater than a preset threshold, delete it; otherwise, retain it as part of the final detection result. The final result will be a list containing all predicted bounding boxes and their confidence scores. Each bounding box will have coordinates, width, height, and confidence score. When the confidence score is greater than a preset confidence score threshold, it is determined that a user has entered a preset area.

[0045] In one feasible implementation, a user is considered to be approaching only when the TOF sensor detects that the distance has shortened to a preset distance, or when the RGB camera detects that a human body has entered a preset area, and the data from the sensors that meet the conditions are consistent for several consecutive frames.

[0046] When the Time-of-Flight (TOF) sensor detects that the distance has shortened to a preset distance, TOF data is continuously acquired for a preset number of frames, and the minimum mean distance and variance of the distance change rate for the user in the consecutive frames are calculated. If the minimum mean distance is less than a preset mean distance threshold, and the variance of the distance change rate is less than a preset variance threshold, the user is considered to be continuously approaching.

[0047] When the RGB camera detects a human entering a preset area, RGB image data for a preset number of frames is continuously acquired. Optical flow is used to calculate the directional consistency of the user's movement trajectory. If the angle change between the user and the ball bag is less than a preset directional change threshold, a user is considered to be approaching. Continuous video frames are captured at a fixed frame rate, and optical flow vectors are mapped to the global coordinate system based on the known position and orientation of the ball bag. Key points (such as hands and feet) of the player's silhouette are extracted using Shi-Tomasi or Harris corner detection, and a set of feature point coordinates is output. The motion vectors of feature points between two consecutive frames are calculated using Farneback dense optical flow or Lucas-Kanade sparse optical flow, and low-confidence feature points are filtered out, retaining only valid motion vectors. The positions of the same feature point in multiple frames are time-series aligned to generate a movement trajectory. The displacement vector between every two adjacent frames is calculated, mapped to the global coordinate system, and the motion direction angle relative to the ball bag is calculated. The direction angles of the most recent N frames are saved, and the absolute value of the difference between adjacent angles is calculated. The system retrieves the maximum change value. If the maximum change value exceeds a preset direction change threshold, it is considered that a user is approaching. The direction change threshold can be dynamically adjusted based on environmental complexity (such as changes in lighting or occlusion).

[0048] In this embodiment, a strategy of dual-sensor high-confidence triggering and single-sensor continuous verification is adopted for player proximity detection on the golf cart. Time-of-Flight (TOF) provides absolute distance information through physical ranging, while the RGB camera provides semantic information (such as the shape of the golf club) through visual recognition. When both are triggered simultaneously, false judgments from a single sensor can be eliminated. Under the single-sensor continuous verification strategy, if one sensor suddenly malfunctions (such as loss of the TOF signal), the other sensor can still pass continuous verification.

[0049] Step S02: Determine whether the user has raised their hand and whether the cabin door of the ball bag is open.

[0050] Step S03: If the hand lifting action exists and the hatch is in the open state, then the stick retrieval action is confirmed to exist.

[0051] In this embodiment, a Time-of-Flight (TOF) sensor and an RGB camera work together to achieve real-time detection of the player's cue-picking motion. The TOF sensor monitors the user's hand movements, while the RGB camera detects the opening of the golf bag door.

[0052] In one feasible implementation, step S02 may further include the following steps: acquiring continuous point cloud frames based on the TOF sensor, extracting hand feature points from the continuous point cloud frames according to a key point extraction algorithm; acquiring the displacement vector of the hand feature points from the current frame to the next frame; if the change in the vertical direction of the displacement vector is greater than a preset change threshold, then confirming the existence of the hand lifting action. The existence of the hand lifting action is confirmed through the above steps.

[0053] In this embodiment, a Time-of-Flight (TOF) sensor is used to capture depth images of the hand and its surrounding environment. Each pixel value in the depth image represents the distance of that point from the sensor. Based on the depth image and the sensor's intrinsic parameters (such as focal length, optical center, etc.), the depth image is converted into a 3D point cloud. Each point cloud point contains three coordinate values ​​(X, Y, Z), representing its position in 3D space. In the initial frame, a keypoint extraction algorithm is used to extract feature points of the hand from the point cloud data, serving as the reference for tracking in subsequent frames. For example, when extracting hand feature points using the ISS3D (Intrinsic Shape Signatures 3D) algorithm, for each point, the covariance matrix in its local neighborhood is calculated. The covariance matrix is ​​then decomposed into eigenvalues ​​λ1, λ2, λ3, and the corresponding eigenvectors. The ISS value (Intrinsic Shape Significance) is calculated based on the eigenvalues. The ISS value is a metric based on the eigenvalues ​​used to evaluate the local geometric saliency of a point. The formula for calculating the ISS value is: ISS = (λ1 / λ2 + a) + (λ2 / λ3 + a) + (λ3 / λ1 + a), where 'a' is a small positive number to avoid a zero denominator. For each point, if its ISS is greater than or equal to the ISS threshold, then that point is considered a hand feature point. In the current frame, the three-dimensional coordinates of each feature point are recorded. Optical flow methods (such as the Lucas-Kanade optical flow method) or trajectory tracking algorithms are applied to find matching feature points between consecutive frames, and the feature points in the current frame are tracked to the next frame. For each successfully tracked feature point, the difference between its new position in the next frame and its position in the current frame is calculated to obtain the displacement vector. If the displacement of a feature point in the vertical direction exceeds a set threshold, it is determined that a hand-raising action has occurred.

[0054] In one feasible implementation, step S02 may include the following steps: The image captured by the RGB camera is input into a pre-trained semantic segmentation model to obtain the pixel-level mask of the hatch. The area of ​​the opening / closing region is determined based on the non-zero pixels in the pixel-level mask. The opening / closing ratio is determined based on the ratio of the area of ​​the opening / closing area to the area of ​​the entire door. If the opening / closing ratio is greater than a preset ratio threshold, the door is determined to be in the open state.

[0055] In this embodiment, different images containing the open and closed states of the hatch are collected in advance and labeled to distinguish the hatch area from other areas. A U-Net model is trained using the prepared dataset. During training, the model learns how to distinguish the hatch area from other background areas and outputs a corresponding pixel-level mask. RGB images containing the hatch are captured using a camera or image sensor. The preprocessed images are input into the trained U-Net model, which outputs a pixel-level mask of the hatch area. Each pixel in the mask image is iterated over, and pixels with a value of 1 (or white) are considered part of the hatch area. The extracted hatch area can be a list, array, or image region containing these pixels. Each pixel in the mask image is iterated over, and during the iteration, for each pixel, its value is checked to see if it is non-zero. If it is, a counter is incremented. This counter will eventually contain the total number of non-zero pixels in the mask.

[0056] The area of ​​the opening / closing region is calculated by multiplying the number of non-zero pixels by the area of ​​each pixel. Based on multiple labeled hatch images, the average area of ​​the hatch region in these images is calculated as the total hatch area. The opening / closing region area is divided by the total hatch area to obtain the opening / closing ratio. An opening / closing ratio threshold is set according to practical application requirements. When the opening / closing ratio is greater than or equal to the threshold, the hatch is determined to be in the open state.

[0057] In another feasible implementation, target detection is performed on the RGB image containing the hatch using a pre-trained target detection model, and the feature information of the hatch bounding box is extracted. The height of the hatch when fully closed is used as a preset height, and the ratio of the bounding box height to the hatch height when fully closed is calculated. If the ratio is greater than a certain threshold, it indicates that the hatch height has increased, and the hatch is determined to be in the open state. If the ratio is less than or equal to the threshold, it indicates that the hatch height has not changed or has decreased, and the hatch is determined to be in the closed state.

[0058] In one feasible implementation, when no clear club-taking action is detected, a sensor scan is initiated at preset time intervals to check whether a club is present in the golf bag compartment.

[0059] Based on the first embodiment of this application, in the third embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 Step S30 may be followed by steps A10 to A40: Step A10: According to the preset target detection algorithm, the cue stick is identified in the point cloud data to obtain the cue stick bounding box, and the target point cloud data of the corresponding area is obtained based on the cue stick bounding box.

[0060] In this embodiment, the obtained club type identification results are verified using TOF data. The YOLOv8 object detection algorithm is used to identify the club in the point cloud data. YOLOv8 outputs the bounding box of the club, which marks the position of the club on the two-dimensional image. Based on the bounding box output by YOLOv8, the point cloud data of the corresponding region is extracted from the three-dimensional point cloud, including the point cloud data of the grip, shaft, and clubhead.

[0061] Step A20: Obtain the normal vector of each point in the target point cloud data, perform cluster analysis on the normal vector, and identify the cylindrical point cloud region with the same normal vector direction as the grip end.

[0062] In this embodiment, methods such as least squares fitting neighborhood plane method or principal component analysis (PCA) are used to calculate the normal vector of each point in the target point cloud. Optionally, the normal vector is obtained using PCA. For each point in the target point cloud and its neighboring points, a covariance matrix is ​​constructed. This covariance matrix reflects the scattering of the neighboring points in various directions. Eigenvalue decomposition is performed on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors. The eigenvector with the largest eigenvalue corresponds to the main scattering direction of the point cloud data, i.e., the direction of maximum variance. The eigenvector with the smallest eigenvalue is selected as the normal vector.

[0063] Use visualization tools (such as PCLVisualizer) to add the calculated normal vectors to the point cloud data to observe the distribution of the normal vectors. In regions where the normal vectors are in the same direction, the normal vectors are distributed in a ring around a central axis, and the point cloud with the normal vector direction perpendicular to the cylinder axis is the part of the point cloud with cylindrical features, which is the grip end part.

[0064] Step A30: Obtain the local density of the target point cloud data and identify the high-density area as the pole head.

[0065] In this embodiment, for the target point cloud data, the neighborhood of each point is defined according to a preset distance radius. The number of points within the neighborhood of each point is calculated as the local density of that point. A point with higher local density means it has more neighboring points and is more likely to be part of the pole head. For each point, the minimum distance to a point with a higher local density than itself is calculated. For the point with the highest density, the farthest distance to all points is taken as the minimum distance. A decision map is plotted with the local density of each point as the x-axis and the minimum distance to a point with a higher density as the y-axis. Based on the outliers on the decision map, points with high local density and relatively far distances to points with higher density are selected as candidate points for the pole head. By connecting these candidate points, the pole head region is formed.

[0066] Step A40: If the distance between the grip and the clubhead is not within the standard length range of the putter, then the preset club type identification model is incrementally trained based on the abnormal data.

[0067] In this embodiment, the distance between the center of mass of the grip end point cloud and the center of mass of the clubhead is calculated to obtain the club length. This length is compared with the standard length range of a putter. If the calculated length value exceeds the standard putter length range, it is marked as an outlier. Outlier measurements are used as hard examples to incrementally update the club type recognition model parameters to improve generalization ability.

[0068] Based on the first embodiment of this application, in the fourth embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 Step S400 may be followed by steps B10 to B40: Step B10: Classify the point cloud data according to the clustering algorithm to obtain multiple clusters.

[0069] Step B20: Determine the occlusion area based on the density and geometric characteristics of the cluster.

[0070] In this embodiment, point cloud data is clustered using a preset clustering algorithm, such as K-means, DBSCAN, or hierarchical clustering, to obtain multiple clusters. Clusters are then filtered based on a point count threshold. All clusters with a point count below the threshold are marked as suspected occluded regions. Then, suspected occluded regions are further filtered based on a density threshold. If the density of a suspected occluded region is also below the density threshold, it is confirmed as an occluded region.

[0071] Step B30: Fit the point cloud data of the unoccluded area according to the preset fitting algorithm to generate a fitting curve.

[0072] In this embodiment, a suitable fitting algorithm is selected based on the characteristics and requirements of the point cloud data, such as least squares method, moving least squares method, surface fitting algorithm, etc. The point cloud data of the unoccluded portion is fitted to obtain a smooth surface or curve. Based on the shape and trend of the fitted surface or curve, it is extended to the occluded area.

[0073] In one alternative implementation, fitting is performed using the moving least squares (MLS) method. The point cloud data is divided into multiple local regions, each containing a certain number of points for local fitting. The size of the regions should be adjusted according to the density of the point cloud data and the degree of local variation. The polynomial order is determined based on the complexity of the point cloud data and the fitting requirements. Lower-order polynomials are suitable for smooth surfaces, while higher-order polynomials can better fit complex local features. For each local region, a local polynomial model is constructed. This local polynomial model fits a polynomial surface centered on a point in the point cloud and within its neighborhood. A weighting function is defined to measure the contribution of each point to the local fitting. Optionally, the weighting function can be a Gaussian function, a polynomial function, etc. An error function is constructed to measure the deviation between the fitted polynomial surface and the point cloud data. Optionally, the error function is the sum of squares of the distances from the points to the fitted surface. By minimizing the error function, the polynomial parameters for the best fit can be found. Initial values ​​are set for the parameters of the local polynomial model, including the coefficients of the polynomial and the parameters of the weighting function. Iterative optimization algorithms (such as gradient descent) are used to minimize the error function. In each iteration, the gradient or derivative of the error function is calculated based on the current parameters, and the parameter values ​​are updated.

[0074] The process continues iteratively until the error function converges to its minimum or the preset number of iterations is reached. Once the iterative optimization is complete, the best-fit polynomial parameters and the fitted surface are output.

[0075] Step B40: Extend the fitted curve to the occluded area to generate point cloud data of the occluded area.

[0076] In this embodiment, interpolation is used to extend the fitted surface to the occluded region. Discrete data points of the known region are obtained, which should include spatial coordinates (e.g., x, y, z) and corresponding attribute values ​​(e.g., depth, temperature, etc.). A suitable interpolation method is selected based on the characteristics and requirements of the data. If the data points are dense and relatively uniformly distributed, inverse distance weighted interpolation (IDW), Kriging interpolation, etc., can be selected; if the data points are few or unevenly distributed, radial basis function (RBF) interpolation can be used.

[0077] In one feasible implementation, the IDW (Inverse Distance Weighted) interpolation method is used to extend the fitted curve to the occluded region. A series of interpolation points are selected within the unoccluded region; these points should be evenly distributed to reflect the surface's variation trend. For each interpolation point, its attribute value is calculated. First, the Euclidean distance between it and all known points is calculated. Based on a preset distance matrix and power parameters, the weight of each known point for the interpolation point is calculated; the weight is the reciprocal of the distance raised to the power of p, where p is the power parameter. Then, the weighted average is used as the interpolation result, i.e., Z = Σ(w i *Z i ) / Σw i Where Z is the estimated attribute value of the point to be interpolated, Z i w is the attribute value of the i-th known point. i These are the corresponding weights. Finally, the calculated attribute values ​​are connected to form a curve to represent the surface changes within the occluded or extended regions. Optionally, appropriate curve fitting methods (such as spline interpolation, polynomial fitting, etc.) can be used to further smooth and optimize the curve. Completing the occluded regions ensures the comprehensiveness and integrity of the data, avoiding inaccurate results or biases caused by missing data points.

[0078] This application provides a green detection device based on object feature recognition, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the green detection method based on object feature recognition in Embodiment 1 above.

[0079] The following is for reference. Figure 5 The diagram illustrates a structural schematic of a green detection device based on object feature recognition suitable for implementing embodiments of this application. The green detection device based on object feature recognition in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, personal digital assistants (PDAs), tablet computers (PADs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The green detection device based on object feature recognition shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0080] like Figure 5As shown, the green detection device based on object feature recognition may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The random access memory 1004 also stores various programs and data required for the operation of the green detection device based on object feature recognition. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the green detection equipment based on object feature recognition to communicate wirelessly or wiredly with other devices to exchange data. Although a green detection equipment based on object feature recognition with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0081] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0082] The green detection device based on object feature recognition provided in this application, employing the green detection method based on object feature recognition in the above embodiments, can solve the technical problem of ensuring the security and integrity of original data during data quality checks. Compared with the prior art, the beneficial effects of the green detection device based on object feature recognition provided in this application are the same as those of the green detection method based on object feature recognition provided in the above embodiments, and other technical features in this green detection device based on object feature recognition are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0083] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0084] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0085] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the green detection method based on object feature recognition in the above embodiments.

[0086] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination thereof.

[0087] The aforementioned computer-readable storage medium may be included in a green detection device based on object feature recognition; or it may exist independently and not assembled into the green detection device based on object feature recognition. The aforementioned computer-readable storage medium carries one or more programs that, when executed by the green detection device based on object feature recognition, cause the green detection device to: upon detecting a user's club-taking action, acquire image data and point cloud data; acquire the texture features of the club based on the image data, and acquire the geometric features of the club based on the point cloud data; concatenate the geometric features and the texture features to obtain a comprehensive feature vector, and determine the club type based on the comprehensive feature vector; if the club type matches a preset target club type, confirm arrival at the green area, and send a stop-follow command to control the golf cart to brake.

[0088] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0089] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0090] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0091] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described green detection method based on object feature recognition. This solves the technical problem of ensuring the security and integrity of original data during data quality checks. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the green detection method based on object feature recognition provided in the above embodiments, and will not be repeated here.

[0092] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A green detection method based on object feature recognition, characterized in that, The green detection method based on object feature recognition includes: When the user's lever-taking action is detected, image data and point cloud data are acquired; The texture features of the cue stick are obtained from the image data, and the geometric features of the cue stick are obtained from the point cloud data. The geometric features and the texture features are concatenated to obtain a comprehensive feature vector, and the club type is determined based on the comprehensive feature vector. If the club type matches the preset target club type, the system confirms arrival at the green area and sends a stop-follow command to control the golf cart to brake. The target club type is a putter.

2. The green detection method based on object feature recognition as described in claim 1, characterized in that, The process includes the following steps prior to acquiring image data and point cloud data: When the distance between the user and the ball bag in the point cloud data is less than a preset distance, and the human confidence score obtained based on the image data is greater than a preset confidence score threshold, the action detection process is triggered. Determine whether the user has made a hand-raising motion and whether the cabin door of the ball bag is open; If the hand lifting action occurs and the hatch is in the open state, then the stick retrieval action is confirmed to have occurred.

3. The green detection method based on object feature recognition as described in claim 2, characterized in that, The step of determining whether the user has raised their hand includes: Continuous point cloud frames are acquired using a TOF sensor, and hand feature points are extracted from these continuous point cloud frames using a key point extraction algorithm. Obtain the displacement vector of the hand feature points from the current frame to the next frame; If the change in the vertical direction of the displacement vector is greater than a preset change threshold, then the hand lifting action is confirmed to exist.

4. The green detection method based on object feature recognition as described in claim 2, characterized in that, The step of determining that the cabin door of the golf cart is in the open state includes: The image captured by the RGB camera is input into a pre-trained semantic segmentation model to obtain the pixel-level mask of the hatch; The area of ​​the opening / closing region is determined based on the non-zero pixels in the pixel-level mask. The opening / closing ratio is determined based on the ratio of the area of ​​the opening / closing area to the area of ​​the entire door. If the opening / closing ratio is greater than a preset ratio threshold, the door is determined to be in the open state.

5. The green detection method based on object feature recognition as described in claim 1, characterized in that, The steps of obtaining the texture features of the golf club based on the image data, obtaining the geometric features of the golf club based on the point cloud data, and concatenating the geometric features and the texture features to obtain a comprehensive feature vector include: The texture features are extracted from the image data according to the texture analysis algorithm; The shape and length features of the point cloud data are extracted based on the edge detection algorithm; The texture features, shape features, and length features are transformed and concatenated using linear discriminant analysis to obtain the comprehensive feature vector.

6. The green detection method based on object feature recognition as described in claim 1, characterized in that, The step of determining the cue type based on the comprehensive feature vector includes: The category score is determined by the sum of the product of the weight matrix and the eigenvector and the bias vector. The category scores are converted into a probability distribution, and the category with the highest probability is taken as the club type.

7. The green detection method based on object feature recognition as described in claim 1, characterized in that, After the step of concatenating the geometric features and the texture features to obtain a comprehensive feature vector, and determining the cue type based on the comprehensive feature vector, the method further includes: According to the preset target detection algorithm, the club is identified in the point cloud data to obtain the club bounding box, and the target point cloud data of the corresponding area is obtained based on the club bounding box; Obtain the normal vector of each point in the target point cloud data, perform cluster analysis on the normal vector, and identify the cylindrical point cloud region with the same normal vector direction as the grip end; Obtain the local density of the target point cloud data, and identify the high-density area as the pole head; If the distance between the grip and the clubhead is not within the standard length range of a putter, then the preset club type identification model is incrementally trained based on the abnormal data.

8. The green detection method based on object feature recognition as described in claim 1, characterized in that, After the step of detecting the user's lever movement and acquiring image data and point cloud data, the method further includes: The point cloud data is classified according to a clustering algorithm to obtain multiple clusters; The occluded area is determined based on the density and geometric characteristics of the cluster; The point cloud data of the unoccluded area is fitted according to the preset fitting algorithm to generate a fitting curve; The fitted curve is extended to the occluded area to generate point cloud data of the occluded area.

9. A green detection device based on object feature recognition, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the green detection method based on object feature recognition as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the green detection method based on object feature recognition as described in any one of claims 1 to 8.

Images