Image-based avoidance control method and device, and storage medium

Abstract

The invention discloses an image-based avoidance control method and device and a storage medium, and relates to the technical field of image processing, and the image-based avoidance control method comprises the steps: analyzing a space-time diagram structure constructed by an image sequence based on an action recognition model, and recognizing the action type of a user in the image sequence; if the action type is a collision early warning action type, generating a risk assessment result according to the motion trail of the user predicted by the space-time diagram structure in combination with the real-time position data of the cart with respect to the user; and based on the risk assessment result, determining an avoidance path of the golf trolley and generating a dynamic avoidance strategy to drive the golf trolley to execute an avoidance action. The technical effect of reducing the collision risk when the golf cart follows the user is achieved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image-based avoidance control method, device, and storage medium. Background Art

[0002] Currently, golf bag carts that automatically follow the user's movements cannot accurately and quickly capture the user's swing movements during the continuous following process, and are easily accidentally injured by the user's swinging behavior or the swinging behavior of other users around them, causing a collision risk.

[0003] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide an image-based avoidance control method, device and storage medium, aiming to solve the technical problem of the risk of collision caused by the golf bag car failing to avoid the user in time when automatically following the user.

[0005] To achieve the above objectives, the present application proposes an image-based avoidance control method, which includes: Analyzing the spatiotemporal graph structure constructed by the image sequence based on the action recognition model to identify the user's action type in the image sequence; If the action type is a collision warning action type, a risk assessment result is generated based on the motion trajectory of the user predicted by the spatiotemporal graph structure and the real-time position data of the golf cart relative to the user; Based on the risk assessment result, an avoidance path of the golf bag car is determined and a dynamic avoidance strategy is generated to drive the golf bag car to perform an avoidance action.

[0006] In one embodiment, the step of analyzing the spatiotemporal graph structure constructed by the image sequence based on the action recognition model and identifying the action type of the user in the image sequence includes: Performing human skeleton key point detection on the image sequence to obtain key point coordinates of the user; The key point coordinates are used as nodes of a space-time graph structure, and the connection relationships between the key point coordinates are used as edges of the space-time graph structure to construct the space-time graph structure; The spatiotemporal graph structure is input into an action recognition model, hierarchical motion features in the spatiotemporal graph structure are extracted through a spatiotemporal graph convolutional network, and the action type of the user is output.

[0007] In one embodiment, if the action type is a collision warning action type, the step of generating a risk assessment result based on the user's motion trajectory predicted by the spatiotemporal graph structure and the real-time position data of the golf bag car relative to the user includes: If the action type is a collision warning action type, analyzing the user's kinematic characteristics based on the user's spatiotemporal graph structure and predicting the user's motion trajectory; Calculating a dynamic safety boundary based on the user's motion trajectory and a safety distance threshold; Based on the real-time position data of the golf bag car relative to the user, and compared with the dynamic safety boundary, a risk assessment result is generated.

[0008] In one embodiment, if the action type is a collision warning action type, the step of analyzing the user's kinematic characteristics based on the user's spatiotemporal graph structure and predicting the user's motion trajectory includes: Extracting the user's kinematic features based on the user's spatiotemporal graph structure and constructing a spatiotemporal feature matrix; Perform convolution operations on the spatiotemporal feature matrix through a spatiotemporal graph convolutional network to capture the continuity characteristics of the user's action, output the predicted probability of the user's action, and predict the user's motion trajectory; or The state transition probability of the user's action is modeled by a discrete Bayesian network to predict the user's movement trajectory.

[0009] In one embodiment, the step of determining the avoidance path of the golf bag car based on the risk assessment result and generating a dynamic avoidance strategy to drive the golf bag car to perform an avoidance action includes: determining a collision risk direction and a collision risk distance in the collision risk direction according to the risk assessment result; Planning an avoidance path for the golf cart based on the collision risk direction and the collision risk distance; Generate the dynamic avoidance strategy based on the avoidance path and the direction and motion posture of the golf cart; Based on the dynamic avoidance strategy, corresponding control instructions are generated to drive the golf bag car to perform corresponding avoidance actions.

[0010] In one embodiment, the step of generating corresponding control instructions based on the dynamic avoidance strategy to drive the golf cart to perform the corresponding avoidance action further includes: The depth camera collects point cloud data of the environment around the golf cart in real time to build a three-dimensional obstacle map; Determining whether there is an obstacle on the avoidance path based on a spatial relationship between the avoidance path in the dynamic avoidance strategy and the obstacle map; If the obstacle exists on the avoidance path, the local path around the obstacle is replanned to avoid the obstacle.

[0011] In one embodiment, before the step of analyzing the spatiotemporal graph structure constructed by the image sequence based on the action recognition model and identifying the action type of the user in the image sequence, the following steps are included: Build an action recognition model based on spatiotemporal graph convolutional network; Collect golf action image sequences and corresponding action type labels to construct training data; Inputting the golf action image sequence of the training data into the action recognition model to generate a predicted action type; The error value between the predicted action type and the corresponding action type label is calculated through the loss function, and the network parameters are updated through back propagation to obtain the trained action recognition model.

[0012] In one embodiment, if the action type is a collision warning action type, before the step of generating a risk assessment result based on the user's motion trajectory predicted by the spatiotemporal graph structure and the real-time position data of the golf bag car relative to the user, the step includes: Using a monocular vision sensor or a binocular vision sensor, collecting a visual image sequence, performing target detection and feature extraction on the visual image sequence, and calculating visual distance data of the golf cart relative to the user; Collecting three-dimensional point cloud data through a laser ranging sensor, and extracting point cloud position data of the golf cart relative to the user through point cloud clustering or feature matching algorithm; The real-time position data is determined by aligning the timestamps and unifying the coordinate system and fusing the visual distance data with the point cloud position data.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes an image-based avoidance control device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the image-based avoidance control method described above.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the image-based avoidance control method described above are implemented.

[0015] The present application provides an image-based avoidance control method, which first analyzes the spatiotemporal graph structure constructed by the image sequence based on the action recognition model to identify the action type of the user in the image sequence; if the action type is a collision warning action type, the risk assessment result is generated based on the user's motion trajectory predicted by the spatiotemporal graph structure and the real-time position data of the golf bag car relative to the user; based on the risk assessment result, the avoidance path of the golf bag car is determined and a dynamic avoidance strategy is generated to drive the golf bag car to perform the avoidance action. The present application uses a visual sensor to capture the image sequence in real time, uses the action recognition model to accurately analyze the user's action type, and can respond quickly when a collision warning action is detected, and generates a risk assessment result based on the real-time position data of the golf bag car relative to the user, and formulates a dynamic avoidance strategy based on this to ensure that the golf bag car can avoid the dangerous area in time and reduce the risk of collision. The present application achieves the technical effect of reducing the collision risk when the golf bag car follows the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A flowchart illustrating the first embodiment of the image-based avoidance control method of the present application is provided; Figure 2 A flowchart illustrating the second embodiment of the image-based avoidance control method of this application is provided; Figure 3 A flowchart of the third embodiment of the image-based avoidance control method of this application is provided; Figure 4 A flowchart illustrating a fourth embodiment of the image-based avoidance control method of the present application is provided; Figure 5 A flowchart of the fifth embodiment of the image-based avoidance control method of the present application is provided; Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the image-based avoidance control method in the embodiment of the present application.

[0019] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0021] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The main solutions of the embodiments of this application are: Currently, golf bag carts that automatically follow the user's movements cannot accurately and quickly capture the user's swing movements during the continuous following process, and are easily accidentally injured by the user's swinging behavior or the swinging behavior of other users around them, causing a collision risk.

[0023] This application uses visual sensors to capture image sequences in real time, uses motion recognition models to accurately analyze user motion types, and can respond quickly when collision warning actions are detected. It also generates risk assessment results based on the real-time position data of the golf cart relative to the user, and formulates dynamic avoidance strategies based on this to ensure that the golf cart can avoid dangerous areas in a timely manner and reduce collision risks.

[0024] It should be noted that the execution entity of this embodiment can be an image-based avoidance control system, a computing service device with data processing, network communication, and program execution capabilities, such as a tablet computer, personal computer, or mobile phone, or a control device capable of implementing the aforementioned functions. This embodiment does not specifically limit this. The following describes this embodiment and the following embodiments using an image-based avoidance control system as the execution entity.

[0025] Example 1 Based on this, this application proposes the first embodiment of the image-based avoidance control method, please refer to Figure 1 , the image-based avoidance control method includes: Step S10 , analyzing the spatiotemporal graph structure constructed by the image sequence based on the action recognition model, and identifying the action type of the user in the image sequence.

[0026] In this embodiment, the image sequence is captured by a visual sensor and is a data stream consisting of multiple frames of continuous images, which reflects the dynamic changes of the user's actions over a period of time and is used to capture the temporal characteristics of the user's actions. A visual sensor is a device that captures environmental image information through optical imaging technology, including depth cameras, monocular cameras, and binocular cameras. The action recognition model is a model built based on a deep learning algorithm, which is used to extract features from image sequences and classify action types. The spatiotemporal graph structure is a structure used to represent user actions. The relationship between the key points of the user's human skeleton in time and space is modeled in the form of a graph, which contains nodes and edges. The nodes represent key points, and the edges represent the connection relationship between key points.

[0027] As an optional implementation method, preprocessing operations are performed on the image sequence captured by the visual sensor, including image grayscale, normalization, denoising and other operations, and the human skeleton key points are performed on the collected image sequence to construct the user's spatiotemporal graph structure. The spatiotemporal graph structure is input into the trained action recognition model, and the spatiotemporal graph structure is analyzed and calculated to extract the hierarchical motion features therein, output the probability distribution of the action type, and select the action type with the highest probability as the final result.

[0028] Optionally, step S10 includes: Step S11 , performing human skeleton key point detection on the image sequence to obtain key point coordinates of the user.

[0029] Human skeleton key point detection is performed on image sequences to extract the spatiotemporal characteristics of human motion, making it easier to convert motion data into a graph structure suitable for deep learning model processing.

[0030] It should be noted that human skeleton key point detection is to locate the positions of key bone nodes of the human body from the image to form a skeleton model to determine the user's posture and gesture.

[0031] Optionally, a human skeleton key point detection algorithm is selected, and deep learning technology is used to accurately locate key parts of the human body in the image by training a convolutional neural network.

[0032] Exemplarily, the image sequence is input into the selected key point detection algorithm frame by frame, and each frame of the image is processed to obtain the key point coordinate information of the human skeleton in each frame of the image, including the three-dimensional position of each key part.

[0033] Step S12: Using the key point coordinates as nodes of a space-time graph structure and the connection relationships between the key point coordinates as edges of the space-time graph structure to construct the space-time graph structure.

[0034] It should be noted that the spatiotemporal graph structure is a structure used to represent the user's body movements. The relationship between the key points of the human skeleton in time and space is modeled in the form of a graph, which includes nodes and edges. Nodes represent key points, and edges represent the connection relationship between key points.

[0035] For example, based on the key point coordinates detected by key point detection, the key points of the human skeleton and their connection relationships in the same frame image are represented as a graph structure in the spatial dimension; in the time dimension, the graph structures in multiple consecutive frame images are connected in chronological order to form a spatiotemporal graph structure.

[0036] Step S13: input the spatiotemporal graph structure into an action recognition model, extract hierarchical motion features in the spatiotemporal graph structure through a spatiotemporal graph convolutional network, and output the user's action type.

[0037] By using the spatiotemporal graph convolutional network, complex motion features are extracted from the spatiotemporal graph structure to achieve efficient classification and recognition of user actions.

[0038] It should be noted that the spatiotemporal graph convolutional network is a deep learning model that combines graph convolutional networks and temporal convolution. It is used to process spatiotemporal graph data. It can perform convolution operations on graph-structured data, automatically learn node and edge features, and capture spatial and temporal correlations between nodes. Hierarchical motion features are features that describe actions at different levels. They include local features, i.e., motion features of individual limbs, and global features, i.e., motion features of the entire body.

[0039] Optionally, the architecture of the spatiotemporal graph convolutional network is designed, including an input layer, multiple spatiotemporal graph convolutional layers, a pooling layer, a fully connected layer, etc., and the parameters of each layer, including the convolution kernel size and the number of channels, are determined to adapt to the characteristics of the spatiotemporal graph structured data.

[0040] For example, the spatiotemporal graph data is fed into the action recognition model. Through the forward propagation of the spatiotemporal graph convolutional network, convolution operations are performed on the nodes and edges in the spatiotemporal graph structure, automatically extracting hierarchical motion features. Based on the extracted hierarchical motion features, the user's action type is classified and identified, and the corresponding action type probability distribution is output. The category with the highest probability is selected as the final action recognition result.

[0041] Step S20: If the action type is a collision warning action type, a risk assessment result is generated based on the motion trajectory of the user predicted by the spatiotemporal graph structure and the real-time position data of the golf bag car relative to the user.

[0042] When it is detected that the user's action type belongs to the collision warning action type, the user's movement trajectory is predicted, and then based on the user's movement trajectory and the real-time position relationship between the golf bag car and the user, the possible collision risk is evaluated and a risk assessment result is generated so that appropriate measures can be taken to avoid the occurrence of a collision.

[0043] In this embodiment, the collision warning action type is a type of action that the user may take that may cause a collision risk, including: when pushing the cart, the user may collide with the golf cart due to excessive force or direction deviation; when turning, if the user does not pay attention to the position of the golf cart, a collision may occur due to excessive movement; if a swing action is performed near the golf cart, the swing trajectory may interfere with the golf cart.

[0044] As an optional implementation, a motion recognition model analyzes the displacement directions of key points of the user's body in an image sequence to determine the direction of the user's motion. The range of the user's motion is measured by calculating the distance changes between key points. The speed and acceleration of the user's motion are derived by differentiating the changes in the key point positions over time. The start and end times of the motion are determined using timestamp information, and the duration of the user's motion is recorded. A laser sensor or camera is used to identify the user's position and determine the relative position of the user and the golf cart. Combining the user's kinematic characteristics and position data, a risk assessment algorithm is used to assess collision risk. The user's future displacement trajectory is predicted based on their motion direction and speed, identifying the collision risk area. This area is then compared with the golf cart's position to determine whether a collision is possible.

[0045] As another optional implementation method for obtaining real-time location data, a positioning device is used to obtain the precise location data of the golf cart and the user in real time, including coordinate positions and relative distances.

[0046] Step S30: determining an avoidance path for the golf bag car based on the risk assessment result and generating a dynamic avoidance strategy to drive the golf bag car to perform an avoidance action.

[0047] Based on the generated risk assessment results, an avoidance strategy is dynamically generated and the golf bag car is driven to perform corresponding avoidance actions, effectively reducing the possibility of collision between the golf bag car and the user. By responding to the risk assessment results in real time, the golf bag car can flexibly adjust its own movement state, avoid potential danger areas in advance, and avoid collision accidents.

[0048] In this embodiment, the dynamic avoidance strategy is an avoidance plan developed based on real-time risk assessment results, including avoidance direction, speed adjustment, and path planning. The avoidance action is the specific action taken by the golf cart to avoid collision with the user, including steering, acceleration, deceleration, and parking.

[0049] As an optional implementation method, based on the dangerous areas where collisions may occur determined in the risk assessment results, combined with the position of the golf bag car and the kinematic characteristics of the user, a path planning algorithm is used to generate an avoidance path, and the avoidance direction and speed adjustment strategy are determined. Then, combined with the direction facing the golf bag car, as well as the speed, acceleration and other motion posture information of the golf bag car, a corresponding dynamic avoidance strategy is generated, and the generated avoidance strategy is converted into specific control instructions, which are sent to the drive system of the golf bag car so that it performs the corresponding avoidance action according to the control instructions.

[0050] This embodiment provides an image-based avoidance control method. This embodiment first captures image sequences in real time through a visual sensor, uses a motion recognition model to accurately analyze the user's motion type, and can respond quickly when a collision warning action is detected. It also generates a risk assessment result based on the real-time position data of the golf bag car relative to the user, and formulates a dynamic avoidance strategy based on this to ensure that the golf bag car can avoid dangerous areas in a timely manner and reduce the risk of collision.

[0051] Based on the first embodiment, the second embodiment of the present application proposes an image-based avoidance control method, referring to Figure 2 , step S20 includes: Step S21 : If the action type is a collision warning action type, the user's kinematic characteristics are analyzed based on the user's spatiotemporal graph structure to predict the user's motion trajectory.

[0052] After identifying that the user's action belongs to the collision warning action type, the kinematic characteristics of the user's action are analyzed to predict the user's motion trajectory and determine the user's subsequent motion direction and range, providing a basis for subsequent dynamic safety boundary calculation and risk assessment, thereby more accurately assessing the collision risk.

[0053] It's important to note that kinematic features are physical quantities that describe the motion of an object, including direction, amplitude, velocity, acceleration, and duration. They are used to comprehensively understand the dynamic patterns of user actions. A motion trajectory is the user's likely movement path over a period of time. For actions like walking, the motion trajectory is the path the user takes in space; for actions like swinging, the motion trajectory is the trajectory of the user's golf club.

[0054] As an optional implementation, the user's kinematic features are extracted from the user's spatiotemporal graph structure, and the user's future motion trajectory is predicted based on the kinematic equation.

[0055] As another optional implementation, sensors installed on the cart or surrounding environment collect real-time data related to the user's movements, including image information, acceleration values, angular velocity values, etc. The collected data is processed and analyzed to extract the kinematic features of the user's movements. Based on these extracted kinematic features, a kinematic model or deep learning algorithm is used to predict the user's motion trajectory.

[0056] For example, image recognition technology is used to analyze the position and posture of various parts of the user's body to calculate the direction and amplitude of the movement. Accelerometer and gyroscope data are used to calculate the speed and acceleration of the movement. Kinematic equations are used to predict the user's future movement trajectory based on the user's current speed, acceleration, and direction.

[0057] Optionally, step S21 includes: Step A10: extracting the user's kinematic features based on the user's spatiotemporal graph structure and constructing a spatiotemporal feature matrix.

[0058] The kinematic features of user actions are extracted and structured, and a spatiotemporal feature matrix is ​​constructed to provide structured input data for subsequent spatiotemporal graph convolutional network processing and action analysis.

[0059] It should be noted that the spatiotemporal feature matrix is ​​a matrix-form data that integrates time and space information. The rows and columns in the matrix represent different time points and spatial dimensions, respectively, and the elements in the matrix correspond to the kinematic eigenvalues ​​in time and space positions.

[0060] As an optional implementation, the user's spatiotemporal graph structure is filtered and denoised, extracting key kinematic features such as velocity, acceleration, and displacement from the preprocessed data. Based on the specified time window and spatial dimensions, the extracted eigenvalues ​​are populated into a matrix to form a spatiotemporal feature matrix. This matrix is ​​then normalized to ensure that the eigenvalues ​​are on the same scale, improving the efficiency and accuracy of subsequent processing.

[0061] For example, for the user's arm swinging action, the time can be divided into several equally spaced time periods, the spatial dimension can be set to different position points of the arm, and then a matrix is ​​constructed based on the kinematic eigenvalues ​​of each position point at each time point.

[0062] Step A20, performing a convolution operation on the spatiotemporal feature matrix through a spatiotemporal graph convolutional network to capture the continuity characteristics of the user's action, output the predicted probability of the user's action, and predict the user's motion trajectory.

[0063] The spatiotemporal graph convolutional network is used to extract action continuity features from the spatiotemporal feature matrix, and then predict the prediction probability and motion trajectory of the user action to achieve more accurate action recognition and trajectory prediction.

[0064] It should be noted that the action continuity feature is a feature that reflects the continuous changes in user actions in time and space.

[0065] Optionally, the user's skeleton key points and their connections are represented as a graph structure, with nodes representing key points and edges representing connections between key points. A spatiotemporal graph convolutional network architecture is designed, including an input layer, multiple spatiotemporal graph convolutional layers, a pooling layer, and a fully connected layer. The spatiotemporal graph convolutional network model is trained using labeled motion data to accurately extract action continuity features and output predicted action probabilities.

[0066] Exemplarily, the spatiotemporal feature matrix is ​​input into a trained spatiotemporal graph convolutional network, and the motion continuity features are extracted through convolution operations. Based on the extracted features, the predicted probability of the user's motion is output, and through further analysis and calculation, the user's motion trajectory is predicted based on the motion continuity features and the current motion state.

[0067] Step A30: Modeling the state transition probability of the user's action through a discrete Bayesian network to predict the user's movement trajectory.

[0068] The state transition probability of user actions is modeled through discrete Bayesian networks, and the user's movement trajectory is predicted using the state transition probability.

[0069] It should be noted that the discrete Bayesian network is a probabilistic graphical model based on Bayes' theorem, used to represent probabilistic dependencies between variables. In user action prediction, the variables can be different states of the user's action, including standing, walking, and running. The probabilistic transition relationships between these states are represented by nodes and directed edges. The state transition probability refers to the probability of the user's action transitioning from one state to another. For example, the probability of transitioning from standing to walking, or from walking to running.

[0070] Optionally, the user's actions are divided into different discrete states. According to the characteristics of the user's actions and domain knowledge, the structure of the discrete Bayesian network is defined, including definite nodes representing different states of the user's actions and directed edges representing the transition relationship between states. The probability parameters in the discrete Bayesian network, i.e., the state transition probability, are learned using historical data.

[0071] For example, the initial state is determined based on the user's current action. Based on the learned state transition probabilities, Bayesian inference is used to calculate the probabilities of the user's possible future states. Based on these probabilities, the user's future action state sequence is predicted. Based on the predicted action state sequence and combined with the kinematic model, the user's future motion trajectory is predicted.

[0072] Step S22: Calculate a dynamic safety boundary based on the user's motion trajectory and the safety distance threshold.

[0073] Based on the predicted user motion trajectory, the dynamic safety boundary is calculated to provide a reference range for safe operation of the golf cart.

[0074] It should be noted that the dynamic safety boundary is a safety boundary calculated in real time based on factors such as the user's movement trajectory and speed. It is used to define the safe distance between the golf bag cart and the user and is dynamically adjusted as the user's movement state changes.

[0075] As an optional implementation method, the safety distance is determined based on the size of the golf cart, the movement speed and the safety needs of the user. A dynamic safety boundary model is constructed based on the determined safety distance with the user's movement trajectory as the center. The position and shape of the dynamic safety boundary are adjusted in real time according to the changes in the user's movement trajectory to ensure that a sufficient safety distance is maintained from the user.

[0076] For example, the safety distance is determined based on the speed of the user and the golf cart. For example, the formula ,in For a safe distance, For user speed, is the speed of the golf cart, It is a preset deceleration threshold to ensure that the golf cart can stop in time in an emergency.

[0077] Step S23: generating a risk assessment result based on the real-time position data of the golf bag car relative to the user and comparing it with the dynamic safety boundary.

[0078] By comparing the real-time location data of the golf cart with the dynamic safety boundary, a risk assessment result is generated to evaluate the current collision risk between the golf cart and the user.

[0079] For example, the real-time position data of the golf cart relative to the user is compared with the dynamic safety boundary to determine whether the position of the golf cart is outside the dynamic safety boundary. Based on the comparison result, the direction and distance of the risk source are determined to generate a risk assessment result.

[0080] This embodiment provides an image-based avoidance control method. Upon identifying a collision warning action, this method analyzes the kinematic characteristics of the user's motion to accurately predict the user's trajectory. Based on the predicted trajectory, a dynamic safety margin is calculated. The golf cart's real-time position relative to the user is compared with the dynamic safety margin, enabling the cart to react quickly and avoid a collision.

[0081] Based on the first embodiment, the third embodiment of the present application proposes an image-based avoidance control method, referring to Figure 3 , step S30 includes: Step S31 : determining a collision risk direction and a collision risk distance in the collision risk direction according to the risk assessment result.

[0082] By analyzing the risk assessment results, the specific direction and distance of the collision risk are determined, providing accurate positioning information for the subsequent generation of avoidance paths and the formulation of dynamic avoidance strategies.

[0083] It should be noted that the collision risk direction refers to the direction in which the user may collide with the golf cart. The collision risk distance is the specific distance between the golf cart and the user in the collision risk direction, that is, the shortest distance between the golf cart and the user where a collision may occur.

[0084] As an optional implementation method, based on the real-time position data of the golf cart and the user, the direction of the user relative to the golf cart is calculated, and the user's movement trajectory is further analyzed to predict the direction in which the golf cart and the user may meet. The position coordinates of the golf cart and the user are used to calculate the Euclidean distance between the two. According to the dynamic safety boundary, the calculated distance is adjusted to determine the collision risk distance.

[0085] As another optional implementation, sensors on the golf cart collect real-time information about the surrounding environment. Combined with relevant data from the risk assessment results, trigonometric functions and vector operations are used to calculate the azimuth of potential collision objects relative to the cart, thereby determining the direction of the collision risk. Based on the sensor data, the distance between the cart and the user, i.e., the collision risk distance, is directly obtained.

[0086] Step S32: planning an avoidance path for the golf bag car based on the collision risk direction and the collision risk distance.

[0087] Based on the direction and distance of the collision risk, an avoidance path for the golf cart is generated to ensure that the golf cart can safely avoid the user and reduce the risk of collision.

[0088] It should be noted that the avoidance path is the driving path planned by the golf cart to avoid collision, which usually includes parameters such as avoidance direction, avoidance distance and avoidance speed.

[0089] For example, the determined collision risk direction and distance are used as input parameters, combined with the current position, speed, acceleration and other information of the golf cart, and the selected path planning algorithm is used for calculation to generate an avoidance path from the current position to the safe area.

[0090] Step S33: Generate the dynamic avoidance strategy according to the avoidance path and the traveling direction and movement posture of the golf bag car.

[0091] For example, the heading angle representing the direction directly in front of the golf cart at the current moment is used as the direction of travel of the golf cart. Based on the motion posture data such as the speed, acceleration, and steering angular velocity data of the golf cart measured by the sensor, the kinematic constraints such as the maximum acceleration and maximum steering angular velocity of the golf cart are determined. According to the path curvature of the avoidance path and the kinematic constraints of the golf cart, the speed distribution on the avoidance path is planned. According to the tangent direction of the path, the target heading angle is calculated, and the steering angular velocity is calculated based on the target heading angle and the current heading angle. The path planning results are combined with motion control parameters such as heading angle, speed, acceleration, etc. to generate a dynamic avoidance strategy.

[0092] Step S34: Based on the dynamic avoidance strategy, a corresponding control instruction is generated to drive the golf bag car to perform a corresponding avoidance action.

[0093] According to the generated dynamic avoidance strategy, the golf bag car is controlled to perform specific avoidance actions to ensure that the golf bag car can safely avoid the user and reduce the risk of collision.

[0094] For example, based on the generated avoidance path, the path information is converted into control instructions such as motor speed and steering angle, and a steering instruction is generated based on the heading angle and steering angular velocity in the dynamic avoidance strategy; a speed instruction is generated based on the speed requirement of the dynamic avoidance strategy; and a braking instruction is generated based on the parking position planned by the dynamic avoidance strategy and sent to the drive system of the golf cart.

[0095] Optionally, the driving system of the golf cart controls the speed and direction of the motor according to the speed instruction; controls the steering system to adjust the steering wheel angle according to the steering instruction; and controls the braking system to perform the parking operation according to the braking instruction.

[0096] Optionally, during the avoidance process, sensors continuously capture feedback data such as the real-time position and speed of the cart and user. Based on this feedback, the avoidance effectiveness is evaluated in real time. If risks are still present in the initial avoidance path, the path is replanned and steering and speed commands are adjusted, dynamically adjusting the avoidance strategy.

[0097] Optionally, step S34 further includes: Step B10: Use a depth camera to collect point cloud data of the environment around the golf cart in real time to build a three-dimensional obstacle map.

[0098] The depth camera collects point cloud data around the golf cart and constructs a three-dimensional obstacle map to provide environmental information for subsequent avoidance path planning, ensuring that the golf cart can avoid obstacles during the avoidance process.

[0099] It should be noted that a depth camera is a device that collects depth information of the environment in real time. It emits a specific pattern of light and measures the time or phase change from emission to reflection to calculate the distance between the object and the camera, generating an image or point cloud data containing depth information. Point cloud data is a data set consisting of a large number of three-dimensional spatial points. Each point contains its coordinate information in three-dimensional space, as well as possible other attribute information. Point cloud data can accurately represent the shape and surface features of the object. The three-dimensional obstacle map is a three-dimensional obstacle distribution map constructed based on point cloud data, which is used to describe the position, shape and size of obstacles in the environment surrounding the golf cart.

[0100] For example, a depth camera collects point cloud data of the golf cart's surroundings in real time at a set frame rate. This data is then pre-processed, such as through denoising and filtering, to improve data quality. A point cloud processing algorithm is then used to segment and identify the point cloud data, extracting obstacle information. A three-dimensional obstacle map is then constructed based on the location and shape of the obstacles.

[0101] Step B20: Based on the spatial relationship between the avoidance path in the dynamic avoidance strategy and the obstacle map, it is determined whether there is an obstacle on the avoidance path.

[0102] Based on the constructed obstacle map, the pre-planned avoidance path is spatially analyzed to determine whether there are obstacles on the path, so as to adjust the avoidance strategy in time.

[0103] For example, the pre-planned avoidance path is spatially overlaid with the constructed three-dimensional obstacle map for analysis, and a spatial geometry algorithm is used to calculate the distance from each point on the avoidance path to the nearest obstacle. If the distance is less than a set safety threshold, it is considered that there is an obstacle on the path.

[0104] Step B30: If the obstacle exists on the avoidance path, replan the local path around the obstacle to avoid the obstacle.

[0105] When an obstacle is found on the avoidance path, the local path is replanned so that the golf cart can avoid the obstacle and continue to drive safely.

[0106] As an optional implementation, a local path planning algorithm, such as the artificial potential field method or the fast-search random tree algorithm, is used to calculate the detour direction based on the location and shape of the obstacle. A safe distance is set around the obstacle to ensure sufficient clearance between the cart and the obstacle, generating a local path that avoids the obstacle. The replanned local path is then connected to the original avoidance path to form a complete new avoidance path.

[0107] For example, by using a fast search random tree algorithm, starting from the current position of the ball bag car, randomly sampling points around the obstacle, and gradually expanding the path until a path is found that avoids the obstacle and reaches the target position.

[0108] This embodiment provides an image-based avoidance control method. First, based on risk assessment results, the method accurately determines the collision risk direction and distance. Then, a path planning algorithm is used to generate an avoidance path for the golf cart. This ensures the cart's safe and efficient operation during the avoidance process, preventing collisions and improving the cart's avoidance efficiency. The cart's drive system accurately executes the corresponding avoidance maneuvers, ensuring the cart accurately follows the planned path.

[0109] Based on the first embodiment, the fourth embodiment of the present application proposes an image-based avoidance control method, referring to Figure 4 , before step S10, including: Step S40: constructing an action recognition model based on a spatiotemporal graph convolutional network.

[0110] Build a deep learning model that can effectively process spatiotemporal graph data as an action recognition model for the identification and classification of golf actions, thereby providing a basis for subsequent action analysis and risk assessment.

[0111] For example, to build an action recognition model, based on the characteristics of golf action recognition, the architecture of a spatiotemporal graph convolutional network is designed, determining parameters such as the number of network layers, the number of nodes per layer, the size of the convolution kernel, and the stride. The input layer is designed to receive spatiotemporal graph data of golf actions. Multiple spatiotemporal graph convolutional layers are stacked, each responsible for extracting spatiotemporal features at a different level. Pooling layers are inserted between convolutional layers to reduce the number of parameters and improve computational efficiency. Finally, a fully connected layer is used to map the extracted features to the space of action types.

[0112] Step S50 : Collect golf action image sequences and corresponding action type labels to construct training data.

[0113] A large number of golf action image sequences and their corresponding action type labels are collected to build a comprehensive training dataset for training action recognition models so that they can accurately identify various golf actions.

[0114] It should be noted that the action type label is a label for classifying the actions represented by the golf action image sequence, and is used to identify the specific type of action, including swinging, putting, chipping, etc.

[0115] As an optional implementation, image sequences are obtained from a public golf action dataset, and each image sequence is labeled with a corresponding action type label. The collected golf action image sequences are then preprocessed by image cropping, scaling, and normalization.

[0116] As another optional implementation of data collection, a high-speed camera is used to shoot a video of a professional golf player's action, and the video is segmented into image sequences.

[0117] Optionally, the preprocessed dataset is divided into a training set, a validation set, and a test set. The training set is used to train the model, the validation set is used to adjust the model's hyperparameters and evaluate the model's performance during training, and the test set is used to ultimately evaluate the performance of the trained model.

[0118] Step S60: Input the golf action image sequence of the training data into the action recognition model to generate a predicted action type.

[0119] Optionally, the image pixel values ​​are normalized to the range [0, 1] or [-1, 1], and a skeleton keypoint detection algorithm is used to extract skeleton keypoints from the image sequence. A spatiotemporal graph structure is constructed based on the skeleton keypoints, where each node represents a keypoint and the edges represent the connections between keypoints.

[0120] For example, a loaded golf action image sequence is fed into a trained action recognition model for forward propagation. Based on the action recognition model, operations such as convolution and pooling are used to extract spatiotemporal features of the image sequence and output a predicted action type.

[0121] Step S70: Calculate the error value between the predicted action type and the corresponding action type label through a loss function, and back-propagate to update the network parameters to obtain the trained action recognition model.

[0122] The loss function calculates the error between the predicted action type and the corresponding action type label, and backpropagation is used to update the network parameters to obtain a trained action recognition model. The loss function is used to measure the difference between the model's prediction and the true label. The backpropagation algorithm transmits the error value back to the network, adjusts the network parameters, and continuously optimizes the model to improve the accuracy of action recognition.

[0123] It's important to note that the loss function measures the difference between the model's predictions and the true labels. The smaller the loss function, the closer the model's predictions are to the true labels, and the better the model's performance. Backpropagation is an algorithm used in deep learning model training to update network parameters based on the gradient of the loss function. Starting from the output layer, the backpropagation algorithm propagates the error back to the input layer layer by layer, calculating the gradient of the parameters at each layer and updating them using gradient descent to gradually bring the model's predictions closer to the true labels.

[0124] Exemplarily, a cross-entropy loss function is used to calculate the error between the predicted action type and the corresponding action type label. A backpropagation algorithm is used to calculate the gradient of the loss function with respect to the network parameters, and an optimization algorithm is used to update the network parameters based on the gradient. Through continuous iterative training, repeating the forward propagation, loss calculation, and backpropagation steps, the model's predicted results gradually approach the true label. When the similarity with the true label reaches a preset threshold, training is stopped, resulting in the trained action recognition model.

[0125] Optionally, during training, periodically evaluate the model's performance using a validation set. Based on the evaluation results on the validation set, adjust the model's hyperparameters to prevent overfitting or underfitting. When the model's performance on the validation set meets the requirements, stop training and obtain a trained action recognition model.

[0126] This embodiment provides an image-based avoidance control method. First, a deep learning model based on a spatiotemporal graph convolutional network is constructed to automatically learn the complex spatiotemporal features of golf action image sequences, thereby improving the accuracy and efficiency of action recognition. The deep learning model is trained using a large number of golf action image sequences and corresponding action type labels, resulting in a highly efficient and accurate action recognition model.

[0127] Based on the first embodiment, the fifth embodiment of the present application proposes an image-based avoidance control method, referring to Figure 5 , before step S20, including: Step S80: collecting a visual image sequence through a monocular vision sensor or a binocular vision sensor, performing target detection and feature extraction on the visual image sequence, and calculating visual distance data of the golf cart relative to the user.

[0128] The visual image sequence is collected, and the visual distance data between the golf cart and the user is calculated through target detection and feature extraction, providing visual information for subsequent position data fusion.

[0129] It should be noted that a monocular vision sensor uses only one camera to capture image information. While relatively low-cost, it lacks depth information and requires additional algorithms or prior knowledge to acquire three-dimensional spatial information. A binocular vision sensor, comprised of two cameras, mimics the stereoscopic vision principle of the human eye to capture depth information and more accurately measure the distance between objects and the sensor. A visual image sequence is a series of images continuously captured by a monocular or binocular vision sensor, arranged in chronological order, that records changes in the environment over a period of time. Object detection involves identifying the user's position and size within an image. Feature extraction involves extracting representative features from the detected object, including shape, color, and texture, for subsequent calculations and analysis.

[0130] As an optional implementation method, a visual sensor is used to collect a visual image sequence and preprocess the image. A deep learning algorithm is used to perform target detection on the preprocessed image, identify the position of the golf cart, extract the outline, color histogram and other features of the golf cart, and calculate the visual distance data between the golf cart and the user using geometric relationships based on the extracted features and camera parameters.

[0131] For example, for binocular vision ranging, the distance is calculated by the parallax principle of the binocular vision sensor. According to the parallax of the two cameras, the formula is used ,in For distance, is the focal length, is the baseline distance, For parallax.

[0132] For example, for monocular vision ranging, the distance is calculated by knowing the target size and the target size in the image. Use the formula ,in, is the actual size of the target, is the size of the target in the image.

[0133] Step S90: Collect three-dimensional point cloud data through a laser ranging sensor, and extract point cloud position data of the golf cart relative to the user through point cloud clustering or feature matching algorithm.

[0134] It should be noted that the laser ranging sensor is a sensor that uses laser technology to measure distance. It calculates the distance between the target and the sensor by emitting a laser beam and measuring the time from the laser beam being emitted to being reflected back. 3D point cloud data is the 3D spatial data of the surrounding environment collected by the laser ranging sensor. It is composed of a large number of points, and each point contains its coordinate information in 3D space. Point cloud clustering is the process of clustering points with similar features in 3D point cloud data. Point cloud clustering can be used to separate the point cloud area where the golf cart is located from the entire point cloud data. Feature matching algorithms are algorithms used to find point cloud areas in point cloud data that match known target features. Matching can be performed based on features such as shape, texture, and color, thereby extracting the user's position in the point cloud data.

[0135] As an optional implementation, a laser ranging sensor is used to collect 3D point cloud data of the surrounding environment at a set frame rate, recording the coordinates of each point. The collected point cloud data is then preprocessed, including filtering and denoising, to remove noise and outliers. A point cloud clustering algorithm or feature matching algorithm is then used to extract the user's point cloud position relative to the golf cart from the point cloud data, which is then converted into the point cloud position of the golf cart relative to the user.

[0136] For example, the point cloud data is clustered using Euclidean distance to identify the user's location within the point cloud data. Key points in the point cloud data are detected using a key point detection algorithm, and target points are matched using a matching algorithm. The point cloud data is converted from the sensor coordinate system to the global coordinate system of the golf cart, and the point cloud position data between the golf cart and the user is calculated.

[0137] Step S100 , by aligning timestamps and unifying coordinate systems, fusing the visual distance data and the point cloud position data, and determining the real-time position data.

[0138] By aligning timestamps and unifying coordinate systems, visual distance data and point cloud location data are fused to generate more accurate real-time location data.

[0139] It's important to note that aligning timestamps involves matching the data collected by the vision sensor and the laser ranging sensor in chronological order to ensure that data from the same moment can be fused. Unifying the coordinate system involves converting the coordinate systems of the two sensors so that they are expressed in the same coordinate system for data fusion.

[0140] As an optional implementation method, timestamps are added to the data collected by the visual sensor and the laser ranging sensor, and the data collected by the two sensors are matched in chronological order using interpolation, fitting and other methods. According to the installation positions and parameters of the two sensors, a coordinate system conversion relationship is established, and the point cloud position data and visual distance data are converted to the same coordinate system. The visual distance data and the point cloud position data are fused using weighted averaging, Kalman filtering and other methods to determine the real-time position data of the golf cart.

[0141] This embodiment provides an image-based avoidance control method. This embodiment first aligns timestamps and unifies coordinate systems to fuse data collected by the visual sensor and the laser ranging sensor, thereby making full use of information from both and improving the accuracy of determining the position of the golf cart.

[0142] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the image-based avoidance control method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0143] The present application provides an image-based avoidance control device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the image-based avoidance control method of the above-mentioned embodiment one.

[0144] Reference below Figure 6 , which shows a schematic structural diagram of an image-based avoidance control device suitable for implementing embodiments of the present application. The image-based avoidance control device in embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, personal digital assistants (PDAs), tablet computers (PADs), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as desktop computers. Figure 6 The image-based avoidance control device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0145] like Figure 6As shown, the image-based avoidance control device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the image-based avoidance control device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to I / O interface 1006: input device 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output device 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003 including, for example, a magnetic tape, hard disk, etc.; and communication device 1009. Communication device 1009 may allow the image-based avoidance control device to communicate with other devices wirelessly or by wire to exchange data. While the figure illustrates an image-based avoidance control device with various systems, it should be understood that implementation or presence of all illustrated systems is not required. More or fewer systems may alternatively be implemented or present.

[0146] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are performed.

[0147] The image-based avoidance control device provided in this application, utilizing the image-based avoidance control method described in the aforementioned embodiment, can address the technical issue of golf bag carts automatically following a user and causing collision risks due to untimely avoidance. Compared to the prior art, the image-based avoidance control device provided in this application offers the same beneficial effects as the image-based avoidance control method described in the aforementioned embodiment. Other technical features of this image-based avoidance control device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0148] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0149] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0150] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the image-based avoidance control method in the above-mentioned embodiment.

[0151] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, radio frequency (RF), etc., or any suitable combination thereof.

[0152] The computer-readable storage medium may be included in the image-based avoidance control device, or may exist independently without being incorporated into the image-based avoidance control device.

[0153] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the image-based avoidance control device, the image-based avoidance control device can write computer program code for performing the operations of the present application in one or more programming languages ​​or a combination thereof. The programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user computer, partially on the user computer, as a stand-alone software package, partially on the user computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, via the Internet using an Internet service provider).

[0154] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0155] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0156] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned image-based avoidance control method. This computer-readable storage medium can address the technical issue of golf bag carts automatically following a user, often resulting in collision risks due to untimely avoidance. Compared to the prior art, the computer-readable storage medium provided in this application offers the same beneficial effects as the image-based avoidance control method provided in the aforementioned embodiments, and therefore will not be elaborated upon here.

[0157] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An image-based avoidance control method, characterized in that: The image-based avoidance control method includes: Analyzing the spatiotemporal graph structure constructed by the image sequence based on the action recognition model to identify the user's action type in the image sequence; If the action type is a collision warning action type, a risk assessment result is generated based on the motion trajectory of the user predicted by the spatiotemporal graph structure and the real-time position data of the golf cart relative to the user; Based on the risk assessment result, an avoidance path of the golf bag car is determined and a dynamic avoidance strategy is generated to drive the golf bag car to perform an avoidance action.

2. The image-based avoidance control method according to claim 1, wherein: The step of analyzing the spatiotemporal graph structure constructed by the action recognition model in the image sequence and identifying the action type of the user in the image sequence includes: Performing human skeleton key point detection on the image sequence to obtain key point coordinates of the user; The key point coordinates are used as nodes of a space-time graph structure, and the connection relationships between the key point coordinates are used as edges of the space-time graph structure to construct the space-time graph structure; The spatiotemporal graph structure is input into an action recognition model, hierarchical motion features in the spatiotemporal graph structure are extracted through a spatiotemporal graph convolutional network, and the action type of the user is output.

3. The image-based avoidance control method according to claim 1, wherein: If the action type is a collision warning action type, the step of generating a risk assessment result based on the user's motion trajectory predicted by the spatiotemporal graph structure and the real-time position data of the golf bag car relative to the user includes: If the action type is a collision warning action type, analyzing the user's kinematic characteristics based on the user's spatiotemporal graph structure and predicting the user's motion trajectory; Calculating a dynamic safety boundary based on the user's motion trajectory and a safety distance threshold; Based on the real-time position data of the golf bag car relative to the user, and compared with the dynamic safety boundary, a risk assessment result is generated.

4. The image-based avoidance control method according to claim 3, wherein: If the action type is a collision warning action type, the step of analyzing the user's kinematic characteristics based on the user's spatiotemporal graph structure and predicting the user's motion trajectory includes: Extracting the user's kinematic features based on the user's spatiotemporal graph structure and constructing a spatiotemporal feature matrix; Perform convolution operations on the spatiotemporal feature matrix through a spatiotemporal graph convolutional network to capture the continuity characteristics of the user's action, output the predicted probability of the user's action, and predict the user's motion trajectory; or The state transition probability of the user's action is modeled by a discrete Bayesian network to predict the user's movement trajectory.

5. The image-based avoidance control method according to claim 1, wherein: The step of determining the avoidance path of the golf bag car based on the risk assessment result and generating a dynamic avoidance strategy to drive the golf bag car to perform an avoidance action includes: determining a collision risk direction and a collision risk distance in the collision risk direction according to the risk assessment result; Planning an avoidance path for the golf cart based on the collision risk direction and the collision risk distance; Generate the dynamic avoidance strategy based on the avoidance path and the direction and motion posture of the golf cart; Based on the dynamic avoidance strategy, corresponding control instructions are generated to drive the golf bag car to perform corresponding avoidance actions.

6. The image-based avoidance control method according to claim 5, wherein: The step of generating corresponding control instructions based on the dynamic avoidance strategy to drive the golf bag car to perform corresponding avoidance actions also includes: The depth camera collects point cloud data of the environment around the golf cart in real time to build a three-dimensional obstacle map; Determining whether there is an obstacle on the avoidance path based on a spatial relationship between the avoidance path in the dynamic avoidance strategy and the obstacle map; If the obstacle exists on the avoidance path, the local path around the obstacle is replanned to avoid the obstacle.

7. The image-based avoidance control method according to claim 1, wherein: Before the step of analyzing the spatiotemporal graph structure constructed by the image sequence based on the action recognition model and identifying the action type of the user in the image sequence, the method includes: Build an action recognition model based on spatiotemporal graph convolutional network; Collect golf action image sequences and corresponding action type labels to construct training data; Inputting the golf action image sequence of the training data into the action recognition model to generate a predicted action type; The error value between the predicted action type and the corresponding action type label is calculated through the loss function, and the network parameters are updated through back propagation to obtain the trained action recognition model.

8. The image-based avoidance control method according to claim 1, wherein: If the action type is a collision warning action type, before the step of generating a risk assessment result based on the user's motion trajectory predicted by the spatiotemporal graph structure and the real-time position data of the golf bag car relative to the user, the step includes: Using a monocular vision sensor or a binocular vision sensor, collecting a visual image sequence, performing target detection and feature extraction on the visual image sequence, and calculating visual distance data of the golf cart relative to the user; Collecting three-dimensional point cloud data through a laser ranging sensor, and extracting point cloud position data of the golf cart relative to the user through point cloud clustering or feature matching algorithm; The real-time position data is determined by aligning the timestamps and unifying the coordinate system and fusing the visual distance data with the point cloud position data.

9. An image-based avoidance control device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the image-based avoidance control method according to any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the image-based avoidance control method according to any one of claims 1 to 8 are implemented.

Images