A method, apparatus and medium for calibrating a golf visual sensor

Abstract

The present application relates to the technical field of computer vision technology. Specifically relates to a kind of golf visual sensor's calibration method, device and medium, its method includes based on the constraint that golf ball space point information is located in horizontal plane, camera external parameter is updated, obtains the camera external parameter after updating;Determine the predicted position information corresponding to face point image coordinate information;Based on the deviation between predicted position information and face point image coordinate information, the validity of updated calibration parameter is judged, and drift determination result is obtained;Based on face point image coordinate information, updated calibration parameter is updated, and corrected calibration parameter is obtained.The present application has the effect of improving the stability and consistency of visual sensor spatial analysis result in dynamic motion analysis scene.

Description

Technical Field

[0001] This invention relates to the technical field of computer vision technology, and in particular to a calibration method, apparatus and medium for a golf vision sensor. Background Technology

[0002] With the development of motion data acquisition and analysis technologies, motion behavior analysis methods based on vision sensors are increasingly being applied in sports training, motion assessment, and related smart devices. Acquiring image information of target objects through vision sensors and performing spatial analysis of this image information in conjunction with calibration parameters is a crucial foundation for motion process analysis.

[0003] In related technologies, visual sensors typically require a calibration process to determine the correspondence between the imaging model and spatial coordinates to ensure the accuracy of subsequent image data processing. In existing technologies, visual sensor calibration is mostly completed during equipment installation or initial use, and the obtained calibration parameters are used as fixed parameters for a long period. However, in practical applications, due to changes in equipment location, differences in the operating environment, or the cumulative effects of long-term operation, the actual imaging state of the visual sensor may change, leading to deviations between the calibration parameters and the true state, thus affecting the accuracy of spatial resolution results. Summary of the Invention

[0004] To improve the stability and consistency of spatial resolution results of vision sensors in dynamic motion analysis scenarios, this invention provides a calibration method, apparatus, and medium for a golf vision sensor.

[0005] The above-mentioned objective of this invention is achieved through the following technical solution:

[0006] A calibration method for a golf vision sensor, the calibration method for the golf vision sensor comprising:

[0007] Acquire an image of a golf ball located on a horizontal plane, determine the spatial point information of the golf ball based on the imaging information of the golf ball image, and update the camera extrinsic parameters based on the constraint that the spatial point information of the golf ball is located on the horizontal plane, to obtain the updated camera extrinsic parameters;

[0008] Based on the camera intrinsic parameters and the updated camera extrinsic parameters, the updated calibration parameters are obtained;

[0009] Acquire the image information of the clubface contact points collected during the swing, and obtain the image coordinate information of the clubface contact points from the image information of the clubface contact points;

[0010] Based on the camera intrinsic parameters, the updated camera extrinsic parameters, and the relative positional relationship between the pole surface contact points, the predicted position information corresponding to the image coordinate information of the pole surface contact points is determined.

[0011] Based on the deviation between the predicted position information and the coordinate information of the pole surface attachment image, the validity of the updated calibration parameters is determined to obtain the drift determination result;

[0012] If the drift determination result indicates that the updated calibration parameters need to be corrected, the updated calibration parameters are updated based on the coordinate information of the pole face image to obtain the corrected calibration parameters.

[0013] By adopting the above technical solution, the calibration parameters of the vision sensor can be continuously corrected and maintained without relying on additional calibration devices. By introducing the imaging information of the target object located on the horizontal plane, the camera's extrinsic parameters are constrained and updated, making the calibration parameters closer to the actual installation state. During the swing dynamics, multiple frames of pole face contact point images are used to form time-consistent error information to determine the validity of the calibration parameters. When a deviation in the calibration parameters is detected, the calibration parameters are corrected based on the coordinate information of the acquired multiple frames of images, thereby ensuring the stability and consistency of the mapping relationship between image coordinates and spatial position, reducing the cumulative error caused by changes in device posture or long-term use, and improving the calibration reliability and data consistency of the vision sensor in dynamic motion scenarios.

[0014] Preferably, acquiring an image of a golf ball located on a horizontal plane and determining spatial point information of the golf ball based on the imaging information of the golf ball image includes:

[0015] Obtain the two-dimensional position of the golf ball in the image and the pixel radius of the golf ball in the image;

[0016] Obtain the preset radius of the golf ball;

[0017] The focal length is determined based on the focal length parameter in the camera intrinsic parameters. The distance to the golf ball is calculated based on the focal length, the preset radius, and the pixel radius. The spatial point information of the golf ball is calculated based on the distance and the two-dimensional position.

[0018] By adopting the above technical solution, the imaging scale information of the golf ball in the monocular image can be used to combine the two-dimensional position in the image space with the physical size constraint. Without introducing an additional ranging device, the spatial distance between the golf ball and the camera can be calculated, and the spatial point information of the golf ball in the camera coordinate system can be further determined. This gives the image information a clear spatial meaning, thereby providing a reliable data foundation for subsequent parameter constraints and calibration updates based on spatial point information, and enhancing the consistency and repeatability of the spatial point calculation process.

[0019] Preferably, the step of updating the camera extrinsic parameters based on the constraint that the golf ball spatial point information is located on the horizontal plane, to obtain the updated camera extrinsic parameters, includes:

[0020] The golf ball spatial point information is converted into target spatial point information in a horizontal coordinate system, wherein the target spatial point information satisfies that the height direction coordinate is zero.

[0021] The correspondence between the spatial point information of the golf ball and the spatial point information of the target is established using the camera extrinsic parameters. The correspondence constraints corresponding to the spatial point information of the golf ball acquired multiple times are solved simultaneously to obtain the result of the simultaneous solution.

[0022] The camera extrinsic parameters are updated based on the results of the simultaneous solution to obtain the updated camera extrinsic parameters.

[0023] By adopting the above technical solution, the spatial point information of the golf ball obtained from image inversion can be introduced with explicit geometric constraints. By uniformly converting the spatial point information into target spatial point information that satisfies the condition that the height direction coordinate is zero, a stable and consistent constraint is established between the spatial points and the actual placement plane. On this basis, the correspondence constraint between spatial points is constructed using the camera extrinsic parameters. By solving the constraints of multiple acquired spatial points simultaneously, the update process of the camera extrinsic parameters is simultaneously constrained by multiple sets of observation information, thereby avoiding random deviations caused by a single observation. This allows the updated camera extrinsic parameters to more realistically reflect the spatial relationship between the camera and the horizontal plane, enhancing the stability and consistency of the camera extrinsic parameter update results and providing a reliable foundation for the accurate use of subsequent calibration parameters.

[0024] Preferably, the step of acquiring the clubface contact point image information collected during the swing and obtaining the image coordinate information of the clubface contact point from the clubface contact point image information includes:

[0025] During the swing, multiple frames of images containing clubface contact points are continuously acquired to obtain multi-frame clubface contact point image information;

[0026] For each frame of the pole face sticker image information, the two-dimensional position of the pole face sticker in the corresponding image is determined, and the pole face sticker image coordinate information corresponding one-to-one with the pole face sticker image information of each frame is obtained.

[0027] By adopting the above technical solution, it is possible to continuously acquire multi-frame image information containing the clubface contact point during the swing dynamic process, and extract the corresponding two-dimensional position coordinates from each frame image, so that the position change of the clubface contact point in the time dimension is completely recorded, thereby forming a sequence of image coordinate information with temporal continuity. This provides a stable data source for subsequent position prediction, deviation calculation and parameter determination based on multi-frame information, avoiding the random influence brought about by relying solely on a single frame image.

[0028] Preferably, determining the predicted position information corresponding to the image coordinate information of the pole surface contact point based on the camera intrinsic parameters, the updated camera extrinsic parameters, and the relative positional relationship between the pole surface contact points includes:

[0029] The relative positional relationship information of the pole face contact points is determined by utilizing the relative positional relationship between the contact points.

[0030] Based on the camera intrinsic parameters, the updated camera extrinsic parameters, and the relative positional relationship information of the pole surface attachment points, the spatial position information of the pole surface attachment points in the camera coordinate system is determined.

[0031] The spatial position information of the pole surface attachment point in the camera coordinate system is mapped to the image plane according to the camera intrinsic parameters and the updated camera extrinsic parameters to obtain the predicted position information of the pole surface attachment point corresponding to the image coordinate information of the pole surface attachment point.

[0032] By adopting the above technical solution, the relative positional relationship information of pole facet points can be constructed using the relative positional relationship between them. This ensures that the geometric relationship between multiple pole facet points is consistently constrained during the calculation process. Based on this, and combined with the updated calibration parameter information, a correspondence is established between the image coordinate information of the pole facet points and the spatial position information in the camera coordinate system. This yields the spatial position information of each pole facet point in the camera coordinate system. Furthermore, based on the updated calibration parameter information, the spatial position information is mapped onto the image plane to obtain the predicted position information of the pole facet points corresponding to the image coordinate information. This allows the predicted position information to reflect the imaging result under the combined effect of the calibration parameters and the geometric relationship of the points, providing a unified comparison benchmark for subsequent deviation calculation and determination of the validity of the calibration parameters.

[0033] Preferably, the step of determining the validity of the updated calibration parameters based on the deviation between the predicted position information and the coordinate information of the pole surface contact image to obtain a drift determination result includes:

[0034] For each frame of clubface contact image captured during the swing, the predicted position information of the corresponding clubface contact in each frame of clubface contact image is compared with the coordinate information of the corresponding clubface contact image in each frame of clubface contact image to determine the corresponding deviation information;

[0035] The deviation information from multiple frames is collected in the frame order of the swing process to obtain multi-frame error information;

[0036] The average value of the multi-frame error information is calculated to obtain error index information;

[0037] The error index information is compared with the preset threshold information to obtain the drift determination result.

[0038] By adopting the above technical solution, the predicted position information of the clubface contact point in each frame during the swing process can be compared with the coordinate information of the clubface contact point image in the corresponding frame to form deviation information. Then, the deviation information of multiple frames is collected in frame order to obtain multi-frame error information. The average value of the multi-frame error information is then calculated to form error index information. The error index information is compared with the preset threshold information to output the drift judgment result. This makes the drift judgment result based on the statistical quantity of multi-frame deviation rather than on the instantaneous deviation of a single frame, thereby reducing the instability of the judgment caused by single-frame noise or occasional recognition fluctuations, and ensuring that the drift judgment process is consistent with the frame sequence information.

[0039] Preferably, when the drift determination result indicates that the updated calibration parameters need to be corrected, updating the updated calibration parameters based on the pole facet image coordinate information to obtain corrected calibration parameters includes:

[0040] When the drift determination result indicates that the updated calibration parameters need to be corrected, based on the constraint conditions formed by the coordinate information of the multi-frame pole surface contact image, the updated calibration parameters are used as the variables to be adjusted, and iterative update processing is performed on the updated calibration parameters until the preset convergence condition is met, and the corrected calibration parameters are obtained.

[0041] By adopting the above technical solution, when the drift determination result indicates that drift has occurred, the coordinate information of multiple frames of clubface contact images acquired during the swing can be directly introduced into the calibration parameter correction process. The correction process uses the actual acquired image coordinate information as input, and while keeping the coordinate information of the multiple frames of clubface contact images unchanged, iterative update processing is performed on the updated calibration parameters. This allows the calibration parameter update process to gradually converge around the same set of input data, thereby obtaining corrected calibration parameters that match the coordinate information of the multiple frames of clubface contact images. This ensures that the calibration parameters after drift occurs can be corrected in a timely manner and maintain consistency.

[0042] Preferably, the calibration method for a golf vision sensor further includes:

[0043] Obtain the joint zero-position parameters and link parameters of the robotic arm to obtain the initial calibration parameters of the robotic arm;

[0044] The positional information of the robotic arm end effector in multiple preset postures is collected, and the initial calibration parameters of the robotic arm are updated to obtain the updated calibration parameters of the robotic arm.

[0045] By adopting the above technical solution, initial calibration parameters describing the kinematic relationship of the robotic arm can be formed based on the acquisition of the joint zero-position parameters and link parameters. By collecting the pose information of the end effector of the robotic arm under multiple preset postures, the pose observation results under actual operation are introduced into the parameter update process. This allows the calibration parameters of the robotic arm to be constrained by multiple posture conditions simultaneously, thereby correcting the initial calibration parameters. The updated calibration parameters are more consistent with the motion characteristics of the robotic arm under actual working conditions, providing a reliable parameter basis for subsequent spatial calculations and collaborative operations based on the pose information of the robotic arm.

[0046] The second objective of this invention is achieved through the following technical solution:

[0047] A calibration device for a golf vision sensor, the calibration device for the golf vision sensor comprising:

[0048] The image acquisition module is used to acquire an image of a golf ball located on a horizontal plane, determine the spatial point information of the golf ball based on the imaging information of the golf ball image, and update the camera extrinsic parameters based on the constraint that the spatial point information of the golf ball is located on the horizontal plane, thereby obtaining the updated camera extrinsic parameters.

[0049] The calibration parameter update module is used to obtain the updated calibration parameters based on the camera intrinsic parameters and the updated camera extrinsic parameters;

[0050] The clubface image information acquisition module is used to acquire clubface image information collected during the swing, and to obtain the image coordinate information of the clubface image from the clubface image information;

[0051] The predicted position information determination module is used to determine the predicted position information corresponding to the image coordinate information of the pole surface point based on the camera intrinsic parameters, the updated camera extrinsic parameters, and the relative positional relationship between the pole surface points;

[0052] The deviation analysis module is used to determine the validity of the updated calibration parameters based on the deviation between the predicted position information and the coordinate information of the pole surface attachment image, and to obtain the drift determination result.

[0053] The calibration parameter correction module is used to update the updated calibration parameters based on the coordinate information of the pole face image when the drift determination result indicates that the updated calibration parameters need to be corrected, so as to obtain the corrected calibration parameters.

[0054] By adopting the above technical solution, the calibration parameters of the vision sensor can be continuously corrected and maintained without relying on additional calibration devices. By introducing the imaging information of the target object located on the horizontal plane, the camera's extrinsic parameters are constrained and updated, making the calibration parameters closer to the actual installation state. During the swing dynamics, multiple frames of pole face contact point images are used to form time-consistent error information to determine the validity of the calibration parameters. When a deviation in the calibration parameters is detected, the calibration parameters are corrected based on the coordinate information of the acquired multiple frames of images, thereby ensuring the stability and consistency of the mapping relationship between image coordinates and spatial position, reducing the cumulative error caused by changes in device posture or long-term use, and improving the calibration reliability and data consistency of the vision sensor in dynamic motion scenarios.

[0055] The above-mentioned objective three of this application is achieved through the following technical solution:

[0056] A non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described calibration method for a golf vision sensor.

[0057] In summary, the present invention has at least one of the following beneficial technical effects:

[0058] 1. It can continuously correct and maintain the calibration parameters of the vision sensor without relying on additional calibration devices. By introducing the imaging information of the target object located on the horizontal plane, the camera's extrinsic parameters are constrained and updated, so that the calibration parameters can be closer to the actual installation state. During the swing dynamics, it uses multi-frame images of the club face to form time-consistent error information to determine the validity of the calibration parameters. When a deviation in the calibration parameters is detected, the calibration parameters are corrected based on the coordinate information of the acquired multi-frame images, thereby ensuring the stability and consistency of the mapping relationship between image coordinates and spatial position, reducing the cumulative error caused by changes in equipment posture or long-term use, and improving the calibration reliability and data consistency of the vision sensor in dynamic motion scenarios. Attached Figure Description

[0059] Figure 1 This is a flowchart of a calibration method for a golf vision sensor according to an embodiment of the present invention.

[0060] Figure 2 This is a flowchart illustrating the implementation of step S10 in a calibration method for a golf vision sensor according to an embodiment of the present invention.

[0061] Figure 3 This is another implementation flowchart of step S10 in a calibration method for a golf vision sensor according to an embodiment of the present invention;

[0062] Figure 4 This is a flowchart illustrating the implementation of step S30 in a calibration method for a golf vision sensor according to an embodiment of the present invention.

[0063] Figure 5 This is a flowchart illustrating the implementation of step S40 in a calibration method for a golf vision sensor according to an embodiment of the present invention.

[0064] Figure 6 This is a flowchart illustrating the implementation of step S50 in a calibration method for a golf vision sensor according to an embodiment of the present invention.

[0065] Figure 7 This is a flowchart illustrating the implementation of step S60 in a calibration method for a golf vision sensor according to an embodiment of the present invention.

[0066] Figure 8 This is a flowchart illustrating the implementation of step S60 and subsequent steps in a calibration method for a golf vision sensor according to an embodiment of the present invention.

[0067] Figure 9 This is a schematic diagram of a calibration device for a golf vision sensor according to an embodiment of the present invention.

[0068] Explanation of reference numerals in the attached figures:

[0069] 1. Image acquisition module; 2. Calibration parameter update module; 3. Patch image information acquisition module; 4. Predicted position information determination module; 5. Deviation analysis module; 6. Calibration parameter correction module. Detailed Implementation

[0070] The present invention will be further described in detail below with reference to the accompanying drawings.

[0071] In one embodiment, such as Figure 1 As shown, this invention discloses a calibration method for a golf vision sensor, specifically including the following steps:

[0072] S10: Acquire an image of a golf ball located on a horizontal plane, determine the spatial point information of the golf ball based on the imaging information of the golf ball image, and update the camera extrinsic parameters based on the constraint that the spatial point information of the golf ball is located on the horizontal plane, so as to obtain the updated camera extrinsic parameters.

[0073] In this embodiment, the golf ball image located on the horizontal plane refers to image data containing a golf ball acquired within the camera's field of view. The golf ball maintains contact with the horizontal plane in its actual placement state. The horizontal plane refers to a horizontal reference plane determined by the direction of gravity, independent of the camera's mounting posture. The horizontal reference plane can be characterized by a plane formed by a stationary liquid surface. The imaging information of the golf ball image refers to the geometric features of the golf ball in the image, reflecting its position and size in the camera's imaging plane. The golf ball spatial point information refers to the spatial representation of the golf ball's position in the camera coordinate system, determined based on the imaging information. The constraint of being located on the horizontal plane means that the coordinate components of the golf ball's spatial point information along the direction of gravity satisfy the planar conditions of the horizontal reference plane in the spatial representation related to the horizontal reference plane. The camera extrinsic parameters refer to parameter information describing the spatial relationship between the camera coordinate system and an external reference coordinate system. The external reference coordinate system uses the horizontal reference plane as a planar reference to characterize the horizontal reference plane's position and orientation in space. The updated camera extrinsic parameters refer to the parameters obtained by adjusting the original camera extrinsic parameters while satisfying the constraint that the golf ball's spatial point information is located on the horizontal plane.

[0074] Specifically, when acquiring an image of a golf ball on a horizontal plane, image data containing the golf ball is collected within the camera's field of view. By processing the imaging information of the golf ball image, the spatial point information of the golf ball corresponding to the golf ball is determined. After determining the spatial point information of the golf ball, the spatial point information of the golf ball is aligned with the planar conditions of the horizontal reference plane by taking advantage of the fact that the golf ball maintains contact with the horizontal reference plane during actual placement. Based on this, the spatial relationship between the camera coordinate system described by the camera extrinsic parameters and the external reference coordinate system is adjusted, thereby completing the update of the camera extrinsic parameters and obtaining the updated camera extrinsic parameters.

[0075] S20: Based on the camera intrinsic parameters and the updated camera extrinsic parameters, obtain the updated calibration parameters.

[0076] In this embodiment, the camera intrinsic parameters refer to the parameter information used to describe the camera imaging geometry and to characterize the imaging mapping relationship between the camera coordinate system and the image plane. The updated camera extrinsic parameters refer to the parameter information obtained after updating the camera extrinsic parameters under the constraint that the spatial point information of the golf ball is located on the horizontal plane. The updated calibration parameters refer to the parameter combination information jointly determined by the camera intrinsic parameters and the updated camera extrinsic parameters, and are used to characterize the mapping relationship between the image plane, the camera coordinate system and the external reference coordinate system.

[0077] Specifically, when the updated calibration parameters are obtained based on the camera intrinsic parameters and the updated camera extrinsic parameters, the camera intrinsic parameters are read from the storage area and the updated camera extrinsic parameters are obtained. The camera intrinsic parameters and the updated camera extrinsic parameters are combined according to a preset parameter combination format. The preset parameter combination format is used to limit the order and field correspondence of the camera intrinsic parameters and the updated camera extrinsic parameters in the same parameter structure. The updated calibration parameters are formed through combination processing and written to the calibration parameter storage area for subsequent prediction position information determination processing.

[0078] S30: Acquire the image information of the clubface contact points collected during the swing, and obtain the image coordinate information of the clubface contact points from the image information of the clubface contact points.

[0079] In this embodiment, the multi-frame clubface contact image information refers to multi-frame image data containing clubface contact points that are continuously acquired during the swing. The image coordinate information of the clubface contact points refers to the coordinate representation of the position of each clubface contact point in the image, determined from the multi-frame clubface contact image information.

[0080] Specifically, during the swing, image data containing clubface contact points are continuously acquired at a preset image acquisition frequency to form multi-frame clubface contact point image information. The multi-frame clubface contact point image information is processed frame by frame to identify the position of the corresponding clubface contact point in the image, and the identified clubface contact point position in each frame is converted into the corresponding image coordinate representation, thereby obtaining the image coordinate information of the clubface contact point that corresponds one-to-one with the clubface contact point image information of each frame.

[0081] S40: Based on the camera intrinsic parameters, the updated camera extrinsic parameters, and the relative positional relationship between the pole surface contact points, determine the predicted position information corresponding to the pole surface contact point image coordinate information.

[0082] In this embodiment, the relative positional relationship between the pole face stickers refers to the fact that when multiple pole face stickers are fixedly set on the pole face, the relative position of each pole face sticker in the pole face coordinate relationship remains unchanged. The predicted position information of the pole face stickers refers to the positional information used to characterize the predicted position of each pole face sticker in the image, which is calculated based on the relative positional relationship between the pole face stickers under the given updated calibration parameters.

[0083] Specifically, after obtaining the updated calibration parameters and the image coordinate information of the pole surface attachments, the relative spatial distribution of each pole surface attachment in the pole surface is determined based on the relative positional relationship between the pole surface attachments. Combined with the camera imaging relationship and spatial mapping relationship described by the updated calibration parameters, the relative spatial distribution is mapped to calculate the predicted position information of the pole surface attachments corresponding to the image coordinate information of the pole surface attachments, so that the image coordinate information of each frame of pole surface attachments corresponds to a set of predicted position results.

[0084] S50: Based on the deviation between the predicted position information and the coordinate information of the pole surface contact image, the validity of the updated calibration parameters is determined, and the drift determination result is obtained.

[0085] In this embodiment, the drift determination result refers to the determination information used to characterize whether the updated calibration parameters still meet the imaging consistency requirements under the current swing process.

[0086] Specifically, during the swing, for each frame of the clubface contact image acquired, the predicted position information of the clubface contact corresponding to that frame and the coordinate information of the clubface contact image are calculated in the same image coordinate system. The position difference calculation is performed by calculating the coordinate difference between the predicted position information and the image coordinate information in the horizontal and vertical directions respectively and combining them to obtain the deviation information corresponding to that frame. Then, the deviation information of multiple frames is collected in the frame order of the swing to form multi-frame error information. After obtaining the multi-frame error information, the average value of the deviation of each frame in the multi-frame error information is calculated to obtain the error index information. The average value calculation process is to accumulate the numerical results of the deviation information of each frame and divide by the number of frames to obtain the overall error level. Finally, the error index information is compared with the preset threshold information. When the error index information exceeds the preset threshold information, a drift judgment result indicating that the calibration parameter has drifted is generated. When the error index information does not exceed the preset threshold information, a drift judgment result indicating that the calibration parameter has not drifted is generated.

[0087] S70: If the drift determination result indicates that the updated calibration parameters need to be corrected, the updated calibration parameters are updated based on the coordinate information of the pole face image to obtain the corrected calibration parameters.

[0088] In this embodiment, the corrected calibration parameters refer to the calibration parameters formed after readjusting the updated calibration parameters in the event of drift, and are used to reflect the imaging relationship and spatial correspondence between the camera and the spatial object in the current installation state.

[0089] Specifically, when the drift determination result indicates that drift has occurred, the coordinate information of the multi-frame pole surface attachment images corresponding to the multi-frame pole surface attachment image information is obtained. The coordinate information of the multi-frame pole surface attachment images is used as a spatial constraint input condition and introduced into the imaging mapping relationship corresponding to the updated calibration parameters. By repeatedly adjusting the imaging mapping relationship, the deviation between the predicted position information of the pole surface attachment calculated based on the updated calibration parameters and the coordinate information of the multi-frame pole surface attachment images is gradually reduced, thereby completing the update processing of the updated calibration parameters and obtaining the corrected calibration parameters.

[0090] In one embodiment, such as Figure 2 As shown, in step S10, an image of a golf ball located on a horizontal plane is acquired, and spatial point information of the golf ball is determined based on the imaging information of the golf ball image, including:

[0091] S101: Obtain the two-dimensional position of the golf ball in the image and the pixel radius of the golf ball in the image from the golf ball image.

[0092] In this embodiment, the two-dimensional position of the golf ball in the image refers to the two-dimensional coordinate representation used to characterize the position of the golf ball in the golf ball image, denoted as p=[x,y,1], where x and y represent the horizontal and vertical coordinates of the golf ball in the image coordinate system, respectively, and the pixel radius of the golf ball in the image refers to the pixel scale representation used to characterize the imaging size of the golf ball in the golf ball image, denoted as r.

[0093] Specifically, when obtaining the two-dimensional position p=[x,y,1] of the golf ball in the image and the pixel radius r of the golf ball in the image, the golf ball image is subjected to golf ball region localization processing to determine the pixel region range of the golf ball in the image. Within the pixel region range, the coordinates of the pixel center point used to represent the position of the golf ball are determined and marked as the two-dimensional position p=[x,y,1]. At the same time, the set of boundary pixels of the golf ball is extracted within the pixel region range, and the pixel distance value is calculated based on the pixel distance relationship between the pixel center point coordinates and the set of boundary pixels. The pixel distance value is selected as the pixel radius r of the golf ball in the image.

[0094] S102: Obtain the preset radius of the golf ball.

[0095] In this embodiment, the preset radius of the golf ball refers to the radius parameter known before the determination of the golf ball's spatial point information, which is used to characterize the actual physical size of the golf ball. It is denoted as s. The preset radius is used as a fixed size parameter in subsequent spatial relationship calculations related to the golf ball image imaging information.

[0096] Specifically, when obtaining the preset radius s of the golf ball, the standard size parameter corresponding to the golf ball is used as the preset radius. Before determining the spatial point information of the golf ball, the preset radius s is read and recorded as a known input parameter. When performing spatial distance calculations based on the pixel radius of the golf ball in the image and the camera intrinsic parameters, the preset radius s is kept unchanged to ensure that the correspondence between the image size of the golf ball and the actual physical size is consistent.

[0097] S103: Determine the focal length based on the focal length parameter in the camera's intrinsic parameters, calculate the distance of the golf ball based on the focal length, preset radius, and pixel radius, and calculate the spatial point information of the golf ball based on the distance and two-dimensional position.

[0098] In this embodiment, the focal length refers to the focal length parameter determined from the camera's intrinsic parameters and used to characterize the camera's imaging ratio, denoted as f. The distance of the golf ball refers to the distance of the golf ball in space relative to the camera, denoted as d. The golf ball spatial point information refers to the spatial point representation used to characterize the spatial position of the golf ball in the camera coordinate system.

[0099] Specifically, when determining the focal length f based on the focal length parameter in the camera's intrinsic parameters, the focal length parameter corresponding to the imaging ratio is read from the camera's intrinsic parameters and used as the input condition for subsequent calculations. Combined with the preset radius s of the golf ball and the pixel radius r of the golf ball in the image, the distance of the golf ball in space is calculated according to the imaging geometry to obtain the distance d of the golf ball. After obtaining the distance d, the distance d is combined with the two-dimensional position p=[x,y,1] of the golf ball in the image, and the two-dimensional position is spatially mapped according to the mapping relationship between the camera coordinate system and the image coordinate system to determine the corresponding spatial point information of the golf ball.

[0100] In one embodiment, such as Figure 3 As shown, in step S10, based on the constraint that the spatial point information of the golf ball is located on the horizontal plane, the camera extrinsic parameters are updated to obtain the updated camera extrinsic parameters, including:

[0101] S104: Convert the spatial point information of the golf ball into the spatial point information of the target in the horizontal coordinate system. The target spatial point information satisfies that the height coordinate is zero.

[0102] In this embodiment, the target spatial point information refers to the spatial point representation obtained by representing the golf ball spatial point information according to the coordinate expression method of the horizontal plane coordinate system. The height direction coordinate is zero, which means that the position component of the target spatial point information in the horizontal plane coordinate system that represents the height direction is zero, so as to represent the position constraint of the golf ball on the horizontal plane.

[0103] Specifically, when converting the spatial point information of the golf ball into the target spatial point information in the horizontal coordinate system, the spatial point information of the golf ball is denoted as P=[X,Y,Z]. Based on the fact that the golf ball is located on the horizontal plane, the height component of the golf ball in the horizontal coordinate system is set to zero, thereby obtaining the target spatial point information P'=[X,Y,0], where X and Y represent the planar position coordinates of the golf ball in the horizontal coordinate system. The target spatial point information P' is used as the horizontal constraint spatial point input adopted in the subsequent camera extrinsic parameter update process.

[0104] S105: Establish the correspondence constraint between the spatial point information of the golf ball and the spatial point information of the target using the camera extrinsic parameters, and solve the correspondence constraint corresponding to the spatial point information of the golf ball acquired multiple times to obtain the result of the simultaneous solution.

[0105] In this embodiment, the correspondence constraint refers to the spatial correspondence condition that the spatial point information of the golf ball and the spatial point information of the target should satisfy under the spatial relationship described by the camera extrinsic parameters. The result of the simultaneous solution refers to the solution result used to characterize the update direction of the camera extrinsic parameters after jointly calculating the correspondence constraints corresponding to the spatial point information of the golf ball obtained multiple times.

[0106] Specifically, when establishing the correspondence constraints between the spatial point information of the golf ball and the spatial point information of the target using the camera extrinsic parameters, the spatial point information of the golf ball is represented as a spatial point P=[X,Y,Z] in the camera coordinate system, and the spatial point information of the target is represented as a spatial point P'=[X,Y,0] in the horizontal coordinate system. Based on the spatial transformation relationship described by the camera extrinsic parameters, a correspondence constraint that satisfies the mapping consistency between spatial points is established. When the spatial point information of the golf ball is acquired multiple times, a correspondence constraint is established between each set of spatial point information of the golf ball and the corresponding target spatial point information. The multiple sets of correspondence constraints are then processed simultaneously. By performing a solution operation on the constraint relationship processed simultaneously, the simultaneous solution result used to characterize the update direction of the camera extrinsic parameters is obtained.

[0107] S106: Update the camera extrinsic parameters according to the results of the simultaneous solution to obtain the updated camera extrinsic parameters.

[0108] In this embodiment, the camera extrinsic parameters are used to characterize the spatial transformation relationship between the camera coordinate system and the horizontal plane coordinate system. The result of the simultaneous solution is used to limit the spatial transformation relationship to satisfy the correspondence consistency between multiple sets of golf ball spatial point information and target spatial point information.

[0109] Specifically, when updating the camera extrinsic parameters according to the results of the simultaneous solution, the spatial point information of the golf ball is represented as a spatial point P=[X,Y,Z] in the camera coordinate system, and the spatial point information of the target is represented as a spatial point P'=[X,Y,0] in the horizontal coordinate system, and the spatial transformation relationship described by the camera extrinsic parameters is satisfied. As an update constraint, R represents the rotation relationship from the camera coordinate system to the horizontal coordinate system, and T represents the displacement relationship from the camera coordinate system to the horizontal coordinate system. For the spatial point information of the golf ball acquired multiple times, corresponding spatial transformation constraint relationships are established respectively, and multiple sets of constraint relationships are processed simultaneously. By adjusting the rotation relationship R and the displacement relationship T, the spatial transformation relationship is made to simultaneously satisfy the correspondence consistency between multiple sets of golf ball spatial point information and target spatial point information, thereby completing the update processing of the camera extrinsic parameters and obtaining the updated camera extrinsic parameters.

[0110] In one embodiment, such as Figure 4As shown, in step S30, the image information of the clubface contact point collected during the swing is acquired, and the image coordinate information of the clubface contact point is obtained from the image information of the clubface contact point, including:

[0111] S301: Continuously acquire multiple frames of images containing clubface contact points during the swing to obtain multi-frame clubface contact point image information.

[0112] In this embodiment, multi-frame clubface contact image information refers to a set of image data that are continuously acquired in chronological order during the same swing, all of which contain clubface contact points, and is used to characterize the continuous imaging state of the clubface contact points during the swing.

[0113] Specifically, when continuously acquiring multiple frames of images containing clubface contact points during the swing, the swing process is continuously imaged according to a preset image acquisition time interval from the start to the end of the swing, and multiple frames of images containing clubface contact points are acquired sequentially. These multiple frames are numbered according to the acquisition order as the i-th frame image, where the value of i ranges from 1 ≤ i ≤ n, and n represents the number of frames in the multiple frames image, where n is a positive integer. The imaging position of the clubface contact points contained in each frame image in the image coordinate system is represented as two-dimensional image coordinates. This results in a multi-frame pole surface sticker image information composed of multiple frames of images and their corresponding pole surface sticker image coordinates.

[0114] S302: For each frame of pole face sticker image information, determine the two-dimensional position of the pole face sticker in the corresponding image, and obtain the pole face sticker image coordinate information that corresponds one-to-one with the pole face sticker image information of each frame.

[0115] In this embodiment, the clubface contact image coordinate information refers to the two-dimensional coordinate representation used to characterize the imaging position of the clubface contact in the corresponding frame image. It corresponds one-to-one with the multi-frame clubface contact image information in time sequence and is used to describe the image position change of the clubface contact during the swing.

[0116] Specifically, when determining the two-dimensional position of the pole face sticker in the corresponding image for each frame of pole face sticker image information, the image content in the i-th frame of pole face sticker image information is processed to determine the sticker position. The pixel region corresponding to the pole face sticker is located in the i-th frame image, and the region is used to characterize the pole face sticker. The pixel center point coordinates of the point imaging position are represented as two-dimensional image coordinates, where xi and yi represent the horizontal and vertical pixel coordinates of the pole surface attachment point in the i-th frame image, respectively. By performing the above two-dimensional position determination processing on the pole surface attachment point image information of multiple frames, the pole surface attachment point image coordinate information corresponding one-to-one with the pole surface attachment point image information of each frame is obtained.

[0117] In one embodiment, such as Figure 5 As shown, in step S40, based on the camera intrinsic parameters, the updated camera extrinsic parameters, and the relative positional relationship between the pole surface contact points, the predicted position information corresponding to the pole surface contact point image coordinate information is determined, including:

[0118] S401: Determine the relative positional relationship information of the clubface contact points by utilizing the relative positional relationship between the contact points.

[0119] In this embodiment, the relative positional relationship information of pole face stickers refers to the geometric representation used to characterize the positional relationship between multiple pole face stickers in the same frame image under the image coordinate system. This positional relationship is used to reflect the fixed relative distribution state of each pole face sticker on the pole face.

[0120] Specifically, when determining the relative positional relationship information of pole face stickers using the relative positional relationship between them, the two-dimensional position of the i-th pole face sticker in the image coordinate system is represented as follows in the pole face sticker image coordinate information corresponding to the same frame of pole face sticker image: The two-dimensional position of the j-th rod surface contact point in the image coordinate system is represented as: Where i and j are used to distinguish different pole surface points in the same frame image, and the value of i ranges from 1 to 1. The range of values ​​for j is 'm' represents the number of identifiable pole facet points in the same frame image, where 'm' is a positive integer. The relative displacement relationship between different pole facet points is calculated based on their two-dimensional positions and expressed as... By calculating and recording the relative displacement relationships between the pole surface stickers in the same frame image, information on the relative positional relationships of the pole surface stickers is formed to describe the fixed geometric distribution characteristics of the pole surface stickers in the image coordinate system.

[0121] S402: Based on the camera intrinsic parameters, the updated camera extrinsic parameters, and the relative positional relationship information of the pole surface contact points, determine the spatial position information of the pole surface contact points in the camera coordinate system.

[0122] In this embodiment, the spatial position information of the pole facet points in the camera coordinate system refers to the spatial point representation used to characterize the three-dimensional position of the pole facet points in the camera coordinate system. The camera intrinsic parameters are used to characterize the imaging geometric parameters between the image coordinate system and the camera coordinate system. The updated camera extrinsic parameters are used to characterize the spatial pose relationship between the camera coordinate system and the reference coordinate system. The relative position relationship information of the pole facet points refers to the geometric constraint information used to characterize the relative displacement relationship between multiple pole facet points. The relative displacement relationship can be represented by the coordinate difference between the points.

[0123] Specifically, when determining the spatial position information of the pole surface attachment point in the camera coordinate system, the image coordinates of the k-th pole surface attachment point in the i-th frame image are represented as follows: Where i represents the frame number, and k represents the pole surface patch number within the same frame image, with the value of k ranging from 1 to 2. Let q represent the number of identifiable pole facet points in a single frame image, and q is a positive integer. The spatial points of the corresponding pole facet points in the i-th frame image in the camera coordinate system are represented as follows: ,in , , These are used to characterize the three coordinate components of a spatial point in the camera coordinate system, and are established based on the camera intrinsic parameters and the updated camera extrinsic parameters. and The projection mapping relationship between them makes each After projection mapping and corresponding To maintain consistency, based on this, relative displacement constraints between the sticker points are constructed according to the relative positional relationship information of the sticker points on the pole face. The image coordinate difference between any two sticker points on the pole face in the i-th frame image is expressed as... and in accordance with Determine the relative displacement, where the subscripts i and j are used to distinguish different pole surface contact points within the same image frame. Use the relative displacement and projection mapping relationship as constraints, and solve them simultaneously to find the values ​​corresponding to each pole surface contact point. Simultaneously satisfying both projection consistency constraints and relative displacement constraints, we can obtain the spatial position information of the pole surface contact point in the camera coordinate system.

[0124] S403: Map the spatial position information of the pole facet point in the camera coordinate system to the image plane according to the camera intrinsic parameters and the updated camera extrinsic parameters, and obtain the predicted position information of the pole facet point corresponding to the image coordinate information of the pole facet point.

[0125] In this embodiment, the predicted position information of the pole face is a two-dimensional position representation obtained by projecting the spatial position information of the pole face in the camera coordinate system onto the image plane based on the updated calibration parameter information. This representation is used to correspond the position of the pole face image coordinate information of the corresponding frame.

[0126] Specifically, when mapping the spatial position information of the pole surface attachments in the camera coordinate system to the image plane, the spatial position information of each pole surface attachment in the i-th frame in the camera coordinate system is represented as follows: Based on the updated calibration parameters containing the camera intrinsic and extrinsic parameters, an imaging mapping relationship from the camera coordinate system to the image plane is established. Imaging mapping processing is then performed on the spatial position information, making the spatial position information correspond to a two-dimensional position representation on the image plane. The mapped two-dimensional position is then represented as the predicted position information of the pole surface contact point. The predicted position information of the pole face patch is consistent with the coordinate information of the pole face patch image corresponding to the i-th frame in terms of frame number and patch number, thus obtaining the predicted position information of the pole face patch corresponding to the coordinate information of the pole face patch image.

[0127] In one embodiment, such as Figure 6 As shown, in step S50, multi-frame error information is obtained based on the deviation between the predicted position information of the pole face and the coordinate information of the pole face image, including:

[0128] S501: For each frame of clubface contact image acquired during the swing, compare the predicted position information of the corresponding clubface contact in each frame with the coordinate information of the corresponding clubface contact in each frame to determine the corresponding deviation information.

[0129] In this embodiment, the deviation information refers to the quantitative representation of the positional difference between the predicted position information of the pole face and the coordinate information of the pole face image in the same frame image, and it corresponds one-to-one with the frame number and the pole face number.

[0130] Specifically, for the i-th frame of the clubface contact image acquired during the swing, the corresponding predicted position information of the clubface contact is represented as follows: The corresponding pole face contact point image coordinate information is represented as follows: By comparing the predicted position information and image coordinate information of the same patch point k within the same frame one by one, the coordinate difference between the two on the image plane is calculated, and the coordinate difference is represented as deviation information:

[0131] By performing the above comparison and difference calculation on each frame of clubface contact image captured during the swing, deviation information corresponding to each frame of clubface contact image is obtained.

[0132] S502: Collect multi-frame deviation information according to the frame order during the swing process to obtain multi-frame error information.

[0133] In this embodiment, multi-frame error information refers to the error data set formed by sequentially combining the deviation information corresponding to each frame according to the time sequence of image acquisition during the swing process. It is used to characterize the temporal change of the clubface contact point deviation during the swing process.

[0134] Specifically, when collecting multi-frame deviation information according to the frame order during the swing, the deviation information for the i-th frame clubface contact image acquired during the swing is represented as follows:

[0135] According to frame number The deviation information corresponding to each frame is arranged and combined sequentially according to the time order, and the deviation information corresponding to different patch numbers k in the same frame is included in the collection result, thereby forming an ordered error sequence containing deviation information of multiple frames, which serves as multi-frame error information.

[0136] S503: Perform average value calculation on the error information of multiple frames to obtain error index information.

[0137] In this embodiment, the error index information refers to the quantitative result used to characterize the overall deviation level, obtained by summarizing and calculating the error information of multiple frames generated during the swing.

[0138] Specifically, when calculating the average value of multi-frame error information, the deviation information corresponding to the i-th frame in the multi-frame error information is represented as follows: Based on this, for the n frames of images acquired during the swing, the deviation information corresponding to each frame is summed according to the frame number, and the summation result is divided by the number of frames n to obtain the average result of the error information of multiple frames in the time dimension. The average result is used as the error index information. .

[0139] S504: Compare the error index information with the preset threshold information to obtain the drift determination result.

[0140] In this embodiment, the drift determination result refers to the determination output formed based on the magnitude relationship between the error index information and the preset threshold information, which is used to characterize whether the current calibration parameter state has shifted.

[0141] Specifically, when comparing the error index information with the preset threshold information, the error index information obtained by averaging the error information from multiple frames is represented as follows: The preset threshold information is represented as T by the error index information. The value is compared with the preset threshold information T. When the condition is met... At that time, drift determination result information indicating that drift has occurred is generated, when the conditions are met. At that time, drift determination result information indicating that no drift has occurred is generated, thereby obtaining the drift determination result.

[0142] In one embodiment, such as Figure 7 As shown, in step S60, when the drift determination result indicates that the updated calibration parameters need to be corrected, the updated calibration parameters are updated based on the coordinate information of the pole face contact image to obtain the corrected calibration parameters, including:

[0143] S601: When the drift determination result indicates that the updated calibration parameters need to be corrected, based on the constraint conditions formed by the coordinate information of the pole surface patch images in multiple frames, the updated calibration parameters are used as the variables to be adjusted, and iterative update processing is performed on the updated calibration parameters until the preset convergence condition is met, and the corrected calibration parameters are obtained.

[0144] In this embodiment, the coordinate information of the multi-frame pole surface attachment images is a set of two-dimensional positions of the pole surface attachments on the image plane in each frame of the command pole process. The coordinate information of the multi-frame pole surface attachment images is used as input for parameter update processing when the drift determination result indicates that drift has occurred. As input means that the coordinate information of the multi-frame pole surface attachment images is written as a known quantity into the subsequent calculation process of constructing corresponding relationship constraints and solving simultaneous problems. The updated calibration parameters refer to the combination result of the camera intrinsic parameters and camera extrinsic parameters obtained after the previous round of calibration parameter update. The corrected calibration parameters refer to the calibration parameter results obtained by resolving through iterative update processing while keeping the coordinate information of the multi-frame pole surface attachment images unchanged.

[0145] Specifically, when the drift determination result indicates that drift has occurred, the coordinate information of the clubface contact points acquired during the swing is collected according to the frame number, and the corresponding clubface contact point coordinates in each frame are represented as p'1, ​​p'2, p'3… respectively. The camera extrinsic parameters obtained from the previous calibration parameter update are represented as R' and T'. p'1, ​​p'2, p'3… and R' and T' are used as known quantities to participate in the subsequent construction and simultaneous solution of corresponding relationship constraints. Based on this, the camera intrinsic parameters and camera extrinsic parameters in the updated calibration parameters are represented as K, R, and T respectively, and K, R, and T are introduced as variables to be adjusted into the simultaneous solution. In the solution process, the simultaneous solution is used to iteratively update the variables to be adjusted under the constraint of the coordinate information of the pole surface image. In each iteration, the camera intrinsic parameter K is kept unchanged and the camera extrinsic parameters R and T are solved and updated. Then, the updated camera extrinsic parameters R and T are kept unchanged and the camera intrinsic parameter K is solved and updated. The iterative update process is performed cyclically in the above alternating update method. When the iterative update reaches a predetermined number of iterations or the error calculated based on p'1, ​​p'2, p'3... is not greater than a preset threshold, the iterative update process ends. The camera intrinsic parameter K and camera extrinsic parameters R and T obtained by the iterative solution are determined as the corrected calibration parameters.

[0146] In one embodiment, such as Figure 8 As shown, after step S60, i.e., a calibration method for a golf vision sensor, it further includes:

[0147] S70: Obtain the joint zero-position parameters and link parameters of the robotic arm to obtain the initial calibration parameters of the robotic arm.

[0148] In this embodiment, the initial calibration parameters of the robotic arm refer to the set of parameters used to characterize the initial pose relationship and link geometry relationship of each joint of the robotic arm under the zero-position reference. The joint zero-position parameters refer to the parameters used to characterize the offset relationship between the zero-position reference of each joint and the actual reading of the joint. The link parameters refer to the parameters used to characterize the link geometry relationship between adjacent joints.

[0149] Specifically, when acquiring the joint zero-position parameters and link parameters of the robotic arm, the joint zero-position parameter table corresponding to the robotic arm is read from the preset parameter storage area, and the zero-position offset value corresponding to each joint is recorded as bg, where the subscript g is used to distinguish different joints. The link parameter table corresponding to the robotic arm is also read from the preset parameter storage area, and the link geometric parameters between adjacent joints are recorded as the link length parameter Lg and the link torsion angle parameter, respectively. Linkage offset parameter dg, link rotation angle parameter Where Lg represents the geometric length relationship between adjacent joint axes, dg represents the angular relationship between adjacent joint axes, and dg represents the offset relationship between adjacent joint axes in the height direction. Characterizing the angular relationship between adjacent joint axes in the rotational direction, based on the joint zero-position offset value bg and the link parameter Lg, dg The parameters are collected and structured to form the initial calibration parameters for the robotic arm.

[0150] S80: Collects pose information of the robotic arm's end effector in multiple preset postures, updates the initial calibration parameters of the robotic arm, and obtains the updated calibration parameters of the robotic arm.

[0151] In this embodiment, the pose information of the end effector of the robotic arm refers to the parameter representation used to characterize the position and attitude state of the end effector in space. Multiple preset poses refer to different spatial attitude states of the end effector of the robotic arm that are preset for calibration. The updated calibration parameters of the robotic arm refer to the parameter results obtained after correcting the initial calibration parameters of the robotic arm under the pose information constraints corresponding to multiple preset poses.

[0152] Specifically, when collecting the pose information of the robotic arm's end effector in multiple preset postures, the robotic arm is driven to move sequentially according to a pre-set posture sequence, so that the end effector is in multiple different spatial posture states. The corresponding end effector pose information is acquired in each preset posture. The pose information of the end effector in the s-th preset posture is represented as Ts, where s represents the preset posture number, and the value of s ranges from... v represents the number of preset postures, where v is a positive integer. After obtaining the pose information corresponding to multiple preset postures, the pose information is introduced as a known quantity into the parameter update process with the initial calibration parameters of the robot arm as unknowns. By adjusting the correspondence between the end effector pose information under multiple preset postures and the end effector pose calculated based on the initial calibration parameters of the robot arm, the initial calibration parameters of the robot arm are updated to obtain the updated calibration parameters of the robot arm that satisfy the constraints of multiple preset postures.

[0153] In one embodiment, a calibration device for a golf vision sensor is provided, which corresponds one-to-one with the calibration method for a golf vision sensor described in the above embodiments. For example... Figure 9 As shown, the calibration device of this golf vision sensor includes an image acquisition module, a calibration parameter update module, a spot image information acquisition module, a predicted position information determination module, a deviation analysis module, and a calibration parameter correction module.

[0154] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0155] Acquire an image of a golf ball located on a horizontal plane, determine the spatial point information of the golf ball based on the imaging information of the golf ball image, and update the camera extrinsic parameters based on the constraint that the spatial point information of the golf ball is located on the horizontal plane, so as to obtain the updated camera extrinsic parameters.

[0156] Based on the camera intrinsic parameters and the updated camera extrinsic parameters, the updated calibration parameters are obtained;

[0157] Acquire the image information of the clubface contact points collected during the swing, and extract the image coordinate information of the clubface contact points from the image information of the clubface contact points;

[0158] Based on the camera intrinsic parameters, the updated camera extrinsic parameters, and the relative positional relationship between the pole surface contact points, the predicted positional information corresponding to the pole surface contact point image coordinate information is determined.

[0159] Based on the deviation between the predicted position information and the coordinate information of the pole surface patch image, the validity of the updated calibration parameters is determined, and the drift determination result is obtained.

[0160] If the drift determination result indicates that the updated calibration parameters need to be corrected, the updated calibration parameters are updated based on the coordinate information of the pole face image to obtain the corrected calibration parameters.

[0161] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A calibration method for a golf vision sensor, characterized in that, The calibration method for the golf vision sensor includes: Acquire an image of a golf ball located on a horizontal plane, determine the spatial point information of the golf ball based on the imaging information of the golf ball image, and update the camera extrinsic parameters based on the constraint that the spatial point information of the golf ball is located on the horizontal plane, to obtain the updated camera extrinsic parameters; Based on the camera intrinsic parameters and the updated camera extrinsic parameters, the updated calibration parameters are obtained; Acquire the image information of the clubface contact points collected during the swing, and obtain the image coordinate information of the clubface contact points from the image information of the clubface contact points; Based on the camera intrinsic parameters, the updated camera extrinsic parameters, and the relative positional relationship between the pole surface contact points, the predicted position information corresponding to the image coordinate information of the pole surface contact points is determined. Based on the deviation between the predicted position information and the coordinate information of the pole surface attachment image, the validity of the updated calibration parameters is determined to obtain the drift determination result; If the drift determination result indicates that the updated calibration parameters need to be corrected, the updated calibration parameters are updated based on the coordinate information of the pole face image to obtain the corrected calibration parameters.

2. The calibration method for a golf vision sensor according to claim 1, characterized in that, The step of acquiring an image of a golf ball located on a horizontal plane and determining spatial point information of the golf ball based on the imaging information of the golf ball image includes: Obtain the two-dimensional position of the golf ball in the image and the pixel radius of the golf ball in the image; Obtain the preset radius of the golf ball; The focal length is determined based on the focal length parameter in the camera intrinsic parameters. The distance to the golf ball is calculated based on the focal length, the preset radius, and the pixel radius. The spatial point information of the golf ball is calculated based on the distance and the two-dimensional position.

3. The calibration method for a golf vision sensor according to claim 1, characterized in that, The camera extrinsic parameters are updated based on the constraint that the golf ball spatial point information is located on the horizontal plane, resulting in updated camera extrinsic parameters, including: The golf ball spatial point information is converted into target spatial point information in a horizontal coordinate system, wherein the target spatial point information satisfies that the height direction coordinate is zero. The correspondence between the spatial point information of the golf ball and the spatial point information of the target is established using the camera extrinsic parameters. The correspondence constraints corresponding to the spatial point information of the golf ball acquired multiple times are solved simultaneously to obtain the result of the simultaneous solution. The camera extrinsic parameters are updated based on the results of the simultaneous solution to obtain the updated camera extrinsic parameters.

4. The calibration method for a golf vision sensor according to claim 1, characterized in that, The process of acquiring clubface contact image information collected during the swing and obtaining image coordinate information of the clubface contact points from the clubface contact image information includes: During the swing, multiple frames of images containing clubface contact points are continuously acquired to obtain multi-frame clubface contact point image information; For each frame of the pole face sticker image information, the two-dimensional position of the pole face sticker in the corresponding image is determined, and the pole face sticker image coordinate information corresponding one-to-one with the pole face sticker image information of each frame is obtained.

5. The calibration method for a golf vision sensor according to claim 1, characterized in that, The step of determining the predicted position information corresponding to the image coordinate information of the pole surface contact point based on the camera intrinsic parameters, the updated camera extrinsic parameters, and the relative positional relationship between the pole surface contact points includes: The relative positional relationship information of the pole face contact points is determined by utilizing the relative positional relationship between the contact points. Based on the camera intrinsic parameters, the updated camera extrinsic parameters, and the relative positional relationship information of the pole surface attachment points, the spatial position information of the pole surface attachment points in the camera coordinate system is determined. The spatial position information of the pole surface attachment point in the camera coordinate system is mapped to the image plane according to the camera intrinsic parameters and the updated camera extrinsic parameters to obtain the predicted position information of the pole surface attachment point corresponding to the image coordinate information of the pole surface attachment point.

6. The calibration method for a golf vision sensor according to claim 1, characterized in that, The determination of the validity of the updated calibration parameters based on the deviation between the predicted position information and the coordinate information of the pole surface attachment image, to obtain a drift determination result, includes: For each frame of clubface contact image captured during the swing, the predicted position information of the corresponding clubface contact in each frame of clubface contact image is compared with the coordinate information of the corresponding clubface contact image in each frame of clubface contact image to determine the corresponding deviation information; The deviation information from multiple frames is collected in the frame order of the swing process to obtain multi-frame error information; The average value of the multi-frame error information is calculated to obtain error index information; The error index information is compared with the preset threshold information to obtain the drift determination result.

7. The calibration method for a golf vision sensor according to claim 1, characterized in that, When the drift determination result indicates that the updated calibration parameters need to be corrected, the updated calibration parameters are updated based on the coordinate information of the pole face contact image to obtain the corrected calibration parameters, including: When the drift determination result indicates that the updated calibration parameters need to be corrected, based on the constraint conditions formed by the coordinate information of the multi-frame pole surface contact image, the updated calibration parameters are used as the variables to be adjusted, and iterative update processing is performed on the updated calibration parameters until the preset convergence condition is met, and the corrected calibration parameters are obtained.

8. The calibration method for a golf vision sensor according to claim 1, characterized in that, The calibration method for the golf vision sensor further includes: Obtain the joint zero-position parameters and link parameters of the robotic arm to obtain the initial calibration parameters of the robotic arm; The positional information of the robotic arm end effector in multiple preset postures is collected, and the initial calibration parameters of the robotic arm are updated to obtain the updated calibration parameters of the robotic arm.

9. A calibration device for a golf vision sensor, characterized in that, The calibration device for the golf vision sensor includes: The image acquisition module is used to acquire an image of a golf ball located on a horizontal plane, determine the spatial point information of the golf ball based on the imaging information of the golf ball image, and update the camera extrinsic parameters based on the constraint that the spatial point information of the golf ball is located on the horizontal plane, so as to obtain the updated camera extrinsic parameters. The calibration parameter update module is used to obtain the updated calibration parameters based on the camera intrinsic parameters and the updated camera extrinsic parameters; The clubface image information acquisition module is used to acquire clubface image information collected during the swing, and to obtain the image coordinate information of the clubface image from the clubface image information; The predicted position information determination module is used to determine the predicted position information corresponding to the image coordinate information of the pole surface point based on the camera intrinsic parameters, the updated camera extrinsic parameters, and the relative positional relationship between the pole surface points; The deviation analysis module is used to determine the validity of the updated calibration parameters based on the deviation between the predicted position information and the coordinate information of the pole surface attachment image, and to obtain the drift determination result. The calibration parameter correction module is used to update the updated calibration parameters based on the coordinate information of the pole face contact image when the drift determination result indicates that the updated calibration parameters need to be corrected, so as to obtain the corrected calibration parameters.

10. A non-transitory computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the calibration method for a golf vision sensor as described in any one of claims 1 to 8.

Images