Image-based lost ball automatic classification method using deep learning algorithm

Abstract

Disclosed is an image-based lost ball automatic classification method comprising: a learning image generation step of generating a brand learning image for learning a brand of a golf ball and a defect learning image for learning a defect; a golf ball brand learning step of training an image classification algorithm about the brand of the golf ball by using the brand learning image; a golf ball defect learning step of training an image segmentation algorithm about the defect in the golf ball by using the defect learning image; a target image generation step of generating at least eight target images from a target lost ball to be classified; a target lost ball brand classification step of classifying a brand of the target lost ball from a target image by using the trained image classification algorithm; and a target lost ball grade classification step of classifying a grade of the target lost ball from the target image by using the trained image segmentation algorithm.

Description

Image-based automatic lost ball classification method using deep learning algorithms

[0001] The present invention relates to a method for automatically classifying the brand and grade of lost balls based on images using a deep learning algorithm.

[0002] Golf balls have grooves called dimples formed on their surface to reduce air resistance and increase distance. The dimples on the golf ball may be formed differently depending on the brand, including the number and shape of the grooves. Furthermore, the shape of the dimples is protected by patents for each manufacturer. Therefore, the brand of the golf ball can be identified from the shape of the dimples without recognizing the brand mark.

[0003] Generally, a lost ball refers to a golf ball that enters a wood or water during a round and cannot be found. The lost balls are collected, recycled, and resold. The selling price of the lost balls may be determined by classifying them into grades based on their brand and surface condition. The brand and surface condition of the lost balls may be inspected visually. The lost balls may be classified into grades such as Grade 1, Grade 2, and Grade 3, for example, based on their brand and surface condition.

[0004] Visual inspection of the aforementioned lost balls requires a significant amount of manpower and time. Furthermore, if the brand is erased or damaged, it may be difficult to accurately identify the brand of the lost balls. Additionally, when determining the grade of the lost balls based on surface condition through visual inspection, the basis for judgment is inconsistent, so the grade may be classified differently depending on the subjectivity of the inspector.

[0005] The present invention aims to provide an image-based automatic classification method for lost balls that can automatically classify the brand and grade of lost balls quickly and accurately using a deep learning algorithm.

[0006] An image-based automatic lost ball classification method according to an embodiment of the present invention for solving the above-mentioned problems comprises: a training image generation step of generating a brand learning image for learning the brand of a golf ball and a defect learning image for learning defects; a golf ball brand learning step of learning the brand of the golf ball for an image classification algorithm using the brand learning image; a golf ball defect learning step of learning the defects of the golf ball for an image segmentation algorithm using the defect learning image; a target image generation step of generating at least 8 target images from a target lost ball to be classified; a target lost ball brand classification step of classifying the brand of the target lost ball from the target images using the learned image classification algorithm; and a target lost ball grade classification step of classifying the grade of the target lost ball from the target images using the learned image segmentation algorithm.

[0007] In addition, the golf ball brand learning step is carried out by matching the brand learning image with the golf ball brand, and golf ball brand classification information can be generated according to the characteristics of the dimples for each golf ball brand.

[0008] In addition, the golf ball defect learning step is performed by matching the defect learning image with the assigned grade, and can generate golf ball defect classification information based on the size of the defect in the defect learning image.

[0009] In addition, the target image generation step can generate 14 target images by taking photos in 6 directions in the face direction and 8 directions in the vertex direction based on the virtual cube that the target lost ball accommodates inside.

[0010] In addition, the target image generation step can generate 8 target images by taking photos in 4 directions in the face direction and 4 directions in the vertex direction based on the tetrahedron that the target lost ball accommodates inside.

[0011] In addition, the target lost ball brand classification step can generate brand information for the target lost ball from the target image.

[0012] In addition, the above-mentioned target lost ball brand classification step can verify the authenticity of the brand displayed on the above-mentioned reused lost ball.

[0013] In addition, the target lost ball grade classification step may classify grades for all of the plurality of target images generated for the target lost ball, and generate the lowest grade among the grades assigned to each of the target images as grade information for the target lost ball.

[0014] The image-based automatic lost ball classification method of the present invention uses a deep learning algorithm to automatically classify the brand and grade of the lost ball from the image of the lost ball, so the brand and grade of the lost ball can be classified quickly and accurately.

[0015] In addition, the image-based automatic lost ball classification method of the present invention classifies the brand of the lost ball using the dimples of the lost ball, so the brand can be classified quickly and accurately even if the brand name is damaged.

[0016] In addition, the image-based automatic lost ball classification method of the present invention classifies the brand of the lost ball using the dimples of the lost ball, so the real brand of the reused lost ball can be accurately identified.

[0017] FIG. 1 is a flowchart of an image-based automatic lost ball classification method according to one embodiment of the present invention.

[0018] Figure 2 is a photograph showing the shape of dimples by brand of golf ball.

[0019] Figure 3 is a photograph showing the shape of a defect in a golf ball.

[0020] Figure 4 is a diagram showing the direction of generating 14 target images from a target lost ball.

[0021] Figure 5 is a diagram showing the valid area of ​​the target image in the target lost ball.

[0022] Figure 6 is a diagram showing the direction of generating 8 target images from a target lost ball.

[0023] Figure 7 is a configuration diagram of an image-based automatic classification system for lost ball brands and grades.

[0024] Hereinafter, an image-based automatic lost ball classification method using a deep learning algorithm according to an embodiment of the present invention will be described in more detail with reference to the embodiments and the attached drawings.

[0025]

[0026] First, an image-based automatic lost ball classification method according to one embodiment of the present invention will be described.

[0027] FIG. 1 is a flowchart of an image-based automatic lost ball classification method according to an embodiment of the present invention. FIG. 2 is a photograph showing the shape of dimples by brand of golf ball. FIG. 3 is a photograph showing the shape of defects in a golf ball. FIG. 4 is a diagram showing the direction of generating 14 target images from a target lost ball. FIG. 5 is a diagram showing the valid area of ​​the target image in the target lost ball. FIG. 6 is a diagram showing the direction of generating 8 target images from a target lost ball.

[0028] An image-based automatic lost ball classification method using a deep learning algorithm according to one embodiment of the present invention may include, with reference to FIGS. 1 to 6, a training image generation step (S10), a golf ball brand learning step (S20), a golf ball defect learning step (S30), a target image generation step (S40), a target lost ball brand classification step (S50), and a target lost ball grade classification step (S60).

[0029] The above image-based automatic lost ball classification method uses a deep learning algorithm to learn the brand and defects of golf balls and can automatically classify the brand and grade of the target lost balls. Therefore, the above image-based automatic lost ball classification method can classify the brand and grade of the target lost balls quickly and accurately.

[0030] The above image-based automatic lost ball sorting method may be applied to target lost balls collected from a golf course and subjected to a surface cleaning process. Here, the target lost balls may refer to golf balls that have entered woods or water during a golf round and cannot be found. Additionally, the target lost balls may include remanufactured lost balls that have had a brand printed on their surface after being newly painted. Furthermore, the automatic sorting may include brand classification and grade classification of the target lost balls. The brand classification may include verifying the genuine brand of remanufactured lost balls among the target lost balls. In the following, the golf ball may refer to an unused golf ball, a lost ball, or a remanufactured lost ball.

[0031] The above-described training image generation step (S10) is a step of generating a training image of a golf ball. The training image may include a brand training image for brand learning and a defect training image for defect learning. The training image generation step (S10) may be generated by photographing a golf ball with a camera at a predetermined angle. The brand training image may be generated using an unused golf ball. Additionally, the brand training image may be generated using a lost ball. However, when generating the brand training image using a lost ball, it may be generated using a surface without defects or a surface where the dimples are not damaged. As shown in FIG. 2, the brand training image may be generated for golf balls of various brands. The brand training image may be generated for multiple golf balls of the same brand. Additionally, the brand training image may be generated as multiple images by photographing a single golf ball from various angles. The brand training image may be generated by matching the brand of the corresponding golf ball to each image.

[0032] The above brand learning images can be generated using all golf balls of currently sold brands or golf balls of well-known brands. Additionally, the above brand learning images can be generated by enlarging the golf ball by a certain ratio based on its actual size.

[0033] Additionally, the defect learning image can be generated using a lost ball containing defects, as shown in FIG. 3. Furthermore, the defect learning image can be generated as an image containing defects of various sizes and shapes. The defect learning image can be generated such that the defect is located approximately in the center. The defect learning image may be an image of a golf ball taken from above with the golf ball positioned so that the defect is at the top. Thus, the defect learning image may be an image containing half of the golf ball.

[0034] The above defects may take the form of scribbles, stains, scratches, or paint peeling formed on the surface of the golf ball. The golf balls may be selected from among lost balls based on the size and shape of the defects. The defect learning images may be graded based on the size of the defects. The defect learning images may be classified into grades 1 to 3 according to the size of the defects. For example, Grade 1 may be assigned when there are no defects. Additionally, Grade 2 may be assigned when the defect size is small, such as a scratch of 0.1 mm or less, and the number of defects is 5 or fewer. Additionally, Grade 3 may be assigned when the defect size is larger than 0.1 mm or the number of defects is 5 or more. Here, the size of the defect may refer to the width of a scratch shape, the diameter of a circle shape, or the width of an approximate square shape. The grades based on the size of the defects may be adjusted as necessary. In addition, the defect learning image may be graded by separating and observing defects when multiple defects are located adjacently. Each defect in the defect learning image may be graded. The defects may be classified into grades after post-processing using thresholds, minimum sizes, etc. Referring to FIG. 3, the defect learning image may be an image containing scratches of 0.1 mm or less (a) or containing multiple defects (b). Additionally, the defect learning image may be generated by enlarging the actual size of the golf ball by a certain ratio.

[0035] In addition, the defect learning images can be generated by matching the grade of the corresponding defect to each. The defect learning images can be formed in multiple numbers for defects of various grades. For example, the defect learning images can be generated by acquiring about 100 lost balls for each grade.

[0036] The above-mentioned training image generation step (S10) may be performed in a device comprising a camera that generates an image necessary for recognizing an image of a golf ball using the camera, a control unit that controls the camera and generates a training image, and a storage unit for the generated training image.

[0037]

[0038] The above golf ball brand learning step (S20) is a step of learning the brand of a golf ball for an image classification algorithm using an image of a golf ball containing dimples for brand learning. The golf ball may have unique dimples formed for each brand. The dimples may refer to grooves arranged on the surface of the golf ball with a certain shape. The dimples generate lift during the flight of the golf ball caused by impact, thereby increasing the flight time and distance. The dimples are designed to have structural elements including arrangement, diameter, planar shape, and depth, and influence the flight characteristics of the golf ball. It can be confirmed that the golf ball has dimples with different structural elements for each brand.

[0039] The golf ball brand learning step (S20) described above can train an image classification algorithm using brand learning images of golf balls for each brand. That is, the golf ball brand learning step (S20) can proceed by matching and providing the golf ball brand learning images with the golf ball brands. The image classification algorithm may be an algorithm such as ResNet, AlexNet, VGG, or LeNet. Additionally, the image classification algorithm may be an algorithm such as ResNext or EfficientNet. For the convenience of training, the image classification algorithm may retrieve weights from a pre-trained model and adjust the class to match the individual data of the training target to facilitate training. The image classification algorithm may use ResNet as a backbone for training.

[0040] The golf ball brand learning step (S20) can derive features regarding structural elements of dimples from brand learning images and match them with the provided brands. Accordingly, the golf ball brand learning step (S20) can generate golf ball brand classification information based on the characteristics of dimples for each golf ball brand. Multiple images of the golf ball may be provided for each brand. For example, at least 20 brand learning images may be provided for each brand. Additionally, 80% of the total number of brand learning images may be used for learning, 10% for verification, and 10% for testing. The golf ball brand learning step (S20) may be repeated until the accuracy in verification and testing is 98% or higher.

[0041] The above golf ball brand learning step (S20) can be performed in a device comprising a camera that generates video images necessary for recognizing provided brand learning images, a control unit that controls the camera while performing learning of an image classification algorithm, and a storage unit that stores generated golf ball brand classification information.

[0042]

[0043] The golf ball defect learning step (S30) described above is a step of training an image segmentation algorithm for defects of a golf ball using defect learning images containing defects. The golf ball defect learning step (S30) may train the image segmentation algorithm by providing defect images of the golf ball for each defect size. The image segmentation algorithm may be an algorithm such as FCN, SegNet, or UNET. The image segmentation algorithm may be trained using ResNet as a backbone.

[0044] The above golf ball defect learning step (S30) can be performed by matching and providing defect learning images with assigned grades. Multiple defect learning images may be provided for each grade. For example, at least 20 defect learning images may be provided for each grade. Additionally, 80% of the total number of defect learning images may be used for learning, 10% for verification, and 10% for testing. The above golf ball brand learning step (S20) may be performed repeatedly so that the accuracy in verification and testing is 98% or higher.

[0045] The above golf ball defect learning step (S30) can derive the size of the defect from the defect learning image and assign a grade. Accordingly, the above golf ball defect learning step (S30) can generate golf ball defect classification information based on the size of the defect in the defect learning image.

[0046] The above golf ball defect learning step (S30) may be performed in a device comprising a camera that generates an image image necessary for recognizing defects in a provided defect learning image, a control unit that controls the camera while performing learning of an image segmentation algorithm, and a storage unit that stores the generated golf ball defect classification information.

[0047]

[0048] The target image generation step (S40) is a step of generating at least 8 target images from the target lost ball to be classified. The target image generation step (S40) can preferably generate 14 target images. Referring to FIG. 4, the target image generation step (S40) (S30) can generate 14 target images by taking photos in 6 directions in the face direction and 8 directions in the vertex direction based on the virtual cube (20) that the target lost ball (10) accommodates inside. Referring to FIG. 5, the target image may be an image that includes an effective area (10a) of 90 degrees, extending 45 degrees upward and downward with respect to the central axis (a) extending from the camera (30) to the target lost ball (10). Accordingly, the entire surface of the target lost ball can be covered by 6 target images taken in 6 directions in the face direction. However, since the target lost ball is spherical, the size of the defect may be captured as smaller than the actual size as it moves further away from the center of the target image. Therefore, the target image may be captured from the direction of the vertices of the cube, and eight additional images may be generated for the area between the six target images. Additionally, the target image generation step (S40) may generate additional target images from other directions if necessary.

[0049] Additionally, the target image generation step (S40), referring to FIG. 6, can generate eight target images by taking photos in four directions in the face direction and four directions in the vertex direction based on the tetrahedron (40) that the target lost ball (10) accommodates inside. Although not specifically illustrated, the target images may be images that include a valid area of ​​120 degrees, with 60 degrees on each side based on a central axis extending from the camera to the target lost ball, similar to the case of a cube. Additionally, the target image generation step (S40) can generate additional target images from other directions if necessary.

[0050] The target image generation step (S40) above can generate a target image while rotating the target lost ball using a separate rotation means. The target image generation step (S40) can generate a target image using a camera after rotating the target lost ball using the rotation means so that the directions corresponding to the six face directions and the eight vertex directions are sequentially facing upward. Although the rotation means is not specifically illustrated, it can rotate the target lost ball along a first vertical direction while supporting both sides based on one of the horizontal central axes of the target lost ball. Additionally, the rotation means can rotate the target lost ball along a second vertical direction perpendicular to the first vertical direction.

[0051] Additionally, the target image generation step (S40) can be generated by taking a target image while the target lost ball is placed on the upper surface of a black plate. Therefore, the target image can be made so that it does not include an image that can be recognized as a defect existing around the target lost ball.

[0052] The above target image generation step (S40) may be performed in a device comprising a camera capable of generating a target image of a target lost ball, a control unit that classifies the brand and grade of the target image using a learned image classification algorithm and an image segmentation algorithm, and a storage unit that stores brand information and grade information of the generated target lost ball. Additionally, the above image generation step may be performed in a device comprising a transfer means for transferring the target lost ball to the lower part of the camera and a rotation means for rotating the target lost ball in a predetermined direction.

[0053]

[0054] The target lost ball brand classification step (S50) is a step of classifying the brand of the target lost ball from the target image using a learned image classification algorithm. The target lost ball brand classification step (S50) may use at least one target image among a plurality of target images generated for the target lost ball. The target lost ball brand classification step (S50) may use at least three target images to improve the accuracy of brand classification. The target lost ball brand classification step (S50) may generate brand information for the target lost ball from the target image.

[0055] In addition, the target lost ball brand classification step (S50) can verify the authenticity of the brand displayed on the remanufactured lost ball. As mentioned above, a remanufactured lost ball refers to a ball that has had its entire surface repainted and a famous brand preferred by golfers newly printed on it. Therefore, although the remanufactured lost ball has the dimples of the original brand, it can be recognized as a ball of a famous brand because a famous brand is printed on it. However, the paint on the remanufactured lost ball peels off after a single use, making it difficult to use further. Since the target lost ball brand classification step (S50) generates brand information using the dimples of the remanufactured lost ball, it enables the determination of the authenticity of the famous brand printed on the surface. The authenticity of the brand can be verified by directly comparing the provided brand information with the printed brand on the remanufactured lost ball.

[0056] The above target lost ball brand classification step (S50) may be performed in a device comprising a camera that recognizes a target image of a target lost ball, a control unit that classifies the brand of the target image using a learned image classification algorithm and generates brand information, and a storage unit that stores the generated brand information of the target lost ball.

[0057]

[0058] The target lost ball grade classification step (S60) is a step of classifying the grade of a target lost ball from a target image using a learned image segmentation algorithm. The target lost ball grade classification step (S60) can classify the grade for all of the multiple target images generated for the target lost ball. The target lost ball is spherical, and defects may be randomly formed at various locations on the spherical surface. Additionally, it is necessary for the grade of the target lost ball to be determined by the largest defect among all defects. The target lost ball grade classification step (S60) can first assign a grade to all generated target images to improve the accuracy of the grade classification. For example, the target lost ball grade classification step (S60) can assign a grade to each of the 14 target images. Next, the target lost ball grade classification step (S60) can classify the lowest grade among the grades assigned to each target image as the grade of the target lost ball. The above target lost ball grade classification step (S60) can generate grade information for the target lost ball.

[0059] The above target lost ball grade classification step (S60) may be performed in a device comprising a camera that recognizes a target image of a target lost ball, a control unit that classifies the grade of the target image using a learned image segmentation algorithm and generates grade information, and a storage unit that stores the generated grade information of the target lost ball.

[0060]

[0061] The following describes, by way of example, an image-based lost ball brand and grade automatic classification system for performing an image-based lost ball brand and grade automatic classification method using a deep learning algorithm according to one embodiment of the present invention.

[0062] Figure 7 is a configuration diagram of an image-based automatic classification system for lost ball brands and grades.

[0063] The above-described lost ball brand and grade automatic classification system may include a camera (100), a conveying means (200), a rotating means (300), a control unit (400), and a storage unit (500). Additionally, the above-described lost ball brand and grade automatic classification system may further include a display unit (600).

[0064]

[0065] The camera (100) may be formed as a general optical camera or an aerial camera. The camera (100) may be a camera capable of generating images with a resolution necessary for recognizing dimples and defects on a golf ball. The camera (100) may generate brand learning images necessary for recognizing dimples on a golf ball from images of golf balls provided for learning necessary for classifying golf ball brands. Additionally, the camera (100) may generate defect learning images necessary for recognizing defects from images of defects on golf balls provided for learning about defects on golf balls. Additionally, the camera (100) may generate target images of a target lost ball. The camera (100) may form multiple target images of a target lost ball being transported downward. That is, the camera (100) may generate target images as the target lost ball rotates, thereby generating target images for the entire surface of the target lost ball.

[0066] The above-mentioned transfer means (200) may be formed as a means for transferring an object, such as a conveyor belt, an LM guide, and a ball screw. The above-mentioned transfer means (200) may transfer the target lost ball to the lower part of the camera (100). The above-mentioned transfer means (200) may further include means for fixing the target lost ball so that it does not move when the camera (100) captures a video image after the target lost ball has been transferred to a predetermined position.

[0067] The rotation means (300) may include a plurality of robot arms and a means for rotating the robot arms. The rotation means (300) may pick up a target lost ball and rotate it so that a predetermined surface faces the bottom of the camera (100). Additionally, the rotation means (300) may position the target lost ball at the bottom of the camera (100) after rotating it. The rotation means (300) may generate a predetermined number of target images of the target lost ball while repeatedly rotating the target lost ball.

[0068] The control unit (400) is connected to the camera (100), the transfer means (200), the rotation means (300), the storage unit (500), and the display unit (600), and can control each operation necessary to classify the brand and grade of the target lost ball. The control unit (400) can perform learning of an image classification algorithm while controlling the camera (100). The control unit (400) can transmit the learned image classification algorithm and the generated golf ball brand classification information to the storage unit (500) for storage. Additionally, the control unit (400) can perform learning of an image segmentation algorithm while controlling the camera (100). The control unit (400) can transmit the learned image segmentation algorithm and the generated golf ball defect classification information to the storage unit (500) for storage.

[0069] The control unit (400) can classify the brand and grade of a target image using a learned image classification algorithm and an image segmentation algorithm. Additionally, the control unit (400) can transmit the brand information and grade information of the generated target lost ball to the storage unit (500).

[0070] The storage unit (500) may be formed to include a memory for storing information. The storage unit (500) may store a learned image classification algorithm transmitted from the control unit (400) and generated golf ball brand classification information. Additionally, the storage unit (500) may store a learned image segmentation algorithm and generated golf ball defect classification information. Additionally, the storage unit (500) may store brand information and grade information of the target lost ball. Additionally, the storage unit (500) may store the target image of the target lost ball.

[0071] The above display unit (600) may be formed to include a display means. The above display unit (600) may display the target image of the target lost ball transmitted from the control unit (400), brand information, and grade information.

[0072]

[0073] The above description is merely one embodiment for implementing an image-based method for automatically classifying lost ball brands and grades using a deep learning algorithm according to one embodiment of the present invention. The present invention is not limited to the above-described embodiment, and the technical spirit of the present invention extends to the scope where any person with ordinary knowledge in the field to which the invention belongs can make various modifications without departing from the gist of the invention as claimed in the following claims.

Claims

1. A training image generation step for generating brand learning images for learning the brand of a golf ball and defect learning images for learning defects, and A golf ball brand learning step for training the brand of the golf ball on an image classification algorithm using the above-mentioned brand learning image, and A golf ball defect learning step for learning the defects of the golf ball to an image segmentation algorithm using the defect learning image above, and A target image generation step that generates at least 8 target images from target lost balls to be classified, and A target lost ball brand classification step that classifies the brand of the target lost ball from the target image using the learned image classification algorithm, and An image-based automatic lost ball classification method characterized by including a target lost ball grade classification step that classifies the grade of the target lost ball from the target image using the learned image segmentation algorithm.

2. In Paragraph 1, An image-based automatic lost ball classification method characterized by the above golf ball brand learning step proceeding by matching and providing the above brand learning image with the brand of the golf ball, and generating golf ball brand classification information according to the characteristics of the above dimples for each of the above golf ball brands.

3. In Paragraph 1, An image-based automatic lost ball classification method characterized by the above golf ball defect learning step proceeding by matching and providing the defect learning image with the assigned grade, and generating golf ball defect classification information according to the size of the defect in the defect learning image.

4. In Paragraph 1, An image-based automatic classification method for lost balls, characterized in that the target image generation step generates 14 target images by taking photos in 6 face directions and 8 vertex directions based on a virtual cube that the target lost ball accommodates inside.

5. In Paragraph 1, An image-based automatic classification method for lost balls, characterized in that the target image generation step generates eight target images by taking photos in four face directions and four vertex directions based on a tetrahedron that accommodates the target lost ball inside.

6. In Paragraph 1, An image-based automatic lost ball classification method characterized by the above-mentioned target lost ball brand classification step generating brand information for the target lost ball from the above-mentioned target image.

7. In Paragraph 1, An image-based automatic lost ball classification method characterized by the above-mentioned target lost ball brand classification step verifying the authenticity of the brand displayed on the above-mentioned reused lost ball.

8. In Paragraph 1, An image-based automatic lost ball classification method characterized by the above target lost ball grade classification step classifying grades for all of a plurality of target images generated for the above target lost ball, and generating the lowest grade among the grades assigned to each of the above target images as grade information for the above target lost ball.

Images