Tea bud screening method adaptive to tea leaf picking machine
Through image recognition model and binocular camera technology, accurate identification and three-dimensional positioning of tea buds are achieved, and the problem of insufficient recognition of existing tea bud picking machines is solved, the picking efficiency and tea quality are improved, and the damage to tea trees is reduced.
Patent Information
- Application Number
- CN202411419077.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-12
- Publication Date
- 2024-11-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing tea bud picking machines have many challenges in technology, efficiency and quality control, and the inability to effectively identify and selectively pick new leaves, resulting in low picking efficiency and large damage to tea trees.
The image recognition model is constructed, and tea bud segmentation and positioning is performed through algorithms such as DeepLabv3+, YOLOv8, ShiTomas and Oriented R-CNN, and the three-dimensional picking points are calculated in combination with a binocular camera to achieve accurate identification and selective picking of tea buds.
It improves the efficiency and quality of tea bud picking, reduces damage to tea trees, and achieves accurate identification and selective picking of tea leaves.
Smart Images

Figure CN118941967A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of image recognition and algorithms, and in particular to a tea bud screening method adapted to a tea picking machine. Background Art
[0002] Tea picking machinery is a crop harvesting machinery that collects new tender tea leaves from the tops of tea trees. Due to low production efficiency and high requirements for tea garden working conditions, selective tea picking machinery has not been widely promoted. The main reason is that there is no recognition of tea buds, resulting in a high impurity rate, and it is impossible to selectively pick new leaves as required, and it is even more impossible to control the shape of the tea leaves. Therefore, the extensive picking method of reciprocating cutting causes great damage to the tea trees and the picking efficiency is low; the existing parallel intelligent tea picking robots face the challenge of ensuring picking efficiency and quality in actual applications, because the existing tea picking machines are still unable to selectively pick new leaves as required;
[0003] In summary, the current tea bud picking machines still face many challenges in terms of technology, efficiency and quality control. Further research is needed on the identification requirements for tea buds to improve picking efficiency and tea quality, while reducing damage to tea trees to meet market demand and tea farmers' production needs. Summary of the invention
[0004] The purpose of the present invention is to overcome the problem of insufficient identification of tea buds in the prior art and to provide a tea bud screening method suitable for tea picking machinery.
[0005] The objective of the present invention is achieved through the following technical solutions:
[0006] A tea bud screening method adapted for a tea picking machine, comprising the following steps:
[0007] S1: Tea recognition model construction: including:
[0008] Image data preparation: Real image data of tea buds are taken on site, including image data of tea buds under different lighting, different angles, different types and other scenes. The data are then flipped horizontally and vertically, and the brightness and contrast are adjusted, randomly cropped, and rotated to enhance the image effect.
[0009] Tea bud segmentation model: Input image data into DeepLabv3+ to obtain images containing only tea buds;
[0010] S2: Two-dimensional picking point positioning: The picking coordinates of the corresponding tea bud position data are obtained through the tea bud position data obtained in step S1, including:
[0011] Picking area detection: based on the image processed by tea bud segmentation, picking area detection and picking point calculation are performed to obtain picking area data;
[0012] Among them, the picking point is calculated as:
[0013] ;
[0014] Among them, (x min ,y min ) is the value of the upper left corner of the regional coordinate value of the tea bud, (x max ,y max ) is the value of the lower right corner of the regional coordinate value of the tea bud, and the coordinate value is calculated (x min ,y min ) and (x max ,y max ) The geometric center of the two values gets the coordinates of the final picking point (x0, y0);
[0015] S3: Calculation of three-dimensional picking points based on binocular cameras: After the two-dimensional picking point positioning is completed, the depth value is calculated through the binocular camera data, and then the three-dimensional picking point coordinates are determined in combination with coordinate conversion;
[0016] The depth value is calculated as:
[0017] ;
[0018] Where z is the depth value, f is the focal length of the camera, T is the baseline length of the two cameras, and d is the disparity value;
[0019] Coordinate transformation: Binocular vision involves the transformation between the two-dimensional pixel coordinates of the image and the three-dimensional spatial coordinates of the object. There are four coordinate systems, including: world coordinate system, camera coordinate system, image coordinate system and pixel coordinate system;
[0020] Pixel coordinate system, an image coordinate system with pixels as units, where the upper left corner of the image is the origin of the coordinate system;
[0021] Image coordinate system, a coordinate system on the camera imaging plane, whose origin is the intersection of the camera optical axis and the imaging plane;
[0022] Camera coordinate system, the camera principal point is the origin of the coordinate system, and the z-axis is the optical axis direction, that is, the focus of the z-axis and the image;
[0023] World coordinate system, the absolute coordinate system of the real world;
[0024] To realize the three-dimensional positioning of tea buds in the world coordinate system, it is necessary to convert the coordinates obtained by the two-dimensional picking point positioning into pixel coordinates (u, v) through pixel coordinates to image coordinates, image coordinates to camera coordinates, and then from camera coordinates to world coordinates. After combining the depth value z conversion, the world coordinates of the picking point (X w , Y w , Z w), world coordinates (X w , Y w , Z w ) is used to input into the picking machine for operation; specifically:
[0025] [ x y 1 ] = K − 1 [ u v 1 ] ;
[0026] [ X c Y c Z c 1 ] = [ R | t ] [ x y 1 z ] ;
[0027] [ X w Y w Z w 1 ] = T c − 1 [ X c Y c Z c 1 ] = [ R t 0 1 ] − 1 [ X c Y c Z c 1 ] ;
[0028] Among them, x, y are the coordinates obtained by locating the two-dimensional picking point, u and v are the pixel coordinate values of the two-dimensional picking point, K is the camera internal parameter matrix, X c , Y c and Z c are the coordinate values of the x, y, and z axes of the camera coordinates after the image coordinates are converted, c is the camera coordinates, [R|t] is the camera external parameter matrix, R is the rotation matrix, t is the translation vector, and T c It is the transformation matrix from the camera coordinate system to the world coordinate system.
[0029] Furthermore, in S2, the two-dimensional picking point positioning can also be: the two-dimensional picking point is obtained by calculating through the tea bud segmentation model and ShiTomas, specifically:
[0030] Through tea bud segmentation DeepLabv3+, the tea buds and background in the tea bud image data are separated to obtain an image containing only tea buds. The single bud, one leaf and two leaves in the tea buds are separated and extracted by combining the tea bud image and the RGB color component range to obtain three binary images of the same size as the original image; then the discrete pixels in the binary image are counted respectively, and only the area with the most pixels is retained to obtain the maximum connected area of the single bud, one leaf and two leaves; the ShiTomas algorithm is used to detect some corner points of the single bud, one leaf and two leaves, and the corner point with the smallest ordinate is selected as the respective picking point. The coordinates of the corner point are the final picking point coordinates (x0, y0).
[0031] Among them, DeepLabv3+ is an existing technology. DeepLabv3+ is a semantic segmentation network that combines dilated convolution, spatial pyramid pooling modules and encoder-decoder structures. DeepLabv3+ is designed to solve two main problems in semantic segmentation: the multi-scale problem of objects and the problem of reduced feature map resolution caused by multiple downsampling of DCNN. By introducing the encoder-decoder structure and the improved backbone network, DeepLabv3+ can better retain boundary information and improve segmentation accuracy. In addition, DeepLabv3+ also uses dilated convolution and spatial pyramid pooling techniques to capture multi-scale contextual information, thereby improving the performance of the model.
[0032] Among them, Shi-Tomasi is an existing technology. The Shi-Tomasi corner detection algorithm is an algorithm for locally detecting interest points and is often used for corner detection and feature point extraction.
[0033] Furthermore, in S2, the two-dimensional picking point positioning can also be: obtaining the two-dimensional picking point by solving the tea bud detection model and the K3M algorithm, specifically:
[0034] Tea bud detection: inputting image data into YOLOv8, positioning through YOLOv8 and obtaining position data of tea buds;
[0035] The image data is tested by tea bud detection, the position of the tea bud in the image is selected, and the image containing only a single tea bud is obtained after cropping. The single tea bud image is binarized, and the maximum connected area is found according to the pixel value. Then, the boundary of the binarized area is iteratively eroded by the K3M algorithm to continuously refine the tea bud area, and finally the skeleton of the tender bud is obtained. The minimum outer bounding rectangle of the skeleton is calculated, and the intersection of the tea bud skeleton and the minimum outer bounding rectangle at the bottom is the coordinate of the final picking point (x0, y0). The K3M algorithm is an algorithm for skeleton extraction, which belongs to the class of algorithms for iterative erosion boundaries. Its basic idea is to assume that the "burning" starts at the boundary of the object in the binary image at the same time, and the object is gradually refined during the burning process, while ensuring that the points that meet certain conditions are retained or "burned" to ensure that the last pixel-wide image left after the burning is the skeleton of the image.
[0036] Among them, YOLOv8 belongs to the existing technology. YOLOv8 is the latest version of the YOLO series model. YOLOv8's innovations include a new backbone network, a new Anchor-Free detection head, and a new loss function. These innovations enable YOLOv8 to perform well in tasks such as object detection and instance segmentation. At the same time, YOLOv8 also inherits some advantages of the YOLO series models, such as simplicity, ease of use, and efficient reasoning.
[0037] Furthermore, in S2, the two-dimensional picking point positioning can also be: obtaining the picking point through the tea bud rotation detection model, the single tender bud segmentation mask map and the traditional image processing solution, specifically:
[0038] Tea bud rotation detection: The image data is input into Oriented R-CNN, and the position data of the tea buds is obtained through the detection frame rotation function in Oriented R-CNN;
[0039] The image data is passed through the tea bud rotation detection model to select the position of the tea bud in the image, and after cropping, an image containing only a single tea bud is obtained. The single tender shoot segmentation is obtained by training with the following data sets: single bud with contour marking, one bud and one leaf, one bud and one leaf front view, and one bud and one leaf back view. After the single tender shoot segmentation, a single tender shoot mask image is obtained. The segmented single tender shoot mask image is binarized, eroded, screened for the maximum connected area, and the lowest point on the left end of the connected area is found, and the ROI area is obtained. Finally, the center coordinates of the ROI area are calculated and taken as the final picking point coordinates (x0, y0).
[0040] The beneficial effects of the present invention are:
[0041] (1) By analyzing tea images, tea buds can be accurately identified. After analysis, the state of the tea buds can be judged, and the picking and picking methods can be selectively selected, thereby improving the picking efficiency and quality of the picked tea buds;
[0042] (2) It can identify each individual tea bud, reduce or avoid extensive picking, improve picking efficiency, and protect the tea trees. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 is a flow chart of the main steps of the method of the present invention;
[0044] Figure 2 Schematic diagram for tea bud identification, (a) is the tea bud outline mark, (b) is the mark box, and (c) is the rotation box mark;
[0045] Figure 3 This is the tea bud sample image data enhancement effect diagram. Figure 3 (1) is the effect diagram of the horizontal and vertical flipping of the tea bud sample image. Figure 3 (2) is the effect diagram of brightness and contrast adjustment of tea bud sample image. Figure 3 (3) is the effect of randomly cropping the tea bud sample image. Figure 3 (4) Tea bud sample image rotation effect diagram;
[0046] Figure 4 This is the DeepLabv3+ network structure diagram in the tea bud segmentation model;
[0047] Figure 5 This is the YOLOv8 network structure diagram in the tea bud detection model;
[0048] Figure 6 This is the Oriented R-CNN network structure diagram in the tea bud rotation detection model;
[0049] Figure 7 Label the resulting map for the picking area;
[0050] Figure 8 Detect overall flow chart for picking area;
[0051] Fig. 9 Complete flowchart for solving picking points for ShiTomas algorithm;
[0052] Fig.10 Complete flowchart for solving the picking point for K3M algorithm;
[0053] Fig.11 It is the single tender shoot segmentation mask map;
[0054] Fig.12 Complete flowchart for solving picking points for traditional image processing;
[0055] Fig.13 This is the flow chart for solving the three-dimensional picking point coordinates. DETAILED DESCRIPTION
[0056] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0057] See also Figure 1 , a tea bud screening method suitable for tea picking machinery is proposed, the method steps are:
[0058] S1: Tea bud recognition: By inputting image data into existing technical means such as DeepLabv3+, YOL0v8, ShiTomas, etc., the model, data processing method and capabilities are included to construct a tea bud recognition model. The constructed tea bud recognition model includes: tea bud segmentation model, tea bud detection model and tea bud rotation detection model.
[0059] See also Figure 2-Figure 3,Image data preparation: Build a model for tea bud recognition, collect tea bud images for model training, and shoot real tea bud data on site, including tea bud image data under different lighting, different angles, different types and other scenes. Label the data set according to the actual picking position. Depending on the selected model, use a variety of labeling methods, such as Figure 2 (a) Contour marking, Figure 2 (b) Box marking, Figure 2 (c) The rotation box is marked as shown. In order to improve the recognition performance of the model in various scenarios, based on the collected tea bud images, image enhancement technology is used to add tea bud images of various scenes, including horizontal and vertical flipping, brightness and contrast adjustment, random cropping, rotation, etc. Figure 3 As shown in (1), (2), (3) and (4).
[0060] See also Figure 4 ,Tea bud segmentation: Tea bud recognition can be achieved by building and using a tea bud segmentation model, mainly using DeepLabv3+. The structure of the tea bud segmentation model is as follows Figure 4 As shown. Input a captured original image, pass through the Xception feature extractor in the encoding part, and obtain a low-order feature map of 1 / 4 size of the original image and a high-order feature map of 1 / 16 size of the original image. The high-order feature map is passed into the ASPP module to obtain context features of different scales. The feature map output by the ASPP module is spliced by channel and then 1x1 convolution is used for dimensionality reduction. The feature map after dimensionality reduction is used as the input of the decoder. The decoder input also includes low-order feature maps. The feature map after dimensionality reduction is upsampled to the same size, spliced by channel, and then 1x1 convolution is performed. Finally, it is restored to the original size through upsampling, and finally an image containing only tea buds is obtained from the original image; the model is combined with the enhanced contour annotation dataset and trained and verified according to the 7:2:1 distribution. The model structure is optimized according to the verification index value, so that the tea bud recognition accuracy is higher and the speed is faster.
[0061] See also Figure 5 ,Tea bud detection: Tea bud recognition is achieved by establishing and using a tea bud detection model. Combined with real-time requirements, YOLOv8 is mainly used. The tea bud detection model structure is as follows Figure 5 As shown in the figure, a tea bud image to be identified is input. The backbone network Backbone first extracts the feature information of the tea bud image at different scales, then fuses the deep and shallow features through the Neck module, and finally selects the specific position of the tea bud through the three detection heads of the Head module. The training model is based on YOLOv8n, combined with the enhanced box annotation dataset according to the 7:2:1 distribution for training and verification, and the model structure is optimized according to the verification index value, so that the tea bud recognition accuracy is higher and the speed is faster.
[0062] See also Figure 6,Tea bud rotation detection: Tea bud recognition is achieved by building and using a tea bud rotation detection model, mainly using Oriented R-CNN. The structure of the tea bud rotation detection model is as follows Figure 6 As shown in the figure, the detection box of the Oriented R-CNN model is rotated, and it can select the object according to its actual direction and shape, thus providing more accurate positioning information. Input a tea bud image to be identified, the backbone network Backbone uses ResNet-50 to extract image feature information, the Neck module uses the feature pyramid network FPN to fuse feature maps of different scales to obtain richer semantic and location information, and the regional candidate network Oriented RPN generates candidate boxes that may contain tea buds on the feature map. After screening by the Head module, the positions of the tea buds that are identified and selected are obtained. The model is combined with the enhanced rotating box annotation dataset for training and verification according to the 7:2:1 distribution, and the model structure is optimized according to the verification index value, so that the tea bud recognition accuracy is higher and the speed is faster.
[0063] S2: Two-dimensional picking point positioning: The picking coordinates of the corresponding tea bud position data are obtained through the tea bud position data obtained in step S1, including: (1) based on tea bud segmentation + picking area detection + picking point calculation; (2) based on tea bud segmentation + ShiTomas algorithm to solve the picking point; (3) based on tea bud detection + K3M algorithm to solve the picking point; (4) based on tea bud rotation detection + single tender bud segmentation mask map + traditional image processing to solve the picking point.
[0064] See also Figure 7-8 , based on tea bud segmentation + picking area detection + picking point calculation: the original tea image is separated from the tea bud and background by the tea bud segmentation model DeepLabv3+, and the image containing only tea buds will be used as the training sample of the picking area detection model. First, the image containing only tea buds is annotated as follows Figure 7 As shown in the figure, the tea bud picking area is selected, and the selected image is input into the detection model for training to obtain a model that can detect the picking area. The overall process of picking area detection is as follows: Figure 8 As shown; Picking point calculation: based on the image processed by tea bud segmentation, picking area detection and picking point calculation are performed to obtain the coordinates of the picking point;
[0065] Among them, the picking point is calculated as:
[0066] ;
[0067] Among them, (x min ,y min ) is the value of the upper left corner of the regional coordinate value of the tea bud, (x max ,y max ) is the value of the lower right corner of the regional coordinate value of the tea bud, and the coordinate value is calculated (xmin ,y min ) and (x max ,y max ) The geometric center of the two values is obtained to obtain the coordinates of the final two-dimensional picking point (x0, y0);
[0068] See also Fig. 9 , based on tea bud segmentation + ShiTomas algorithm to solve the picking point: the original tea image is separated from the tea bud and background by the tea bud segmentation model DeepLabv3+, and a picture containing only tea buds is obtained. Combining the tea bud picture and the RGB color component range, the single bud, one leaf and two leaves in the tea bud are separated and extracted to obtain three binary images of the same size as the original image; then the discrete pixels in the binary image are counted respectively, and only the area with the most pixels is retained to obtain the maximum connected area of the single bud, one leaf and two leaves; the ShiTomas algorithm is used to detect some corner points of the single bud, one leaf and two leaves, and the corner point with the smallest ordinate is selected as the respective picking point, and the coordinates of the corner point are the picking point coordinates; the complete process of ShiTomas algorithm to solve the picking point calculation is as follows Fig. 9 shown.
[0069] See also Fig.10 , based on tea bud detection + K3M algorithm to solve the picking point: the original tea image is framed by the tea bud detection model to select the position of the tea bud in the picture, and then cropped to get a picture containing only a single tea bud. First, the single tea bud picture is binarized, and the maximum connected area is found according to the pixel value. Then, the K3M method is used to iteratively erode the binary area boundary to continuously refine the tea bud area, and finally the bud skeleton is obtained. The minimum outer bounding rectangle of the skeleton is calculated, and the intersection of the tea bud skeleton and the minimum outer bounding rectangle at the bottom is the picking point. The complete process of the K3M algorithm to solve the picking point is as follows Fig.10 shown.
[0070] See also Fig.11 , based on tea bud rotation detection + single tender shoot segmentation mask image + traditional image processing to solve the picking point: the original tea image is framed by the tea bud rotation detection model to select the position of the tea bud in the picture, and after cropping, a picture containing only a single tea bud is obtained. The single tender shoot segmentation model is trained with the single bud, one bud and one leaf, one bud and one leaf front view, and one bud and one leaf back view data sets after contour marking. The basic model used is DeepLabv3+. After segmentation, a single tender shoot mask image is obtained. The segmentation results are shown as follows Fig.11 As shown in the figure. The mask image obtained by segmentation is binarized, eroded, the maximum connected area is screened, the lowest point on the left end of the connected area is found, the ROI area is obtained, and finally the center coordinates of the ROI area are taken as the final picking point. The complete process of solving the picking point by traditional image processing is as follows Fig.12 shown.
[0071] S3. Calculation of three-dimensional picking points based on binocular cameras: After the two-dimensional picking point positioning is completed, the depth value is calculated through the binocular camera data, and then the three-dimensional picking point coordinates are determined in conjunction with the coordinate conversion;
[0072] See also Fig.13 , which is a flowchart for solving the three-dimensional picking point coordinates in step S3. Turn on the binocular camera, and the left and right cameras start to collect tea bud images. After the left camera image is acquired, it enters the tea bud recognition model in turn, and then obtains the two-dimensional picking point coordinates of the tea buds through step S2. The images collected by the left and right cameras are subjected to stereo correction, stereo matching and depth value calculation in turn, and then combined with the already obtained two-dimensional picking point coordinates, the corresponding three-dimensional picking point coordinates are calculated.
[0073] After determining the 2D picking point, the 3D picking point is solved based on the binocular camera. The key to solving the 3D picking point with the binocular camera is to calculate the depth value, which is based on three steps: camera calibration, stereo correction, and stereo matching.
[0074] Depth value calculation:
[0075] Camera calibration mainly uses the checkerboard method to obtain the camera's intrinsic matrix, distortion coefficient, extrinsic matrix and other parameters as well as the distance between the two cameras, the conversion matrix, and the translation matrix;
[0076] Stereo correction: based on the parameters obtained from camera calibration, the left and right views are distorted and corrected to ensure that the corresponding points of the binocular images are aligned horizontally.
[0077] Stereo matching, matching the left and right images captured by the binocular camera, determining the correspondence between corresponding pixels, and obtaining the disparity value between the images. The disparity value can be calculated by the RAFT or SGBM model. Depth value calculation, after obtaining the disparity value, the disparity value needs to be converted into a depth value. The conversion formula is:
[0078] ;
[0079] Among them, z is the depth value, f is the focal length of the camera, T represents the baseline length of the two cameras, and d is the disparity value.
[0080] Convert 3D world coordinates:
[0081] Binocular vision involves the conversion between the two-dimensional pixel coordinates of an image and the three-dimensional spatial coordinates of an object, including four coordinate systems: world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system.
[0082] Pixel coordinate system, an image coordinate system with pixels as units, where the upper left corner of the image is the origin of the coordinate system;
[0083] Image coordinate system, a coordinate system on the camera imaging plane, whose origin is the intersection of the camera optical axis and the imaging plane; camera coordinate system, the camera principal point is the origin of the coordinate system, and the z-axis is the optical axis direction, that is, the focus of the z-axis and the image;
[0084] World coordinate system, the absolute coordinate system of the real world.
[0085] To realize the three-dimensional positioning of tea buds in the world coordinate system, it is necessary to convert pixel coordinates to image coordinates. The conversion formula is:
[0086] [ x y 1 ] = K − 1 [ u v 1 ] ;
[0087] Among them, K is the camera internal parameter matrix, x, y are the horizontal and vertical coordinates of the two-dimensional picking point coordinates, u, v are the coordinate values of the pixel coordinates converted from the coordinates of the two-dimensional picking point;
[0088] The image coordinates are converted to camera coordinates, and the conversion formula is:
[0089] [ X c Y c Z c 1 ] = [ R | t ] [ x y 1 z ] ;
[0090] Where [R|t] is the camera external parameter matrix, R is the rotation matrix, t is the translation vector, X c , Y c and Z c are the coordinate values of the x, y, and z axes of the camera coordinates after the conversion corresponding to the image coordinates brought in, and c is the camera coordinates;
[0091] The camera coordinates are converted to world coordinates using the following conversion formula:
[0092] [ X w Y w Z w 1 ] = T c − 1 [ X c Y c Z c 1 ] = [ R t 0 1 ] − 1 [ X c Y c Z c 1 ] ;
[0093] Among them, Tc is the transformation matrix from the camera coordinate system to the world coordinate system, X w , Y w and Z w The coordinate values of the x, y, and z axes of the three-dimensional world coordinates after the conversion corresponding to the camera coordinates brought in;
[0094] The coordinates of the two-dimensional picking point are pixel coordinates (u, v). After combining with the depth value Z, the three-dimensional world coordinates of the picking point are finally obtained (X w , Y w , Z w ), world coordinates (X w , Y w , Z w ) is used to input into tea picking machinery for picking operations.
[0095] This embodiment is used to adapt to the problem that the current tea picking machinery has insufficient recognition of tea buds. The state of the tea buds is judged through image analysis and algorithms, and the picking machinery is controlled to accurately pick the tea buds.
[0096] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.
Claims
1. A tea bud screening method suitable for tea picking machinery, It is characterized in that The method steps include: S1: Tea Identification: Includes: Image data preparation: Take tea bud images, including tea bud image data under different lighting, different angles, and different types of scenes, and perform horizontal and vertical flipping, brightness and contrast adjustment, random cropping, and rotation on the data to enhance the image effect; Tea bud segmentation: Input the image data into DeepLabv3+ to obtain images containing only tea buds; S2: Two-dimensional picking point positioning: The picking coordinates of the corresponding tea bud position data are obtained through the tea bud position data obtained in step S1, including: Picking area detection: based on the image processed by tea bud segmentation, picking area detection and picking point calculation are performed to obtain picking area data; Among them, the two-dimensional picking point is calculated as: ; Among them, (x min ,y min ) is the value of the upper left corner of the regional coordinate value of the tea bud, (x max ,y max ) is the value of the lower right corner of the regional coordinate value of the tea bud, and the coordinate value is calculated (x min ,y min ) and (x max ,y max ) The geometric center of the two values gets the coordinates of the final picking point (x 0 ,y 0 ); S3: Calculation of three-dimensional picking points: After the two-dimensional picking point positioning is completed, the depth value is calculated through the binocular camera data, and then the three-dimensional picking point coordinates are determined in conjunction with coordinate conversion; The depth value is calculated as: ; Where z is the depth value, f is the focal length of the camera, T is the baseline length of the two cameras, and d is the disparity value; Coordinate conversion: The coordinates obtained by locating the two-dimensional picking point are converted into pixel coordinates (u, v). After combining the depth value z conversion, the world coordinates of the picking point are finally obtained (X w , Y w , Z w ), world coordinates (X w , Y w , Z w ) is used to input into the picking machine for operation; specifically: ; ; ; Among them, x, y are the coordinates obtained by locating the two-dimensional picking point, u and v are the pixel coordinate values of the two-dimensional picking point, K is the camera internal parameter matrix, X c , Y c and Z c are the coordinate values of the x, y, and z axes of the camera coordinates after the image coordinates are converted, c is the camera coordinates, [R|t] is the camera external parameter matrix, R is the rotation matrix, t is the translation vector, and T c It is the transformation matrix from the camera coordinate system to the world coordinate system.
2. A tea bud screening method adapted for a tea picking machine according to claim 1, It is characterized in that In S2, the two-dimensional picking point positioning can also be: obtaining the two-dimensional picking point by calculating through tea bud segmentation and ShiTomas, specifically: Tea bud segmentation DeepLabv3+ is used to separate the tea buds and background in the tea bud image data to obtain an image containing only tea buds. The single bud, one leaf and two leaves in the tea buds are separated and extracted by combining the tea bud image and the RGB color component range to obtain three binary images of the same size as the original image. Then, the discrete pixels in the binary image are counted and only the area with the most pixels is retained to obtain the maximum connected area of the single bud, one leaf and two leaves. The ShiTomas algorithm is used to detect some corner points of the single bud, one leaf and two leaves, and the corner point with the smallest ordinate is selected as the picking point. The coordinates of the corner point are the final picking point coordinates (x 0 ,y 0 ).
3. A tea bud screening method adapted for a tea picking machine according to claim 1, It is characterized in that In S2, the two-dimensional picking point positioning can also be: obtaining the two-dimensional picking point by tea bud detection and K3M algorithm solution, specifically: Tea bud detection: inputting image data into YOLOv8, positioning through YOLOv8 and obtaining position data of tea buds; The image data is subjected to tea bud detection, the position of the tea bud in the image is framed, and the image containing only a single tea bud is obtained after cropping. The single tea bud image is binarized, and the maximum connected area is found according to the pixel value. Then, the K3M algorithm is used to iteratively erode the binary area boundary to continuously refine the tea bud area, and finally the bud skeleton is obtained. The minimum outer bounding rectangle of the skeleton is calculated, and the intersection of the tea bud skeleton and the minimum outer bounding rectangle at the bottom is the coordinate of the final picking point (x 0 ,y 0 ).
4. A tea bud screening method adapted for a tea picking machine according to claim 1, It is characterized in that In S2, the two-dimensional picking point positioning can also be: obtaining the picking point through tea bud rotation detection, single tender bud segmentation mask map and traditional image processing solution, specifically: The tea bud rotation detection: inputting the image data into Oriented R-CNN, and obtaining the position data of the tea buds through the detection frame rotation function in Oriented R-CNN; The image data is passed through the tea bud rotation detection frame to select the position of the tea bud in the image, and the image containing only a single tea bud is obtained after cropping. The single tender shoot segmentation is: single bud after contour marking, one bud and one leaf, one bud and one leaf front view, one bud and one leaf back view data set training; after the single tender shoot segmentation, the single tender shoot mask image is obtained. The segmented single tender shoot mask image is subjected to binarization, corrosion, maximum connected area screening, finding the lowest point on the left end of the connected area and obtaining the ROI area, and finally the center coordinates of the ROI area are calculated as the final picking point coordinates (x 0 ,y 0 ).
Citation Information
Patent Citations
Machine vision-based famous high-quality tea picking point position information acquisition method
CN112861654A
Machine vision-based famous high-quality tea tender shoot identification and picking point positioning method
CN114708208A
