A multi-resolution blade sample appearance feature extraction method

Abstract

The present application relates to a kind of multi-resolution blade sample appearance feature extraction method, belong to blade detection technical field.The present application method is by panoramic image sensor to collect the whole blade appearance feature image, image is handled by ROI segmentation, remove background and extract the whole blade ROI and blade main vein ROI;In the whole blade ROI, the number of industrial microscope sampling point and position data information are calculated;The sampling point to main vein distance is not less than the sampling point of distance threshold l And the intersection ratio IOU of blade of the sampling point is not greater than threshold p is removed;According to known panoramic image sensor internal parameter, by coordinate conversion, sampling point pixel coordinate is converted into world coordinate, and the moving distance L of each sampling point panoramic image sensor sampling is calculated;The pulse number P required by motor is calculated when panoramic image sensor moves to sampling point;By the pulse number required by motor, panoramic image sensor is moved to sampling point position, and the blade appearance feature high-resolution image data of sampling point is extracted.

Description

Technical Field

[0001] This invention relates to a method for extracting the appearance features of multi-resolution leaf samples, belonging to the field of leaf detection technology. Background Technology

[0002] Currently, blade grading in my country mainly relies on manual grading. This method requires a large number of personnel, and training blade grading operators is time-consuming and costly. Moreover, due to human factors, the grading accuracy and efficiency are low. Therefore, there is an urgent need for machine grading methods that meet industrial needs to replace manual grading. Machine grading methods require blade appearance features, and currently, this is mainly achieved by acquiring images of blade appearance features using wide-angle cameras. However, the acquired images have low resolution and insufficient accuracy, which is not conducive to subsequent blade grading using image recognition algorithms. Summary of the Invention

[0003] This invention addresses the problem of leaf sample appearance feature extraction in leaf grading by proposing a multi-resolution leaf sample appearance feature extraction method. This method uses a multi-resolution leaf sample appearance feature extraction device to extract the overall appearance features of the leaf and the high-resolution appearance features of the leaf midrib, replacing the method of obtaining leaf appearance features by manual touch and visual observation. This reduces labor costs, avoids interference from human factors, and improves the accuracy and efficiency of leaf appearance feature extraction.

[0004] A method for extracting appearance features from multi-resolution leaf samples, employing a multi-resolution leaf sample appearance feature extraction device, includes the following specific steps:

[0005] 1) Acquire images of the entire blade's appearance features using a panoramic image sensor;

[0006] 2) Perform ROI segmentation on the image, remove the background, and extract the entire leaf ROI and the leaf midrib ROI;

[0007] 3) Calculate the number and location data of industrial microscope sampling points within the entire blade ROI;

[0008] 4) Ensure the distance from the sampling point to the main vein is not less than the distance threshold. l Sampling points whose Intersection over Union (IOU) is not greater than a threshold p are removed; where the IOU is the ratio of the area occupied by the leaf to the area of ​​the panoramic image sensor sampled image; and the distance threshold is... l The specific threshold is 1-2cm. l The threshold p is 0.5, depending on the size of the different types of leaves.

[0009] 5) Based on the known focal length, rotation matrix, and translation matrix of the panoramic image sensor, the pixel coordinates of the sampling points are converted into world coordinates through coordinate transformation, and the moving distance L of the panoramic image sensor for each sampling point is calculated.

[0010] 6) Calculate the number of pulses P required for the motor to move the panoramic image sensor to the sampling point based on the moving distance L, transmission ratio i, pitch K, and motor step angle B;

[0011] 7) Drive the imaging device moving component to move the panoramic image sensor to the sampling point position by the number of pulses required by the motor, and extract high-resolution image data of the blade appearance features at the sampling point.

[0012] Step 2) ROI segmentation processing method: Using a target detection algorithm, the entire leaf ROI and the leaf main vein ROI are selected in the background, and the pixel coordinates of the upper left and lower right corners of the ROI box are calculated.

[0013] The method for calculating the number and location data of industrial microscope sampling points in step 3) is as follows:

[0014] The pixel coordinates of the two diagonal vertices of the entire leaf ROI bounding box are A1(X1, Y1) and A2(X2, Y2). The resolution of the panoramic image sensor is a×b. (The formula is used to...) Calculate the number of sampling points using the formula. , Calculate the pixel coordinates of all sampled points, where % represents the modulo operation.

[0015] Step 4)

[0016] Distance S from sampling point to main vein m,n The calculation method is as follows

[0017] The pixel coordinates of the two diagonal vertices of the leaf midrib ROI are A3(X3, Y3) and A4(X4, Y4), which can be determined using the formula... Calculate the distance from the sampling point to the main vein;

[0018] The method for calculating the blade crossover ratio (IOU) is as follows:

[0019] The resolution of the panoramic image sensor is a×b, which is determined by the formula... Calculate the blade crossover ratio (IOU) at the sampling points.

[0020] The formula for calculating the number of pulses P required by the motor in step 6) is as follows: .

[0021] The multi-resolution leaf sample appearance feature extraction device includes an imaging device, a moving device, a driving component, and a controller 21. The moving device includes an imaging device moving component, a support 12, and a leaf sample moving component. The imaging device moving component is located at the top of the support 12, and the leaf sample moving component is located at the bottom of the support 12. The leaf sample 25 is placed on the leaf sample moving component. The imaging device is vertically mounted on the imaging device moving component. Both the imaging device moving component and the leaf sample moving component are connected to the driving component. Both the driving component and the imaging device are data-connected to the controller 21.

[0022] The imaging device includes an industrial microscope 7 and a panoramic image sensor 11. The industrial microscope 7 is vertically mounted on the moving component of the imaging device. The lens 28 of the industrial microscope 7 is located at the bottom of the industrial microscope 7. The annular aperture 27 of the industrial microscope 7 is fitted around the outer ring of the lens 28. The panoramic image sensor 11 is fixedly mounted on the outer side of the top of the industrial microscope 7. Both the industrial microscope 7 and the panoramic image sensor 11 are connected to the controller 21.

[0023] Furthermore, the top of the industrial microscope 7 is mounted on the top of the mobile device via an industrial microscope stand 9.

[0024] The moving components of the imaging device include a transverse screw I1, a transverse screw II, a horizontal screw 5, a horizontal guide rail I6, a horizontal guide rail II, a horizontal screw block 10, a transverse guide rail I, and a transverse guide rail II16.

[0025] Horizontal guide rails I and II16 are fixedly mounted parallel to each other at the top of the bracket 12. Horizontal screws I1 and II are mounted parallel to each other at the top of the bracket 12. Horizontal screw I1 is parallel to horizontal guide rail I and is located above horizontal guide rail I. Horizontal screw II is parallel to horizontal guide rail II16 and is located above horizontal guide rail II16. Horizontal screw 5, horizontal guide rail I6, and horizontal guide rail II are mounted parallel to each other between horizontal screws I1 and II. Horizontal screw 5 is located between horizontal guide rail I6 and II. The industrial microscope holder 9 is mounted on horizontal screw 5 via horizontal screw block 10. Horizontal screws I1, II, and 5 are all connected to the driving component.

[0026] Furthermore, one end of the horizontal screw 5 is connected to the bearing bracket I3 via bearing I4, and the other end of the horizontal screw 5 is connected to the bearing bracket II via bearing II. The bearing bracket I3, bearing bracket II, horizontal guide rail I6, and both ends of the horizontal guide rail II are threadedly connected to the horizontal screw I1 and the horizontal screw II via transverse screw blocks 2. The two ends of the bearing bracket I3 are fixedly connected to the inner walls of the horizontal guide rail I6 and the horizontal guide rail II, respectively. The two ends of the bearing bracket II are fixedly connected to the inner walls of the horizontal guide rail I6 and the horizontal guide rail II, respectively.

[0027] The moving components of the blade sample device include a lifting plate 20, a lifting motor frame II, a lifting screw I23, a lifting screw II, a lifting screw block I24, a lifting screw block II, and a blade fixing frame 26. The lifting screw I23 and the lifting screw II are respectively vertically arranged at the bottom of the support 12. The lifting screw block I24 is sleeved on the lifting screw I23 and is threadedly connected to the lifting screw I23. The lifting screw block II is sleeved on the lifting screw II and is threadedly connected to the lifting screw II. The lifting plate 20 is horizontally arranged at the bottom of the support 12. The two ends of the lifting plate 20 are fixedly connected to the lifting screw block I24 and the lifting screw block II, respectively. The blade sample 25 is fixedly arranged on the lifting plate 20 through the blade fixing frame 26.

[0028] The driving components include a transverse movement motor I, a transverse movement motor II17, a rotary motor 8, a horizontal movement motor 13, a lifting motor I, and a lifting motor II19. The transverse movement motor I is fixedly mounted on the top of the support 12 via a transverse motor frame I, and is located at the end of the transverse screw I1, with a driving connection between the transverse movement motor I and the transverse screw I1. The transverse movement motor II17 is fixedly mounted on the top of the support 12 via a transverse motor frame II18, and is located at the end of the transverse screw II, with a driving connection between the transverse movement motor II17 and the transverse screw II. The industrial microscope 7 is mounted in the middle of the industrial microscope frame 9 via a rotating shaft. The rotary motor 8 is fixed to the outside of the industrial microscope frame 9 via a rotary motor bracket, and the output shaft of the rotary motor 8 is connected to... The rotating shafts are located on the same straight line and the output shaft of the rotary motor 8 is fixedly connected to the rotating shaft. The horizontal moving motor 13 is fixedly mounted on the bearing bracket II through the horizontal motor frame. The horizontal moving motor 13 is driven by the horizontal screw 5. The lifting motor I is fixedly mounted at the bottom of the bracket 12 through the lifting motor frame I22 and is located at the side end of the lifting screw I23. The lifting motor I is driven by the lifting screw I23. The lifting motor II is fixedly mounted at the bottom of the bracket 12 through the lifting motor frame II and is located at the side end of the lifting screw II. The lifting motor II is driven by the lifting screw II. The transverse moving motor I, transverse moving motor II 17, rotary motor 8, horizontal moving motor 13, lifting motor I and lifting motor II 19 are all connected to the controller 21.

[0029] A driven gear I is fixedly mounted at the end of the transverse screw I1, and a motor drive gear I is fixedly mounted on the output shaft of the transverse moving motor I, with the motor drive gear I meshing with the driven gear I for transmission. A driven gear II is fixedly mounted at the end of the transverse screw II, and a motor drive gear II is fixedly mounted on the output shaft of the transverse moving motor II17, with the motor drive gear II meshing with the driven gear II for transmission. A driven gear III14 is fixedly mounted at the end of the horizontal screw 5, and a motor drive gear III15 is fixedly mounted on the output shaft of the horizontal moving motor 13, with the motor drive gear III15 meshing with the driven gear III14 for transmission. A driven gear IV is fixedly mounted at the bottom end of the lifting screw I23, and a motor drive gear IV is fixedly mounted on the output shaft of the lifting motor I, with the motor drive gear IV meshing with the driven gear IV for transmission. A driven gear V is fixedly mounted at the bottom end of the lifting screw II, and a motor drive gear V is fixedly mounted on the output shaft of the lifting motor II, with the motor drive gear V meshing with the driven gear V for transmission.

[0030] The beneficial effects of this invention are:

[0031] (1) The present invention utilizes a multi-resolution leaf sample appearance feature extraction device to extract multi-resolution leaf appearance feature image data, including the overall appearance features of the leaf and the high-resolution appearance features of the leaf main vein, replacing the method of obtaining leaf appearance features by manual touch and visual observation, reducing labor costs and avoiding human interference, and improving the accuracy and efficiency of leaf appearance feature extraction.

[0032] (2) The present invention uses the image data information of the appearance features of each part of the leaf extracted by the multi-resolution leaf sample appearance feature extraction device and the image recognition technology to classify the leaf. This can avoid the training and management costs caused by manual classification, thereby greatly reducing the manual cost of leaf classification. It can also reduce the interference of human subjective factors and improve the accuracy and speed of leaf classification. Attached Figure Description

[0033] Figure 1 A schematic diagram of the multi-resolution leaf sample appearance feature extraction device (front view);

[0034] Figure 2 A schematic diagram of the multi-resolution leaf sample appearance feature extraction device (rear view).

[0035] Figure 3 This is a schematic diagram of the imaging device structure;

[0036] In the figure, 1-Horizontal screw I, 2-Horizontal screw block, 3-Bearing bracket I, 4-Bearing I, 5-Horizontal screw, 6-Horizontal guide rail I, 7-Industrial microscope, 8-Rotary motor, 9-Industrial microscope stand, 10-Horizontal screw block, 11-Panoramic image sensor, 12-Support, 13-Horizontal moving motor, 14-Driven gear III, 15-Motor transmission gear III, 16-Horizontal guide rail II, 17-Horizontal moving motor II, 18-Horizontal motor bracket II, 19-Lifting motor II, 20-Lifting plate, 21-Controller, 22-Lifting motor bracket I, 23-Lifting screw I, 24-Lifting screw block I, 25-Blade sample, 26-Blade fixing bracket, 27-Annular aperture, 28-Lens;

[0037] Figure 4 This is a flowchart of the present invention;

[0038] Figure 5 The image was captured by a panoramic image sensor.

[0039] Figure 6 This is a ROI diagram of the entire leaf;

[0040] Figure 7 This is a ROI diagram of the leaf midrib;

[0041] Figure 8 This is a sampling point map of the panoramic image sensor;

[0042] Figure 9 A panoramic image of the sensor sampling points after removing unnecessary sampling points;

[0043] Figure 10 High-resolution images of the collected leaves;

[0044] Figure 11 High-resolution image of leaf veins collected;

[0045] Figure 12 This is a high-resolution image of the midrib of the leaf. Detailed Implementation

[0046] The present invention will be further described in detail below with reference to specific embodiments, but the scope of protection of the present invention is not limited to the content described.

[0047] Example 1: As Figure 1-2As shown, a multi-resolution leaf sample appearance feature extraction device includes an imaging device, a moving device, and a driving component. The moving device includes an imaging device moving component, a support 12, and a leaf sample device moving component. The imaging device moving component is located at the top of the support 12, and the leaf sample device moving component is located at the bottom of the support 12. The leaf sample 25 is located on the leaf sample device moving component. The imaging device is vertically located on the imaging device moving component. Both the imaging device moving component and the leaf sample device moving component are connected to the driving component.

[0048] The multi-resolution blade sample appearance feature extraction device also includes a controller 21, and the driving component and imaging device are all data connected to the controller 21.

[0049] The controller controls the driving component to move the imaging device moving component of the moving device in a plane, thereby moving the imaging device in a plane. The controller also controls the driving component to move the blade sample moving component up and down to achieve vertical translation of the blade sample. The imaging device is used to extract blade texture image data, which is then transmitted to the controller and then to the computer for data analysis.

[0050] like Figure 3 As shown, the imaging device includes an industrial microscope 7 and a panoramic image sensor 11. The industrial microscope 7 is vertically mounted on the moving component of the imaging device. The lens 28 of the industrial microscope 7 is mounted at the bottom of the industrial microscope 7. The annular aperture 27 of the industrial microscope 7 is fitted around the outer ring of the lens 28. The panoramic image sensor 11 is fixedly mounted on the outer side of the top of the industrial microscope 7. Both the industrial microscope 7 and the panoramic image sensor 11 are connected to the controller 21.

[0051] The top of the industrial microscope 7 is mounted on the top of the mobile device via the industrial microscope stand 9;

[0052] The controller controls the panoramic image sensor to capture images of the leaf sample, acquire image data of the entire leaf, and transmit the image data to the computer. Based on the image of the leaf sample, the computer uses an image segmentation algorithm to remove the background, segment the leaf outline, and calculate the number and location data of the industrial microscope sampling points.

[0053] The ring aperture provides a light source, and the industrial microscope captures the appearance characteristics of the leaf, such as the main vein pattern, leaf shape, and color, through the lens.

[0054] Example 2: The multi-resolution leaf sample appearance feature extraction device in this example is basically the same as the multi-resolution leaf sample appearance feature extraction device in Example 2, except that: the moving components of the imaging device include a transverse screw I1, a transverse screw II, a horizontal screw 5, a horizontal guide rail I6, a horizontal guide rail II, a horizontal screw block 10, a transverse guide rail I, and a transverse guide rail II16.

[0055] Horizontal guide rails I and II16 are fixedly mounted parallel to each other at the top of the bracket 12. Horizontal screws I1 and II are mounted parallel to each other at the top of the bracket 12. Horizontal screw I1 is parallel to horizontal guide rail I and is located above horizontal guide rail I. Horizontal screw II is parallel to horizontal guide rail II16 and is located above horizontal guide rail II16. Horizontal screw 5, horizontal guide rail I6, and horizontal guide rail II are mounted parallel to each other between horizontal screws I1 and II. Horizontal screw 5 is located between horizontal guide rail I6 and II. The industrial microscope holder 9 is mounted on horizontal screw 5 via horizontal screw block 10. Horizontal screws I1, II, and 5 are all connected to the driving component.

[0056] One end of the horizontal screw 5 is connected to the bearing bracket I3 via bearing I4, and the other end of the horizontal screw 5 is connected to the bearing bracket II via bearing II. The bearing bracket I3, bearing bracket II, horizontal guide rail I6, and both ends of the horizontal guide rail II are threadedly connected to the horizontal screw I1 and the horizontal screw II via transverse screw blocks 2. The two ends of the bearing bracket I3 are fixedly connected to the inner walls of the horizontal guide rail I6 and the horizontal guide rail II, respectively. The two ends of the bearing bracket II are fixedly connected to the inner walls of the horizontal guide rail I6 and the horizontal guide rail II, respectively.

[0057] The moving components of the blade sample device include a lifting plate 20, a lifting motor frame II, a lifting screw I23, a lifting screw II, a lifting screw block I24, a lifting screw block II, and a blade fixing frame 26. The lifting screw I23 and the lifting screw II are respectively vertically arranged at the bottom of the support 12. The lifting screw block I24 is sleeved on the lifting screw I23 and is threadedly connected to the lifting screw I23. The lifting screw block II is sleeved on the lifting screw II and is threadedly connected to the lifting screw II. The lifting plate 20 is horizontally arranged at the bottom of the support 12. The two ends of the lifting plate 20 are fixedly connected to the lifting screw block I24 and the lifting screw block II, respectively. The blade sample 25 is fixedly arranged on the lifting plate 20 through the blade fixing frame 26.

[0058] The drive components include a horizontal movement motor I, a horizontal movement motor II17, a rotary motor 8, a horizontal movement motor 13, a lifting motor I, and a lifting motor II19. Horizontal movement motor I is fixedly mounted on the top of the support 12 via a horizontal motor frame I, and is located at the end of the horizontal screw I1, with a driving connection between them. Horizontal movement motor II17 is fixedly mounted on the top of the support 12 via a horizontal motor frame II18, and is located at the end of the horizontal screw II, with a driving connection between them. The industrial microscope 7 is mounted in the middle of the industrial microscope frame 9 via a rotating shaft. The rotary motor 8 is fixed to the outside of the industrial microscope frame 9 via a rotating motor bracket. The output shaft of the rotary motor 8 is connected to the rotating... The moving shafts are located on the same straight line and the output shaft of the rotary motor 8 is fixedly connected to the rotating shaft. The horizontal moving motor 13 is fixedly mounted on the bearing bracket II through the horizontal motor frame. The horizontal moving motor 13 is connected to the horizontal screw 5 through transmission. The lifting motor I is fixedly mounted at the bottom of the bracket 12 through the lifting motor frame I22 and is located at the side end of the lifting screw I23. The lifting motor I is connected to the lifting screw I23 through transmission. The lifting motor II is fixedly mounted at the bottom of the bracket 12 through the lifting motor frame II and is located at the side end of the lifting screw II. The lifting motor II is connected to the lifting screw II through transmission. The transverse moving motor I, transverse moving motor II 17, rotary motor 8, horizontal moving motor 13, lifting motor I and lifting motor II 19 are all connected to the controller 21.

[0059] A driven gear I is fixedly mounted at the end of a transverse screw I1, and a motor drive gear I is fixedly mounted on the output shaft of a transverse moving motor I, with the motor drive gear I meshing with the driven gear I for transmission. A driven gear II is fixedly mounted at the end of a transverse screw II, and a motor drive gear II is fixedly mounted on the output shaft of a transverse moving motor II17, with the motor drive gear II meshing with the driven gear II for transmission. A driven gear III14 is fixedly mounted at the end of a horizontal screw 5, and a motor drive gear III15 is fixedly mounted on the output shaft of a horizontal moving motor 13, with the motor drive gear III15 meshing with the driven gear III14 for transmission. A driven gear IV is fixedly mounted at the bottom end of a lifting screw I23, and a motor drive gear IV is fixedly mounted on the output shaft of a lifting motor I, with the motor drive gear IV meshing with the driven gear IV for transmission. A driven gear V is fixedly mounted at the bottom end of a lifting screw II, and a motor drive gear V is fixedly mounted on the output shaft of a lifting motor II, with the motor drive gear V meshing with the driven gear V for transmission.

[0060] A method for extracting appearance features from multi-resolution leaf samples (see...) Figure 4 The following steps are taken using a multi-resolution leaf sample appearance feature extraction device:

[0061] 1) Place the blade sample to be tested on the lifting plate and fix it with the blade fixing bracket; the computer controls the transverse movement motor I and transverse movement motor II to drive the transverse screw I and transverse screw II to rotate synchronously through the controller, and the horizontal movement motor drives the horizontal screw to rotate, so that the industrial microscope is directly above the blade sample to be tested; the controller controls the rotary motor to drive the industrial microscope to rotate, so that the panoramic image sensor on the outside of the industrial microscope is perpendicular to the lifting plate and aligned with the blade; the controller controls the panoramic image sensor to take pictures of the blade sample and acquire an image of the appearance features of the entire blade;

[0062] 2) Perform ROI segmentation on the image to remove the background and extract the entire leaf ROI and the leaf vein ROI; the ROI segmentation method is as follows: use the target detection algorithm to select the entire leaf ROI and the leaf vein ROI in the background, and calculate the pixel coordinates of the upper left and lower right corners of the ROI box.

[0063] 3) Calculate the number and location data of industrial microscope sampling points within the entire blade ROI;

[0064] The calculation method for the number and location data of industrial microscope sampling points is as follows:

[0065] The pixel coordinates of the two diagonal vertices of the entire leaf ROI bounding box are A1(X1, Y1) and A2(X2, Y2). The resolution of the panoramic image sensor is a×b. (The formula is used to...) Calculate the number of sampling points using the formula. , Calculate the pixel coordinates of all sampled points, where % represents the modulo operation;

[0066] 4) Ensure the distance from the sampling point to the main vein is not less than the distance threshold. l Sampling points whose Intersection over Union (IOU) is not greater than a threshold p are removed; where the IOU is the ratio of the area occupied by the leaf to the area of ​​the panoramic image sensor sampled image; and the distance threshold is... l The specific threshold is 1-2cm. l The threshold p is 0.5, depending on the size of the different types of leaves.

[0067] Distance S from sampling point to main vein m,n The calculation method is as follows

[0068] The pixel coordinates of the two diagonal vertices of the leaf midrib ROI are A3(X3, Y3) and A4(X4, Y4), which can be determined using the formula... Calculate the distance from the sampling point to the main vein;

[0069] The method for calculating the blade crossover ratio (IOU) is as follows:

[0070] The resolution of the panoramic image sensor is a×b, which is determined by the formula... Calculate the blade intersection-union ratio (IOU) at the sampling points;

[0071] 5) Based on the known parameters of the panoramic image sensor, such as focal length, rotation matrix, and translation matrix, the pixel coordinates of the sampling point are multiplied with the known parameter matrix of the panoramic image sensor using the coordinate transformation formula to obtain the corresponding world coordinates. The moving distance L of the panoramic image sensor for each sampling point is then calculated.

[0072] 6) Based on the travel distance L, transmission ratio i, pitch K, and motor step angle B, calculate the number of pulses P required for the panoramic image sensor to move to the sampling point; the formula for calculating the number of motor pulses P is as follows: ;

[0073] 7) By controlling the number of pulses required by the motor, the controller controls the rotary motor to drive the industrial microscope to rotate, so that the lens of the industrial microscope is perpendicular to the lifting plate and aligned with the blade; the controller controls the ring aperture to open to provide a light source and controls the lifting motor I and lifting motor II to drive the lifting screw I and lifting screw II to rotate synchronously, thereby driving the lifting plate to move up and down to adjust the distance between the lens of the industrial microscope and the blade sample until the industrial microscope image is clear.

[0074] Based on the number and location data of the sampling points of the industrial microscope, the controller controls the transverse movement motor I and transverse movement motor II to drive the transverse screw I and transverse screw II to rotate synchronously. The horizontal movement motor drives the horizontal screw to rotate, moving the plane of the industrial microscope to the position of the first sampling point. The industrial microscope then takes a picture of the blade sample and acquires the first sampling image data.

[0075] The controller controls the transverse movement motor I and transverse movement motor II to synchronously drive the transverse screw I and transverse screw II to rotate. The horizontal movement motor drives the horizontal screw to rotate, moving the plane of the industrial microscope to the position of the next sampling point. The industrial microscope takes pictures of the blade sample and acquires the next sampling image data. The data acquisition is repeated until all sampling point image data is extracted.

[0076] Example 3: A method for extracting appearance features from multi-resolution leaf samples (see Example 4) Figure 4 The multi-resolution leaf sample appearance feature extraction device of Example 2 was used, and the specific steps are as follows:

[0077] 1) Place the blade sample to be tested on the lifting plate and fix it with the blade fixing bracket; the computer controls the transverse movement motor I and transverse movement motor II to synchronously drive the transverse screw I and transverse screw II to rotate, and the horizontal movement motor drives the horizontal screw to rotate, so that the industrial microscope is directly above the blade sample to be tested; the controller controls the rotary motor to drive the industrial microscope to rotate, so that the panoramic image sensor on the outside of the industrial microscope is perpendicular to the lifting plate and aligned with the blade; the controller controls the panoramic image sensor to take pictures of the blade sample, and acquire an image of the entire blade's appearance features (see...). Figure 5 );

[0078] 2) Perform ROI segmentation on the image, remove the background, and extract the entire leaf ROI and the leaf midrib ROI. This involves using an object detection algorithm to select the entire leaf ROI and the leaf midrib ROI from the background, and calculating the pixel coordinates of the top-left and bottom-right corners of the ROI bounding box. Figure 6 and Figure 7 As shown;

[0079] 3) Within the entire blade ROI, calculate the number and location data of industrial microscope sampling points (see...). Figure 8 ):

[0080] The pixel coordinates of the two diagonal vertices of the entire leaf ROI bounding box are A1(100, 99) and A2(524, 231). The resolution of the panoramic image sensor is 60×44. (The formula is used to...) The number of sampling points was calculated to be 21; using the formula , Calculate the pixel coordinates of all sampling points (see Table 1), where % represents the modulo operation;

[0081] Table 1

[0082] ;

[0083] 4) Remove unnecessary sampling points: Ensure that the distance from each sampling point to the main vein is not less than the distance threshold. l Sampling points and leaf intersection-union ratio (IOU) not greater than threshold p are removed (see Figure 9 ); where the blade crossover ratio (IOU) is the ratio of the area occupied by the blade to the area of ​​the image sampled by the panoramic image sensor; where the distance threshold l The value is 100 pixels, and the threshold p is;

[0084] Distance S from sampling point to main vein i,j The calculation method is as follows

[0085] The two diagonal vertex pixel coordinates of the leaf midrib ROI are A3 (169, 137) and A4 (404, 151), which can be obtained using the formula... Calculate the distance from the sampling point to the main vein (see Table 2);

[0086] Table 2

[0087] ;

[0088] The method for calculating the blade crossover ratio (IOU) is as follows:

[0089] The panoramic image sensor has a resolution of 60×44, as shown by the formula. Calculate the blade crossover ratio (IOU) at the sampling points (see Table 3);

[0090] Table 3

[0091] ;

[0092] Based on the data in Tables 3 and 4 above, some sampling points can be removed to avoid sampling useless points (see Table 4). The superscript * indicates the sampling points that have been removed.

[0093] Table 4

[0094] ;

[0095] 5) Based on the known parameters of the panoramic image sensor, such as focal length, rotation matrix, and translation matrix, the pixel coordinates of the sampling point are multiplied by the known parameter matrix of the panoramic image sensor using the coordinate transformation formula to obtain the corresponding world coordinates. The moving distance L of the panoramic image sensor for each sampling point is then calculated.

[0096] 6) Based on the travel distance L, transmission ratio i, pitch K, and motor step angle B, calculate the number of pulses P required for the panoramic image sensor to move to the sampling point; the formula for calculating the number of motor pulses P is:

[0097] ;

[0098] 7) Based on the number of pulses required by the motor, the controller controls the rotary motor to drive the industrial microscope to rotate, so that the lens of the industrial microscope is perpendicular to the lifting plate and aligned with the blade; the controller controls the ring aperture to open to provide a light source and controls the lifting motor I and lifting motor II to drive the lifting screw I and lifting screw II to rotate synchronously, thereby driving the lifting plate to move up and down to adjust the distance between the lens of the industrial microscope and the blade sample until the industrial microscope image is clear.

[0099] 8) Based on the number and location data of the industrial microscope sampling points, the controller controls the transverse movement motor I and transverse movement motor II to synchronously drive the transverse screw I and transverse screw II to rotate, and the horizontal movement motor drives the horizontal screw to rotate, moving the industrial microscope plane to the position of the first sampling point. The industrial microscope then takes a picture of the leaf sample and acquires the first sampling image data.

[0100] 9) The controller synchronously drives the transverse movement motors I and II to rotate the transverse screws I and II, while the horizontal movement motor drives the horizontal screw to rotate, moving the industrial microscope plane to the next sampling point. The industrial microscope then photographs the leaf sample, acquiring the next sampling image data. This data acquisition process is repeated until all sampling point image data extraction is complete. The acquired high-resolution image of the leaf is shown below. Figures 10-12 The images shown are high-resolution images of the mesophyll sampling point (430, 165), the branch vein sampling point (370, 165), and the main vein sampling point (310, 165), respectively.

[0101] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for extracting appearance features from multi-resolution leaf samples, characterized in that, The following are the specific steps of using a multi-resolution leaf sample appearance feature extraction device: 1) Acquire images of the entire blade's appearance features using a panoramic image sensor; 2) Perform ROI segmentation on the image, remove the background, and extract the entire leaf ROI and the leaf midrib ROI; 3) Calculate the number and location data of industrial microscope sampling points within the entire blade ROI; The method for calculating the number and location data of sampling points for the industrial microscope: The pixel coordinates of the two diagonal vertices of the entire leaf ROI bounding box are A1(X1, Y1) and A2(X2, Y2). The resolution of the panoramic image sensor is a×b. (The formula is used to...) Calculate the number of sampling points using the formula. , Calculate the pixel coordinates of all sampled points, where % represents the modulo operation; 4) Ensure the distance from the sampling point to the main vein is not less than the distance threshold. l Sampling points whose intersection-union ratio (IOU) with the leaf blade is not greater than the threshold p are removed; The blade crossover ratio (IOU) is the ratio of the area occupied by the blade to the area of ​​the image sampled by the panoramic image sensor; the distance threshold is... l The value is 1-2 cm, and the threshold p is 0.

5. 5) Based on the known focal length, rotation matrix, and translation matrix of the panoramic image sensor, the pixel coordinates of the sampling points are converted into world coordinates through coordinate transformation, and the moving distance L of the panoramic image sensor for each sampling point is calculated. 6) Based on the travel distance L, transmission ratio i, pitch K, and motor step angle B, calculate the number of pulses P required for the panoramic image sensor to move to the sampling point; the formula for calculating the number of motor pulses P is: ； 7) Drive the imaging device moving component to move the panoramic image sensor to the sampling point position by the number of pulses required by the motor, and extract high-resolution image data of the blade appearance features at the sampling point.

2. The method for extracting the appearance features of multi-resolution leaf samples according to claim 1, characterized in that: Step 2) ROI segmentation processing method: Using a target detection algorithm, the entire leaf ROI and the leaf main vein ROI are selected in the background, and the pixel coordinates of the upper left and lower right corners of the ROI are calculated.

3. The method for extracting the appearance features of multi-resolution leaf samples according to claim 1, characterized in that: Step 4) Distance S from sampling point to main vein m,n The calculation method is as follows: The pixel coordinates of the two diagonal vertices of the leaf midrib ROI are A3(X3, Y3) and A4(X4, Y4), which can be determined using the formula... Calculate the distance from the sampling point to the main vein; The method for calculating the blade crossover ratio (IOU) is as follows: Through formula Calculate the blade crossover ratio (IOU) at the sampling points.

4. The method for extracting the appearance features of multi-resolution leaf samples according to claim 1, characterized in that: The multi-resolution leaf sample appearance feature extraction device includes an imaging device, a moving device, a driving component, and a controller (21). The moving device includes an imaging device moving component, a support (12), and a leaf sample moving component. The imaging device moving component is located at the top of the support (12), and the leaf sample moving component is located at the bottom of the support (12). The leaf sample (25) is placed on the leaf sample moving component, and the imaging device is vertically placed on the imaging device moving component. Both the imaging device moving component and the leaf sample moving component are connected to the driving component. Both the driving component and the imaging device are connected to the controller (21) via data. The controller (21) is connected to an external computer. The imaging device includes an industrial microscope (7) and a panoramic image sensor (11). The industrial microscope (7) is vertically mounted on the moving component of the imaging device. The lens (28) of the industrial microscope (7) is mounted at the bottom of the industrial microscope (7). The annular aperture (27) of the industrial microscope (7) is mounted on the outer ring of the lens (28). The panoramic image sensor (11) is fixedly mounted on the outer side of the top of the industrial microscope (7). Both the industrial microscope (7) and the panoramic image sensor (11) are connected to the controller (21).

5. The method for extracting the appearance features of multi-resolution leaf samples according to claim 4, characterized in that: The top of the industrial microscope (7) is set on the top of the mobile device via an industrial microscope stand (9); The moving components of the imaging device include a transverse screw I (1), a transverse screw II, a horizontal screw (5), a horizontal guide rail I (6), a horizontal guide rail II, a horizontal screw block (10), a transverse guide rail I, and a transverse guide rail II (16). Horizontal guide rail I and horizontal guide rail II (16) are fixedly mounted on the top of the bracket (12) in parallel. Horizontal screw I (1) and horizontal screw II are mounted on the top of the bracket (12) in parallel. Horizontal screw I (1) is parallel to horizontal guide rail I and is located above horizontal guide rail I. Horizontal screw II is parallel to horizontal guide rail II (16) and is located above horizontal guide rail II (16). Horizontal screw (5), horizontal guide rail I (6) and horizontal guide rail II are mounted in parallel between horizontal screw I (1) and horizontal screw II. Horizontal screw (5) is located between horizontal guide rail I (6) and horizontal guide rail II. The top of the industrial microscope stand (9) is mounted on the horizontal screw (5) through a horizontal screw block (10). Horizontal screw I (1), horizontal screw II and horizontal screw (5) are all connected to the driving component. One end of the horizontal screw (5) is connected to the bearing bracket I (3) through bearing I (4), and the other end of the horizontal screw (5) is connected to the bearing bracket II through bearing II. The two ends of the bearing bracket I (3), bearing bracket II, horizontal guide rail I (6) and horizontal guide rail II are all threadedly connected to the horizontal screw I (1) and horizontal screw II through the transverse screw block (2). The two ends of the bearing bracket I (3) are fixedly connected to the inner sidewalls of the horizontal guide rail I (6) and the horizontal guide rail II, respectively. The two ends of the bearing bracket II are fixedly connected to the inner sidewalls of the horizontal guide rail I (6) and the horizontal guide rail II, respectively.

6. The method for extracting the appearance features of multi-resolution leaf samples according to claim 5, characterized in that: The moving components of the blade sample device include a lifting plate (20), a lifting motor frame II, a lifting screw I (23), a lifting screw II, a lifting screw block I (24), a lifting screw block II, and a blade fixing frame (26). The lifting screw I (23) and the lifting screw II are respectively vertically set at the bottom of the support (12). The lifting screw block I (24) is sleeved on the lifting screw I (23) and the lifting screw block I (24) is threadedly connected to the lifting screw I (23). The lifting screw block II is sleeved on the lifting screw II and the lifting screw block II is threadedly connected to the lifting screw II. The lifting plate (20) is horizontally set at the bottom of the support (12). The two ends of the lifting plate (20) are fixedly connected to the lifting screw block I (24) and the lifting screw block II, respectively. The blade sample (25) is fixedly set on the lifting plate (20) through the blade fixing frame (26).

7. The method for extracting the appearance features of multi-resolution leaf samples according to claim 6, characterized in that: The driving components include a transverse movement motor I, a transverse movement motor II (17), a rotary motor (8), a horizontal movement motor (13), a lifting motor I, and a lifting motor II (19). The transverse movement motor I is fixedly mounted on the top of the support (12) via a transverse motor frame I. The transverse movement motor I is located at the end of the transverse screw I (1) and is connected to the transverse screw I (1) via transmission. The transverse movement motor II (17) is fixedly mounted on the top of the support (12) via a transverse motor frame II (18). The transverse movement motor II (17) is located at the end of the transverse screw II and is connected to the transverse screw II via transmission. The industrial microscope (7) is mounted in the middle of the industrial microscope frame (9) via a rotating shaft. The rotary motor (8) is fixed on the outside of the industrial microscope frame (9) via a rotary motor bracket. The output of the rotary motor (8) is... The shaft and the rotating shaft are on the same straight line and the output shaft of the rotary motor (8) is fixedly connected to the rotating shaft. The horizontal moving motor (13) is fixedly mounted on the bearing frame II through the horizontal motor frame. The horizontal moving motor (13) is connected to the horizontal screw (5) through transmission. The lifting motor I is fixedly mounted at the bottom of the bracket (12) through the lifting motor frame I (22) and the lifting motor I is located at the side end of the lifting screw I (23). The lifting motor I is connected to the lifting screw I (23) through transmission. The lifting motor II is fixedly mounted at the bottom of the bracket (12) through the lifting motor frame II and the lifting motor II is located at the side end of the lifting screw II. The lifting motor II is connected to the lifting screw II through transmission. The horizontal moving motor I, the horizontal moving motor II (17), the rotary motor (8), the horizontal moving motor (13), the lifting motor I and the lifting motor II (19) are all connected to the controller (21).

8. The method for extracting the appearance features of multi-resolution leaf samples according to claim 7, characterized in that: A driven gear I is fixedly installed at the end of the transverse screw I (1), and a motor drive gear I is fixedly installed on the output shaft of the transverse moving motor I. The motor drive gear I meshes with the driven gear I for transmission. A driven gear II is fixedly installed at the end of the transverse screw II, and a motor drive gear II is fixedly installed on the output shaft of the transverse moving motor II (17). The motor drive gear II meshes with the driven gear II for transmission. A driven gear III (14) is fixedly installed at the end of the horizontal screw (5), and a motor drive gear III (15) is fixedly installed on the output shaft of the horizontal moving motor (13). The motor drive gear III (15) meshes with the driven gear III (14). A driven gear IV is fixedly installed at the bottom end of the lifting screw I (23), and a motor drive gear IV is fixedly installed on the output shaft of the lifting motor I. The motor drive gear IV meshes with the driven gear IV for transmission. A driven gear V is fixedly installed at the bottom end of the lifting screw II, and a motor drive gear V is fixedly installed on the output shaft of the lifting motor II. The motor drive gear V meshes with the driven gear V for transmission.

Images