Preprocessing method of forest canopy hemisphere image

Abstract

The invention provides a forest canopy hemisphere image preprocessing method, which comprises the steps of inputting a labeled image set into a reference detection model for training to obtain an optimized target detection model, and inputting a test set comprising a plurality of forest canopy hemisphere images, performing target detection on the forest canopy hemisphere image contained in the test set based on the target detection model to determine an effective calculation region hp and a north pointing point np of the forest canopy hemisphere image, and calculating a relative rotation angle theta nv based on a detection frame determined by the effective calculation region hp and the north pointing point np and a vertex coordinate v of the forest canopy hemisphere image; and performing rotation operation on the forest canopy hemisphere image and outputting the forest canopy hemisphere image corrected by the north pointing point np. According to the invention, the processing efficiency and accuracy of the image preprocessing of the forest canopy hemisphere image are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer image vision processing, and in particular to a preprocessing method for a hemispherical image of a forest canopy. Background Art

[0002] The forest canopy structure is a visual indicator of the forest's appearance. It not only affects the degree of solar radiation intercepted by vegetation, but also affects climate characteristics such as wind speed, air temperature and humidity, soil evaporation, and soil temperature, thereby regulating the interaction between plants and the environment, and has an important impact on ecosystem material and energy exchange, biodiversity, climate change, and the storage of greenhouse gases such as carbon dioxide. Studying the forest canopy structure is a prerequisite for forest carbon sink measurement, and digital image analysis of forest canopy hemispherical images is a key technology for forest carbon sink measurement. Digital image analysis based on forest canopy hemispherical images includes performing image preprocessing (hereinafter referred to as "image preprocessing" or "preprocessing") on the original forest canopy hemispherical images collected, and performing image segmentation and image calculation on the preprocessed images. How to efficiently and accurately perform image preprocessing on the original forest canopy hemispherical images is the prerequisite for subsequent image segmentation and image calculation, so as to accurately measure the forest carbon sink of forests in a specific area.

[0003] However, the problem of how to effectively determine the effective calculation area and accurately determine the north point in the image preprocessing based on the original forest canopy hemispherical image has not been effectively solved, which poses a greater challenge to the subsequent image segmentation and image calculation of the forest canopy hemispherical image, and will cause large errors in the calculation of the forest canopy porosity and leaf area index obtained in the final calculation. For a large number of forest canopy hemispherical images, the effective calculation area can only be extracted after completing the north point correction and removing the invalid calculation area such as the surrounding black area in the forest canopy hemispherical image that does not contain the forest canopy hemispherical image.

[0004] Ginseng Figure 1As shown, the original forest canopy hemispherical image 12 includes a large number of black image areas and an effective calculation area hp located at the center of the original image and including the forest canopy hemispherical image, and the effective calculation area hp includes a circular forest canopy hemispherical image hp. Due to the interference of various factors such as random shooting angles and different shooting times when taking photos, the angles of the effective calculation areas hp included in different original forest canopy hemispherical images 12 will be extremely random. Since the original forest canopy hemispherical image 12 includes pixels of the sky area and pixels including the forest canopy area, and it is necessary to effectively determine the north reference point in each effective calculation area hp, otherwise it will cause interference between the pixels including the sky area and the pixels including the forest canopy area respectively included in the multiple original forest canopy hemispherical images 12 formed by shooting the forest canopy of the same area at the same time, and even there are pixels that cannot effectively separate the sky area and the pixels including the forest canopy area, which makes it impossible to analyze and calculate the canopy porosity and leaf area index of the forest through the forest canopy hemispherical image 12, and thus it is impossible to accurately perform forest carbon sink measurement.

[0005] Although there is an existing technology for image preprocessing after image acquisition based on convolutional neural network (CNN), in the preprocessing process of distinguishing pixels containing the sky area and pixels containing the forest canopy area in the forest canopy hemispherical image, due to the extremely irregular shape of the edge of the forest canopy area and the random discrete characteristics of the pixels, there is still a technical problem of huge computational overhead in the process of image preprocessing using convolutional neural network, and the correction and confirmation of the effective calculation area and the north point are not effective.

[0006] It should be noted that the above introduction to the background technology is only for the convenience of a clear and complete description of the technical solutions of the present application and for the convenience of understanding by those skilled in the art. It cannot be considered that the above technical solutions are well known to those skilled in the art just because these solutions are described in the background technology section of the present application. In view of this, it is necessary to improve the image preprocessing technical solutions of the original forest canopy hemispherical images in the prior art to solve the above problems. Summary of the invention

[0007] The purpose of the present invention is to disclose a preprocessing method for a forest canopy hemispherical image, so as to solve the aforementioned technical problems existing in the image preprocessing process of the forest canopy hemispherical image in the prior art, and in particular to realize how to quickly extract the effective calculation area hp and the correction of the north point np of the original forest canopy hemispherical image, thereby improving the processing efficiency and accuracy of the image preprocessing of the forest canopy hemispherical image.

[0008] To achieve the above-mentioned object of the invention, the present invention provides a method for preprocessing a hemispherical image of a forest canopy, comprising:

[0009] S1, acquiring multiple hemispherical images of the original forest canopy based on an imaging device to form an original image set;

[0010] S2, performing data annotation on each original forest canopy hemispherical image contained in the original image set to form an annotated image set including a plurality of annotated images;

[0011] S3, inputting the labeled image set into the yoloV5 benchmark detection model for training to obtain the optimized target detection model CanopyYolo;

[0012] S4, inputting a test set including several forest canopy hemispherical images, and performing target detection on the forest canopy hemispherical images included in the test set based on the target detection model CanopyYolo to determine the effective calculation area hp and the north point np of the forest canopy hemispherical images;

[0013] S5, calculating the relative rotation angle θ based on the vertex coordinates v of the detection frame determined by the effective calculation area hp and the north point np and the forest canopy hemispherical image nv ;

[0014] S6, based on the relative rotation angle θ nv A rotation operation is performed on the forest canopy hemispherical image to output a forest canopy hemispherical image corrected by the north point np.

[0015] As a further improvement of the present invention, the data labeling of each original forest canopy hemispherical image contained in the original image set is specifically as follows:

[0016] The target annotation of the original forest canopy hemispherical image is performed by using LabelMe annotation software to annotate the effective calculation area hp and the north point np in the original forest canopy hemispherical image, and the original forest canopy hemispherical image after target annotation is saved in the yolo format.

[0017] As a further improvement of the present invention, the imaging device is a camera equipped with a fisheye lens, and the hemispherical image of the primeval forest canopy consists of sky pixels and vegetation pixels.

[0018] As a further improvement of the present invention, the inputting of the annotated image set into the yoloV5 benchmark detection model for training to obtain the optimized target detection model CanopyYolo is specifically:

[0019] After the annotated image set is input into the yoloV5 benchmark detection model, the data-annotated original forest canopy hemispherical images contained in the annotated image set are divided into a training set, a validation set and a test set in a ratio of 8:1:1, wherein the training set and the validation set are annotated images with annotations, and the test set is an unannotated image. The training set and the labels of the training set, the validation set and the labels of the validation set are used as model inputs, and the annotated images with annotations are divided into positive samples TP and negative samples TN based on the detection performance index through a classifier. The test set containing several forest canopy hemispherical images is detected through the optimized target detection model CanopyYolo to obtain a detection frame containing a valid calculation area hp and a north point np.

[0020] As a further improvement of the present invention, the relative rotation angle θ nv Calculated by the following formula:

[0021]

[0022] Among them, parameter a, parameter b, and parameter c are the side lengths of the triangle formed by parameter o, parameter n, and parameter v, respectively. The parameter o is the coordinate of the center of the effective calculation area hp, the parameter n is the coordinate of the north point np, and the parameter v is the vertex coordinate of the hemispherical image of the forest canopy.

[0023] As a further improvement of the present invention,

[0024] The calculation formula of the parameter a is as follows:

[0025]

[0026] The calculation formula of the parameter b is as follows:

[0027]

[0028] The calculation formula of the parameter c is as follows:

[0029]

[0030] The parameter x and the parameter y are the coordinate values ​​of the parameter o, the parameter n, and the parameter v along the horizontal coordinate and the vertical coordinate in the hemispherical image of the forest canopy, respectively.

[0031] As a further improvement of the present invention, the calculation formula of the center coordinate o of the effective calculation area hp is as follows:

[0032] (ox,oy)=(x center-hp ,y center-hp );

[0033] The calculation formula of the north point np is as follows:

[0034] (nx,ny)=(x center-np ,y center-np );

[0035] The calculation formula of the vertex coordinate v of the forest canopy hemispherical image is as follows:

[0036] (vx,vy)=(ox,oy-h hp / 2).

[0037] As a further improvement of the present invention, the step S6 is specifically as follows: taking the center coordinate o of the effective calculation area hp as the center, the forest canopy hemispherical image is rotated based on the relative rotation angle θ nv A rotation operation is performed, and the rotated forest canopy hemispherical image is cropped according to the coordinate range formed by the detection frame, so as to finally output the forest canopy hemispherical image corrected by the north point np.

[0038] As a further improvement of the present invention, the coordinate range formed by the detection frame is specifically:

[0039]

[0040] Among them, the parameter w hp and parameter h hp are the detection box width and detection box height respectively, parameter w is the width of the original forest canopy hemisphere image, parameter h is the height of the original forest canopy hemisphere image, and the symbol · is the product operator.

[0041] As a further improvement of the present invention, the effective calculation area hp and the north point np are realized by a fast scanning line algorithm;

[0042] The fast scan line algorithm is specifically:

[0043] Set the brightness threshold T;

[0044] Perform traversal scanning on the effective calculation area hp contained in the forest canopy hemispherical image included in the test set along the row and column directions. When the brightness of the pixel point is greater than the brightness threshold T, save the coordinates of the effective calculation area hp and the north point np;

[0045] The brightness threshold T is 20-40.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] Through this application, the processing efficiency and accuracy of image preprocessing of forest canopy hemispherical images are improved, and assistance is provided for subsequent forest canopy hemispherical image segmentation tasks, significantly reducing the computational overhead of correcting the effective calculation area and the north point. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a schematic diagram of a hemispherical image of the original forest canopy before image preprocessing;

[0049] Figure 2 The forest canopy hemispherical image preprocessing method of the present invention is used to Figure 1 Schematic diagram of the hemispherical image of the forest canopy after performing the north point NP correction;

[0050] Figure 3 Based on Figure 2 Schematic diagram of the effective calculation area hp of the extracted hemispherical image of the forest canopy, and Figure 3 It is represented as the valid calculation area hp after image segmentation has been performed and only contains squares;

[0051] Figure 4 It is an overall flow chart of the preprocessing method of the forest canopy hemispherical image of the present invention;

[0052] Figure 5 It is a specific flow chart of the fast scan line algorithm;

[0053] Figure 6 Schematic diagram of the R curve for determining the target detection model CanopyYolo. DETAILED DESCRIPTION

[0054] The present invention is described in detail below in conjunction with the various embodiments shown in the accompanying drawings, but it should be noted that these embodiments are not limitations of the present invention, and any equivalent transformations or substitutions in functions, methods, or structures made by ordinary technicians in the field based on these embodiments are all within the scope of protection of the present invention.

[0055] Ginseng Figures 1 to 4 As shown, a specific implementation method of a forest canopy hemispherical image preprocessing method (hereinafter referred to as "preprocessing method") disclosed in the present application. The forest canopy hemispherical image (hereinafter referred to as "image" or "hemispherical image") obtained by the preprocessing method is used to realize how to collect the original forest canopy hemispherical image (hereinafter referred to as "original hemispherical image") and perform preprocessing operations on the collected original hemispherical image to output the forest canopy hemispherical image 12 (i.e., the preprocessed image) corrected by the north point np to meet the subsequent image segmentation and image calculation processing of the aforementioned preprocessed image. Since the imaging device (for example, a fisheye camera, model DMC-LX5) needs to collect multiple times at different time points and different locations in the forest to calculate the canopy porosity and leaf area index, the image needs to be rotated through the preprocessing method of the present invention, and the north point np of the image (i.e., Figure 1The multiple forest canopy images taken within a period of time can only be accurately calculated and analyzed after the correction processing of the reference numeral 13 in the figure) is performed. The north compass 13 contained in the randomly taken and acquired forest canopy hemispherical image 12 will obviously not be uniform and strictly coincide with the arrow N, so the forest canopy hemispherical image 12 needs to be rotated. The forest canopy hemispherical image 12 contains a hemispherical image 10 composed of sky pixels and vegetation pixels. It is generally selected in the morning and evening on cloudy or sunny days, and the forest type can be broad-leaved forest, coniferous forest or mixed forest.

[0056] The leaf area index is used to describe the canopy porosity. There is a specific mathematical model correspondence between the leaf area index and the canopy porosity. Since the present application is not intended to protect the technical solution of how to calculate the leaf area index and the canopy porosity, the specific calculation process of the leaf area index and the canopy porosity and the specific mathematical model correspondence formed by them are not elaborated in detail in the present application. The preprocessing method disclosed in the present invention includes the following steps S1 to S6.

[0057] First, execute step S1, obtain multiple original forest canopy hemispherical images 12 based on an imaging device to form an original image set. Specifically, the imaging device is a camera equipped with a fisheye lens, and the original forest canopy hemispherical image (i.e., the original image) is composed of sky pixels and vegetation pixels. Specifically, Figure 1 The area where the black pixels in the original image 10 are located is characterized as vegetation pixels, and the area where the white pixels are located is characterized as sky pixels. Figure 1 The original image 10 and the subsequent preprocessed image (refer to Figure 3 (as shown in Figure 2) can reflect the canopy porosity and leaf area index of the forest, thus accurately calculating the forest carbon sink. Figure 1 As shown, the original forest canopy hemispherical image 12 includes a square effective calculation area hp, and the direction of the arrow N is the direction of the north point np. The radius of the original forest canopy hemispherical image 12 is R. If the north point np of a certain original forest canopy hemispherical image 12 is not along the direction of the arrow N, it is necessary to rotate the original forest canopy hemispherical image 12 with the center O1 as the rotation reference point. The aforementioned rotation angle is the relative rotation angle θ described below. nv .

[0058] Figure 1 The square area enclosed by the white dotted line in the figure is the effective calculation area hp. The area where the black pixels outside the effective calculation area hp are located is not the forest canopy image and is considered an invalid calculation area, and needs to be removed through image segmentation.

[0059] Then, step S2 is executed to perform data annotation on each original forest canopy hemispherical image contained in the original image set to form a labeled image set containing a plurality of labeled images. The data annotation of each original forest canopy hemispherical image 12 contained in the original image set is specifically as follows: target annotation is performed on the original forest canopy hemispherical image 12 by using LabelMe annotation software to annotate the effective calculation area hp and the north point np in the original forest canopy hemispherical image 12, and the original forest canopy hemispherical image 12 after target annotation is saved in yolo format.

[0060] The target is annotated by using LabelMe annotation software to outline a circular canopy image 10, and the pixels contained in the canopy image represent the forest canopy image. The annotated images contained in the annotated image set are original images that are annotated and carry the valid calculation area hp and the north point np. At this time, the multiple annotated images contained in the annotated image set need to be optimized and trained before the final optimized target detection model CanopyYolo can be obtained. LabelMe annotation software is a QT-based cross-platform image annotation tool written in Python. It can be used to annotate common visual tasks such as classification, detection, segmentation, key points, etc., and supports export in VOC (Visual Object Classes) format and COCO (Common Objects in Context) format.

[0061] Then, execute step S3, input the annotated image set into the yoloV5 benchmark detection model for training to obtain the optimized target detection model CanopyYolo. The yoloV5 benchmark detection model uses Mosaic image enhancement to increase data diversity, and adaptive anchor frame calculation improves detection accuracy, but the existing yoloV5 benchmark detection model still has many congenital defects such as low detection accuracy and limitation on input image size when detecting annotated images. For this reason, the present application needs to train an optimized target detection model CanopyYolo to facilitate the detection of canopy images, so as to facilitate the calculation of the final canopy porosity and leaf area index.

[0062] Specifically, the aforementioned input of the annotated image set into the yoloV5 benchmark detection model for training to obtain the optimized target detection model CanopyYolo is specifically as follows: after the annotated image set is input into the yoloV5 benchmark detection model, the data-annotated original forest canopy hemispherical image 12 contained in the annotated image set is divided into a training set, a validation set and a test set in a ratio of 8:1:1, wherein the training set and the validation set are annotated images with annotations, and the test set is an unannotated image. The training set and the labels of the training set, the validation set and the labels of the validation set are used as model inputs, and the annotated images with annotations are divided into positive samples TP and negative samples TN based on the detection performance index by a classifier, and the test set containing several forest canopy hemispherical images is detected by the optimized target detection model CanopyYolo to obtain a detection frame containing a valid calculation area hp and a north point np.

[0063]

[0064]

[0065] Ginseng Figure 6 As shown in the table above, in this application, the configuration parameters for the deep learning experimental environment are shown in the table above. The configuration parameters for deep learning of the yoloV5 benchmark detection model are set as follows: The learning rate is set to 1e -5 The batch size is set to 16 and the model training rounds are 100 epochs. Adam is used as the optimizer and binary cross entropy is used as the loss function. Dropout is set to 0.5 to prevent overfitting. Deep learning uses a set of 1254 annotated images of mixed forests to train the yoloV5 benchmark detection model. Figure 6 The horizontal axis in the middle is the confidence interval (Confidence), and is specifically 0.95 (95%).

[0066] This application introduces the following performance detection indicators: Accuracy, Precision, Recall, F1-score and mean Average Precision (mAP) to predict the forest canopy hemispherical images contained in the test set, extract positive and negative samples, and train the optimized target detection model CanopyYolo, thereby improving the accuracy of the target detection results of the forest canopy hemispherical images contained in the test set by the optimized target detection model CanopyYolo.

[0067] The accuracy calculation formula is:

[0068] The accuracy calculation formula is:

[0069] The recall calculation formula is:

[0070] In the above calculation formula, TP: the classifier predicts correctly, that is, the label is a positive example, and the classifier predicts a positive sample; TN: the classifier predicts correctly, that is, the label is a negative example, and the classifier predicts a negative sample; FP: the classifier predicts incorrectly, the label is a negative example, and the classifier predicts a positive sample; FN: the classifier predicts incorrectly, the classifier predicts a negative sample, and the label is a positive example.

[0071] The calculation formula of F1-score is:

[0072] The calculation formula of the mean average precision is:

[0073]

[0074] Where C is the number of categories, AP i is the average precision of the i-th category, the number of categories C is usually 2, and i is a positive integer greater than or equal to 1.

[0075] Then, execute step S4, input a test set containing several forest canopy hemispherical images, and perform target detection on the forest canopy hemispherical images contained in the test set based on the target detection model CanopyYolo to determine the effective calculation area hp and north point np of the forest canopy hemispherical images.

[0076] The effective calculation area hp and the north point np are realized by the fast scanning line algorithm. The fast scanning line algorithm is as follows: set the brightness threshold T; perform traversal scanning on the effective calculation area hp contained in the forest canopy hemispherical image included in the test set along the row and column directions. When the pixel brightness is greater than the brightness threshold T, save the coordinates of the effective calculation area hp and the north point np; the brightness threshold T is 20 to 40. Figure 5 As shown, the fast scan line algorithm is further described in detail as follows.

[0077] start.

[0078] Step 401: Scan the effective calculation area hp along the row direction and the column direction.

[0079] Step 402: Determine whether the brightness of the pixels in the effective calculation area hp is greater than the brightness threshold T. If so, jump to step 403; if not, jump to step 401, thereby eliminating the pixels in the effective calculation area hp whose brightness is less than the brightness threshold T.

[0080] Step 403: When the brightness of the pixel points in the effective calculation area hp is greater than the brightness threshold T, the scanning is suspended, and the coordinates of the scanned pixel points at this time are P(x, y).

[0081] Step 404: Record the pixel point where the pixel point coordinates P(x, y) are located.

[0082] Step 405: Obtain the brightness of the adjacent pixel point P(x+1,y), and the aforementioned adjacent pixel point may be adjacent pixel points along the row direction or adjacent pixel points along the column direction.

[0083] Step 406: Determine whether the brightness of the adjacent pixel point P(x+1,y) is greater than the brightness threshold T. If so, jump to step 407, if not, jump again to step 402. This process continues until all pixel points in the effective calculation area hp are completely scanned.

[0084] Step 407: Terminate scanning.

[0085] Finish.

[0086] Then, step S5 is executed to calculate the relative rotation angle θ based on the vertex coordinates v of the detection frame determined by the effective calculation area hp and the north point np and the forest canopy hemispherical image. nv In this application, the center O1, the north point np and the vertex coordinate v are combined to calculate the angle (relative angle) between the vertex coordinate v and the north point np, thereby calculating the relative rotation angle θ nv , to rotate the forest canopy hemispherical image clockwise or counterclockwise by a relative rotation angle θ nv , and finally complete the correction of the north point np contained in the forest canopy hemispherical image.

[0087] Specifically, the relative rotation angle θ nv Calculated by the following formula:

[0088] Among them, parameter a, parameter b, and parameter c are the side lengths of the triangle formed by parameters o, parameter n, and parameter v respectively. Parameter o is the coordinate of the center of the effective calculation area hp, parameter n is the coordinate of the north point np, and parameter v is the vertex coordinate of the hemispherical image of the forest canopy.

[0089] The calculation formula of parameter a is as follows:

[0090] The calculation formula of parameter b is as follows:

[0091] The calculation formula of parameter c is as follows:

[0092] Among them, parameter x and parameter y are the coordinate values ​​of parameter o, parameter n, and parameter v along the horizontal and vertical coordinates in the hemispherical image of the forest canopy, respectively. The coordinate values ​​of the horizontal and vertical coordinates are Figure 2 The coordinate values ​​of the hemispherical image of the forest canopy formed by the north point np correction.

[0093] The calculation formula for the center coordinate o of the effective calculation area hp is as follows:

[0094] (ox,oy)=(x center-hp ,y center-hp );

[0095] The calculation formula of the north point np is as follows:

[0096] (nx,ny)=(x center-np ,y center-np );

[0097] The calculation formula of the vertex coordinate v of the forest canopy hemispherical image is as follows:

[0098] (vx,vy)=(ox,oy-h hp / 2).

[0099] Finally, step S6 is performed, based on the relative rotation angle θ nv The forest canopy hemispherical image is rotated to output a forest canopy hemispherical image corrected by the north point np. Figure 3 The specific step S6 is: taking the center coordinate o of the effective calculation area hp as the center (that is, Figure 2 The center of the circle is O1), and the hemispherical image of the forest canopy is based on the relative rotation angle θ nv The rotation operation is performed, and the rotated forest canopy hemispherical image is cropped according to the coordinate range formed by the detection frame, so as to finally output the forest canopy hemispherical image 12a corrected by the north point np, which contains the north point np13 and Figure 1 The arrow N in coincides.

[0100] The coordinate range formed by the detection frame is specifically:

[0101]

[0102] Among them, the parameter w hp and parameter h hp are the detection frame width and the detection frame height respectively, the parameter w is the width of the original forest canopy hemispherical image 12, and the parameter h is the height of the original forest canopy hemispherical image 12. The aforementioned width and height are the original forest canopy hemispherical image 12. Figure 1 or Figure 2The width and height in , the symbol · is the product operator. The aforementioned detection box is the area where the effective calculation area hp is located.

[0103] In summary, the preprocessing method disclosed in this application significantly improves the processing efficiency and accuracy of image preprocessing of forest canopy hemispherical images, provides assistance for subsequent forest canopy hemispherical image segmentation tasks, and significantly reduces the computational overhead of correcting the effective calculation area hp and the north point np.

[0104] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

[0105] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0106] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for preprocessing a hemispherical image of a forest canopy, characterized in that: include: S1, acquiring multiple hemispherical images of the original forest canopy based on an imaging device to form an original image set; S2, performing data annotation on each original forest canopy hemispherical image contained in the original image set to form an annotated image set including a plurality of annotated images; S3, inputting the labeled image set into the yoloV5 benchmark detection model for training to obtain the optimized target detection model CanopyYolo; S4, inputting a test set including several forest canopy hemispherical images, and performing target detection on the forest canopy hemispherical images included in the test set based on the target detection model CanopyYolo to determine the effective calculation area hp and the north point np of the forest canopy hemispherical images; S5, calculating the relative rotation angle θ based on the vertex coordinates v of the detection frame determined by the effective calculation area hp and the north point np and the forest canopy hemispherical image nv ; S6, based on the relative rotation angle θ nv A rotation operation is performed on the forest canopy hemispherical image to output a forest canopy hemispherical image corrected by the north point np.

2. The pretreatment method according to claim 1, characterized in that The data labeling of each original forest canopy hemispherical image contained in the original image set is specifically as follows: The target annotation of the original forest canopy hemispherical image is performed by using LabelMe annotation software to annotate the effective calculation area hp and the north point np in the original forest canopy hemispherical image, and the original forest canopy hemispherical image after target annotation is saved in the yolo format.

3. The pretreatment method according to claim 1, characterized in that: The imaging device is a camera equipped with a fisheye lens, and the hemispherical image of the primeval forest canopy consists of sky pixels and vegetation pixels.

4. The pretreatment method according to claim 1, characterized in that: The specific steps of inputting the labeled image set into the yoloV5 benchmark detection model for training to obtain the optimized target detection model CanopyYolo are as follows: After the annotated image set is input into the yoloV5 benchmark detection model, the data-annotated original forest canopy hemispherical images contained in the annotated image set are divided into a training set, a validation set and a test set in a ratio of 8:1:1, wherein the training set and the validation set are annotated images with annotations, and the test set is an unannotated image. The training set and the labels of the training set, the validation set and the labels of the validation set are used as model inputs, and the annotated images with annotations are divided into positive samples TP and negative samples TN based on the detection performance index through a classifier. The test set containing several forest canopy hemispherical images is detected through the optimized target detection model CanopyYolo to obtain a detection frame containing a valid calculation area hp and a north point np.

5. The pretreatment method according to claim 1, characterized in that: The relative rotation angle θ nv Calculated by the following formula: Among them, parameter a, parameter b, and parameter c are the side lengths of the triangle formed by parameter o, parameter n, and parameter v, respectively. The parameter o is the coordinate of the center of the effective calculation area hp, the parameter n is the coordinate of the north point np, and the parameter v is the vertex coordinate of the hemispherical image of the forest canopy.

6. The pretreatment method according to claim 5, characterized in that: The calculation formula of the parameter a is as follows: The calculation formula of the parameter b is as follows: The calculation formula of the parameter c is as follows: The parameter x and the parameter y are the coordinate values ​​of the parameter o, the parameter n, and the parameter v along the horizontal coordinate and the vertical coordinate in the hemispherical image of the forest canopy, respectively.

7. The pretreatment method according to claim 6, characterized in that: The calculation formula of the center coordinate o of the effective calculation area hp is as follows: (ox,oy)=(x center-hp ,y center-hp ); The calculation formula of the north point np is as follows: (nx,n)=(x center-np ,the center-np ): The calculation formula of the vertex coordinate v of the forest canopy hemispherical image is as follows: (vx,vy)=(ox,oy-hhp / 2).

8. The pretreatment method according to claim 7, characterized in that: The step S6 specifically includes: taking the center coordinate o of the effective calculation area hp as the center, rotating the forest canopy hemispherical image based on the relative rotation angle θ nv A rotation operation is performed, and the rotated forest canopy hemispherical image is cropped according to the coordinate range formed by the detection frame, so as to finally output the forest canopy hemispherical image corrected by the north point np.

9. The pretreatment method according to claim 8, characterized in that: The coordinate range formed by the detection frame is specifically: Among them, the parameter w hp and parameter h hp are the detection box width and detection box height respectively, parameter w is the width of the original forest canopy hemisphere image, parameter h is the height of the original forest canopy hemisphere image, and the symbol · is the product operator.

10. The pretreatment method according to claim 1, characterized in that: The effective calculation area hp and the north point np are realized by a fast scanning line algorithm; The fast scan line algorithm is specifically: Set the brightness threshold T; Perform traversal scanning on the effective calculation area hp contained in the forest canopy hemispherical image included in the test set along the row and column directions. When the brightness of the pixel point is greater than the brightness threshold T, save the coordinates of the effective calculation area hp and the north point np; The brightness threshold T is 20-40.

Images