Single tree detection method and device, computer program product and parallel computing system

Abstract

The invention provides a single tree detection method and device, a computer program product and a parallel computing system, and relates to the field of forestry remote sensing monitoring. The method comprises the steps of obtaining a to-be-analyzed image of a target area; wherein the to-be-analyzed image comprises a plurality of tree connected domains; generating a tree connected domain subset according to the plurality of tree connected domains; for each tree connected domain in the tree connected domain subset, calculating the fitness of each basic template sequence to the tree connected domain; wherein the basic template sequence comprises a plurality of templates with different sizes; determining an optimal template combination according to the fitness and each basic template sequence; wherein the optimal template combination is used for performing individual tree detection on the target area. According to the method, the connected domain subset is generated according to the tree connected domain, and the optimal template combination is determined according to the connected domain subset and the preset template, so that the accuracy and the generation efficiency of the optimal template combination are improved, and the individual tree detection efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of forestry remote sensing monitoring, in particular, to a single tree detection method, device, computer program product and parallel computing system. BACKGROUND

[0002] The forest on earth is an important part of the ecological system, is an important link in the carbon cycle in nature, has the functions of protecting water and soil resources, sand fixation and wind prevention, maintaining climate stability, etc. For human production and life, forest is also an important agriculture.

[0003] Detecting and counting the forest in remote sensing image (single tree detection) is an indispensable work in researching the number change of tree population and supporting the formulation of tree protection policy, which has been widely concerned because of its potential to replace the traditional field sampling investigation method. Among them, template matching is one of the commonly used methods in single tree detection field, which has the advantages of simple process, high efficiency and wide application range.

[0004] However, when using the existing template matching method for single tree detection based on remote sensing image, the size of the single tree template needs to be manually selected, and the hardware resources are not fully utilized, resulting in low efficiency of single tree detection. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide a single tree detection method, device, computer program product and parallel computing system, which generates a connected domain subset according to the tree connected domain, determines the optimal template combination according to the connected domain subset and the preset template, and uses the optimal template combination for single tree detection, improves the accuracy and generation efficiency of the optimal template combination, and thus improves the efficiency of single tree detection.

[0006] In a first aspect, the embodiments of the present application provide a single tree detection method, which comprises: acquiring a target area of an image to be analyzed; wherein the image to be analyzed comprises a plurality of tree connected domains; generating a tree connected domain subset according to the plurality of tree connected domains; calculating the fitness of each basic template sequence to the tree connected domain for each tree connected domain in the tree connected domain subset; wherein the basic template sequence comprises a plurality of templates with different sizes; determining the optimal template combination according to the fitness and each basic template sequence; wherein the optimal template combination is used for single tree detection of the target area.

[0007] In the embodiments of the present application, the tree connected domains subset is generated by the plurality of tree connected domains, so as to generate a subset with relatively small quantity and capable of representing the sample distribution in the image to be analyzed, thereby reducing the data quantity for calculating the fitness. Further, for each tree connected domain in the subset, the fitness corresponding to each basic template sequence is calculated, and the optimal template combination is determined according to the fitness, so that the optimal template combination is automatically generated by the fitness calculation algorithm, thereby improving the accuracy and generation efficiency of the optimal template combination.

[0008] In some embodiments, the fitness of each basic template sequence to the tree connected domain is calculated, including: calculating a template filling rate according to the pixel quantity of the template in the basic template sequence and the pixel quantity of the tree connected domain; calculating a template matching degree based on the similarity between each template in the basic template sequence and the tree connected domain; calculating a template use cost of the basic template sequence according to a preset parameter sequence; and calculating the fitness of the basic template sequence to the tree connected domain by the template filling rate, the template matching degree and the template use cost.

[0009] In the embodiments of the present application, since the template filling rate, the template matching degree and the template use cost reflect the fitness of the template sequence to the tree connected domain from different angles, the fitness of the basic template sequence to the tree connected domain is calculated by the template filling rate, the template matching degree and the template use cost, thereby improving the accuracy of the fitness calculation of the template sequence to the tree connected domain, and improving the accuracy and generation efficiency of the optimal template combination, which can not only improve the efficiency of single tree detection, but also improve the accuracy of single tree detection.

[0010] In some embodiments, the fitness of the basic template sequence to the tree connected domain is calculated by the template filling rate, the template matching degree and the template use cost, including: calculating the fitness of the basic template sequence to the tree connected domain according to the formula ; wherein, represents the fitness of the kth basic template sequence to the ith tree connected domain, FR represents the template filling rate, SR represents the template matching degree, and TC represents the template use cost, , , represents a weight factor.

[0011] In the embodiments of the present application, the final fitness is calculated by assigning different weight factors to the parameters such as the template filling rate, the template matching degree and the template use cost, which reflects the influence of these parameters on the fitness calculation, thereby improving the accuracy and flexibility of the fitness calculation.

[0012] In some embodiments, wherein the optimal template combination is determined based on the fitness and each base template sequence, the method comprises: a counting step of counting the number of times each base template sequence achieves the maximum fitness on the tree connected domain; a sorting step of sorting the base template sequences in descending order according to the number of times to obtain sorted base template sequences; a cross combination step of cross combining two adjacent base template sequences to generate a plurality of new template sequences; and a fitness calculation step of calculating the new fitness of each new template sequence on each tree connected domain in the subset of tree connected domains; and determining the optimal template combination based on the new fitness of each new template sequence on the tree connected domain.

[0013] In the embodiments of the present application, considering that the base template sequence is usually not the optimal result, the plurality of base template sequences are sorted in descending order according to the number of times of the maximum fitness of the base template sequence, and two adjacent base template sequences are cross combined to form a new template sequence, and the new fitness of the new template sequence is calculated again, and finally the optimal template combination is determined based on the new fitness of the new template sequence. In this process, the accuracy of the finally generated optimal template combination is improved by optimizing and iterating the base template sequence based on the fitness, which not only improves the efficiency of single tree detection, but also improves the accuracy of single tree detection.

[0014] In some embodiments, wherein the optimal template combination is determined based on the new fitness of each new template sequence on the tree connected domain, the method comprises: for each new template sequence, the counting step, the sorting step, the cross combination step and the fitness calculation step are executed in a loop according to the new fitness, until one template sequence is generated after the cross combination step as the optimal template combination.

[0015] In the embodiments of the present application, since only one template combination is finally needed for single tree detection, one template combination is finally generated based on a plurality of base template sequences through loop iteration. And the new template sequence generated each time is formed by cross combining the template sequence of the last time, not simply selected from the template sequence of the last time, so the accuracy of the finally generated optimal template combination is improved, which not only improves the efficiency of single tree detection, but also improves the accuracy of single tree detection.

[0016] In some embodiments, for the sorted base template sequences, two adjacent base template sequences are cross combined to generate a plurality of new template sequences, comprising: if it is determined that the codes at the corresponding positions in the two adjacent base template sequences are consistent, the codes are taken as the codes at the corresponding positions in the new template sequence; and if it is determined that the codes at the corresponding positions in the two adjacent base template sequences are inconsistent, the codes in the two adjacent base template sequences are randomly selected as the codes at the corresponding positions in the new template sequence.

[0017] In the embodiments of the present application, if it is determined that the codes of the corresponding positions in the two adjacent basic template sequences are consistent, it indicates that the template is necessary in either template sequence, and thus the code is taken as the code of the corresponding position in the new template sequence, ensuring the consistency of the template and laying a foundation for the subsequent generation of the optimal template combination. If it is determined that the codes of the corresponding positions in the two adjacent basic template sequences are inconsistent, it indicates that the template can be selected, and thus randomness is introduced through random selection, thereby increasing the diversity of the new template sequence and preventing the algorithm from converging to a local optimal solution too early. Therefore, through the cross combination, the accuracy and adaptability of the optimal template combination generated finally are improved.

[0018] In some embodiments, for the sorted basic template sequences, the two adjacent basic template sequences are cross combined to generate a plurality of new template sequences, and the method further includes: if it is determined that the plurality of new template sequences include the same new template sequence, the repeated new template sequence is removed.

[0019] In the embodiments of the present application, the repeated new template sequence is removed, so that the finally retained new template sequence is unique, which not only reduces the amount of calculation data, but also improves the accuracy of the optimal template combination generated finally.

[0020] In some embodiments, the tree connected domain subset is generated according to the tree connected domain, including: partitioning the tree connected domain based on the area size of the tree connected domain according to a preset partition rule to generate a plurality of tree connected domain candidate sets; for each tree connected domain candidate set, extracting the same number of tree connected domains as a tree connected domain candidate subset; and generating the tree connected domain subset based on each tree connected domain candidate subset.

[0021] In the embodiments of the present application, the tree connected domains are partitioned according to the area size through the preset partition rule, and sampling is performed from each partition to form the tree connected domain subset. In this process, a subset with a relatively small number of samples is generated to represent the sample distribution in the image to be analyzed, so as to generate the template combination most suitable for the image to be analyzed, and at the same time, the processing efficiency is increased by avoiding the use of all tree connected domains in the image to be analyzed. Thus, the efficiency of single tree detection is improved.

[0022] In a second aspect, an embodiment of the present application provides a parallel computing system, which comprises a task queue, a process scheduler and a plurality of processing cores; wherein the task queue is configured to store fitness calculation tasks; wherein the fitness calculation tasks are generated according to any one of the embodiments of the first aspect; the process scheduler is configured to be communicatively connected with the task queue and the plurality of processing cores respectively; to monitor the state of the task queue and the load state of each processing core, and to assign the fitness calculation tasks to idle processing cores according to a pre-designed calculation mode; and the processing cores are configured to execute the fitness calculation tasks.

[0023] In the embodiments of the present application, the process scheduler assigns the fitness calculation tasks to the idle processing cores, so that a plurality of fitness calculation tasks can be processed simultaneously, the computing resources are fully utilized, and the computing efficiency is improved.

[0024] In some embodiments, the pre-designed calculation mode comprises a multi-connected domain parallel calculation mode and a multi-image parallel calculation mode; the fitness calculation task comprises a plurality of fitness calculation subtasks; the fitness calculation subtask comprises an image identifier; and assigning the fitness calculation task to the idle processing core according to the pre-designed calculation mode comprises: if the pre-designed calculation mode is the multi-connected domain parallel calculation mode, the process scheduler is configured to extract the fitness calculation subtasks from the task queue in sequence and assign the fitness calculation subtasks to any idle processing core; and if the pre-designed calculation mode is the multi-image parallel calculation mode, the process scheduler is configured to determine a target processing core according to the image identifier and assign the fitness calculation subtask to the corresponding target processing core.

[0025] In the embodiments of the present application, the process scheduler is configured with different assignment rules for different pre-designed calculation modes, so that the computing resources are fully utilized based on different scenarios, and the computing efficiency is improved.

[0026] In a third aspect, an embodiment of the present application provides an electronic device, which comprises a processor and a memory; the memory stores machine readable instructions executable by the processor; and the machine readable instructions, when executed by the processor, can execute the method steps of any one of the embodiments of the first aspect.

[0027] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which comprises computer program instructions stored thereon; and the computer program instructions, when executed by a processor, execute the method steps of any one of the embodiments of the first aspect.

[0028] In a fifth aspect, an embodiment of the present application provides a computer program product, which comprises computer program instructions; and the computer program instructions, when executed by a processor, execute the method steps of any one of the embodiments of the first aspect.

[0029] Other features and advantages of the present application will be set forth in the following description, and in part will be apparent from the description, or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly introduced as follows, and it should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0031] Figure 1 A flowchart of a single tree detection method provided by the embodiments of the present application; Figure 2 A sampling diagram of forming a tree connected domain subset provided by the embodiments of the present application; Figure 3 A principle diagram of template embedding and template embedding position sequence encoding provided by the embodiments of the present application; Figure 4 A diagram of forming a new template sequence by template cross combination provided by the embodiments of the present application; Figure 5 A structural diagram of a parallel computing system provided by the embodiments of the present application; Fig. 6(a) is a principle diagram of a multi-connected domain parallel computing mode provided by the embodiments of the present application; Fig. 6(b) is a principle diagram of a multi-image parallel computing mode provided by the embodiments of the present application; Figure 7 A structural diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0032] The embodiments of the technical solutions of the present application will be described in detail below with reference to the drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and therefore only serve as examples, and cannot limit the protection scope of the present application.

[0033] It should be noted that all the technical and scientific terms used herein have the same meaning as that generally understood by the person skilled in the art of the present application; the terms used herein are only for the purpose of describing the specific embodiments, and are not intended to limit the present application; the terms "include" and "have" in the specification and claims of the present application and their any variants are intended to cover the non-exclusive inclusion.

[0034] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise explicitly and specifically limited.

[0035] Forest trees on earth are an important part of the ecological system and an important part of carbon cycling in nature, with the functions of protecting water and soil resources, sand fixation and wind prevention, and maintaining climate stability. For human production and life, forest trees are also important agricultural and industrial resources. Sustainable and recyclable development, utilization and protection of forest resources are important research directions in the fields of agriculture, ecology and environmental science.

[0036] Detecting and counting trees (single tree detection) in remote sensing images is an indispensable work in studying the number change of tree population and supporting the formulation of tree protection policies, because it has the potential to replace the traditional field sampling investigation method and has attracted widespread attention. Among them, template matching is one of the commonly used methods in single tree detection, which has the advantages of simple process, high efficiency and wide application range.

[0037] Existing single tree detection methods based on template matching often place templates in images, calculate the similarity of templates and images at each position, and determine the position with high similarity as the detected single tree. In order to match single tree crowns of different sizes in the image, i.e. to determine the template size, it is necessary to combine the prior knowledge of tree crown diameter in the study area to predefine a combination of templates of different sizes, and to carry out pre-experiments on a certain amount of verification data set to determine the template combination with the best result and use it for subsequent calculation. However, this process requires a lot of manpower and time. In addition, when calculating template filling, existing algorithms often calculate templates one by one in a certain order, and do not involve how to parallelize the calculation, so they cannot effectively utilize hardware resources, resulting in the need to spend a lot of time when the number of images to be calculated is large.

[0038] Therefore, when using the template matching method for single tree detection based on remote sensing images, the size of the single tree template needs to be manually selected, and the hardware resources are not fully utilized, resulting in low efficiency of single tree detection.

[0039] To solve the above problems, the present application provides a single tree detection method, device, computer program product and parallel computing system, which generates a connected domain subset according to the tree connected domain, determines the optimal template combination according to the connected domain subset and the preset template, and uses the optimal template combination for single tree detection to improve the efficiency of single tree detection.

[0040] Figure 1A flowchart of a single tree detection method provided by an embodiment of the present application is shown. It can be understood that the single tree detection method provided by the embodiment of the present application can be applied to a server; wherein the server can be an application server or a Web server.

[0041] As shown in Figure 1 , the method comprises: Step S101, obtaining a to-be-analyzed image of a target region; wherein the to-be-analyzed image comprises a plurality of tree connected domains.

[0042] The target region refers to a specific forest geographical range that needs to be analyzed and information extracted by remote sensing.

[0043] In the specific implementation process, the target region is detected by a sensor (usually mounted on a satellite, an airplane or a drone) to obtain the to-be-analyzed image of the target region.

[0044] In order to obtain the tree candidate region in the to-be-analyzed image, an instance segmentation is performed on the to-be-analyzed image by using a segmentation algorithm to identify the pixel region in the to-be-analyzed image that may contain trees, so as to extract the region marked as trees as the tree candidate region.

[0045] Therefore, the tree candidate region refers to a set of pixel regions in the to-be-analyzed image that are preliminarily determined to possibly contain trees by using an image segmentation algorithm.

[0046] After obtaining the tree candidate region, a connected component calculation is performed on the tree candidate region by using the cv2.connectedComponents() function in OpenCV to generate a plurality of tree connected domains.

[0047] The tree connected domain refers to each independent and connected pixel set obtained after the connected component analysis of the tree candidate region. Each tree connected domain represents a cluster of single or multiple tree crowns of different sizes.

[0048] It should be noted that the segmentation algorithm can be an instance segmentation algorithm based on deep learning, a watershed algorithm, threshold segmentation, edge detection, etc. The segmentation algorithm can be determined according to the actual scene.

[0049] Step S102, generating a tree connected domain subset according to the plurality of tree connected domains.

[0050] Considering that the image to be analyzed can contain a large number of tree connected domains, and there are connected domains with similar size and shape in the numerous tree connected domains. Therefore, in order to avoid increasing the processing efficiency by calculating all the tree connected domains contained in the image to be analyzed, and to generate a relatively small number of subsets that can represent the sample distribution in the image to be analyzed, all the tree connected domains are sorted according to the area size, and all the sorted tree connected domains are partitioned according to a preset interval size. Wherein, the sorting of all the tree connected domains according to the area size can be sorting according to the area from large to small, or sorting according to the area from small to large.

[0051] For each partition, a preset number of tree connected domains are randomly sampled to generate a tree connected domain subset, so that the size of the tree connected domain contained in the generated tree connected domain subset is uniformly distributed, which is convenient for subsequent generation of the optimal template combination of the image to be analyzed by the fitness calculation algorithm.

[0052] Wherein, the values of the preset interval size and the randomly sampled preset number can be adaptively set according to actual conditions, which are not limited in the present application.

[0053] Therefore, in some embodiments, generating a tree connected domain subset according to the tree connected domain includes: partitioning the tree connected domain according to a preset partitioning rule based on the area size of the tree connected domain, generating a plurality of tree connected domain candidate sets; for each tree connected domain candidate set, extracting the same number of tree connected domains as a tree connected domain candidate subset; and generating a tree connected domain subset based on each tree connected domain candidate subset.

[0054] For example, if the image to be analyzed contains 300 tree connected domains, the 300 tree connected domains can be divided into 5 intervals according to the area size after sorting, each interval contains 60 tree connected domains, and then 10 tree connected domains are randomly sampled from each interval to form a tree connected domain subset containing 50 tree connected domains.

[0055] If multiple tree connected domain subsets need to be formed, they can be generated by sampling from all the tree connected domains of the image to be analyzed in a replaceable manner.

[0056] For example, after forming a tree connected domain subset, the 10 tree connected domains sampled from each interval are put back into the corresponding interval, and then 10 tree connected domains are randomly sampled from each interval to form a new tree connected domain subset containing 50 tree connected domains.

[0057] Figure 2 A sampling schematic diagram for forming a tree connected domain subset provided by an embodiment of the present application is shown in FIG. 1. Figure 2As shown, four tree connected domain subsets are obtained by sampling all the tree connected domains included in the image to be analyzed.

[0058] In an alternative embodiment, the machine learning model can be trained in advance based on sample tree connected domains and sample tree connected domain subsets to generate a neural network model of tree connected domain subsets, so that after obtaining a plurality of tree connected domains, the tree connected domains are input into the neural network model of tree connected domain subsets to generate tree connected domain subsets.

[0059] In step S103, the fitness of each base template sequence to the tree connected domain is calculated for each tree connected domain in the tree connected domain subset; wherein the base template sequence includes a plurality of templates of different sizes.

[0060] In step S104, the optimal template combination is determined according to the fitness and each base template sequence; wherein the optimal template combination is used for single tree detection on the target region.

[0061] The base template sequence is generated by sampling from a template library containing templates of different sizes. Therefore, the base template sequence includes a plurality of templates of different sizes.

[0062] The base template sequence includes three parameters: template sequence length, template embedding position sequence, and embedded template size. The template sequence length represents the number of templates included in the base template sequence. The template embedding position sequence is defined by a binary code (which can be randomly initialized or manually defined by the user according to prior knowledge), wherein 1 indicates that a template is embedded at this position, and 0 indicates that no template is embedded at this position. The sequence length of the template embedding position sequence is equal to the number of templates contained in the template library. The embedded template size represents the size of the template used at each embedded position in the base template sequence.

[0063] Figure 3 A schematic diagram of the principle of template embedding and template embedding position sequence coding provided by the embodiments of the present application is shown in Figure 3 As shown, it is assumed that the predefined template library contains 6 templates with sizes [1px, 2px, 3px, 4px, 5px, 6px], wherein px represents pixels. The 1px, 2px, 4px, and 6px templates are embedded into the base template sequence, so that the base template sequence includes 4 templates with embedded template sizes of 1px, 2px, 4px, and 6px, respectively, and the template embedding position sequence is 110101.

[0064] In the implementation process, in order to be able to directly read the corresponding template from the template library to construct the basic template sequence, first, a plurality of template embedding position sequences are initialized randomly or manually defined by a user according to prior knowledge, and then the corresponding template is read from the template library based on the template embedding position sequence to constitute the basic template sequence.

[0065] For example, two groups of randomly initialized template embedding position sequences are 110101 and 101111, wherein 1 represents that the position has template embedding, and 0 represents that there is no template embedding. According to the embedding principle from large to small, the two final basic template sequences and the embedded template size are [6, 5, 3, 1] (corresponding to 110101) and [6, 4, 3, 2, 1] (corresponding to 101111) respectively, and the template sequence lengths are 4 and 5 respectively.

[0066] Among them, considering that the larger the crown is, the easier it is to be identified in segmentation and matching, therefore, the templates in the generated basic template sequence are embedded from large to small, so that in template matching, matching large templates first can quickly locate the main target, improve the efficiency of single tree detection, and also avoid the interference of too many false responses in local areas caused by small templates.

[0067] After obtaining the tree connected domain subset, the basic template sequence is read, and for each tree connected domain in the tree connected domain subset, the fitness of each basic template sequence to the tree connected domain is calculated. Then, according to the fitness and each basic template sequence, the optimal template combination is determined.

[0068] Among them, the fitness is a quantitative index for measuring the comprehensive performance of the basic template sequence as a detector when matching the tree connected domain.

[0069] In the implementation process, the fitness of each basic template sequence to the tree connected domain is calculated by the fitness calculation formula (the fitness calculation formula mentioned in the following embodiment), and then the optimal template combination is determined according to the fitness and each basic template sequence.

[0070] According to the above content, a plurality of tree connected domain subsets can be obtained by means of replaceable sampling, therefore, if there are multiple tree connected domain subsets corresponding to a to-be-analyzed image, the fitness of each basic template sequence to each tree connected domain in each tree connected domain subset is calculated, and then the optimal template combination is determined based on the fitness and each basic template sequence. That is, each tree connected domain subset is combined into a large tree connected domain subset, and then the fitness of each basic template sequence to each tree connected domain in the large tree connected domain subset is calculated to determine the optimal template combination.

[0071] After obtaining the optimal template combination, a template matching based method can be used to detect individual trees in the target region, and the specific process is as follows: The optimal template combination is used to scan the image to be analyzed in a sliding window. At each scanning position, each template in the optimal template combination is matched with the local region under the image window, and a similarity index (such as a correlation coefficient, mutual information, etc.) is calculated. For each position, a series of similarity values from multiple templates in the template combination will be obtained. The series of similarity values are fused (for example, the maximum value or the average value is taken) to form a comprehensive "matching response map". In the matching response map, the local peak value (i.e. the point with extremely high response value) usually corresponds to the center position of the single tree crown. By setting a reasonable threshold and finding the local maximum value, each individual tree in the target region is finally located and identified.

[0072] In the embodiments of the present application, a subset of tree connected domains is generated from multiple tree connected domains, so as to generate a subset with relatively small number of samples and representing the distribution of the image to be analyzed, thereby reducing the amount of data used for calculating the fitness. Further, for each tree connected domain in the subset, the fitness of each basic template sequence is calculated, and the optimal template combination is determined according to the fitness. The optimal template combination is automatically generated by the fitness calculation algorithm, thereby improving the accuracy and generation efficiency of the optimal template combination.

[0073] In some embodiments, calculating the fitness of each basic template sequence to the tree connected domain comprises: calculating a template filling rate according to the number of pixels of the template in the basic template sequence and the number of pixels of the tree connected domain; calculating a template matching degree based on the similarity between each template in the basic template sequence and the tree connected domain; calculating a template use cost of the basic template sequence according to a preset parameter sequence; and calculating the fitness of the basic template sequence to the tree connected domain by the template filling rate, the template matching degree and the template use cost.

[0074] In one implementation, the fitness calculation includes three parts: template filling rate (FR), template matching degree (SR) and template use cost (TC).

[0075] The template filling rate FR is used to measure how many pixels of the tree connected domain are covered by the template. That is, the proportion of the pixels of the tree connected domain filled in the template coverage area. Specifically, the template filling rate is calculated by the ratio of the number of filled template pixels to the total number of pixels of the tree connected domain.

[0076] For example, if the template is a circle and the tree connected domain is a polygon, the template filling rate is obtained by calculating the proportion of the circle in the polygon when the template fills the tree connected domain.

[0077] Since the fitness is calculated for the basic template sequence to the tree connected domain, and the basic template sequence includes multiple templates, the template filling rate refers to the ratio of the pixels of all templates in a basic template sequence to the pixels of the tree connected domain.

[0078] The template matching degree SR is used to measure the similarity between a template and the tree connected domain. High matching degree helps to ensure that a tree connected domain is matched by only one most suitable template, instead of being segmented by multiple small templates.

[0079] In an embodiment, the template matching degree can be calculated by a Structural Similarity Index Measurement (SSIM) formula, i.e.:

[0080] wherein, represents the pixels of the template region, represents the pixels of the tree connected domain covered by the template x, represents the average gray scale of the pixels of the template, represents the average gray scale of the pixels of the tree connected domain, represents the standard deviation of the pixel gray scale of the template, represents the standard deviation of the pixel gray scale of the tree connected domain, and represents an empirical constant, which is usually a small value to avoid 0 in the numerator and denominator, and an example is , .

[0081] Finally, the SSIM values calculated by matching all templates in the basic template sequence to the tree connected domain are arithmetically averaged to obtain the template matching degree of the basic template sequence on the tree connected domain.

[0082] It should be understood that other similarity calculation methods can also be used to calculate the template matching degree, such as the square difference matching method, the Bhattacharyya distance, the Histogram of Oriented Gradients (HOG) similarity, and feature point matching, etc.

[0083] Considering that a template with a smaller size has more appropriate positions for filling the tree connected domain (to improve the SR), and using a smaller template can improve the filling rate (to improve the FR), a penalty term is added to the fitness calculation to avoid the algorithm tending to use smaller templates.

[0084] In an embodiment of the present application, the template use cost TC is a penalty term, which avoids overfilling the tree connected domain by multiple small templates to cause the calculated fitness to be artificially high.

[0085] The template use cost TC is calculated by a set of preset parameter sequences defined according to different template sizes, wherein the preset parameter sequences are a negative value sequence. The length of the negative value sequence is equal to the number of templates in the template library, and one-to-one correspondence exists, and a smaller template corresponds to a smaller numerical value.

[0086] For example, in the template library defined above, 6 templates with template sizes of [1px, 2px, 3px, 4px, 5px, 6px] are included, and the corresponding negative value sequence can be defined as [-0.3, -0.25, -0.2, -0.15, -0.1, -0.05]. If the basic template sequence used for calculation is 110101, i.e., [6, 5, 3, 1], the calculated TC value is: .

[0087] Finally, the fitness of the basic template sequence to the tree connected domain is calculated by the template filling rate, the template matching degree, and the template use cost.

[0088] In an embodiment, the fitness of the basic template sequence to the tree connected domain is calculated according to the formula ; wherein, represents the fitness of the kth basic template sequence to the ith tree connected domain, FR represents the template filling rate, SR represents the template matching degree, and TC represents the template use cost, , , represents a weight factor.

[0089] Based on the above embodiment, a plurality of basic template sequences are included, and therefore, for each basic template sequence, the fitness of each basic template sequence to each tree connected domain needs to be calculated by using the above method, so as to evaluate the performance of each basic template sequence by the fitness.

[0090] In the embodiment of the present application, since the template filling rate, the template matching degree, and the template use cost reflect the fitness of the template sequence to the tree connected domain from different angles, the fitness of the basic template sequence to the tree connected domain is calculated by the template filling rate, the template matching degree, and the template use cost, which improves the accuracy of the fitness calculation of the template sequence to the tree connected domain, thereby improving the accuracy and generation efficiency of the optimal template combination, which not only improves the efficiency of single tree detection, but also improves the accuracy of single tree detection.

[0091] In some embodiments, wherein the optimal template combination is determined according to the fitness and each base template sequence, the method comprises: a counting step of counting the number of times each base template sequence achieves the maximum fitness on each tree connected domain; a sorting step of sorting the base template sequences from large to small according to the number of times to obtain sorted base template sequences; a cross combination step of cross combining two adjacent base template sequences to generate a plurality of new template sequences; and a fitness calculation step of calculating the new fitness of each new template sequence on each tree connected domain in the tree connected domain subset; and determining the optimal template combination based on the new fitness of each new template sequence on the tree connected domain.

[0092] Based on the above embodiments, the base template sequence is multiple, and the fitness of each base template sequence on each tree connected domain is calculated. Therefore, the fitness of the same base template sequence on different tree connected domains may be different; similarly, the fitness of different base template sequences on the same tree connected domain may also be different.

[0093] Therefore, in order to find the optimal template combination, the number of times each base template sequence achieves the maximum fitness on each tree connected domain is counted, which facilitates understanding the fitness of each base template sequence.

[0094] In order to optimize the template combination, the base template sequences are sorted from large to small according to the number of times of their corresponding maximum fitness to obtain sorted base template sequences.

[0095] For example, the tree connected domain subset includes 100 tree connected domains, the base template sequence is 3, which are base template sequence a, base template sequence b and base template sequence c, and the base template sequences a, b and c achieve 20 times, 30 times and 50 times of maximum fitness respectively. Then, the base template sequences are sorted from large to small according to the number of times of maximum fitness, and the sorted base template sequences are cba.

[0096] For the sorted base template sequences, two adjacent base template sequences are cross combined to generate a plurality of new template sequences. Then, the new fitness of each new template sequence on each tree connected domain in the tree connected domain subset is calculated. Finally, the optimal template combination is determined based on the new fitness of each new template sequence on the tree connected domain.

[0097] Since only one optimal template combination is finally determined, for each new template sequence, the above counting step, sorting step, cross combination step and fitness calculation step are executed according to the new fitness in a loop until one template sequence is generated as the optimal template combination after the cross combination step.

[0098] Figure 4 A schematic diagram of forming new template sequences by template crossover combination provided in an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the basic template sequences include five, which are sorted in descending order according to the number of their corresponding maximum fitness, to obtain the sorted basic template sequences. For the sorted basic template sequences, two adjacent basic template sequences are combined to form four new template sequences. Therefore, if there are N basic template sequences, N-1 new template sequences are generated for the next step calculation. After iterative cycles, a new template sequence is generated as the optimal template combination. Figure 4

[0099] In an embodiment of the present application, considering that the basic template sequences are usually not the optimal result, the multiple basic template sequences are sorted in descending order according to the number of their maximum fitness, and two adjacent basic template sequences are combined to form new template sequences, and the new fitness of the new template sequences is calculated again, and finally the optimal template combination is determined based on the new fitness of the new template sequences. In this process, the accuracy of the finally generated optimal template combination is improved by optimizing and iterating the basic template sequences based on the fitness, which not only improves the efficiency of single tree detection, but also improves the accuracy of single tree detection.

[0100] Moreover, since only one template combination is needed for single tree detection, one template combination is finally generated based on multiple basic template sequences through iterative cycles. Moreover, the new template sequence generated each time is formed by crossover combination based on the template sequence of the last time, rather than simply selected from the template sequence of the last time, thus improving the accuracy of the finally generated optimal template combination, which not only improves the efficiency of single tree detection, but also improves the accuracy of single tree detection.

[0101] In some embodiments, for the sorted basic template sequences, two adjacent basic template sequences are combined to generate multiple new template sequences, including: if it is determined that the codes at the corresponding positions in the two adjacent basic template sequences are consistent, the codes are taken as the codes at the corresponding positions in the new template sequence; and if it is determined that the codes at the corresponding positions in the two adjacent basic template sequences are inconsistent, the codes in the two adjacent basic template sequences are randomly selected as the codes at the corresponding positions in the new template sequence.

[0102] Continuing to refer to Figure 4 , the sequence crossover rule for forming new template sequences by crossover combination is as follows: if the codes at a certain position in the basic template sequence a and the basic template sequence b are both 1 or 0, then the position in the generated new template sequence c is also 1 or 0; and if the codes at a certain position are different, then the position in the generated new template sequence c has a 50% probability of becoming 1 or 0, respectively.

[0103] ​In the embodiments of the present application, if it is determined that the codes of the corresponding positions in the two adjacent basic template sequences are consistent, it indicates that the template is necessary in either template sequence, and thus the code is taken as the code of the corresponding position of the new template sequence, ensuring the consistency of the template and laying a foundation for the subsequent generation of the optimal template combination. If it is determined that the codes of the corresponding positions in the two adjacent basic template sequences are inconsistent, it indicates that the template can be selected, and thus the randomness is introduced through the random selection, thereby increasing the diversity of the new template sequence and preventing the algorithm from converging to a local optimal solution too early. Therefore, through the cross combination, the accuracy and adaptability of the optimal template combination generated finally are improved.

[0104] In some embodiments, for the sorted basic template sequences, the two adjacent basic template sequences are cross combined to generate a plurality of new template sequences, and the method further includes: if it is determined that the plurality of new template sequences include the same new template sequence, the repeated new template sequence is removed.

[0105] In the embodiments of the present application, the repeated new template sequence is removed, so that the new template sequence finally reserved is unique, which not only reduces the amount of calculation data, but also improves the accuracy of the optimal template combination generated finally.

[0106] Based on the above embodiments, it can be known that when the optimal template combination is determined, the fitness of each tree connected domain in the tree connected domain subset needs to be calculated by using the plurality of basic template sequences. In the actual single tree detection scene, a tree connected domain usually corresponds to dozens of template sizes, and thus the number of templates in the basic template sequence is large, and the corresponding calculation amount is also large.

[0107] Although the above method has reduced the tree connected domains that need to be calculated by constructing the tree connected domain subset, thereby improving the efficiency of generating the optimal template combination, if the fitness of a plurality of to-be-analyzed images needs to be calculated at the same time to perform single tree detection, the calculation amount will also be doubled.

[0108] Therefore, to further improve the calculation efficiency, the parallel computing mode is designed to realize fast calculation in the embodiments of the present application. The main purpose of the parallel computing is to make full use of the calculation resources of the mainstream multi-core processor, especially the cluster computing unit, to realize the fast and efficient implementation of the single tree detection method of the present application.

[0109] Figure 5 The structure schematic diagram of a parallel computing system provided in the embodiments of the present application is as follows: Figure 5As shown in FIG. 5, the parallel computing system 50 includes a task queue 501, a process scheduler 502, and a plurality of processing cores 503; the task queue 501 is configured to store fitness calculation tasks; the fitness calculation tasks are generated according to the tree connected domain subsets in any one of the above embodiments; the process scheduler 502 is configured to be communicatively connected with the task queue and the plurality of processing cores respectively; the process scheduler 502 is configured to monitor the state of the task queue and the load state of each processing core, and assign the fitness calculation tasks to the idle processing cores 503 according to a pre-designed calculation mode; and the processing cores 503 are configured to execute the fitness calculation tasks.

[0110] After the tree connected domain subsets of the images to be analyzed are constructed, all the obtained tree connected domains to be calculated are summarized to form a sample pool. It should be noted that when the tree connected domains are stored in the sample pool, the topological structure of each tree connected domain, the image number (image identifier) to which the tree connected domain belongs, and the position information of the tree connected domain in the image to which the tree connected domain belongs are recorded.

[0111] In an embodiment, a process scheduler is constructed by using the multiprocessing.pool() function in python, which records the maximum number of processor cores and allocates processor core resources to each process.

[0112] When a tree connected domain needs to be calculated, the process scheduler adds the fitness calculation task of the tree connected domain to be calculated to the task queue, and allocates an idle processing core to calculate the calculation task at the front of the task queue.

[0113] In the embodiments of the present application, the fitness calculation tasks are allocated to the idle processing cores by the process scheduler, so that multiple fitness calculation tasks can be processed simultaneously, the computing resources are fully utilized, and the computing efficiency is improved.

[0114] To adapt to different computing scenarios, two computing modes are proposed in the embodiments of the present application: a multi-connected domain parallel computing mode and a multi-image parallel computing mode.

[0115] The multi-connected domain parallel computing mode is suitable for scenarios in which the number of images to be analyzed is small and the processor core resources are relatively sufficient, and all processor cores can be fully utilized by using the computing mode.

[0116] In this mode, a sample pool is constructed, and the tree connected domains in the tree connected domain subsets of all images to be analyzed are stored in the sample pool.

[0117] The multi-image parallel computing mode is suitable for tasks in which the number of images to be analyzed is large (the number of images to be analyzed is much larger than the number of processor cores available for calculation), and different processor cores are used to calculate different images to be analyzed, which facilitates subsequent statistics and analysis.

[0118] In this mode, a sample pool is constructed for each image to be analyzed, and the tree connected domains in the tree connected domain subset of each image to be analyzed are stored in the corresponding sample pool.

[0119] Therefore, in some embodiments, the pre-design computation mode includes a multi-connected domain parallel computation mode and a multi-image parallel computation mode; the fitness computation task includes a plurality of fitness computation subtasks; the sub-fitness computation task includes image identification; and the fitness computation task is assigned to the idle processing core according to the pre-design computation mode, including: if the pre-design computation mode is the multi-connected domain parallel computation mode, the process scheduler is configured to extract the fitness computation subtasks from the task queue in sequence and assign the fitness computation subtasks to any currently idle processing core; and if the pre-design computation mode is the multi-image parallel computation mode, the process scheduler is configured to determine the target processing core according to the image identification and assign the fitness computation subtask to the corresponding target processing core.

[0120] FIG. 6(a) is a schematic diagram of a multi-connected domain parallel computation mode provided by an embodiment of the present application, as shown in FIG. 6(a), without distinguishing which image to be analyzed the tree connected domain belongs to, after a processing core completes the computation, the process scheduler continues to assign the new fitness computation task corresponding to the tree connected domain to the processing core from the task queue until the task queue is empty.

[0121] FIG. 6(b) is a schematic diagram of a multi-image parallel computation mode provided by an embodiment of the present application, as shown in FIG. 6(b), this mode constructs an independent sample pool for each image to be analyzed when computing, and each sample pool corresponds to a processing core. When assigning the fitness computation task, the process scheduler will only assign the tree connected domain in the corresponding sample pool to each processing core. If the number of sample pools is greater than the number of processing cores, after a processing core completes the computation of the tree connected domain of the corresponding image to be analyzed, the process scheduler will assign the tree connected domain of a new image to be analyzed to the processing core for further computation until all data processing is completed.

[0122] In the embodiments of the present application, the process scheduler is configured with different assignment rules for different pre-design computation modes, so as to fully utilize the computing resources based on different scenarios and improve the computing efficiency.

[0123] In summary, the beneficial effects of the present application are as follows: The present application provides a template size self-selection method, which can automatically select the most suitable template size sequence and is applied to single tree detection tasks based on template matching method, to solve the problem of selecting template size through a large amount of manual pre-experiment and supervised data in the prior art, and improve the automation degree and detection efficiency of the detection process.

[0124] The application designs a fitness calculation method (including template filling rate (FR), template matching degree (SR) and template use cost (TC)) as a self-optimization index of the algorithm. The fitness calculation method combines the filling rate and the matching degree, two important accuracy indicators, and adds the template use cost as a penalty term to alleviate the problem that the template matching algorithm tends to use smaller templates to obtain higher filling rate and matching degree, so that the finally obtained template sequence has better global fitness.

[0125] The application designs a self-optimization algorithm to optimize the template sequence and output the optimal result. The method generates new template sequences by crossing combination of template sequences from high to low fitness to realize the optimization of template combination. The algorithm process is simple and efficient, and can be deployed on multi-core computing devices using parallel computing method to further enhance the computing efficiency.

[0126] The application designs a parallel computing method to realize efficient deployment of template filling process on multi-core computing devices. The method includes two computing modes, multi-connected domain parallel computing mode and multi-image parallel computing mode. It can be applied to different data volume and computing resource scenarios. Through the process scheduler, each processing core is allocated tree connected domain computing tasks one by one to fully and efficiently utilize computing hardware resources. At the same time, the sample pool design of tree connected domain position and topological morphological features makes the computing process not need to repeatedly traverse the whole image area to be analyzed to find the tree connected domain, thereby effectively utilizing the hardware resources and improving the computing efficiency.

[0127] Figure 7 The electronic device structure schematic diagram provided by the embodiment of the application is shown in FIG. 1. Figure 7 As shown in FIG. 1, the electronic device includes a processor 701, a memory 702 and a bus 703; wherein the processor 701 and the memory 702 complete mutual communication through the bus 703. The processor 701 is used to call the program instruction in the memory 702 to execute the method provided by each method embodiment.

[0128] The processor 701 can be an integrated circuit chip with a signal processing capability. The processor 701 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), or the like; or can be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The processor 701 can implement or execute the various methods, steps, and logical block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor, or the processor can also be any conventional processor.

[0129] The memory 702 can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0130] The embodiments of the present application provide a computer program product, including: computer program instructions, the computer program instructions being executed by a processor to perform the method provided in each of the above method embodiments.

[0131] The embodiments of the present application provide a computer readable storage medium, including: computer program instructions stored on the computer readable storage medium, the computer program instructions being executed by a processor to perform the method provided in each of the above method embodiments.

[0132] The computer readable storage medium can be realized by any type of volatile or nonvolatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0133] In the embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented in other manners. The described device embodiments are merely schematic, and the division of units is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and there can be electric, mechanical or other forms.

[0134] In addition, the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0135] In addition, each functional module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0136] The above only describes the embodiments of the present application and is not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A single tree detection method, characterized by, The method comprises: obtaining a to-be-analyzed image of a target region; wherein the to-be-analyzed image comprises a plurality of tree connected domains; generating a subset of tree connected domains according to the plurality of tree connected domains; calculating, for each tree connected domain in the subset of tree connected domains, a fitness of each base template sequence to the tree connected domain; wherein the base template sequence comprises a plurality of templates of different sizes; determining an optimal template combination according to the fitness and each base template sequence; wherein the optimal template combination is used for single tree detection of the target region.

2. The method of claim 1, wherein, The calculation of the fitness of each base template sequence to the tree connected domain comprises: calculating a template filling rate according to the number of pixels of a template in the base template sequence and the number of pixels of the tree connected domain; calculating a template matching degree based on the similarity between each template in the base template sequence and the tree connected domain; calculating a template use cost of the base template sequence according to a preset parameter sequence; calculating the fitness of the base template sequence to the tree connected domain by the template filling rate, the template matching degree, and the template use cost.

3. The method of claim 2, wherein, The calculation of the fitness of the base template sequence to the tree connected domain by the template filling rate, the template matching degree, and the template use cost comprises: According to the formula calculate fitness of the base template sequence to the tree connected domain; wherein, represents the fitness of the kth base template sequence for the ith tree connected domain, FR represents the template filling rate, SR represents the template matching degree, TC represents the template use cost, , , represents the weight factor.

4. The method of claim 1, wherein, wherein, The determination of the optimal template combination according to the fitness and each base template sequence comprises: a statistical step: counting the number of times each base template sequence achieves the maximum fitness on the tree connected domain; a sorting step: sorting the base template sequences from large to small according to the number of times to obtain sorted base template sequences; a cross combination step: for the sorted base template sequences, cross combining two adjacent base template sequences to generate a plurality of new template sequences; a fitness calculation step: for each tree connected domain in the subset of tree connected domains, calculating a new fitness of each new template sequence to the tree connected domain; determining the optimal template combination based on the new fitness of each new template sequence to the tree connected domain.

5. The method of claim 4, wherein, wherein, The determination of the optimal template combination based on the new fitness of each new template sequence to the tree connected domain comprises: for each new template sequence, cyclically executing the statistical step, the sorting step, the cross combination step, and the fitness calculation step according to the new fitness, until one template sequence is generated as the optimal template combination after the cross combination step.

6. The method of claim 4, wherein, The cross combination of two adjacent base template sequences to generate a plurality of new template sequences comprises: if it is determined that the codes at the corresponding positions of the two adjacent base template sequences are consistent, then the codes are taken as the codes at the corresponding positions of the new template sequence; if it is determined that the codes at the corresponding positions of the two adjacent base template sequences are inconsistent, then randomly selecting the codes at the corresponding positions of the two adjacent base template sequences as the codes at the corresponding positions of the new template sequence.

7. The method of claim 6, wherein, The method further comprises: If the new template sequences include the same new template sequence, the repeated new template sequence is removed.

8. The method according to any one of claims 1 to 7, characterized in that, The method further comprises: The method further comprises: The method further comprises: The method further comprises:

9. A parallel computing system, characterized by The method further comprises: The system comprises a task queue, a process scheduler and a plurality of processing cores. The task queue is configured to store fitness calculation tasks, wherein the fitness calculation tasks are generated according to the tree connected domain subsets in the method of any one of claims 1-8. The process scheduler is configured to be communicatively connected with the task queue and the plurality of processing cores, to monitor the state of the task queue and the load state of each processing core, and to assign the fitness calculation tasks to idle processing cores according to a preset calculation mode.

10. The system of claim 9, wherein, The processing cores are configured to execute the fitness calculation tasks. The preset calculation mode comprises a multi-connected domain parallel calculation mode and a multi-image parallel calculation mode. The fitness calculation tasks comprise a plurality of fitness calculation subtasks. The fitness calculation subtasks comprise image identification. The process scheduler is configured to extract the fitness calculation subtasks from the task queue in sequence and assign the fitness calculation subtasks to any idle processing core if the preset calculation mode is the multi-connected domain parallel calculation mode. The process scheduler is configured to determine a target processing core according to the image identification and assign the fitness calculation subtask to the corresponding target processing core if the preset calculation mode is the multi-image parallel calculation mode.

Images