A litchi male and female flower counting method based on a point-to-point network and a bayesian loss

Abstract

This invention discloses a method for counting male and female flowers of litchi based on point-to-point networks and Bayesian loss. Addressing the linear perspective problem in images, a feature pyramid network and a lightweight attention module are introduced into the backbone network. In the loss function construction part, the problem of predicting whether a litchi flower exists at a certain point in the image is transformed into a Bayesian estimation problem. Utilizing Bayesian loss, which is highly effective in dense target counting, the update direction of the network parameters is supervised, accurately obtaining the number of male and female flowers in the image. This provides a decision-making reference for flowering period regulation, flower thinning, and yield prediction.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and more specifically to a method for counting male and female flowers of litchi based on point-to-point networks and Bayesian loss. Background Technology

[0002] Flowering is a crucial stage in the growth and development of litchi. Male flowers produce pollen, female flowers produce nectar, and the ovary develops into the fruit after fertilization. Litchi flowering generally involves the opening of the first batch of male flowers, the opening of female flowers, and then the opening of the second batch of male flowers. In some cases, female flowers may open first, and the order of opening may be related to the order of flower differentiation. An excessive number of male flowers in the first batch consumes a large amount of nutrients, leading to a low proportion of female flowers, a short flowering period, and ultimately, a low fruit set. A large number of male flowers in the second batch deplete the nutrients of the fruiting branches, putting the fertilized ovary at a disadvantage in nutrient competition, resulting in fruit drop during flowering. Therefore, thinning the male flowers in both batches can reduce competition for nutrients, decrease fruit drop during flowering, and thus increase litchi yield. Information on flowering intensity is crucial for guiding flower thinning. Efficient and accurate flower counting technology can effectively help orchard owners make flower thinning decisions, contribute to strengthening research on flowering period regulation, predicting yield, and achieving high and stable production.

[0003] Deep learning provides a highly effective technique for the intelligent detection and counting of flowers. Flowers in natural scenes are generally densely packed, and current mainstream counting methods include those based on predicted bounding box detection and those based on density map estimation. Among methods based on predicted bounding box detection, Ye et al. designed a male flower counting method based on a polyphyletic loss function to address the accurate counting problem of litchi flowers. This loss function makes the predicted bounding boxes and labeled boxes closer, resulting in more accurate target detection. However, due to the difficulty in accurately labeling dense, small targets, the dense flowers interfere with each other during feature extraction, leading to poor model optimization and post-prediction processing (non-maximum suppression). For the detection and counting of dense targets, density map estimation methods have been continuously proposed. Lin et al. added two branches with 3*3 and 11*11 convolutional kernel sizes to a multi-column convolutional neural network (MCNN) to estimate the density of litchi male flowers, making it more adaptable to scale changes compared to the original network. However, density map estimation methods are not only computationally intensive and difficult to use in real-time counting, but also generate density maps with low clarity, making it impossible to locate each target. Furthermore, both domestic and international research has focused on counting male litchi flowers, while there has been almost no research on counting the smaller female flowers.

[0004] Therefore, how to provide a method for counting male and female flowers of litchi based on point-to-point networks and Bayesian loss is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a method for counting male and female flowers of litchi based on point-to-point networks and Bayesian loss, which can accurately obtain the number of male and female flowers in an image. The present invention is the first to detect and count the small target of female litchi flowers in natural scenes.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for counting male and female flowers of litchi based on point-to-point networks and Bayesian loss includes:

[0008] S1: Obtain images of litchi flowering period and perform preprocessing;

[0009] S2: Randomly crop the preprocessed litchi flowering period image until the cropped area contains the target;

[0010] S3: Input the image with the target into the litchi male and female flower counting model, extract features through a multi-scale attention backbone network, and output a feature map after feature enhancement and fusion through a feature pyramid network and a lightweight attention module.

[0011] S4: Calculate the confidence and Euclidean distance of the initial prediction points in the feature map, and calculate the cost matrix using the confidence and Euclidean distance of the initial prediction points;

[0012] S5: Perform Hungarian matching on the initial prediction points based on the cost matrix to obtain the final prediction points and background, calculate the cross-entropy loss, Euclidean loss and Bayesian loss, and thus obtain the total loss;

[0013] S6: Supervised optimization of model parameters using total loss yields a well-trained litchi male and female flower counting model;

[0014] S7: Input the litchi time series data into the trained litchi male and female flower counting model to obtain the flower count statistics.

[0015] Preferably, step S3 specifically includes:

[0016] S301: The image with the target is passed through the 4th, 7th, 10th and 13th layers of the VGG16 network to output the first VGG feature map, the second VGG feature map, the third VGG feature map and the fourth VGG feature map respectively;

[0017] S302: The fourth VGG feature map is processed by a lightweight attention module, convolution, upsampling, another lightweight attention module, and another convolution to obtain the third FPN feature map;

[0018] S303: The third VGG feature map is passed through a lightweight attention module and convolution, and then connected to the feature map after upsampling of the fourth VGG feature map. After further upsampling, passing through a lightweight attention module, and convolution, the second FPN feature map is obtained.

[0019] S304: The second VGG feature map is passed through a lightweight attention module and convolution, and then connected to the feature map after upsampling of the third VGG feature map. After further upsampling, passing through a lightweight attention module, and convolution, the first FPN feature map is obtained.

[0020] S305: Select the first FPN feature map, the second FPN feature map, or the third FPN feature map as the output feature map.

[0021] Preferably, the lightweight attention module includes a channel attention module and a spatial attention module;

[0022] The specific processing steps of the channel attention module include:

[0023] The input feature map is subjected to average pooling and max pooling respectively to obtain the average pooling feature map and the max pooling feature map;

[0024] The average pooling feature map and the max pooling feature map are fed into a shared multilayer perceptron network to generate the final channel attention feature map.

[0025] The spatial attention module's specific processing steps include:

[0026] Average pooling and max pooling are performed on the channel attention feature maps, respectively.

[0027] The two feature maps generated by average pooling and max pooling are concatenated;

[0028] The final spatial attention feature map is generated by convolution on the concatenated feature map.

[0029] Preferably, the S1 preprocessing specifically includes:

[0030] S101: Crop the acquired images of the lychee flowering period and remove images without lychee flowers;

[0031] S102: Label the cropped image with data points and save the center point coordinates of each target to the labeling file.

[0032] Preferably, the cost matrix C is calculated using the following formula:

[0033]

[0034] Where ||·||2 represents the Euclidean distance, P and For a pair of matching points, pi The coordinates of the truth value. For reference point The confidence score is given by τ, which is the balancing factor, M represents the initial number of predicted points, and N represents the ground truth in the image, which is consistent with the final number of predicted points.

[0035] Preferably, the formula for calculating cross-entropy loss is:

[0036]

[0037] Among them, L cls The cross-entropy loss is used, where M represents the initial number of predicted points and N represents the final number of predicted points. β represents the confidence level of the initial prediction point, and β represents the weight of balancing negative suggestions;

[0038] The formula for calculating Euclidean loss is:

[0039]

[0040] Among them, L loc p represents Euclidean loss. i The coordinates of the truth value. To match p i The final predicted point;

[0041] The formula for calculating Bayesian loss is:

[0042]

[0043]

[0044]

[0045] Among them, L Bayes Let x represent the Bayesian loss, F represent the absolute error, and x represent the absolute error. m Represents a pixel in the image, y n Let z represent the nth flower. n Indicates flower y n The actual coordinates, N represents the total number of flowers, which is consistent with the final predicted number of points, p(x m ) represents x m The probability of a flower appearing at a given location, p(y) n |x m ) indicates that in x m There is a flower y at the place n The probability, D pred This represents the prediction plot, where σ represents the Gaussian radius.

[0046] The formula for calculating total loss is:

[0047] L=λ1L cls+λ2L loc +λ3L Bayes

[0048] Where L represents the total loss, and λ1, λ2, and λ3 are the coefficients of the loss function.

[0049] Preferably, it further includes:

[0050] S8: Conduct correlation analysis based on the statistical results of flower quantity.

[0051] A computer device, comprising:

[0052] Memory, used to store computer programs;

[0053] A processor is configured to implement the steps of the litchi male and female flower counting method based on point-to-point networks and Bayesian loss when executing the computer program.

[0054] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the litchi male and female flower counting method based on a point-to-point network and Bayesian loss.

[0055] The present invention has the following effects:

[0056] 1) This invention can accurately obtain the number of male and female flowers in an image and directly provide the specific coordinates of each target. By utilizing the coordinate information of the target, subsequent higher-level flowering period analysis tasks can be performed (such as flowering density analysis, flowering duration analysis of a certain flower, etc.).

[0057] 2) The problem of predicting whether a lychee flower exists at a certain point in the image is transformed into a Bayesian estimation problem. Bayesian loss is used to supervise the expected count of a flower at each pixel, so that the predicted position of each flower is as close as possible to the true coordinates.

[0058] 3) Introduce Feature Pyramid Network (FPN) and Lightweight Attention Module (CBAM) into the backbone network. FPN performs multi-scale feature reuse, and CBAM enhances the network's attention and response to dense regions. These two improvement strategies can effectively overcome the impact of linear perspective on the counting accuracy of natural scenes. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0060] Figure 1 This is a schematic flowchart of a litchi male and female flower counting method based on point-to-point network and Bayesian loss provided by the present invention.

[0061] Figure 2 This is a structural diagram of the litchi male and female flower counting model of the present invention.

[0062] Figure 3a The test results for male flowers on a sunny day were 78.

[0063] Figure 3b The test results for male flowers on a cloudy day were 33.

[0064] Figure 3c The female flowers were tested on a sunny day, and the number was 35.

[0065] Figure 3d The number of female flowers tested was 39, taken on a rainy day. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] This invention discloses a method for counting male and female flowers of litchi based on point-to-point networks and Bayesian loss, such as... Figure 1 As shown, it includes:

[0068] S1: Obtain images of litchi flowering period and perform preprocessing;

[0069] S2: Randomly crop the preprocessed litchi flowering period image until the cropped area contains the target;

[0070] S3: Input the image with the target into the litchi male and female flower counting model, extract features through a multi-scale attention backbone network, and output a feature map after feature enhancement and fusion through a feature pyramid network and a lightweight attention module.

[0071] S4: The two convolutional branches of the feature map input calculate the confidence and Euclidean distance of the initial prediction points of the feature map, and calculate the cost matrix using the confidence and Euclidean distance of the initial prediction points;

[0072] S5: Perform Hungarian matching on the initial prediction points based on the cost matrix to obtain the final prediction points and background, calculate the cross-entropy loss, Euclidean loss and Bayesian loss, and thus obtain the total loss;

[0073] S6: Supervised optimization of model parameters using total loss yields a well-trained litchi male and female flower counting model;

[0074] S7: Input the litchi time series data into the trained litchi male and female flower counting model to obtain the flower count statistics.

[0075] The litchi flowering period images are randomly captured images of litchi flowers, used to construct the training and testing sets for the counting network. To ensure good generalization ability of the trained model, the images were taken from morning to evening, covering various lighting conditions, shooting scales, and different weather conditions (such as rain, cloudy, and sunny days). The litchi time series data consists of time series images tracked and captured for each litchi tree, covering the inflorescence stage, flowering stage, and fruit ripening stage. These images are taken daily to analyze the opening status and patterns of male and female flowers in different litchi varieties.

[0076] Sometimes, images captured by a PTZ camera inevitably contain other branches. We manually filter out the regions of each target branch, setting the pixel values ​​of non-target branch regions to 0, to ensure that the detected flower count comes from the target branch as much as possible. The litchi male and female flower counting model is a network model based on a point-to-point network and Bayesian loss (P2PNet-Bayes network).

[0077] Based on the above embodiments, in this embodiment, the size of the acquired litchi flowering period image is 2560×1440 pixels. Because litchi flowers are small, densely packed, and heavily overlapping, labeling the entire image is very difficult. Therefore, the preprocessing of the litchi flowering period image specifically includes:

[0078] After cropping each original image into four new images of 1280×720 pixels, the images were filtered, and images without lychee flowers were removed, resulting in a total of 11,351 images of male flowers and 5,526 images of female flowers.

[0079] The cropped image was annotated with data points, and the center point coordinates of each target were preserved in the annotation file. To compare accuracy with cutting-edge algorithms, various annotation files were generated, including txt, h5, mat, and npy files.

[0080] A total of 11,351 male flower images were labeled, containing 319,568 male flower targets, with an average of 28.15 male flowers per image; 5,526 female flower images were labeled, containing 81,973 female flower targets, with an average of 14.83 female flowers per image. The labeled data were randomly divided into the training set and the validation set of the network in an 8:2 ratio.

[0081] Based on the above embodiments, in this embodiment, for the linear perspective problem of images in orchard scenes, a Feature Pyramid Network (FPN) and a Lightweight Attention Module (CBAM) are introduced into the backbone network. In the loss function construction part, the prediction of whether a point in the image contains a lychee flower is transformed into a Bayesian estimation problem, and a Bayesian loss that is effective in the field of dense object counting is introduced.

[0082] Specifically, the network structure is as follows: Figure 2 As shown, since the counting object of the present invention is a typical small target, in order not to change the pixel size of the target, each input original image is randomly cropped to generate 4 input images of size 256×256 with the target, and then input into the feature extraction network. Figure 2 In the input scale, 8 indicates a batch size of 8, 3 represents an RGB three-channel image, 1280 is the image length, 720 is the image width, and 32 after random cropping means that each original input image yields 4 smaller images, for a total of 32 images. This step avoids excessive resizing that would make the target pixels too small, and also achieves a data augmentation effect.

[0083] The multi-scale attention backbone network still uses the first 13 layers of P2PNet's vgg16_bn. Due to perspective issues in the orchard scene, the target scale varies significantly. To address this, this paper introduces Feature Fusion Network (FPN) for feature fusion. FPN primarily solves the multi-scale problem in object detection. Through simple changes to network connections, it can significantly improve the performance of small object detection without substantially increasing the computational cost of the original model. The lower-level second VGG feature map has relatively little semantic information but accurate target location; the higher-level fourth VGG feature map has richer semantic information but coarser target location. By fusing features from these different layers, a better prediction effect is achieved.

[0084] The specific steps of S3 include:

[0085] S301: The image with the target is passed through layers 4, 7, 10, and 13 of the VGG16 network, which output the first VGG feature map (32×128×128×128), the second VGG feature map (32×256×64×64), the third VGG feature map (32×512×32×32), and the fourth VGG feature map (32×512×16×16). In this case, the first value in parentheses, 32, represents the number of images, the second value represents the number of channels, and the third and fourth values ​​represent the pixel size of the image. The following FPN feature maps are the same as those here.

[0086] S302: The fourth VGG feature map is processed by a lightweight attention module, convolution, upsampling, another lightweight attention module, and another convolution to obtain the third FPN feature map (32×256×16×16).

[0087] S303: The third VGG feature map is passed through a lightweight attention module and convolution, and then connected to the feature map after upsampling of the fourth VGG feature map. After further upsampling, passing through a lightweight attention module, and convolution, the second FPN feature map (32×256×32×32) is obtained.

[0088] S304: The second VGG feature map is passed through a lightweight attention module and convolution, and then connected to the feature map after upsampling of the third VGG feature map. After further upsampling, passing through a lightweight attention module, and convolution, the first FPN feature map (32×256×64×64) is obtained.

[0089] S305: Select a first FPN feature map, a second FPN feature map, or a third FPN feature map as the output feature map. To balance the resolution of the feature map and the computational cost of subsequent networks, this embodiment selects the second FPN feature map as the final output feature map and inputs it into the subsequent networks. Furthermore, this invention adds a lightweight attention module (CBAM) before each two-dimensional convolutional layer of the FPN to enhance the network's attention to and response to important features.

[0090] The lightweight attention module includes a channel attention module and a spatial attention module;

[0091] The specific processing steps of the channel attention module include:

[0092] The input feature map is subjected to average pooling and max pooling respectively to obtain the average pooling feature map and the max pooling feature map;

[0093] The average pooling feature map and the max pooling feature map are fed into a shared multilayer perceptron network to generate the final channel attention feature map.

[0094] The spatial attention module's specific processing steps include:

[0095] Average pooling and max pooling are performed on the channel attention feature maps, respectively.

[0096] The two feature maps generated by average pooling and max pooling are concatenated;

[0097] The final spatial attention feature map is generated by convolution on the concatenated feature map.

[0098] Based on the above embodiments, in this embodiment, the strategy for generating prediction points is to first generate a sufficient number of initial prediction points, and then obtain the final prediction points through a certain matching strategy.

[0099] A prediction point is taken every 8 pixels in a grid pattern, resulting in 1024 initial prediction points on a 256×256 image. The feature map is then processed through two convolutional branches: a confidence branch and a distance branch, to obtain the confidence scores of the M initial prediction points and their Euclidean distance to the nearest ground truth (GT).

[0100] For the confidence branch, after a set of convolutional layers, the output is the confidence score normalized by Softmax. For the distance branch, due to the inherent translation invariance of convolutional layers, it uses the offset of the predicted point coordinates as the output. Assume a reference point p. j =(x j ,y j ) j∈M The most recent GT is p i =(x i ,y i ) i∈N The Euclidean distance is calculated. After obtaining the confidence score and Euclidean distance, the cost matrix is ​​calculated based on these. M initial prediction points are matched one-to-one with N ground truths (GTs) to determine N final prediction points, where the GT value matches the number of final prediction points. The final prediction points contain coordinate location information. Utilizing the target's coordinate information, subsequent higher-order flowering period analysis tasks can be performed (e.g., flowering density analysis, flower blooming duration analysis, etc.).

[0101] In this embodiment, the Hungarian matching algorithm is used when matching the predicted point with the ground truth (GT). When calculating the cost matrix, both the distance between pixels and the confidence level of the candidate points are considered. This ensures that through continuous training iterations, the predicted candidate points can gradually approach the labeled target points with the highest possible confidence.

[0102] C is a pairwise matching cost matrix of shape N×M, which quantifies the "distance" between a pair of matching points. Formally, the cost matrix C is defined by the following formula (1):

[0103]

[0104] Where ||·||² represents the Euclidean distance, and P and For a pair of matching points, p i Let GT be the coordinates. For reference point The confidence score. τ is a balancing factor used to balance the effect from pixel distance, with a preferred value of 2.

[0105] The final matching result is that the matched predicted points can be represented as a positive set P. pos ={pξ(i)|i∈{1,…,N}}, and set P neg ={pξ(i)|i∈{N+1,…,M} is marked as negative and set as background.

[0106] Based on the above embodiments, in this embodiment, the matching result divides the M initial prediction points into N final prediction points and MN background points, using cross-entropy loss L. cls To supervise the classification of predicted points against the background, Euclidean loss L0 is used. loc To monitor the accuracy of the predicted point location.

[0107]

[0108]

[0109] In formula (2), p represents the confidence level of the initial prediction point. In formula (3), p i For GT, To match p i The final prediction point, β represents the weight of balancing negative recommendations.

[0110] This invention transforms the problem of predicting whether a point in a graph contains a lychee flower into a Bayesian estimation problem. In the preceding operations, N prediction points have been obtained. The pixel values ​​at these prediction points are set to 1, and the pixel values ​​of the remaining points are set to 0, resulting in a prediction graph D. pred The probability of a target existing at each pixel in the prediction image can be represented by the pixel value of that point. Each pixel in the prediction image is considered a sample observation; the probability of a flower existing at that point is taken as the prior probability, represented by a uniform distribution of the ground truth (GT), meaning each point could be one of N flowers or the background. Then, based on the observation and the prior probability, the posterior probability of a flower existing at that point can be estimated. The posterior probability is essentially an estimate of the true distribution GT. Therefore, the difference between the posterior probability and the true probability can be used as the loss to optimize the model. The specific reasoning and calculation are as follows:

[0111] Formula (4) substitutes the probability of a flower appearing at a certain pixel into the Bayesian estimation:

[0112]

[0113] Where, x m Represents a pixel in the image, y n Let z represent the nth flower. n Indicates y n The actual coordinates of this flower, and the total number of flowers is N. p(xm ) represents x m The probability of a flower appearing at a given location. p(y) n |x m ) indicates that in x m There is a flower y at the place n The probability of x. Because x m The flower observed at a given location could be any one of N flowers, so the prior distribution p(y) n ) is represented by a uniform distribution 1 / N, then p(x m |y n ) represents flower y n Appears in x m The probability at a given location is represented by a two-dimensional Gaussian distribution in this invention:

[0114] N(x m ;z n ,σ 2 | 2×2 (5)

[0115] The specific form of the Gaussian distribution is:

[0116]

[0117] In other words, the distance z n The farther away, the more... n The probability of it appearing here is smaller. Next, we use the prediction graph D. pred and posterior probability p(y) n |x m To calculate y n The expected presence of this flower in the prediction chart.

[0118]

[0119] Prediction Figure D pred There are m points, and the value of each point represents the probability that a flower exists at that point. E[c] in formula (7) n ] represents y n This flower appears in the expected value across the entire forecast chart. Representing point x m For y n Contribution count, c n To be with y n Related total count. Because y n It belongs to GT, so y n The probability of this flower actually appearing is 1, and the Bayesian loss can be expressed as shown in formula (8).

[0120]

[0121] F here represents the absolute error. In summary, the Bayesian loss in this method infers the probability of a flower existing in each ground truth (GT) by generating a prediction map based on the predicted points, and expects this probability to be close to 1.

[0122] The total loss of the network is:

[0123] L=λ1L cls +λ2L loc +λ3L Bayes (9)

[0124] λ1, λ2, and λ3 are the coefficients of the loss function, which adjust the importance of each term in the overall loss. The optimal values ​​are λ1 = 1, λ2 = 0.0002, and λ3 = 0.02. Finally, through training and optimization, the predicted points are dynamically updated during the training process, becoming increasingly closer to the true target.

[0125] Based on the above embodiments, this embodiment further includes:

[0126] S8: Based on the statistical results of the number of flowers, a correlation analysis was conducted. Specifically, based on the statistical data, a correlation analysis was conducted on factors such as flowering period and fruit set rate. The results showed that there was a significant positive correlation between the initial fruit set rate and the number of male and female flowers, but a significant negative correlation between the fertilization rate and the number of male and female flowers. Therefore, flowers should be thinned scientifically.

[0127] Regarding performance metrics, Mean Absolute Error (MAE) and Mean Squared Error (MSE) are used as evaluation metrics, defined as follows:

[0128]

[0129]

[0130] K represents the number of test images, N k and C k Let represent the number of ground truths (GTs) and the number of predictions for the k-th image, respectively.

[0131] This method achieves good results in dense counting of male and female flowers, enabling high-precision counting and localization in natural scenes. The detection results of male and female flowers under various weather conditions are shown below. Figures 3a-3d As shown.

[0132] The algorithm P2PNet-Bayes of this invention was compared with state-of-the-art counting algorithms on a dataset of male and female flowers. The results are shown in Table 1, with the best performance indicated in bold and the second best performance indicated in underline.

[0133] Table 1 compares the present invention with leading counting methods on a dataset of male and female flowers.

[0134]

[0135] As shown in Table 1, the method of this invention achieves the best counting performance on both male and female flower datasets. Compared to P2PNet, the MAE for male flower counting is reduced by 0.63, and the MSE is reduced by 1.74. Compared to CANet, the MAE and MSE for female flower counting are also reduced.

[0136] This invention provides a computer device, comprising:

[0137] Memory, used to store computer programs;

[0138] A processor is used to implement the steps of a litchi male and female flower counting method based on a point-to-point network and Bayesian loss as described above when executing a computer program.

[0139] For a description of the computer device provided by the present invention, please refer to the above method embodiments; the present invention will not be described in detail here.

[0140] The computer-readable storage medium of this invention stores a computer program, which, when executed by a processor, implements the steps of the litchi male and female flower counting method based on a peer-to-peer network and Bayesian loss as described above. The storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] For a description of the computer-readable storage medium provided by the present invention, please refer to the above method embodiments; the present invention will not be described again here.

[0142] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0143] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for counting male and female flowers of litchi based on point-to-point networks and Bayesian loss, characterized in that, include: S1: Obtain images of litchi flowering period and perform preprocessing; S2: Randomly crop the preprocessed litchi flowering period image until the cropped area contains the target; S3: Input the image with the target into the litchi male and female flower counting model, extract features through a multi-scale attention backbone network, and output a feature map after feature enhancement and fusion through a feature pyramid network and a lightweight attention module. S4: Calculate the confidence and Euclidean distance of the initial prediction points in the feature map, and calculate the cost matrix using the confidence and Euclidean distance of the initial prediction points; S5: Perform Hungarian matching on the initial prediction points based on the cost matrix to obtain the final prediction points and background, calculate the cross-entropy loss, Euclidean loss and Bayesian loss, and thus obtain the total loss; S6: Supervised optimization of model parameters using total loss yields a well-trained litchi male and female flower counting model; S7: Input the litchi time series data into the trained litchi male and female flower counting model to obtain the flower count results; The specific steps of S3 include: S301: The image with the target is passed through the 4th, 7th, 10th and 13th layers of the VGG16 network to output the first VGG feature map, the second VGG feature map, the third VGG feature map and the fourth VGG feature map respectively; S302: The fourth VGG feature map is processed by a lightweight attention module, convolution, upsampling, another lightweight attention module, and another convolution to obtain the third FPN feature map; S303: The third VGG feature map is passed through a lightweight attention module and convolution, and then connected to the feature map after upsampling of the fourth VGG feature map. After further upsampling, passing through a lightweight attention module, and convolution, the second FPN feature map is obtained. S304: The second VGG feature map is passed through a lightweight attention module and convolution, and then connected to the feature map after upsampling of the third VGG feature map. After further upsampling, passing through a lightweight attention module, and convolution, the first FPN feature map is obtained. S305: Select the first FPN feature map, the second FPN feature map, or the third FPN feature map as the output feature map.

2. The method for counting male and female flowers of litchi based on point-to-point networks and Bayesian loss according to claim 1, characterized in that, The lightweight attention module includes a channel attention module and a spatial attention module; The specific processing steps of the channel attention module include: The input feature map is subjected to average pooling and max pooling respectively to obtain the average pooling feature map and the max pooling feature map; The average pooling feature map and the max pooling feature map are fed into a shared multilayer perceptron network to generate the final channel attention feature map. The spatial attention module's specific processing steps include: Average pooling and max pooling are performed on the channel attention feature maps, respectively. The two feature maps generated by average pooling and max pooling are concatenated; The final spatial attention feature map is generated by convolution on the concatenated feature map.

3. The method for counting male and female flowers of litchi based on point-to-point networks and Bayesian loss according to claim 1, characterized in that, S1 preprocessing specifically includes: S101: Crop the acquired images of the lychee flowering period and remove images without lychee flowers; S102: Label the cropped image with data points and save the center point coordinates of each target to the labeling file.

4. The method for counting male and female flowers of litchi based on point-to-point networks and Bayesian loss according to claim 1, characterized in that, The formula for calculating the cost matrix C is: ； in, Represents Euclidean distance. and For a pair of matching points, The coordinates of the truth value. For reference point The confidence score is given by τ, which is the balancing factor, M represents the initial number of predicted points, and N represents the ground truth in the image, which is consistent with the final number of predicted points.

5. The method for counting male and female flowers of litchi based on point-to-point networks and Bayesian loss according to claim 1, characterized in that, The formula for calculating cross-entropy loss is: ； in, The cross-entropy loss is used, where M represents the initial number of predicted points and N represents the final number of predicted points. The confidence level of the initial prediction point. This indicates the weighting of negative recommendations. The formula for calculating Euclidean loss is: ； in, Indicates European-style loss. The coordinates of the truth value. To match The final predicted point; The formula for calculating Bayesian loss is: ； ； in, Let x represent the Bayesian loss, F represent the absolute error, and x represent the absolute error. m Represents a pixel in the image, y n Let z represent the nth flower. n Indicates flower y n The actual coordinates, N represents the total number of flowers, which is consistent with the final predicted number of points, p(x m ) represents x m The probability of a flower appearing at a given location, p(y) n |x m ) indicates that in x m There is a flower y at the place n The probability, D pred This represents the prediction graph. Represents the Gaussian radius; The formula for calculating total loss is: ； Where L represents the total loss, and λ1, λ2, and λ3 are the coefficients of the loss function.

6. The method for counting male and female flowers of litchi based on point-to-point networks and Bayesian loss according to claim 1, characterized in that, Also includes: S8: Conduct correlation analysis based on the statistical results of flower quantity.

7. A computer device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the litchi male and female flower counting method based on a point-to-point network and Bayesian loss as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the litchi male and female flower counting method based on a point-to-point network and Bayesian loss as described in any one of claims 1 to 6.

Images