Fruit maturity determination method, fruit maturity visualization method and device

Abstract

The present disclosure provides a method for determining the maturity of a fruit, a method for visualizing the maturity of a fruit, and a device, and relates to the technical field of image processing and data analysis. The method comprises: obtaining hyperspectral images corresponding to fruits at multiple harvest stages, and performing solid component detection on the fruits at the harvest stages to obtain fruit solid component data corresponding to the hyperspectral images; comparing the fruit solid component data with a preset maturity determination threshold, and performing maturity annotation on the fruit hyperspectral images based on the comparison result to obtain a fruit maturity data set; training a target detection model based on the fruit maturity data set to obtain a fruit maturity determination model; obtaining a target hyperspectral image corresponding to the fruit to be determined, and performing maturity determination on the target hyperspectral image using the fruit maturity determination model to obtain a maturity determination result of the fruit to be determined. The technical solution in the present disclosure can improve the efficiency and accuracy of fruit maturity determination.

Description

Background Art

[0002] In modern agricultural production, maturity is a key factor in determining fruit quality. If the fruit is picked before it is fully mature, it may taste sour and astringent; if it is picked too late, it may become over-mature and taste worse. Therefore, accurately judging the maturity of the fruit before harvesting is of great significance to ensure the quality of the fruit, extend the shelf life, and improve the post-harvest processing value of the fruit.

[0003] In the related art, the maturity of fruits is usually judged manually, such as observing the color change of fruits, feeling their hardness by touch, tasting fruits to determine whether their taste has reached the mature state, etc. These methods are highly subjective and inefficient.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the embodiments of the present disclosure is to provide a fruit maturity determination method, a fruit maturity visualization method, a fruit maturity determination device, a fruit maturity visualization device, an electronic device and a computer-readable storage medium, thereby improving the efficiency and accuracy of fruit maturity determination at least to a certain extent.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to a first aspect of an embodiment of the present disclosure, a method for determining fruit maturity is provided, comprising: obtaining hyperspectral images corresponding to fruits at multiple harvest stages, and performing solid component detection on the fruits at the harvest stages to obtain fruit solid component data corresponding to the hyperspectral images; comparing the fruit solid component data with a preset maturity determination threshold, and performing maturity annotation on the fruit hyperspectral images based on the comparison result to obtain a fruit maturity data set; training a target detection model based on the fruit maturity data set to obtain a fruit maturity determination model; obtaining a target hyperspectral image corresponding to the fruit to be determined, performing maturity determination on the target hyperspectral image using the fruit maturity determination model to obtain a maturity determination result of the fruit to be determined.

[0008] In some example embodiments of the present disclosure, based on the aforementioned scheme, the solid composition of the fruits at the harvest stage is detected to obtain fruit solid composition data corresponding to the hyperspectral image, including: using a portable detector to detect the soluble solid content and dry matter content of the fruits at the harvest stage, and using the soluble solid content and dry matter content as the fruit solid composition data corresponding to the hyperspectral image.

[0009] In some example embodiments of the present disclosure, based on the aforementioned scheme, the fruit hyperspectral image is labeled with maturity based on the comparison result to obtain a fruit maturity dataset, including: when the fruit solid component data is not less than the maturity judgment threshold, the corresponding fruit hyperspectral image is labeled as mature; when the fruit solid component data is less than the maturity judgment threshold, the corresponding fruit hyperspectral image is labeled as unripe; and the fruit maturity dataset is constructed based on the labeled fruit hyperspectral image.

[0010] In some example embodiments of the present disclosure, based on the aforementioned scheme, the fruit maturity dataset is constructed based on the annotated fruit hyperspectral image, including: performing geometric transformation, contrast modification and noise reduction processing on the annotated fruit hyperspectral image to obtain the fruit maturity dataset.

[0011] In some example embodiments of the present disclosure, based on the aforementioned scheme, the target detection model is trained based on the fruit maturity dataset to obtain a fruit maturity determination model, including: optimizing the backbone network and loss function of the YOLOv5 model to obtain a YOLOv5 optimization model; training the YOLOv5 optimization model based on the fruit maturity dataset to obtain a fruit maturity determination model.

[0012] In some example embodiments of the present disclosure, based on the aforementioned scheme, the backbone network and loss function of the YOLOv5 model are optimized, including: introducing a convolutional layer attention module on the backbone network to enhance the feature extraction capability of the backbone network for the hyperspectral image of the fruit; optimizing the loss function of the bounding box in the YOLOv5 model using complete intersection-over-union (IoU) to obtain a complete intersection-over-union (IoU) loss function.

[0013] In some example embodiments of the present disclosure, based on the above scheme, the complete intersection-over-union loss function is obtained as follows:

[0014]

[0015] Among them, LOSS CIoU represents the complete intersection-over-union loss function, IoU represents the intersection-over-union ratio of the bounding box, b represents the center point of the predicted bounding box, and b gtrepresents the center point of the true bounding box, ρ 2 (b,b gt ) represents the square of the Euclidean distance between the center point of the predicted bounding box and the center point of the true bounding box, c represents the diagonal length of the minimum enclosing rectangle containing the predicted bounding box and the true bounding box, α is a hyperparameter, and v represents the absolute value of the aspect ratio difference between the predicted bounding box and the true bounding box.

[0016] According to a second aspect of an embodiment of the present disclosure, a method for visualizing fruit maturity is provided, comprising: obtaining hyperspectral images corresponding to fruits at multiple harvest stages, and performing solid component detection on the fruits at the harvest stages to obtain fruit solid component data corresponding to the hyperspectral images; comparing the fruit solid component data with a preset maturity judgment threshold, and performing maturity annotation on the fruit hyperspectral images based on the comparison result to obtain a fruit maturity data set; training a target detection model based on the fruit maturity data set to obtain a fruit maturity judgment model; obtaining a target hyperspectral image and position information corresponding to each fruit in a fruit tree area based on a preset shooting path, performing maturity judgment on the target hyperspectral image using the fruit maturity judgment model to obtain a maturity judgment result corresponding to each fruit; fusing and splicing the maturity judgment result and position information corresponding to each fruit with the target hyperspectral image to obtain a fruit maturity distribution map corresponding to the fruit tree area, so as to visualize the fruit maturity of the fruit tree area.

[0017] According to a third aspect of an embodiment of the present disclosure, a device for determining the maturity of a fruit is provided, the device comprising: a solid component detection module, for acquiring hyperspectral images corresponding to fruits at multiple harvest stages, and performing solid component detection on the fruits at the harvest stages to obtain fruit solid component data corresponding to the hyperspectral images; a data set acquisition module, for comparing the fruit solid component data with a preset maturity determination threshold, and performing maturity annotation on the fruit hyperspectral images based on the comparison result to obtain a fruit maturity data set; a determination model training module, for training a target detection model based on the fruit maturity data set to obtain a fruit maturity determination model; a maturity determination module, for acquiring a target hyperspectral image corresponding to the fruit to be determined, performing maturity determination on the target hyperspectral image using the fruit maturity determination model to obtain a maturity determination result of the fruit to be determined.

[0018] According to a fourth aspect of an embodiment of the present disclosure, a fruit maturity visualization device is provided, the device comprising: a solid component detection module, used to obtain hyperspectral images corresponding to fruits at multiple harvest stages, and perform solid component detection on the fruits at the harvest stages to obtain fruit solid component data corresponding to the hyperspectral images; a data set acquisition module, used to compare the fruit solid component data with a preset maturity judgment threshold, and perform maturity annotation on the fruit hyperspectral images based on the comparison result to obtain a fruit maturity data set; a judgment model training module, used to train a target detection model based on the fruit maturity data set to obtain a fruit maturity judgment model; a regional maturity judgment module, used to obtain a target hyperspectral image and position information corresponding to each fruit in a fruit tree area based on a preset shooting path, perform maturity judgment on the target hyperspectral image using the fruit maturity judgment model, and obtain a maturity judgment result corresponding to each fruit; a visualization display module, used to fuse and splice the maturity judgment result and position information corresponding to each fruit with the target hyperspectral image to obtain a fruit maturity distribution map corresponding to the fruit tree area, so as to perform fruit maturity visualization on the fruit tree area.

[0019] According to a fifth aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the above-mentioned method for determining fruit maturity or the method for visualizing fruit maturity is implemented.

[0020] According to a sixth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for determining fruit maturity or the method for visualizing fruit maturity as described above is implemented.

[0021] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0022] The fruit maturity determination method in the example embodiment of the present disclosure, on the one hand, performs maturity annotation on the fruit hyperspectral image based on the fruit solid component data of the fruit, which not only makes the annotation of the fruit maturity data set more accurate, but also enables the trained fruit maturity determination model to take into account the intrinsic indicators of the fruit, thereby improving the accuracy of fruit maturity determination; on the other hand, the established fruit maturity determination model can automatically process the target hyperspectral image and quickly give the maturity determination result, thereby reducing manpower requirements and improving the efficiency of fruit maturity determination; on still another hand, the use of hyperspectral images can complete the maturity determination of the fruit without touching the fruit, thereby avoiding fruit damage and ensuring the fruit quality.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0025] Figure 1 A schematic diagram showing an exemplary system architecture of a method for determining fruit maturity according to an embodiment of the present disclosure is shown.

[0026] Figure 2 A flow chart of a method for determining fruit maturity according to some embodiments of the present disclosure is schematically shown.

[0027] Figure 3 A structural diagram of a YOLOv5 optimization model according to some embodiments of the present disclosure is schematically shown.

[0028] Figure 4 A flowchart of a method for visualizing fruit maturity according to some embodiments of the present disclosure is schematically shown.

[0029] Figure 5 A block diagram of a device for determining the maturity of a fruit according to some embodiments of the present disclosure is schematically shown.

[0030] Figure 6 A block diagram of a fruit maturity visualization device according to some embodiments of the present disclosure is schematically shown.

[0031] Figure 7 A schematic diagram of the structure of a computer system of an electronic device according to some embodiments of the present disclosure is schematically shown.

[0032] Figure 8 A schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure is schematically shown.

[0033] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0034] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0035] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0036] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0037] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete and will fully convey the concept of the example embodiments to those skilled in the art.

[0038] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0039] Furthermore, the drawings are only schematic illustrations and are not necessarily drawn to scale. The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0040] Maturity is a key factor in determining fruit quality. During the ripening process, fruits undergo a series of physiological changes such as color change, pulp softening, and increased sugar content, which make the fruits more delicious and taste better. If the fruit is picked too early, it may taste sour and astringent, and the taste may be hard; if it is picked too late, it may cause the fruit to be overripe, the taste may become soft, and the flavor may deteriorate. Therefore, accurately judging the maturity of the fruit is crucial to ensuring the quality of the fruit. Secondly, maturity also affects the storage resistance of the fruit. The cell structure and tissue structure of fruits with appropriate maturity are more stable, and they can better resist the influence of the external environment, thereby extending the shelf life. On the contrary, fruits that are not mature enough or overripe are more easily damaged during storage and transportation, resulting in a decrease in quality. In addition, maturity is also directly related to the post-harvest processing value of the fruit. Fruits with appropriate maturity have richer nutrients and more unique flavors, and are more suitable for making processed products such as juice and jam. However, overripe or over-raw fruits may not meet the quality requirements of processed products. Therefore, accurately judging the maturity of the fruit before harvesting is of great significance for ensuring fruit quality, extending the shelf life, and improving the post-harvest processing value.

[0041] In the related art, the maturity of fruits is usually judged manually, such as observing the color change of fruits, feeling their hardness by touch, tasting fruits to determine whether their taste has reached the mature state, etc. These methods are highly subjective and inefficient.

[0042] In order to solve all or part of the technical problems in the above-mentioned related technologies, in an exemplary embodiment of the present disclosure, a method for determining fruit maturity is firstly provided. Figure 1 A schematic diagram showing an exemplary system architecture of a method for determining fruit maturity according to an embodiment of the present disclosure is shown.

[0043] like Figure 1As shown, the system architecture 100 may include one or more of the terminal devices 101, 102, 103, a network 104, a server 105 and portable detectors 106, 107 represent fruit tree areas, such as orchards, fruit tree cultivation areas, etc. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or fiber optic cables, etc. The terminal devices 101, 102, 103 may be various devices for collecting hyperspectral images of fruits, including but not limited to unmanned vehicles, drones, image collectors, self-propelled devices, cameras and other devices with hyperspectral image acquisition functions. The portable detector 106 can be used to detect the solid content of fruits at the harvest stage. It should be understood that Figure 1 The number of terminal devices, networks, servers and portable detectors in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers.

[0044] The fruit maturity determination method provided in the embodiment of the present disclosure can be executed based on the above-mentioned terminal device and server. For example, the preset terminal devices 101, 102, and 103 can be used to perform image acquisition in the fruit tree area 107 to obtain hyperspectral images corresponding to fruits at multiple harvesting stages. The terminal devices 101, 102, and 103 can send the hyperspectral images corresponding to the fruits at the harvesting stages to the server 105 through the network 104; at the same time, the operator uses the portable detector 106 to detect the solid component of the fruit at the harvesting stage, obtain the fruit solid component data corresponding to the hyperspectral image, and input the fruit solid component data into the server 105. After receiving the hyperspectral image and the fruit solid component data, the server 105 compares the fruit solid component data with the preset maturity determination threshold, and annotates the fruit hyperspectral image with maturity based on the comparison result to obtain a fruit maturity data set, and then trains the target detection model based on the fruit maturity data set to obtain a fruit maturity determination model. When the fruit in the fruit tree area 107 needs to be judged for maturity, the terminal devices 101, 102, and 103 obtain the target hyperspectral image corresponding to the fruit to be judged, and send the target hyperspectral image to the server 105 through the network 104. Finally, the server 105 uses the fruit maturity judgment model to judge the maturity of the target hyperspectral image to obtain the maturity judgment result of the fruit to be judged. However, it is easy for those skilled in the art to understand that the above application scenario is only used for example, and the obtained fruit judgment model can be used in other suitable fruit maturity judgment scenarios, and this exemplary embodiment is not limited to this.

[0045] Figure 2 The following is a schematic diagram of a process for determining the maturity of a fruit according to some embodiments of the present disclosure. Figure 2 As shown, the fruit maturity determination method may include the following steps:

[0046] Step S210, obtaining hyperspectral images corresponding to fruits at multiple harvest stages, and performing solid component detection on the fruits at the harvest stages to obtain fruit solid component data corresponding to the hyperspectral images;

[0047] Step S220, comparing the fruit solid component data with a preset maturity judgment threshold, and marking the maturity of the fruit hyperspectral image based on the comparison result to obtain a fruit maturity data set;

[0048] Step S230, training the target detection model based on the fruit maturity data set to obtain a fruit maturity determination model;

[0049] Step S240, obtaining a target hyperspectral image corresponding to the fruit to be determined, and using a fruit maturity determination model to perform maturity determination on the target hyperspectral image to obtain a maturity determination result of the fruit to be determined.

[0050] According to the fruit maturity determination method in the above-mentioned example embodiment, on the one hand, the maturity annotation of the fruit hyperspectral image is performed based on the fruit solid component data of the fruit, which not only makes the annotation of the fruit maturity data set more accurate, but also enables the trained fruit maturity determination model to take into account the intrinsic indicators of the fruit, thereby improving the accuracy of fruit maturity determination; on the other hand, the established fruit maturity determination model can automatically process the target hyperspectral image and quickly give the maturity determination result, thereby reducing manpower requirements and improving the efficiency of fruit maturity determination; on another hand, the use of hyperspectral images can complete the maturity determination of the fruit without touching the fruit, thereby avoiding fruit damage and ensuring the quality of the fruit.

[0051] Next, the method for determining the maturity of a fruit in an exemplary embodiment will be further described.

[0052] In step S210, hyperspectral images corresponding to fruits at multiple harvest stages are acquired, and solid components of the fruits at the harvest stages are detected to obtain fruit solid component data corresponding to the hyperspectral images.

[0053] Among them, hyperspectral images can refer to images that can capture the reflection or transmission information of target objects in multiple narrow bands of the electromagnetic spectrum. Solid component detection can refer to quantifying the solid mass percentage or detecting specific chemical components in fruit samples. Fruit solid component data can refer to other fruit solid component data such as soluble solid content, dry matter content, vitamin content, organic acid content, mineral content, etc.

[0054] Specifically, according to the fruit growth cycle and maturity rules, multiple hyperspectral images of single fruits under different condition combinations in the natural environment of the fruit tree area during the harvest period can be obtained. Among them, different condition combinations can be random combinations of different distances (close distance and long distance), different lighting (strong light and weak light), and different targets (single fruit and multiple fruits). Then, the detection instrument is used to measure indicators such as soluble solids (such as sugar, protein, cellulose, etc.) and dry matter. By obtaining the hyperspectral images corresponding to the fruits at the harvest stage, the fruit maturity determination model obtained later can better understand and learn the relationship between the spectral characteristics of the hyperspectral images at the fruit harvest stage and the maturity of the fruit, so that when the fruit maturity determination model is used to determine the maturity of the fruit at the fruit harvest period, the determination effect is more accurate and more practical.

[0055] In some embodiments, solid content detection is performed on fruits at the harvest stage to obtain fruit solid content data corresponding to the hyperspectral image, including the following steps: using a portable detector to detect the soluble solid content and dry matter content of fruits at the harvest stage, and using the soluble solid content and dry matter content as the fruit solid content data corresponding to the hyperspectral image.

[0056] Among them, the soluble solids of fruits can represent all organic substances soluble in water in fruits, such as sugars (such as glucose, fructose, sucrose, etc.), amino acids, organic acids, vitamins, etc. The soluble solids content is an important indicator for evaluating the sweetness, maturity and nutritional value of fruits. The dry matter content can represent the proportion of the weight of the remaining material after the fruit is dried to a constant weight to the original fresh weight of the fruit, which can reflect the cumulative amount of nutrients in the fruit.

[0057] Furthermore, the soluble solids content and dry matter content of the fruit are measured by a portable detector. These data represent the fruit solid composition data of the fruit, which can make the subsequent maturity labeling more accurate, thereby improving the judgment accuracy and practicality of the fruit maturity determination model.

[0058] In step S220, the fruit solid component data is compared with a preset maturity judgment threshold, and the maturity of the fruit hyperspectral image is annotated based on the comparison result to obtain a fruit maturity data set.

[0059] Among them, the preset maturity judgment threshold can be the maturity judgment threshold of a specific fruit. For example, the soluble solid content of the fruit is 6.5%, and the dry matter content is 15%. Of course, in other embodiments of the present disclosure, the maturity judgment threshold can also be data corresponding to other fruits. By combining objective solid component detection with hyperspectral image features, the maturity state of the fruit can be more accurately reflected. As a direct indicator of maturity, the fruit solid component data is mutually verified with the hyperspectral image features, which reduces the error of a single judgment method and improves the accuracy and reliability of image annotation.

[0060] In some embodiments, the fruit hyperspectral image is labeled with maturity based on the comparison result to obtain a fruit maturity data set, including: when the fruit solid component data is not less than the maturity judgment threshold, the corresponding fruit hyperspectral image is labeled as mature; when the fruit solid component data is less than the maturity judgment threshold, the corresponding fruit hyperspectral image is labeled as immature; and a fruit maturity data set is constructed based on the labeled fruit hyperspectral image.

[0061] Specifically, a data annotation tool was used to label the fruit maturity dataset, accurately draw a border around the target, and mark the maturity of the fruit. The annotated dataset is clearly divided into two categories: "mature" and "immature", which is convenient for subsequent analysis and model training. Structured datasets help improve the learning efficiency and accuracy of the fruit maturity determination model and reduce confusion and errors during model training.

[0062] In some embodiments, constructing a fruit maturity dataset based on the annotated fruit hyperspectral image includes: performing geometric transformation, contrast modification and noise reduction on the annotated fruit hyperspectral image to obtain the fruit maturity dataset.

[0063] Among them, geometric change can refer to the operation of changing the pixel position in the image, which is used to correct the geometric shape or perspective problem of the image, so that the image can be scaled, rotated, flipped, cropped, distorted, etc. according to specific requirements. Contrast modification can refer to improving the visual clarity and detail visibility of the image by adjusting the degree of brightness difference in the image. Noise reduction processing can refer to reducing or eliminating noise in the image.

[0064] Furthermore, by performing geometric transformation, contrast modification and noise reduction on fruit hyperspectral images, the data set can be enhanced, the number of samples can be increased, and the generalization ability and robustness of the fruit maturity determination model can be improved.

[0065] In step S230, the target detection model is trained based on the fruit maturity data set to obtain a fruit maturity determination model.

[0066] Among them, the target detection model can represent a model used to automatically detect and locate the positions of one or more specific targets from an image, and further identify the categories of these targets. Exemplarily, the target detection model can be a YOLO (You Only Look Once) model, or a Region-based Convolutional Neural Network (R-CNN) model, or a Single Shot MultiBox Detector (SSD) model. Of course, in other embodiments of the present disclosure, the target detection model can also be other suitable models.

[0067] In some embodiments, the target detection model is trained based on a fruit maturity data set to obtain a fruit maturity determination model, including: optimizing the backbone network and loss function of the YOLOv5 model to obtain a YOLOv5 optimization model; training the YOLOv5 optimization model based on the fruit maturity data set to obtain a fruit maturity determination model.

[0068] YOLOv5 is the fifth version of the YOLO (You Only Look Once) series of target detection models and is a real-time target detection algorithm. The backbone network can represent the part of the YOLOv5 model used to extract image features. By optimizing the backbone network and loss function of the YOLOv5 model, the resulting YOLOv5 optimized model can extract more effective image feature information.

[0069] In addition, before training the YOLOv5 optimization model, the fruit maturity dataset can be divided into training set, validation set and test set according to a preset ratio, such as 8:1:1, and then the divided dataset can be used to train, validate and test the YOLOv5 optimization model. Specifically, the divided validation set is used to optimize the model, and the model accuracy, mean average precision, precision and recall are selected as evaluation indicators based on the test set data to establish the optimal fruit maturity determination model. The YOLOv5 optimization model can be used as follows Figure 3 As shown, it may include a channel attention module and a spatial attention module, input features F and Refined features F″.

[0070] In some embodiments, the backbone network and loss function of the YOLOv5 model are optimized, including: introducing a convolutional layer attention module on the backbone network to enhance the backbone network's feature extraction capability for fruit hyperspectral images; optimizing the loss function of the bounding box in the YOLOv5 model using complete intersection-over-union (IoU) to obtain a complete intersection-over-union (IoU) loss function.

[0071] Among them, the convolutional layer attention module can dynamically enhance the network's attention to important features by assigning different weights to different regions in the input image, while suppressing irrelevant or noise information. The convolutional layer attention module can include a channel attention module and a spatial attention module. The channel attention module can weight the features according to the importance of each channel, and the spatial attention module can focus on which areas in the feature map are more important. By combining the attention module and the spatial attention module, the model can more effectively pay attention to the key features in the input image that are related to the determination of fruit maturity, especially when processing hyperspectral images, which are rich in content but may have messy information, and can further enhance the feature extraction capability of the backbone network.

[0072] The Complete Intersection over Union (CIoU) loss function can represent an optimized loss function for bounding box regression in object detection tasks. Introducing the complete intersection over union loss function in the YOLOv5 model can more effectively optimize the accuracy of bounding box prediction, thereby improving the overall detection performance of the model.

[0073] In some embodiments, the above complete intersection-over-union loss function is shown in the following formula (1):

[0074]

[0075] Among them, LOSS CIoU represents the complete intersection-over-union loss function, IoU represents the intersection-over-union ratio of the bounding box, b represents the center point of the predicted bounding box, and b gt represents the center point of the true bounding box, ρ 2 (b,b gt ) represents the square of the Euclidean distance between the center point of the predicted bounding box and the center point of the true bounding box, c represents the diagonal length of the minimum bounding rectangle containing the predicted bounding box and the true bounding box, α is a hyperparameter, and v represents the absolute value of the difference in aspect ratio between the predicted bounding box and the true bounding box.

[0076] Specifically, the full intersection-over-union loss function not only considers the ratio of the overlapped area of ​​the predicted bounding box to the true bounding box to the total area (IoU), but also additionally incorporates the distance between the center point of the bounding box and the difference in aspect ratio between the predicted bounding box and the true bounding box to penalize the mismatch in size and position. Compared with the traditional IoU loss or GIoU (Generalized IoU) loss, the full intersection-over-union loss can more comprehensively evaluate the degree of match between the predicted box and the true box, making the predicted bounding box closer to the position of the true bounding box, better handling non-overlapping or occlusion situations, more effectively guiding model learning, reducing the positioning error of the bounding box, and thus improving the accuracy of target detection.

[0077] In step S240, a target hyperspectral image corresponding to the fruit to be determined is obtained, and a maturity determination model is used to perform maturity determination on the target hyperspectral image to obtain a maturity determination result of the fruit to be determined.

[0078] Specifically, a hyperspectral imaging device is used to obtain a hyperspectral image of the fruit to be judged, and then the preprocessed hyperspectral image is input into a previously trained fruit maturity judgment model. The model automatically extracts features related to fruit maturity from the image, such as color, texture, spectral reflectance, etc., and determines the maturity of the fruit based on these features.

[0079] In some embodiments, the fruit solid component data can be compared with the preset maturity judgment standard to classify the fruit into different maturity levels, such as immature, early-ripe, mature, over-ripe, etc. Then, the fruit hyperspectral image is labeled with maturity based on the maturity level to obtain a fruit maturity data set, so that the trained fruit maturity judgment model can determine the maturity level of the fruit according to the target hyperspectral image.

[0080] In the above-mentioned example embodiment, the method for determining the maturity of a fruit is, on the one hand, based on the fruit solid component data of the fruit, used to mark the maturity of the fruit hyperspectral image, which not only makes the annotation of the fruit maturity data set more accurate, but also enables the trained fruit maturity determination model to consider the intrinsic indicators of the fruit, thereby improving the accuracy of the fruit maturity determination; on the other hand, the established fruit maturity determination model is capable of automatically processing the target hyperspectral image and quickly giving the maturity determination result, thereby reducing the manpower demand and improving the efficiency of the fruit maturity determination; on yet another hand, the use of hyperspectral images can complete the maturity determination of the fruit without touching the fruit, thereby avoiding fruit damage and ensuring the fruit quality.

[0081] Furthermore, by performing geometric transformation, contrast modification and noise reduction on hyperspectral images, the image quality is improved, the constructed fruit maturity dataset is more accurate and robust, the generalization ability of the model is increased, and it is suitable for fruit maturity judgment under different lighting, distances and fruit morphologies; in addition, the optimized YOLOv5 model enhances the extraction of hyperspectral image features by embedding an attention module in the backbone network, so that the model can more finely distinguish the fruit maturity characteristics, making the judgment more adaptable to changes in the natural environment, and improving the accuracy of maturity judgment. The trained YOLOv5 optimized model can more accurately determine the maturity of a single fruit plant; moreover, the introduction of the complete intersection-over-union loss function in the YOLOv5 optimization model not only considers the overlap between the predicted box and the true box, but also adds penalty terms for the distance from the center point of the predicted box and the aspect ratio, so that the predicted bounding box is closer to the true bounding box in position and shape, reducing misjudgment and improving positioning accuracy.

[0082] Figure 4 The following is a schematic diagram of a process flow of a fruit maturity visualization method according to some embodiments of the present disclosure. Figure 4 As shown, the fruit maturity visualization method may include the following steps:

[0083] Step S410, obtaining hyperspectral images corresponding to fruits at multiple harvest stages, and performing solid component detection on the fruits at the harvest stages to obtain fruit solid component data corresponding to the hyperspectral images;

[0084] Step S420, comparing the fruit solid component data with a preset maturity judgment threshold, and marking the maturity of the fruit hyperspectral image based on the comparison result to obtain a fruit maturity data set;

[0085] Step S430, training the target detection model based on the fruit maturity data set to obtain a fruit maturity determination model;

[0086] Step S440, based on the preset shooting path, obtaining the target hyperspectral image and position information corresponding to each fruit in the fruit tree area, and using the fruit maturity determination model to perform maturity determination on the target hyperspectral image to obtain the maturity determination result corresponding to each fruit;

[0087] Step S450, the maturity determination result and location information corresponding to each fruit plant are fused and spliced ​​with the target hyperspectral image to obtain a fruit maturity distribution map corresponding to the fruit tree area, so as to visualize the fruit maturity of the fruit tree area.

[0088] Specifically, the maturity of individual fruits in a fruit tree area can be intuitively displayed through color coding or other visual indicators (such as heat maps, etc.) combined with a fruit tree area map, so that operators can quickly identify which areas of fruit have reached the best time to pick and which areas still need to wait, thereby improving the efficiency of fruit tree management and the accuracy of fruit picking. In addition, the maturity distribution map can also help plan the most optimized picking path, reduce unnecessary walking and repetitive labor, save time and labor costs, and reduce damage to immature fruits.

[0089] Furthermore, in some embodiments, when the ratio of the number of mature fruits in a specific area to the number of all fruits in the area exceeds a preset threshold, the fruit trees in the area can be determined to be mature fruit trees, and the preset threshold can be 0.95, 0.96, or other suitable values. This allows the operator to quickly locate specific mature fruit trees and arrange for priority picking in a targeted manner.

[0090] It should be noted that, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0091] In some embodiments, a fruit maturity visualization system is also provided, which may include: a positioning module, a navigation module, a path planning module, a hyperspectral image acquisition module, an image processing module, a maturity determination module, and a display module. The fruit maturity visualization system may be arranged on a suitable device such as a self-propelled device or an unmanned vehicle.

[0092] Among them, the positioning module and the hyperspectral image acquisition module are connected to receive Beidou satellite positioning signals and obtain the precise location information when taking the current hyperspectral image in real time, thereby providing spatial reference coordinates for subsequent data analysis and ensuring the traceability of the acquisition position of each image.

[0093] The navigation module can be used to guide the carrying equipment of the fruit maturity visualization system to move autonomously in the fruit tree area according to a preset path planning.

[0094] The path planning module can be used to design and optimize the moving path of the carrying equipment of the fruit maturity visualization system in the fruit tree area according to the map, layout and fruit tree density of the fruit tree area, so as to cover all key areas, reduce duplication or omission and improve work efficiency.

[0095] The hyperspectral image acquisition module can be equipped with a hyperspectral camera, which can capture hyperspectral images of specific bands of fruits. These hyperspectral images can include invisible spectral bands such as visible light range and near infrared, which can reflect changes in the internal components of the fruit, such as maturity indicators such as sugar and moisture.

[0096] The image processing module can perform a series of preprocessing on the collected hyperspectral images, such as denoising, enhancement, correction, feature extraction, etc. It can also segment individual fruits in the hyperspectral images, thereby providing high-quality image data for subsequent maturity determination.

[0097] The maturity determination module can use the trained target detection model to analyze the optimized hyperspectral image to identify and determine the maturity of individual fruits.

[0098] The display module can be used to fuse and splice the maturity judgment results and location information corresponding to each fruit with the target hyperspectral image to obtain a fruit maturity distribution map corresponding to the fruit tree area, so as to visualize the fruit maturity of the fruit tree area.

[0099] The fruit maturity visualization system in the above embodiment can realize automatic and high-precision judgment of fruit maturity, and can also intuitively display the maturity of each fruit in the fruit tree area, so that the operator can quickly locate specific ripe fruits, thereby realizing refined management of fruit trees.

[0100] Next, in the embodiment of the present disclosure, a device for determining the maturity of a fruit is also provided. Figure 5 As shown in , the fruit maturity determination device 500 can be composed of a solid component detection module 501, a data set acquisition module 502, a determination model training module 503 and a maturity determination module 504, wherein: the solid component detection module 501 can be used to obtain hyperspectral images corresponding to fruits at multiple harvest stages, and perform solid component detection on the fruits at the harvest stages to obtain fruit solid component data corresponding to the hyperspectral images; the data set acquisition module 502 can be used to compare the fruit solid component data with a preset maturity determination threshold, and perform maturity annotation on the fruit hyperspectral image based on the comparison result to obtain a fruit maturity data set; the determination model training module 503 can be used to train a target detection model based on the fruit maturity data set to obtain a fruit maturity determination model; the maturity determination module 504 can be used to obtain a target hyperspectral image corresponding to the fruit to be determined, perform maturity determination on the target hyperspectral image using the fruit maturity determination model, and obtain a maturity determination result of the fruit to be determined.

[0101] In the embodiment of the present disclosure, a fruit maturity visualization device is also provided. Figure 6As shown in , the fruit maturity visualization device 600 can be composed of a solid component detection module 501, a data set acquisition module 502, a judgment model training module 503, a regional maturity judgment module 601 and a visualization display module 602, wherein: the solid component detection module 501 can be used to obtain hyperspectral images corresponding to fruits at multiple harvest stages, and perform solid component detection on fruits at harvest stages to obtain fruit solid component data corresponding to the hyperspectral images; the data set acquisition module 502 can be used to compare the fruit solid component data with a preset maturity judgment threshold, and perform maturity annotation on the fruit hyperspectral image based on the comparison result to obtain a fruit maturity data set; the judgment model training module 503 can be used to obtain a regional maturity judgment module 601 and a visualization display module 602, wherein: the solid component detection module 501 can be used to obtain hyperspectral images corresponding to fruits at multiple harvest stages, and perform solid component detection on fruits at harvest stages to obtain fruit solid component data corresponding to the hyperspectral images; the data set acquisition module 502 can be used to compare the fruit solid component data with a preset maturity judgment threshold, and perform maturity annotation on the fruit hyperspectral image based on the comparison result to obtain a fruit maturity data set; The model training module 503 can be used to train the target detection model based on the fruit maturity data set to obtain a fruit maturity determination model; the regional maturity determination module 601 can be used to obtain the target hyperspectral image and position information corresponding to each fruit in the fruit tree area based on a preset shooting path, and use the fruit maturity determination model to perform maturity determination on the target hyperspectral image to obtain the maturity determination result corresponding to each fruit; the visualization display module 602 can be used to fuse and splice the maturity determination result and position information corresponding to each fruit with the target hyperspectral image to obtain a fruit maturity distribution map corresponding to the fruit tree area, so as to visualize the fruit maturity of the fruit tree area.

[0102] It should be noted that the specific details of each part of the above-mentioned fruit maturity determination device have been described in detail in the implementation method of the fruit maturity determination method part. The undisclosed details can be found in the implementation method part, and thus will not be repeated here.

[0103] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above-mentioned fruit maturity determination method or fruit maturity visualization method is also provided.

[0104] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware embodiments, complete software embodiments (including firmware, microcode, etc.), or embodiments combining hardware and software aspects, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0105] Refer to the following Figure 7 hereinafter describes an electronic device 700 according to such an embodiment of the present disclosure. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0106] like Figure 7As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include, but are not limited to: the at least one processing unit 710, the at least one storage unit 720, a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710), and a display unit 740.

[0107] The storage unit stores program codes, which can be executed by the processing unit 710, so that the processing unit 710 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 710 can perform the following steps: Figure 2 In step S210, hyperspectral images corresponding to fruits at multiple harvest stages are obtained, and solid component detection is performed on the fruits at the harvest stages to obtain fruit solid component data corresponding to the hyperspectral images; in step S220, the fruit solid component data is compared with a preset maturity judgment threshold, and maturity annotation is performed on the fruit hyperspectral images based on the comparison result to obtain a fruit maturity data set; in step S230, a target detection model is trained based on the fruit maturity data set to obtain a fruit maturity judgment model; in step S240, a target hyperspectral image corresponding to the fruit to be judged is obtained, and maturity judgment is performed on the target hyperspectral image using the fruit maturity judgment model to obtain a maturity judgment result of the fruit to be judged.

[0108] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 721 and / or a cache memory unit 722 , and may further include a read-only memory unit (ROM) 723 .

[0109] The storage unit 720 may also include a program / utility 724 having a set (at least one) of program modules 725, such program modules 725 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0110] Bus 730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0111] The electronic device 700 may also communicate with one or more external devices 770 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or may communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 760. As shown, the network adapter 760 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0112] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0113] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of the present specification.

[0114] refer to Figure 8 As shown, a program product 800 for implementing the above-mentioned fruit maturity determination method or fruit maturity visualization method according to an embodiment of the present disclosure is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0115] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0116] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0117] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, electromagnetic waves, etc., or any suitable combination of the foregoing.

[0118] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0119] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0120] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0121] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0122] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for determining fruit maturity, characterized in that: include: Acquire hyperspectral images corresponding to fruits at multiple harvest stages, and perform solid component detection on the fruits at the harvest stages to obtain fruit solid component data corresponding to the hyperspectral images; The fruit solid component data is compared with a preset maturity judgment threshold, and the fruit hyperspectral image is labeled with maturity based on the comparison result to obtain a fruit maturity data set; Training the target detection model based on the fruit maturity data set to obtain a fruit maturity determination model; Based on the preset shooting path, the target hyperspectral image and position information corresponding to each fruit in the fruit tree area are obtained, and the maturity determination result corresponding to each fruit is obtained by using the fruit maturity determination model to determine the maturity of the target hyperspectral image; The maturity determination result and location information corresponding to each fruit plant are fused and spliced ​​with the target hyperspectral image to obtain a fruit maturity distribution map corresponding to the fruit tree area, so as to visualize the fruit maturity of the fruit tree area.

2. The method for determining the degree of fruit maturity according to claim 1, wherein: The solid component detection of the fruits at the harvest stage to obtain the fruit solid component data corresponding to the hyperspectral image includes: The soluble solid content and dry matter content of the fruits at the harvest stage are detected by using a portable detector, and the soluble solid content and dry matter content are used as fruit solid component data corresponding to the hyperspectral image.

3. The method for determining the degree of fruit maturity according to claim 1, wherein: The step of labeling the fruit hyperspectral image for maturity based on the comparison result to obtain a fruit maturity data set includes: When the fruit solid content data is not less than the maturity determination threshold, the corresponding fruit hyperspectral image is marked as mature; When the fruit solid content data is less than the maturity determination threshold, the corresponding fruit hyperspectral image is marked as unripe; The fruit maturity dataset is constructed based on the annotated fruit hyperspectral image.

4. The method for determining the degree of fruit maturity according to claim 3, wherein: The step of constructing the fruit maturity dataset based on the annotated fruit hyperspectral image comprises: The annotated fruit hyperspectral image is subjected to geometric transformation, contrast modification and noise reduction processing to obtain the fruit maturity data set.

5. The method for determining the degree of fruit maturity according to claim 1, wherein: The object detection model is trained based on the fruit maturity data set to obtain a fruit maturity determination model, including: Optimize the backbone network and loss function of the YOLOv5 model to obtain the YOLOv5 optimized model; The YOLOv5 optimization model is trained based on the fruit maturity data set to obtain a fruit maturity determination model.

6. The method for determining the degree of fruit maturity according to claim 5, wherein: The optimization of the backbone network and loss function of the YOLOv5 model includes: Introducing a convolutional layer attention module into the backbone network to enhance the backbone network's ability to extract features from the fruit hyperspectral image; The loss function of the bounding box in the YOLOv5 model is optimized using complete intersection-over-union (IoU) to obtain a complete intersection-over-union (IoU) loss function.

7. The method for determining the degree of maturity of a fruit according to claim 6, wherein: The obtained complete intersection-over-union loss function is: in, represents the complete intersection-over-union loss function, represents the intersection-over-union ratio of the bounding boxes, represents the center point of the predicted bounding box, represents the center point of the true bounding box, represents the square of the Euclidean distance between the center point of the predicted bounding box and the center point of the true bounding box, c represents the diagonal length of the smallest bounding rectangle that contains the predicted bounding box and the true bounding box, is a hyperparameter, Represents the absolute value of the difference in aspect ratio between the predicted bounding box and the true bounding box.

8. A device for determining the maturity of a fruit, characterized in that: include: A solid component detection module is used to obtain hyperspectral images corresponding to fruits at multiple harvest stages, and perform solid component detection on the fruits at the harvest stages to obtain fruit solid component data corresponding to the hyperspectral images; A data set acquisition module is used to compare the fruit solid component data with a preset maturity judgment threshold, and to mark the maturity of the fruit hyperspectral image based on the comparison result to obtain a fruit maturity data set; A determination model training module is used to train the target detection model based on the fruit maturity data set to obtain a fruit maturity determination model; A maturity determination module is used to obtain the target hyperspectral image and position information corresponding to each fruit in the fruit tree area based on a preset shooting path, and use the fruit maturity determination model to perform maturity determination on the target hyperspectral image to obtain the maturity determination result corresponding to each fruit; The maturity determination result and location information corresponding to each fruit plant are fused and spliced ​​with the target hyperspectral image to obtain a fruit maturity distribution map corresponding to the fruit tree area, so as to visualize the fruit maturity of the fruit tree area.

Images