Weight determination and fruit maturity determination method and device and electronic equipment

Abstract

The embodiment of the invention provides a weight determination method, a fruit maturity determination method, a weight determination device, a fruit maturity determination device and electronic equipment. The method comprises the steps that multiple sets of fruit information of a to-be-detected fruit are acquired, the fruit information comprises a color image and depth data, and different sets of fruit information are acquired for the to-be-detected fruit at different visual angles; for each group of fruit information in the plurality of groups of fruit information, performing image segmentation on a color image in the group of fruit information, and determining a single view angle area corresponding to the group of fruit information based on a segmentation result and depth data in the group of fruit information; based on the single-view-angle areas corresponding to the multiple sets of fruit information, determining the overall area, and based on the segmentation results corresponding to the multiple sets of fruit information, determining the particle density of the to-be-detected fruit; and based on the overall area and the particle density, determining weight information of the to-be-measured fruit. According to the scheme, the weight information of the to-be-measured fruit can be accurately determined by combining the multi-view fruit information and utilizing the depth data in the multi-view fruit information.

Description

Technical Field

[0001] This invention relates to the field of agricultural information sensing technology, and more specifically to a method and apparatus for determining weight, a method and apparatus for determining fruit maturity, electronic equipment, storage media, and computer program products. Background Technology

[0002] In modern crop cultivation and production management, fruit weight is one of the key indicators for measuring crop growth status, assessing yield, guiding harvesting timing, and formulating sales strategies. The following uses grape weight prediction as an example to describe the problems existing in current technology. It should be understood that grapes are only an example; other similar grain crops that require weight determination (such as lychees) also face the following problems. Determining the weight of a single grape or bunch is a necessary operation for measuring crop growth status and plays a decisive role in guiding harvesting timing.

[0003] In yield assessment scenarios, weight information is mainly obtained through two methods: manual sampling and weighing, and image-based estimation, which involves capturing images of grapes through cameras, analyzing the morphological characteristics of the fruit using computer vision technology, and calculating the weight using empirical formulas or statistical models.

[0004] However, both of these methods have significant drawbacks. Manual sampling results have large errors, and frequent harvesting and sampling can affect the normal growth and commercial quality of grapes, resulting in low efficiency and high cost. The existing images have too limited a perspective, making it difficult to accurately determine the true shape and spatial structure of grapes and thus impossible to accurately determine the weight of a single grape or bunch. Summary of the Invention

[0005] The present invention was proposed in view of the above-mentioned problems. The present invention provides a method and apparatus for determining weight, a method and apparatus for determining fruit ripeness, an electronic device, a storage medium, and a computer program product.

[0006] According to one aspect of the present invention, a weight determination method is provided, the method comprising: acquiring multiple sets of fruit information of a fruit to be tested, the fruit information including color images and depth data, different sets of fruit information being acquired from different perspectives for the fruit to be tested, the fruit to be tested being a fruit containing fruit particles; for each set of fruit information, performing image segmentation on the color image in the set of fruit information to obtain a segmentation result of the fruit to be tested, and determining a single-view area corresponding to the set of fruit information based on the segmentation result and the depth data in the set of fruit information, wherein the single-view area includes the single-view fruit area of ​​the fruit to be tested and / or the single-view particle area of ​​at least some of the fruit particles of the fruit to be tested; determining an overall area based on the single-view areas corresponding to each of the multiple sets of fruit information, and determining a particle density of the fruit to be tested based on the segmentation results corresponding to each of the multiple sets of fruit information, wherein the overall area includes the overall fruit area of ​​the fruit to be tested and / or the overall particle area of ​​at least some of the fruit particles; and determining weight information of the fruit to be tested based on the overall area and particle density, the weight information including the fruit weight of the fruit to be tested and / or the particle weight of at least some of the fruit particles.

[0007] For example, the overall area includes the overall particle area of ​​each of the at least some fruit particles, and the weight information includes the particle weight of each of the at least some fruit particles; the weight information of the fruit to be tested is determined based on the overall area and particle density, including: for each fruit particle in the at least some fruit particles, calculating the product of the overall particle area, particle density and preset correction coefficient of the fruit particle, so as to obtain the product result as the particle weight of the fruit particle, and the preset correction coefficient is used to indicate the theoretical weight of a single fruit particle.

[0008] For example, the overall area includes the overall particle area of ​​all fruit particles in the fruit to be tested, and the weight information includes the fruit weight. Before determining the weight information of the fruit to be tested based on the overall area and particle density, the method further includes: determining the number of fruit particles in the fruit to be tested based on the segmentation result; determining the weight information of the fruit to be tested based on the overall area and particle density, including: summing the individual particle weights of all fruit particles in the fruit to be tested according to the number of particles, and obtaining the summed result as the fruit weight, wherein the particle weight of each fruit particle in the fruit to be tested is equal to the product of the overall particle area, particle density and a preset correction coefficient of that fruit particle, and the preset correction coefficient is used to indicate the theoretical weight of a single fruit particle.

[0009] For example, the segmentation result includes a first image region where each fruit particle of the fruit to be tested is located. Based on the segmentation results corresponding to multiple sets of fruit information, the particle density of the fruit to be tested is determined, including: determining the average spacing between the fruit particles of the fruit to be tested based on the distance between the first image regions in all segmentation results; determining the particle density based on the average spacing, wherein the particle density is inversely proportional to the average spacing.

[0010] For example, the segmentation result includes a first image region where each fruit particle of the fruit to be tested is located, and the single-view area includes the single-view particle area of ​​at least some of the fruit particles. Based on the segmentation result and the depth data in the set of fruit information, determining the single-view area corresponding to the set of fruit information includes: for each fruit particle in at least some of the fruit particles, based on the depth data, converting the two-dimensional pixel area of ​​the first image region where the fruit particle is located into a three-dimensional physical area to obtain the single-view particle area of ​​the fruit particle; and / or, the segmentation result includes a second image region where the fruit to be tested is located, and the single-view area includes the single-view fruit area of ​​the fruit to be tested. Based on the segmentation result and the depth data in the set of fruit information, determining the single-view area corresponding to the set of fruit information includes: based on the depth data, converting the two-dimensional pixel area of ​​the second image region into a three-dimensional physical area to obtain the single-view fruit area of ​​the fruit to be tested.

[0011] For example, determining the overall area based on the single-view area corresponding to each of the multiple sets of fruit information includes: fusing the single-view areas corresponding to each of the multiple sets of fruit information to obtain the overall area.

[0012] For example, image segmentation is performed on the color images in the set of fruit information to obtain the segmentation result of the fruit to be tested, including: performing target detection on the color images in the set of fruit information to determine the target detection region where the fruit to be tested is located; performing semantic segmentation on the target detection region to segment a second image region where the fruit to be tested is located from the target detection region; and performing instance segmentation on the second image region to segment a first image region where each fruit particle of the fruit to be tested is located from the second image region; wherein the segmentation result includes the first image region or includes the first image region and the second image region.

[0013] For example, multiple sets of fruit information are acquired by a color depth camera moving horizontally along the field of view within the planting area of ​​the fruit to be tested, thereby acquiring multiple sets of fruit information, wherein the field of view of any two adjacent views partially overlap with each other.

[0014] According to another aspect of the present invention, a method for determining fruit maturity is also provided, including the weight determination method described above, wherein the weight information includes fruit weight, and the method for determining fruit maturity further includes: determining the maturity of the fruit to be tested based on the fruit weight and a preset correlation model, wherein the preset correlation model is used to indicate the correlation between a preset fruit weight and a preset maturity.

[0015] According to another aspect of the present invention, a weight determination device is also provided, comprising: an acquisition module for acquiring multiple sets of fruit information of a fruit to be tested, the fruit information including color images and depth data, wherein different sets of fruit information are acquired from different viewing angles of the fruit to be tested, and the fruit to be tested is a fruit containing fruit particles; and a segmentation module for performing image segmentation on the color image of each set of fruit information to obtain a segmentation result of the fruit to be tested, and determining a single-view area corresponding to the set of fruit information based on the segmentation result and the depth data in the set of fruit information, wherein the single-view area includes the weight of the fruit to be tested. The system comprises: a first determining module, which determines the overall area based on the single-view area corresponding to each of multiple sets of fruit information, and determines the particle density of the fruit to be tested based on the segmentation results corresponding to each of the multiple sets of fruit information, wherein the overall area includes the overall fruit area of ​​the fruit to be tested and / or the overall particle area of ​​each of at least some of the fruit particles; and a second determining module, which determines the weight information of the fruit to be tested based on the overall area and the particle density, wherein the weight information includes the fruit weight of the fruit to be tested and / or the particle weight of each of at least some of the fruit particles.

[0016] According to another aspect of the present invention, a fruit maturity determination device is also provided, including the weight determination device described above, wherein the weight information includes the fruit weight, and the fruit maturity determination device further includes: a third determination module, used to determine the maturity of the fruit to be tested based on the fruit weight and a preset correlation model, wherein the preset correlation model is used to indicate the correlation between the preset fruit weight and the preset maturity.

[0017] According to another aspect of the present invention, an electronic device is also provided, including a processor and a memory, wherein the memory stores computer program instructions, which are executed by the processor to perform the weight determination method or the fruit ripeness determination method as described above.

[0018] According to some embodiments of the present invention, a storage medium is also provided, on which program instructions are stored, which are used to execute the above-described weight determination method or fruit maturity determination method when running.

[0019] According to some embodiments of the present invention, a computer program product is also provided, including computer program instructions, which, when run, are used to execute the above-described weight determination method or fruit ripeness determination method.

[0020] By employing the above technical solution, multiple sets of fruit information can be collected from multiple angles. Combined with image segmentation technology and depth data, the overall area and particle density of the fruit under test can be accurately obtained, thereby comprehensively determining the weight information of the fruit. This solution is an automated and intelligent weight determination method with high accuracy and efficiency, and low operating costs. Furthermore, this solution, through multi-view image acquisition and depth data fusion, can solve the problem of incomplete shape reconstruction caused by the limitations of traditional single-view images. Using the segmentation results of depth data and color images, the actual physical area of ​​the fruit and / or fruit particles under test can be determined, establishing a correlation between two-dimensional pixel area and true physical size. Determining the overall area through multi-angle fruit information improves the robustness of area estimation. Simultaneously, combining the distribution density of fruit particles enables multi-factor joint modeling of shape, area, and density, thereby significantly improving the accuracy of fruit weight estimation. Attached Figure Description

[0021] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0022] Figure 1 A schematic flowchart of a weight determination method according to an embodiment of the present invention is shown;

[0023] Figure 2 A schematic diagram showing the segmentation result of a fruit to be tested according to an embodiment of the present invention;

[0024] Figure 3 A schematic block diagram of a weight determination device according to an embodiment of the present invention is shown; and

[0025] Figure 4 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0027] In modern crop cultivation and production management, fruit weight is a key indicator for measuring crop growth status, assessing yield, guiding harvesting timing, and formulating sales strategies. Traditionally, obtaining this weight information relies primarily on manual sampling and weighing or estimation methods based on single-view images. These existing methods are typically calculated on-site by growers or detection systems, or using fixed models. However, manual sampling results in significant errors, and frequent harvesting and sampling can negatively impact normal fruit growth and market quality, leading to low efficiency and high costs. Single-view image estimation methods rely on a limited perspective, while fruits typically exhibit an irregular three-dimensional distribution. Single-view images struggle to fully capture the true shape and spatial structure of the fruit, resulting in significant deviations in fruit area or volume estimations, making it impossible to accurately determine the weight of the entire fruit or the weight of individual fruit particles. Furthermore, pixel area obtained solely from single-view images cannot accurately reflect the actual physical size of the fruit, especially when fruits are stacked or obscured, significantly reducing the correlation between two-dimensional pixel area and true weight. In addition, existing methods typically rely on only a single factor (such as area or number of particles) to estimate weight, which is not comprehensive enough, resulting in a large deviation between the estimated results and the actual weight, making it difficult to meet the actual needs of precision agriculture.

[0028] To at least partially address the aforementioned problems, embodiments of the present invention provide a weight determination method. Using this method, when detecting the fruit weight and / or particle weight of a fruit to be tested, multiple sets of fruit information can be acquired from multiple angles, namely multiple sets of color images and multiple sets of depth data. By combining image segmentation technology and depth data, the actual physical area (i.e., overall area) and the distribution density of fruit particles (i.e., particle density) can be accurately obtained, thereby comprehensively determining the weight information of the fruit to be tested. This method, by combining fruit information from multiple perspectives and utilizing the depth data, can accurately determine the fruit weight and / or particle weight of the fruit to be tested.

[0029] Figure 1A schematic flowchart illustrating a weight determination method 100 according to an embodiment of the present invention is shown. The weight determination method 100 described herein can be applied to any electronic device with data processing and / or instruction execution capabilities, i.e., it is executed by an electronic device. This electronic device may include, but is not limited to, personal computers, servers, mobile terminals, etc. Exemplarily and not limitingly, the electronic device may include a monitoring system such as one deployed in a smart agricultural orchard, i.e., the weight determination method 100 is executed by a monitoring system deployed at the crop planting site. The weight determination method described herein is used to detect the fruit weight and / or grain weight of a crop, and the weight detection result (i.e., the weight information of the fruit to be measured) is obtained through multi-view fruit information analysis.

[0030] Step S110: Obtain multiple sets of fruit information for the fruit to be tested. The fruit information includes color images and depth data. Different sets of fruit information are collected from different perspectives for the fruit to be tested. The fruit to be tested is a fruit with fruit particles.

[0031] The fruit to be tested is a fruit containing fruit particles. Exemplarily, the fruit to be tested can be, but is not limited to, clustered or bunch-like fruits such as grapes, cherries, and lychees. Each set of fruit information includes a color (RGB) image and depth data. The fruit information can be acquired using a data acquisition device, which can be a color depth (RGBD) camera or other similar data acquisition device capable of simultaneously acquiring color images and depth data, such as a binocular stereo vision camera. Different sets of fruit information are acquired from different perspectives of the fruit to be tested. It can be understood that the color image and depth data in the same set of fruit information are acquired from the same perspective of the fruit to be tested. Exemplarily, a mobile robot equipped with a data acquisition device can collect fruit information from different perspectives while moving around the fruit to be tested. The mobile robot can collect fruit information from multiple perspectives during inspection to maximize both inspection speed and the acquisition of fruit morphology data. Alternatively, fixed data acquisition devices positioned at different locations can be used to collect fruit information from different perspectives. The electronic device for performing the weight determination method 100 can be communicatively connected to a data acquisition device, or the data acquisition device can be included within the electronic device for performing the weight determination method 100, so that the processor of the electronic device can acquire the fruit information acquired by the data acquisition device.

[0032] Step S120: For each group of fruit information in the multiple groups of fruit information, perform image segmentation on the color image in the group of fruit information to obtain the segmentation result of the fruit to be tested, and determine the single-view area corresponding to the group of fruit information based on the segmentation result and the depth data in the group of fruit information. The single-view area includes the single-view fruit area of ​​the fruit to be tested and / or the single-view particle area of ​​at least some of the fruit particles of the fruit to be tested.

[0033] Image segmentation can be performed on the color images of each group of fruit information to obtain the segmentation results of the fruit under test from various viewpoints. For example, during the actual growth of crops, there may be overlapping parts of other fruits in the color image of the fruit under test, requiring identification and segmentation to obtain information about the fruit itself. For example, a single string of fruits can be segmented using an object detection model and a semantic segmentation model, and each fruit particle in the fruit under test can be segmented using an instance segmentation model. Alternatively, traditional image processing algorithms can be used to segment the color image to obtain the fruit under test and / or fruit particles. Those skilled in the art should understand that any existing or future image segmentation scheme can be applied to the embodiments of this invention, and will not be elaborated further here. The segmentation result may include a second image region where the fruit under test is located and / or a first image region where each fruit particle of the fruit under test is located, i.e., it includes the position information of the fruit under test in the color image and / or the position information of each fruit particle of the fruit under test in the color image. For each group of fruit information, based on the corresponding segmentation result and the depth data in that group of fruit information, the single-view area at the corresponding viewpoint of that group of fruit information can be determined. Single-view area can be understood as the three-dimensional surface area determined based on fruit information from a corresponding viewpoint, i.e., the three-dimensional surface area under that viewpoint. Single-view area can include the single-view fruit area of ​​the fruit under test and / or the single-view particle area of ​​all or part of the fruit particles individually. Single-view fruit area is the three-dimensional surface area of ​​the entire fruit under test under the corresponding viewpoint. Single-view particle area is the three-dimensional surface area of ​​the fruit particles under the corresponding viewpoint.

[0034] Step S130: Based on the single-view area corresponding to each of the multiple sets of fruit information, determine the overall area, and based on the segmentation results corresponding to each of the multiple sets of fruit information, determine the particle density of the fruit to be tested. The overall area includes the overall fruit area of ​​the fruit to be tested and / or at least the overall particle area of ​​each of the fruit particles.

[0035] The overall area can be understood as the three-dimensional surface area obtained by combining the single-view areas from multiple perspectives. The overall area can include the total fruit area of ​​the fruit under test and / or at least the total particle area of ​​each individual fruit particle. The total fruit area of ​​the fruit under test is the three-dimensional surface area of ​​the entire fruit in three-dimensional physical space. The total particle area of ​​each fruit particle is the three-dimensional surface area of ​​that fruit particle in three-dimensional physical space. The overall area can be estimated by combining the single-view areas corresponding to multiple sets of fruit information to obtain a fruit area and / or particle area that is closer to the true value in three-dimensional physical space. Particle density is the distribution density of fruit particles in three-dimensional physical space, i.e., how many fruit particles are per unit area, and is an indicator characterizing the density of fruit particle distribution. For example, particle density can be determined by analyzing the particle distribution characteristics of the fruit under test, such as the average spacing between fruit particles.

[0036] Step S140: Based on the overall area and particle density, determine the weight information of the fruit to be tested. The weight information includes the weight of the fruit to be tested and / or the individual particle weight of at least some of the fruit particles of the fruit to be tested.

[0037] In one embodiment, the overall area may include the total fruit area of ​​the fruit to be tested, and the weight information includes the fruit weight of the fruit to be tested. In this case, the overall fruit area can be converted into fruit weight by combining a pre-trained weight estimation model (which may be called the first weight estimation model) or a physical formula (which may be called the first physical formula). The input of the weight estimation model is the overall fruit area, and the output is the corresponding fruit weight. For example, the fruit weight can be determined by multiplying the overall fruit area, particle density, and a preset correction coefficient. The preset correction coefficient is pre-set and represents the theoretical weight of a single fruit particle. The preset correction coefficient can be obtained based on experience or experimentation and is related to the variety of the fruit to be tested. In another embodiment, the overall area may include the total fruit area of ​​the fruit to be tested, and the weight information includes the individual particle weights of all the fruit particles of the fruit to be tested. In this case, the overall fruit area can be converted into fruit weight by combining a pre-trained weight estimation model or a physical formula. The specific implementation method can be understood by referring to the above embodiments. The number of fruit particles can be obtained by counting the fruit particles based on the segmentation results, and the quotient of the fruit weight and the number of particles can be determined as the individual particle weights of all the fruit particles of the fruit to be tested. The particle weights of the fruit particles calculated by this scheme are equal to each other, which is a relatively average calculation method. When the total area includes the total fruit area of ​​the fruit to be tested, and the weight information includes the fruit weight of the fruit to be tested and the individual weight of each fruit particle, the fruit weight and the individual weight of each fruit particle can be determined using the methods described in the above embodiments, and will not be repeated here.

[0038] In one embodiment, the overall area includes the overall particle area of ​​at least some of the fruit particles, and the weight information includes the particle weight of at least some of the fruit particles. In this case, the overall particle area can be converted into the particle weight of the fruit particles by combining a pre-trained weight estimation model (which may be called a second weight estimation model) or a physical formula (which may be called a second physical formula). The input of the weight estimation model is the overall particle area, and the output is the particle weight of the corresponding fruit particle. For example, the particle weight of the fruit particle can be determined by multiplying the overall particle area, particle density, and a preset correction coefficient. As mentioned above, the preset correction coefficient is pre-defined and represents the theoretical weight of a single fruit particle. This scheme can determine the particle weight of a single fruit particle, and in determining this weight, the fruit particles can be optionally filtered to determine the particle weight of all or some of them. For example, for fruit particles whose overall particle area is less than a preset area threshold, it is not necessary to determine their weight; instead, the weight can be determined only for fruit particles whose overall particle area is greater than or equal to the preset area threshold. As another example, the weight can be determined for the fruit particles selected by the user in response to the user's selection operation on a color image. In another embodiment, the overall area includes the total particle area of ​​each individual fruit particle, and the weight information includes the fruit weight. In this case, the overall particle area can be converted into the particle weight of the fruit particle using a pre-trained weight estimation model or physical formula. The specific implementation can be understood by referring to the embodiments described above. The fruit weight can be determined by summing the particle weights of the fruit particles based on the number of fruit particles. When the overall area includes the total particle area of ​​at least some of the fruit particles, and the weight information includes the particle weight of at least some of the fruit particles and the fruit weight of the fruit to be measured, the particle weight and fruit weight can be determined using the methods described in the embodiments above, and will not be repeated here.

[0039] According to the weight determination method of this invention, multiple sets of fruit information can be collected from multiple angles, combined with image segmentation technology and depth data, to accurately obtain the overall area and particle density of the fruit to be measured, and then comprehensively determine the weight information of the fruit. This scheme is an automated and intelligent weight determination scheme with high accuracy and efficiency and low operating cost. In addition, this scheme can solve the problem of incomplete shape restoration caused by the limitation of traditional single-view images by multi-view image acquisition and depth data fusion. By using the segmentation results of depth data and color images, the actual physical area of ​​the fruit and / or fruit particles to be measured can be determined, and a correlation between two-dimensional pixel area and real physical size can be established. Determining the overall area by fruit information from multiple angles can improve the robustness of area estimation. At the same time, by combining the distribution density of fruit particles, multi-factor joint modeling of shape, area and density can be achieved, thereby significantly improving the accuracy of fruit weight estimation.

[0040] According to an embodiment of the present invention, the overall area includes the overall particle area of ​​all fruit particles of the fruit to be tested, and the weight information includes the fruit weight. Before determining the weight information of the fruit to be tested based on the overall area and particle density, the method further includes: determining the number of fruit particles in the fruit to be tested based on the segmentation result; determining the weight information of the fruit to be tested based on the overall area and particle density, including: summing the particle weights of all fruit particles of the fruit to be tested according to the number of particles, and obtaining the summation result as the fruit weight, wherein the particle weight of each fruit particle of the fruit to be tested is equal to the product of the overall particle area, particle density and a preset correction coefficient of the fruit particle, and the preset correction coefficient is used to indicate the theoretical weight of a single fruit particle.

[0041] For example, a bunch of fruits to be tested can be selected, and color images and depth data of the bunch of fruits from multiple different angles can be acquired by a moving color depth camera. The acquired multi-view color images are processed to identify and segment each individual fruit particle, with each fruit particle corresponding to a fruit particle region (referred to as the first image region in this document). Based on the segmentation results, the position and number of each fruit particle can be determined. Based on the segmentation results and depth data, the overall particle area of ​​each fruit particle in at least a portion of the fruit particles can be calculated. Based on the segmentation results of multiple color images, the particle density of the bunch of fruits can be calculated. As mentioned above, at least a portion of the fruit particles can be filtered based on the overall particle area or selected by the user. Of course, optionally, all fruit particles of the fruit to be tested can be selected by default as the aforementioned at least a portion of the fruit particles. The particle weight of each fruit particle can be determined using the following formula: Particle weight of fruit particle = Overall particle area × Particle density × Preset correction coefficient.

[0042] Using the above technical solution, the weight of a single fruit particle can be determined based on the overall particle area, particle density, and preset correction coefficient. This solution can achieve accurate weight determination of individual fruit particles, making it convenient for users to conduct targeted observations of fruit particles.

[0043] According to an embodiment of the present invention, the overall area includes the overall particle area of ​​all fruit particles of the fruit to be tested, and the weight information includes the fruit weight. Before determining the weight information of the fruit to be tested based on the overall area and particle density, the method further includes: determining the number of fruit particles in the fruit to be tested based on the segmentation result; determining the weight information of the fruit to be tested based on the overall area and particle density, including: summing the particle weights of all fruit particles of the fruit to be tested according to the number of particles, and obtaining the summation result as the fruit weight, wherein the particle weight of each fruit particle of the fruit to be tested is equal to the product of the overall particle area, particle density and a preset correction coefficient of the fruit particle, and the preset correction coefficient is used to indicate the theoretical weight of a single fruit particle.

[0044] As described above, the acquired multi-view color images can be processed to identify and segment each individual fruit particle, with each fruit particle corresponding to a fruit particle region. Based on the segmentation results, the location and number of each fruit particle can be determined. For example, the number of individual fruit particles that can be clearly distinguished from the current viewpoint can be counted in the first image region of the segmentation results, i.e., the particle count. Optionally, the total number of fruit particles can be counted, or incomplete fruit particles (e.g., fruit particles whose overall particle area is less than a preset area threshold) can be filtered out to temporarily exclude them from the particle count calculation. Based on the segmentation results and depth data, the overall particle area of ​​each fruit particle can be calculated. Based on the segmentation results, the particle density of the fruit cluster can be calculated. The fruit weight of the fruit to be tested can be determined using the following formula: n represents the number of fruit grains.

[0045] Using the above technical solution, the fruit weight can be determined based on the overall particle area, particle density, particle number, and preset correction coefficient of each fruit particle. This solution determines the fruit weight by summing the particle weights of local fruit particles, effectively avoiding the influence of gaps between fruit particles on the accuracy of fruit weight determination, thereby helping to further improve the accuracy and reliability of weight determination.

[0046] For example, the segmentation result includes a first image region where each fruit particle of the fruit to be tested is located. Based on the segmentation results corresponding to multiple sets of fruit information, the particle density of the fruit to be tested is determined, including: determining the average spacing between the fruit particles of the fruit to be tested based on the distance between the first image regions in all segmentation results; determining the particle density based on the average spacing, wherein the particle density is inversely proportional to the average spacing.

[0047] Figure 2 A schematic diagram showing the segmentation result of a fruit to be tested according to an embodiment of the present invention. Figure 2In the illustrated embodiment, the fruit to be tested is a grape. For example... Figure 2 As shown, the first image region is the pixel region of each visible individual grape grain identified from the color image at the current viewing angle. Exemplarily, the segmentation result may also include a second image region containing the fruit to be tested. Figure 2 In the illustrated embodiment, the second image region can be the pixel region of the identified "whole bunch of grapes," which is typically an irregularly shaped area. The first and / or second image regions can be marked on the color image using a mask pattern.

[0048] For each group of fruit information, the average distance between the first image regions in the color image of that group of fruit information can be determined based on the segmentation result. This average distance can be the average nearest neighbor distance, the global average pairwise distance, etc. The distance between any two first image regions can be represented by, for example, the distance between the center points of these two first image regions. The average distance between the first image regions is the average distance between fruit particles from the current viewpoint. The average spacing between fruit particles can be determined by comprehensively considering the average distances corresponding to multiple groups of fruit information. For example, the average distances corresponding to multiple groups of fruit information can be further averaged to obtain the average spacing between fruit particles. Alternatively, the median of the average distances corresponding to multiple groups of fruit information can be taken to obtain the average spacing between fruit particles. Of course, other methods can also be used to determine the average spacing, which will not be elaborated here.

[0049] Particle density can be determined based on the average spacing, and particle density is inversely proportional to the average spacing. A larger average spacing results in sparser particle distribution and lower particle density. Conversely, a smaller average spacing results in denser particle distribution and higher particle density.

[0050] Using the above technical solution, the particle density can be determined by the average spacing between fruit particles. This solution only requires the distance between the first image regions to determine the particle density. The outline shape of the fruit and fruit particles does not affect the determination of the density. This solution is very suitable for determining the density of naturally formed, blurred, and irregularly shaped targets (i.e., the target fruit to be tested).

[0051] For example, the segmentation result includes a first image region where each fruit particle of the fruit to be tested is located, and the single-view area includes the single-view particle area of ​​at least some of the fruit particles. Based on the segmentation result and the depth data in the group of fruit information, the single-view area corresponding to the group of fruit information is determined, including: for each fruit particle in at least some of the fruit particles, based on the depth data, converting the two-dimensional pixel area of ​​the first image region where the fruit particle is located into a three-dimensional physical area to obtain the single-view particle area of ​​the fruit particle.

[0052] In some embodiments, clustered fruits, such as blueberries, are detected. Those skilled in the art should understand that the application technique of this weight determination method is the same for both clustered and bunch-like fruits; changes in fruit type do not affect the implementation of this method, and will not be elaborated further here. A movable color depth image camera device can be used, moving at a constant speed around the target blueberry cluster. It can pause at multiple equally spaced different viewpoints to acquire a set of data (i.e., fruit information), each set containing a high-resolution color image and corresponding depth data. For each set of data, an image segmentation model can first be run on the color image to detect the overall outline of the blueberry cluster and the boundaries of each individual blueberry grain within the cluster, which are marked as a second image region and several first image regions, respectively. Subsequently, the pixel coordinates of any first image region can be mapped to three-dimensional physical space by combining the depth data from the current viewpoint; that is, the two-dimensional pixel area of ​​the first image region can be converted into a three-dimensional physical area. The converted three-dimensional physical area is the single-view grain area of ​​the corresponding fruit grain.

[0053] The above technical solution converts the two-dimensional pixel area of ​​the first image region containing the fruit particle into a three-dimensional physical area using depth data to obtain the single-view particle area of ​​the fruit particle. This solution is simple to implement and has high accuracy in determining the area.

[0054] For example, the segmentation result includes the second image region where the fruit to be tested is located, and the single-view area includes the single-view fruit area of ​​the fruit to be tested. Based on the segmentation result and the depth data in the set of fruit information, the single-view area corresponding to the set of fruit information is determined, including: based on the depth data, converting the two-dimensional pixel area of ​​the second image region into a three-dimensional physical area to obtain the single-view fruit area of ​​the fruit to be tested.

[0055] Similar to the transformation of the first image region, the pixel coordinates of the second image region can be mapped to three-dimensional physical space by combining the depth data from the current viewpoint. In other words, the two-dimensional pixel area of ​​the second image region can be converted into a three-dimensional physical area. The resulting three-dimensional physical area is the single-view fruit area of ​​the fruit being tested.

[0056] The above technical solution converts the two-dimensional pixel area of ​​the second image region containing the fruit to be tested into a three-dimensional physical area using depth data, thereby obtaining the single-view fruit area. This solution is simple to implement and provides high accuracy in determining the area.

[0057] For example, the segmentation result includes a first image region where each fruit particle of the fruit to be tested is located, and the single-view area includes the single-view particle area of ​​at least some of the fruit particles. Based on the segmentation result and the depth data in the set of fruit information, the single-view area corresponding to the set of fruit information is determined, including: for each fruit particle in at least some of the fruit particles, based on the depth data, converting the two-dimensional pixel area of ​​the first image region where the fruit particle is located into a three-dimensional physical area to obtain the single-view particle area of ​​the fruit particle; and the segmentation result includes a second image region where the fruit to be tested is located, and the single-view area includes the single-view fruit area of ​​the fruit to be tested. Based on the segmentation result and the depth data in the set of fruit information, the single-view area corresponding to the set of fruit information is determined, including: based on the depth data, converting the two-dimensional pixel area of ​​the second image region into a three-dimensional physical area to obtain the single-view fruit area of ​​the fruit to be tested.

[0058] In this embodiment, the area conversion of the first image region and the second image region can be performed based on the depth data to obtain the single-view particle area of ​​the fruit particles and the single-view fruit area of ​​the fruit to be tested. The implementation method and advantages of this embodiment can be understood by referring to the description above, and will not be repeated here.

[0059] For example, determining the overall area based on the single-view area corresponding to each of the multiple sets of fruit information includes: fusing the single-view areas corresponding to each of the multiple sets of fruit information to obtain the overall area.

[0060] Specifically, when the single-view area includes the single-view particle area of ​​at least some fruit particles, and the overall area includes the overall particle area of ​​at least some fruit particles, the total single-view particle area (i.e., the single-view particle area under multiple views) of each fruit particle can be fused to obtain the overall particle area of ​​that fruit particle. When the single-view area includes the single-view fruit area of ​​the fruit under test, and the overall area includes the overall fruit area of ​​the fruit under test, the total single-view fruit area (i.e., the single-view fruit area under multiple views) can be fused to obtain the overall fruit area of ​​the fruit under test. When the single-view area is only the visible area under the corresponding view, fusion can include matching and splicing based on the relationship between views to obtain the overall area. When the single-view area includes both the visible and invisible areas under the corresponding view, and the visible and invisible areas can form the area of ​​the entire surface of the corresponding fruit or fruit particle under test, fusion can include averaging the single-view areas to obtain the overall area.

[0061] By adopting the above technical solution, the overall area is obtained by fusing the single-view areas from different perspectives. This multi-view fusion method can further improve the robustness of area estimation.

[0062] For example, image segmentation is performed on the color images in the set of fruit information to obtain the segmentation result of the fruit to be tested, including: performing target detection on the color images in the set of fruit information to determine the target detection region where the fruit to be tested is located; performing semantic segmentation on the target detection region to segment a second image region where the fruit to be tested is located from the target detection region; and performing instance segmentation on the second image region to segment a first image region where each fruit particle of the fruit to be tested is located from the second image region; wherein the segmentation result includes the first image region or includes the first image region and the second image region.

[0063] In some embodiments, in the scenario of automated analysis of bunches of fruit, taking grapes as an example, the image segmentation process defined in this solution can be specifically implemented as follows: First, a single-view image of grapes collected in an orchard environment is acquired. The trained object detection model is used to scan the image, locate and select the target detection region where the bunch of grapes is located. Then, the above-mentioned target detection region is cropped and input into the semantic segmentation model. This model classifies at the pixel level and accurately segments the complete pixel set of the whole bunch of grapes from the region that may contain branches, leaves, and supports, thus obtaining the second image region representing the whole bunch of grapes to be tested. Then, this region is further processed by the instance segmentation model. This model can identify and distinguish all independent fruit particles that are in contact or overlap in the bunch of grapes. The region corresponding to each independent fruit particle is the first image region. Finally, the segmentation result output by the system can contain these first image region sets refined to the single particle level, providing the most basic unit data for subsequent area and density calculations. For example, object detection models can be region-based convolutional neural networks (R-CNN), fast region-based convolutional neural networks (Fast R-CNN), and You Only Look Once (YOLO). Semantic segmentation models can be fully convolutional networks (FCN), U-Net, and SegNet. Instance segmentation models can be masked region-based convolutional neural networks (Mask R-CNN), fully convolutional instance-aware semantic segmentation (FCIS), Segmenting Objects by Locations (SOLO) series, and the general 2-second segmentation model (Segment Anything in 2 seconds, SAM2). Object detection models, semantic segmentation models, and instance segmentation models are not limited to the types mentioned above. It should be understood that models that can achieve the same type of function are all included in this category.

[0064] By employing the above technical solution, object detection models, semantic segmentation models, and instance segmentation models can be combined to segment the second image region containing the fruit and the first image region containing the fruit particles. Compared to traditional image processing methods, this multi-model combined approach can achieve high-precision and rapid segmentation of fruits with irregular shapes. Furthermore, the models are trainable and transferable, facilitating subsequent upgrades and expansions, and helping to handle the detection of fruits with more diverse shapes, growth stages, or varieties.

[0065] For example, multiple sets of fruit information are acquired by a color depth camera moving horizontally along the field of view within the planting area of ​​the fruit to be tested, thereby acquiring multiple sets of fruit information, wherein the field of view of any two adjacent views partially overlap with each other.

[0066] Collecting fruit information from multiple perspectives is crucial. For example, a technician can operate a handheld color depth camera between the rows of fruit being tested, or a patrol vehicle (i.e., the aforementioned mobile robot) can be used with the color depth camera mounted on it. The method of collecting fruit information is not limited to these methods; any method that allows the color depth camera's field of view to move horizontally is acceptable. To obtain complete multi-perspective data for the cluster of fruit, the operator aligns the camera's field of view with the fruit, maintaining a roughly horizontal height, and smoothly moves the camera along a nearly horizontal arc or straight path. During the movement, the camera pauses at several predetermined or algorithm-triggered discrete positions, acquiring a color image and its corresponding depth data at each location, thus obtaining a set of fruit information. This process is repeated until multiple sets of fruit information covering different sides of the fruit have been acquired. Crucially, when planning or executing this horizontal movement acquisition path, it is essential to ensure that the field of view of the camera corresponding to two adjacent acquisition points partially overlaps. This overlapping view design not only ensures that every local feature of the grape bunch can be captured in images from at least two adjacent viewpoints, providing a data foundation for possible subsequent area fusion, but also ensures that fruit parts that may be missed due to occlusion during single-view image segmentation can be supplemented in images from adjacent viewpoints. This ensures the overall integrity of the collected multi-view fruit information and lays a reliable data foundation for subsequent accurate 3D reconstruction and parameter calculation.

[0067] According to another aspect of the present invention, a method for determining fruit maturity is provided, including the weight determination method 100 described above, wherein the weight information includes fruit weight, and the method for determining fruit maturity further includes: determining the maturity of the fruit to be tested based on the fruit weight and a preset correlation model, wherein the preset correlation model is used to indicate the correlation between a preset fruit weight and a preset maturity.

[0068] Fruit weight can be used to determine fruit maturity, facilitating the observation of fruit development progress and harvesting timing. A pre-defined correlation model can be pre-set, which can be a pre-trained neural network model. The input of this neural network model is the fruit weight, and the output is the fruit maturity. The pre-defined correlation model can also be recorded in a file in tabular form, for example, different rows in the table record different pre-defined fruit weights and the corresponding pre-defined maturity levels. In this way, the maturity of the fruit to be tested can be determined by looking up the table. Of course, the present invention is not limited to the above embodiments; any model that can indicate the correlation between pre-defined fruit weight and pre-defined maturity can be used as the pre-defined correlation model. For example, while determining fruit maturity, it can be graded according to a standard, that is, maturity can be divided into pre-defined levels, and when the fruit weight falls within a specified range corresponding to any pre-defined level, it is determined that the pre-defined level has been reached.

[0069] By using the above technical solution, the ripeness of the fruit can be determined by its weight, which can provide growers with guidance on the fruit's development progress and harvesting timing, thus facilitating their management.

[0070] According to another aspect of the present invention, a weight determining device is provided. Figure 3 A schematic block diagram of a weight determination device 300 according to an embodiment of the present invention is shown. Figure 3 As shown, the device 300 may include an acquisition module 310, a segmentation module 320, a first determination module 330, and a second determination module 340.

[0071] The acquisition module 310 is used to acquire multiple sets of fruit information of the fruit to be tested. The fruit information includes color images and depth data. Different sets of fruit information are collected from different perspectives for the fruit to be tested. The fruit to be tested is a fruit with fruit particles.

[0072] The segmentation module 320 is used to perform image segmentation on the color image in each of the multiple sets of fruit information to obtain the segmentation result of the fruit to be tested, and to determine the single-view area corresponding to the set of fruit information based on the segmentation result and the depth data in the set of fruit information, wherein the single-view area includes the single-view fruit area of ​​the fruit to be tested and / or the single-view particle area of ​​at least some of the fruit particles of the fruit to be tested.

[0073] The first determining module 330 is used to determine the overall area based on the single-view area corresponding to each of the multiple sets of fruit information, and to determine the particle density of the fruit to be tested based on the segmentation results corresponding to each of the multiple sets of fruit information, wherein the overall area includes the overall fruit area of ​​the fruit to be tested and / or at least the overall particle area of ​​each of the fruit particles.

[0074] The second determining module 340 is used to determine the weight information of the fruit to be tested based on the overall area and particle density. The weight information includes the fruit weight of the fruit to be tested and / or at least the individual particle weight of some of the fruit particles.

[0075] According to another aspect of the present invention, a fruit maturity determination device is provided, including the weight determination device described above, wherein the weight information includes the fruit weight, and the fruit maturity determination device further includes: a third determination module, used to determine the maturity of the fruit to be tested based on the fruit weight and a preset correlation model, wherein the preset correlation model is used to indicate the correlation between the preset fruit weight and the preset maturity.

[0076] According to another aspect of the present invention, an electronic device is also provided. Figure 4 A schematic block diagram of an electronic device 400 according to an embodiment of the present invention is shown, such as Figure 4 As shown, the electronic device 400 may include a processor 410 and a memory 420. The memory 420 stores a computer program, and the processor 410 executes the computer program to implement the aforementioned weight determination method or fruit ripeness determination method.

[0077] According to another aspect of the present invention, a storage medium is also provided. It stores a computer program / instructions, which, when executed by a processor, implement the aforementioned weight determination method or fruit ripeness determination method. The storage medium may, for example, include a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0078] According to another aspect of the present invention, a computer program product is also provided, including computer program instructions, which, when executed, are used to perform the weight determination method or the fruit ripeness determination method as described above.

[0079] Those skilled in the art can understand the specific implementation schemes and beneficial effects of the above-mentioned weight determination method or fruit maturity determination method by reading the relevant descriptions. For the sake of brevity, they will not be repeated here.

[0080] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0081] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0083] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0084] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0085] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0086] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0087] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the weight determination device or fruit ripeness determination device according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0088] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0089] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A weight determination method characterized by, include: Multiple sets of fruit information for the fruit to be tested are obtained. The fruit information includes color images and depth data. Different sets of fruit information are collected from different perspectives for the fruit to be tested. The fruit to be tested is a fruit with fruit particles. For each set of fruit information in the multiple sets of fruit information, the color image in the set of fruit information is segmented to obtain the segmentation result of the fruit to be tested, and based on the segmentation result and the depth data in the set of fruit information, the single-view area corresponding to the set of fruit information is determined, wherein the single-view area includes the single-view fruit area of ​​the fruit to be tested and / or the single-view particle area of ​​at least some of the fruit particles of the fruit to be tested. Based on the single-view area corresponding to each of the multiple sets of fruit information, the overall area is determined, and based on the segmentation results corresponding to each of the multiple sets of fruit information, the particle density of the fruit to be tested is determined, wherein the overall area includes the overall fruit area of ​​the fruit to be tested and / or the overall particle area of ​​each of the at least some fruit particles. Based on the overall area and the particle density, the weight information of the fruit to be tested is determined, and the weight information includes the fruit weight of the fruit to be tested and / or the particle weight of each of the at least some fruit particles.

2. The weight determination method according to claim 1, characterized in that, The overall area includes the overall particle area of ​​each of the at least some fruit particles, and the weight information includes the particle weight of each of the at least some fruit particles; Determining the weight information of the fruit to be tested based on the overall area and the particle density includes: For each of the at least some fruit particles, the product of the overall particle area, the particle density, and a preset correction coefficient is calculated to obtain the particle weight of the fruit particle. The preset correction coefficient is used to indicate the theoretical weight of a single fruit particle.

3. The weight determination method according to claim 1 or 2, characterized in that, The overall area includes the overall particle area of ​​all the fruit particles of the fruit to be tested, and the weight information includes the weight of the fruit. Before determining the weight information of the fruit to be tested based on the overall area and the particle density, the method further includes: Based on the segmentation results, the number of fruit particles in the fruit to be tested is determined. Determining the weight information of the fruit to be tested based on the overall area and the particle density includes: Based on the number of particles, the individual particle weights of all the fruit particles in the test fruit are summed to obtain the summed result as the fruit weight. The particle weight of each fruit particle in the test fruit is equal to the product of the overall particle area of ​​the fruit particle, the particle density, and a preset correction coefficient. The preset correction coefficient is used to indicate the theoretical weight of a single fruit particle.

4. The weight determination method according to claim 1 or 2, characterized in that, The segmentation result includes a first image region where each fruit particle of the fruit to be tested is located. The step of determining the particle density of the fruit to be tested based on the segmentation results corresponding to each of the multiple sets of fruit information includes: Based on the distance between the first image regions in all the segmentation results, the average spacing between the fruit particles of the fruit to be tested is determined. The particle density is determined based on the average spacing, and the particle density is inversely proportional to the average spacing.

5. The weight determination method according to claim 1 or 2, characterized in that, The segmentation result includes a first image region where each fruit particle of the fruit to be tested is located, and the single-view area includes the single-view particle area of ​​each of the at least some fruit particles. Determining the single-view area corresponding to the group of fruit information based on the segmentation result and the depth data in the group of fruit information includes: For each of the at least some fruit particles, based on the depth data, the two-dimensional pixel area of ​​the first image region where the fruit particle is located is converted into a three-dimensional physical area to obtain the single-view particle area of ​​the fruit particle. And / or, The segmentation result includes the second image region where the fruit to be tested is located, and the single-view area includes the single-view fruit area of ​​the fruit to be tested. Determining the single-view area corresponding to the set of fruit information based on the segmentation result and the depth data in the set of fruit information includes: Based on the depth data, the two-dimensional pixel area of ​​the second image region is converted into a three-dimensional physical area to obtain the single-view fruit area of ​​the fruit to be tested.

6. The weight determination method according to claim 1 or 2, characterized by, The determination of the overall area based on the single-view area corresponding to each of the multiple sets of fruit information includes: The single-view area corresponding to each of the multiple sets of fruit information is fused to obtain the overall area.

7. The weight determination method according to claim 1 or 2, characterized by, The step of performing image segmentation on the color image in the set of fruit information to obtain the segmentation result of the fruit to be tested includes: Target detection is performed on the color image in the set of fruit information to determine the target detection area where the fruit to be tested is located; Semantic segmentation is performed on the target detection region to obtain a second image region containing the fruit to be tested from the target detection region; The second image region is segmented to obtain the first image region containing each fruit particle of the fruit to be tested from the second image region; The segmentation result includes the first image region or includes both the first image region and the second image region.

8. The weight determination method according to claim 1 or 2, characterized by, The multiple sets of fruit information were obtained through the following methods: A color depth camera moves horizontally within the planting area of ​​the fruit to be tested, with the center of the field of view moving in the horizontal direction, to collect fruit information from multiple perspectives, thereby obtaining multiple sets of fruit information. The field of view ranges of any two adjacent perspectives in the multiple perspectives partially overlap with each other.

9. A method of determining the ripeness of a fruit, characterized in that Including the weight determination method as described in any one of claims 1-8, wherein the weight information includes the fruit weight, and the fruit maturity determination method further includes: Based on the fruit weight and a preset correlation model, the maturity of the fruit to be tested is determined. The preset correlation model is used to indicate the correlation between the preset fruit weight and the preset maturity.

10. A weight determination apparatus, characterized by, include: The acquisition module is used to acquire multiple sets of fruit information of the fruit to be tested. The fruit information includes color images and depth data. Different sets of fruit information are acquired from different perspectives for the fruit to be tested. The fruit to be tested is a fruit with fruit particles. The segmentation module is used to perform image segmentation on the color image in each of the multiple sets of fruit information to obtain the segmentation result of the fruit to be tested, and to determine the single-view area corresponding to the set of fruit information based on the segmentation result and the depth data in the set of fruit information, wherein the single-view area includes the single-view fruit area of ​​the fruit to be tested and / or the single-view particle area of ​​at least some of the fruit particles of the fruit to be tested. The first determining module is used to determine the overall area based on the single-view area corresponding to each of the multiple sets of fruit information, and to determine the particle density of the fruit to be tested based on the segmentation results corresponding to each of the multiple sets of fruit information, wherein the overall area includes the overall fruit area of ​​the fruit to be tested and / or the overall particle area of ​​each of the at least some fruit particles. The second determining module is used to determine the weight information of the fruit to be tested based on the overall area and the particle density, wherein the weight information includes the fruit weight of the fruit to be tested and / or the particle weight of each of the at least some fruit particles.

11. A fruit ripeness determination apparatus characterized by comprising: Including the weight determination device as described in claim 10, wherein the weight information includes the fruit weight, the fruit ripeness determination device further includes: The third determining module is used to determine the maturity of the fruit to be tested based on the fruit weight and a preset correlation model, wherein the preset correlation model is used to indicate the correlation between the preset fruit weight and the preset maturity.

12. An electronic device comprising a processor and a memory, characterized in that The memory stores computer program instructions, which, when executed by the processor, are used to perform the weight determination method as described in any one of claims 1-8 or the fruit maturity determination method as described in claim 9.

13. A storage medium on which program instructions are stored, characterized in that, The program instructions, when executed, are used to perform the weight determination method as described in any one of claims 1-8 or the fruit maturity determination method as described in claim 9.

14. A computer program product comprising computer program instructions, characterised in that, The computer program instructions, when executed, are used to perform the weight determination method as described in any one of claims 1-8 or the fruit maturity determination method as described in claim 9.

Images