Honey pomelo growth process monitoring system and method based on machine vision

Abstract

The invention discloses a honey pomelo growth process monitoring system and method based on machine vision, and the system comprises a honey pomelo growth environment obtaining unit which is used for obtaining the environment information of the environment where honey pomelo is located, and transmitting the environment information of the environment where the honey pomelo is located to a honey pomelo first growth change unit; the honey pomelo first growth change unit is used for determining a first growth change amount of the honey pomelo in a target time period based on the environment information and transmitting the first growth change amount of the honey pomelo in the target time period to the honey pomelo growth process processing unit, and the target time period comprises a first moment and a second moment. According to the invention, the technical problem of inaccurate monitoring of the growth process of the honey pomelo caused by single monitoring method of the growth process of the honey pomelo in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision-based honey pomelo growth process monitoring, and in particular to a machine vision-based honey pomelo growth process monitoring system and method. Background Art

[0002] In recent years, with the continuous advancement of information technology and the vigorous development of smart agricultural technology, the growth process monitoring of modern agriculture has become a key technology in many important agricultural science and technology development directions. Intelligent monitoring of the growth process of pomelo plays a certain role in the development of smart agriculture. Intelligent monitoring of the growth process of pomelo can determine the growth of pomelo and save a lot of manpower and financial resources. However, at present, the growth process of pomelo is monitored by image classification or sensor monitoring of pomelo, and the monitoring of the growth process of pomelo is inaccurate. Therefore, a system and method for monitoring the growth process of pomelo based on machine vision are proposed to solve the technical problem that the growth process monitoring method of pomelo in related technologies is single, resulting in inaccurate monitoring of the growth process of pomelo. Summary of the invention

[0003] The embodiments of the present invention provide a system and method for monitoring the growth process of honey pomelo based on machine vision, which solves the technical problem that the monitoring method for the growth process of honey pomelo in the related art is single, resulting in inaccurate monitoring of the growth process of honey pomelo.

[0004] According to one aspect of an embodiment of the present invention, a machine vision-based honey pomelo growth process monitoring system is provided, which may include: a honey pomelo growth environment acquisition unit, a honey pomelo first growth change unit, a honey pomelo image information acquisition unit and a honey pomelo growth process processing unit.

[0005] The honey pomelo growth environment acquisition unit is used to obtain environmental information of the environment in which the honey pomelo is located, and transmit the environmental information of the environment in which the honey pomelo is located to the honey pomelo first growth change unit.

[0006] The first growth change unit of honey pomelo is used to determine the first growth change amount of honey pomelo in a target time period based on environmental information, and transmit the first growth change amount of honey pomelo in the target time period to the honey pomelo growth process processing unit, wherein the target time period includes a first moment and a second moment.

[0007] The honey pomelo image information acquisition unit is used to acquire the first image information and the second image information of the honey pomelo in the target time period, and transmit the first image information and the second image information to the honey pomelo growth process processing unit.

[0008] The honey pomelo growth process processing unit is used to input the first image information and the second image information into the target convolutional neural network, obtain the first classification result of the first image information and the second classification result of the second image information, determine the second growth change of the honey pomelo based on the first classification result and the second classification result, and determine the target growth value of the honey pomelo based on the first growth change and the second growth change of the honey pomelo.

[0009] Optionally, based on environmental information, determining the first growth change of the honey pomelo in the target time period includes: measuring a first growth value of the honey pomelo at a first moment and a second growth value of the honey pomelo at a second moment under the environmental information; and determining the first growth change of the honey pomelo in the target time period based on the first growth value and the second growth value.

[0010] Optionally, determining a first growth change amount of the honey pomelo in the target time period based on the first growth value and the second growth value includes: determining a difference between the second growth value and the first growth value as the first growth change amount of the honey pomelo in the target time period.

[0011] Optionally, before inputting the first image information and the second image information into the target convolutional neural network to obtain a first classification result of the first image information and a second classification result of the second image information, the system also includes: a historical image acquisition unit, used to acquire historical image information of honey pomelo; a classification unit, used to classify the historical image information of honey pomelo according to the diameter size of the honey pomelo to obtain a classification data set of honey pomelo.

[0012] Optionally, the target convolutional neural network includes: an input layer, a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a third convolutional layer, a third pooling layer, a fourth convolutional layer, a fourth pooling layer, a first cascade layer, a second cascade layer, a first flattening layer, a second flattening layer, a first fully connected layer, a second fully connected layer, a first classification layer, a second classification layer and an average classification layer.

[0013] Optionally, the processing process of the target convolutional neural network is as follows: the first cascade layer cascades the features after the first pooling layer and the features after the third pooling layer to obtain a first fused feature; the second cascade layer cascades the features after the second pooling layer and the features after the fourth pooling layer to obtain a second fused feature; the first flattening layer flattens the first fused feature to obtain a first flattened feature; the second flattening layer flattens the second fused feature to obtain a second flattened feature; the first fully connected layer processes the first flattened feature to obtain a first processed feature; the second fully connected layer processes the second flattened feature to obtain a second processed feature; the first classification layer classifies the first processed feature to obtain a third classification result; the second classification layer classifies the second processed feature to obtain a fourth classification result; the averaging layer averages the third classification result and the fourth classification result to obtain a first classification result or a second classification result.

[0014] Optionally, determining a second growth change of the honey pomelo based on the first classification result and the second classification result includes: obtaining the second growth change of the honey pomelo based on the type of honey pomelo corresponding to the maximum value in the first classification result and the type of honey pomelo corresponding to the maximum value in the first classification result.

[0015] Optionally, based on the first growth change and the second growth change of the honey pomelo, a target growth value of the honey pomelo is determined, including: based on the first growth change and the second growth change, a first gap value is determined; when the first gap value is less than a preset threshold, the growth value of the first growth change at the second moment is determined as the target growth value of the honey pomelo.

[0016] In the embodiment of the present invention, the environmental information of the environment in which the honey pomelo is located is obtained through the honey pomelo growth environment acquisition unit, and the environmental information of the environment in which the honey pomelo is located is transmitted to the honey pomelo first growth change unit; the honey pomelo first growth change unit determines the first growth change amount of the honey pomelo in the target time period based on the environmental information, and transmits the first growth change amount of the honey pomelo in the target time period to the honey pomelo growth process processing unit, wherein the target time period includes a first moment and a second moment; the honey pomelo image information acquisition unit obtains the first image information and the second image information of the honey pomelo in the target time period, and transmits the first image information and the second image information to the honey pomelo growth process processing unit; The pomelo growth process processing unit inputs the first image information and the second image information into the target convolutional neural network to obtain a first classification result of the first image information and a second classification result of the second image information, and determines the second growth change of the pomelo based on the first classification result and the second classification result, and determines the target growth value of the pomelo based on the first growth change and the second growth change of the pomelo, thereby solving the technical problem that the growth process monitoring method of the pomelo in the related art is single, resulting in inaccurate monitoring of the growth process of the pomelo, and achieves the technical effect of combining the image classification and sensor monitoring method to monitor the growth process of the pomelo and accurately monitoring the growth process of the pomelo.

[0017] Advantages of the present invention:

[0018] Compared with the existing technology, the present invention obtains the growth change value of the honey pomelo through a sensor, and processes the image information through an image classification algorithm to obtain the growth change value of the honey pomelo, compares the growth change value of the honey pomelo obtained by the sensor with the growth change value of the honey pomelo obtained by the image classification algorithm, and obtains the target growth value of the honey pomelo. By combining the two methods, the growth process of the honey pomelo is made more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0020] Figure 1 is a structural block diagram of a honey pomelo growth process monitoring system based on machine vision according to an embodiment of the present invention;

[0021] Figure 2 is a structural block diagram of a target convolutional neural network according to an embodiment of the present invention;

[0022] Figure 3 The present invention is a flowchart of a method for monitoring the growth process of honey pomelo based on machine vision according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] Example 1

[0026] According to an embodiment of the present invention, a machine vision-based honey pomelo growth process monitoring system is provided. Figure 1 FIG. 1 is a structural block diagram of a honey pomelo growth process monitoring system based on machine vision according to an embodiment of the present invention. Figure 1 As shown, a honey pomelo growth process monitoring system based on machine vision may include: a honey pomelo growth environment acquisition unit, a honey pomelo first growth change unit, a honey pomelo image information acquisition unit and a honey pomelo growth process processing unit.

[0027] The honey pomelo growth environment acquisition unit is used to obtain environmental information of the environment in which the honey pomelo is located, and transmit the environmental information of the environment in which the honey pomelo is located to the honey pomelo first growth change unit.

[0028] In this embodiment, the environmental information of the environment in which the honey pomelo is located is obtained through the soil sensor and the temperature sensor, and the obtained environmental information of the environment in which the honey pomelo is located is transmitted to the first growth change unit of the honey pomelo.

[0029] The first growth change unit of honey pomelo is used to determine the first growth change amount of honey pomelo in a target time period based on environmental information, and transmit the first growth change amount of honey pomelo in the target time period to the honey pomelo growth process processing unit, wherein the target time period includes a first moment and a second moment.

[0030] In this embodiment, according to the environmental information of the environment in which the honey pomelo is located, the first growth change of the honey pomelo in the target time period is obtained, and according to the growth amount of the honey pomelo at the first moment and the growth amount of the honey pomelo at the second moment, the first growth change of the target time period is obtained, wherein the target time period includes the time period between the first moment and the second moment, wherein the first moment is a specific time point, and the second moment is a specific time point, for example, the first moment may be 15:20 on September 20, 2019, and the second moment may be 15:20 on September 28, 2019.

[0031] The honey pomelo image information acquisition unit is used to acquire the first image information and the second image information of the honey pomelo in the target time period, and transmit the first image information and the second image information to the honey pomelo growth process processing unit.

[0032] In this embodiment, the first image information and the second image information of the honey pomelo in the target time period are captured from multiple angles by a drone, wherein the first image information is the image information captured at the first moment, and the second image information is the image information captured at the second moment.

[0033] The honey pomelo growth process processing unit is used to input the first image information and the second image information into the target convolutional neural network, obtain the first classification result of the first image information and the second classification result of the second image information, determine the second growth change of the honey pomelo based on the first classification result and the second classification result, and determine the target growth value of the honey pomelo based on the first growth change and the second growth change of the honey pomelo.

[0034] In this embodiment, the first image information and the second image information are input into the target convolutional neural network to obtain a first classification result of the first image information and a second classification result of the second image information. According to the first classification result and the second classification result, the second growth change of the honey pomelo is obtained. According to the first growth change and the second growth change of the honey pomelo, the target growth value of the honey pomelo is obtained.

[0035] Optionally, based on environmental information, determining the first growth change of the honey pomelo in the target time period includes: measuring a first growth value of the honey pomelo at a first moment and a second growth value of the honey pomelo at a second moment under the environmental information; and determining the first growth change of the honey pomelo in the target time period based on the first growth value and the second growth value.

[0036] In this embodiment, in the growth environment of honey pomelo, a first growth value of the honey pomelo at a first moment and a second growth value of the honey pomelo at a second moment are measured according to a fruit growth monitoring sensor, and a first growth change of the honey pomelo in a target time period is obtained according to the first growth value and the second growth value, wherein the first growth value and the second growth value may be the diameter or circumference of the honey pomelo, and the first growth value and the second growth value of the embodiment of the present invention are the diameters of the honey pomelo.

[0037] Optionally, determining a first growth change amount of the honey pomelo in the target time period based on the first growth value and the second growth value includes: determining a difference between the second growth value and the first growth value as the first growth change amount of the honey pomelo in the target time period.

[0038] In this embodiment, when the second growth of the honey pomelo is 17 cm and the first growth value is 12 cm, the difference between the second growth value of 17 cm and the first growth value of 12 cm of the honey pomelo is obtained, and the first growth change of the honey pomelo in the target time period is 5 cm.

[0039] Optionally, before the first image information and the second image information are input into the target convolutional neural network to obtain a first classification result of the first image information and a second classification result of the second image information, it also includes: a historical image acquisition unit, used to obtain historical image information of honey pomelo; a classification unit, used to classify the historical image information of honey pomelo according to the diameter size of the honey pomelo to obtain a classification data set of honey pomelo.

[0040] In this embodiment, historical image information of honey pomelo is obtained, and the historical image information of honey pomelo is divided into 5 categories according to the diameter of honey pomelo, and 5 classification data sets of honey pomelo are obtained. The data of the first classification data set is that the diameter of honey pomelo is 6-10 cm, the data of the second classification data set is that the diameter of honey pomelo is 10-12 cm, the data of the third classification data set is that the diameter of honey pomelo is 12-16 cm, the data of the fourth classification data set is that the diameter of honey pomelo is 16-20 cm, and the data of the fifth classification data set is that the diameter of honey pomelo is more than 20 cm. The first classification data set is named M-1, the second classification data set is named M-2, the third classification data set is named M-3, the fourth classification data set is named M-4, and the fifth classification data set is named M-5. The first classification data set contains 256 images, the second classification data set contains 306 images, the third classification data set contains 242 images, the fourth classification data set contains 375 images, and the fifth classification data set contains 106 images. In this embodiment, the images of the first classification data set, the second classification data set, the third classification data set, the fourth classification data set and the fifth classification data set are randomly divided into a training group, a verification group and a test group according to the proportion (80%:10%:10%).

[0041] Optionally, the target convolutional neural network includes: an input layer, a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a third convolutional layer, a third pooling layer, a fourth convolutional layer, a fourth pooling layer, a first cascade layer, a second cascade layer, a first flattening layer, a second flattening layer, a first fully connected layer, a second fully connected layer, a first classification layer, a second classification layer and an average classification layer.

[0042] In this embodiment, the target convolutional neural network includes 18 layers, namely, input layer, first convolutional layer, first pooling layer, second convolutional layer, second pooling layer, third convolutional layer, third pooling layer, fourth convolutional layer, fourth pooling layer, first cascade layer, second cascade layer, first flattening layer, second flattening layer, first fully connected layer, second fully connected layer, first classification layer, second classification layer and average classification layer. The structure of the first input layer is 256×256×3, the structure of the first convolutional layer and the third convolutional layer is the same as 256×256×32, the structure of the second convolutional layer and the fourth convolutional layer is the same as 256×256×64, and the pooling layer is the maximum pooling layer. Among them, the convolutional layer is also called the feature extraction layer, which mainly extracts features from the input image. The convolutional layer generally uses multiple convolution kernels smaller than the image size for convolution operations, and performs convolution operations on the input image through these convolution kernels to obtain a feature map. The convolution operation is actually an inner product, and the process is shown in the following formula: in, represents the output of the convolutional layer, which is the jth feature map of the mth layer. Represents the input image, which is the i-th feature map of the m-1th layer, represents the convolution kernel, that is, the jth convolution kernel of the i-th feature map convolution generated by the m-th layer and the m-1-th layer, Represents the bias, that is, the bias of the jth feature map of the mth layer. i∈Ri represents all the previous feature maps connected, and f{·} represents the activation function. The pooling layer is also called the downsampling layer. The pooling layer often appears alternately with the convolution layer in the convolutional neural network. After the convolution operation, the amount of data increases greatly compared to the previous one. If the amount of data is retained, then as the number of network layers increases, the amount of data will increase greatly, which will cause data redundancy. Connecting the pooling operation after the convolution operation can play a role in dimensionality reduction, reduce the number of parameters of the entire network, and ensure local invariance and extract the main information of the feature map. The process is shown in the following formula: in represents the output of the pooling layer, represents the input pooling layer feature, down represents the downsampling operation on the input, and f(·) represents the activation function. Indicates bias. The pooling layer mainly includes the maximum pooling layer (Maxpooling Layer), the average pooling layer (Meanpooling Layer), etc. The maximum pooling operation refers to using a sliding window to take the maximum value of all values ​​in the image as the pooling result; the average pooling operation refers to using a sliding window to take the average value of all values ​​in the image as the pooling result.

[0043] The cascade layer fuses the features of different layers through the concatenate function to improve the classification accuracy of the image.

[0044] The flattening layer converts the multi-dimensional feature map output by the cascade layer into a one-dimensional vector.

[0045] The fully connected layer outputs the features of the flattened layer to a specific dimension.

[0046] The first classification layer and the second classification layer use the Softmax classifier, where the formula of the Softmax classifier algorithm is as follows: Among them, β is the classification Z represented by a numerical value α , α∈(0,β], λ represents a category in α, g λ Represents the value of this category, P(Z λ ) is the probability of this classification.

[0047] Optionally, the processing process of the target convolutional neural network is as follows: the first cascade layer cascades the features after the first pooling layer and the features after the third pooling layer to obtain a first fused feature; the second cascade layer cascades the features after the second pooling layer and the features after the fourth pooling layer to obtain a second fused feature; the first flattening layer flattens the first fused feature to obtain a first flattened feature; the second flattening layer flattens the second fused feature to obtain a second flattened feature; the first fully connected layer processes the first flattened feature to obtain a first processed feature; the second fully connected layer processes the second flattened feature to obtain a second processed feature; the first classification layer classifies the first processed feature to obtain a third classification result; the second classification layer classifies the second processed feature to obtain a fourth classification result; the averaging layer averages the third classification result and the fourth classification result to obtain a first classification result or a second classification result.

[0048] In this embodiment, Figure 2 is a structural block diagram of a target convolutional neural network according to an embodiment of the present invention, such as Figure 2 As shown in the figure, the first cascade layer cascades the features after the first pooling layer and the features after the third pooling layer to obtain the first fused features; the second cascade layer cascades the features after the second pooling layer and the features after the fourth pooling layer to obtain the second fused features; the first flattening layer flattens the first fused features to obtain the first flattened features; the second flattening layer flattens the second fused features to obtain the second flattened features; the first fully connected layer processes the first flattened features to obtain the first processed features; the second fully connected layer processes the second flattened features to obtain the second processed features; the first classification layer classifies the first processed features to obtain the third classified features. The first classification result and the second classification result are specific values. For example, the first classification result can be 0.02, 0.04, 0.92, 0.01 and 0.01, and the second classification result can be 0.00, 0.03, 0.02, 0.93 and 0.02.

[0049] Optionally, determining a second growth change of the honey pomelo based on the first classification result and the second classification result includes: obtaining the second growth change of the honey pomelo based on the type of honey pomelo corresponding to the maximum value in the first classification result and the type of honey pomelo corresponding to the maximum value in the first classification result.

[0050] In this embodiment, the second growth change of the pomelo is determined by the difference between the median value of the type of pomelo corresponding to the maximum value in the second classification result and the median value of the type of pomelo corresponding to the maximum value in the first classification result. For example, the median value of the type of pomelo corresponding to the maximum value in the second classification result is 18 cm, and the median value of the type of pomelo corresponding to the maximum value in the first classification result is 14 cm. After subtracting the difference, the second growth change is 4 cm.

[0051] Optionally, based on the first growth change and the second growth change of the honey pomelo, a target growth value of the honey pomelo is determined, including: based on the first growth change and the second growth change, a first gap value is determined; when the first gap value is less than a preset threshold, the growth value of the first growth change at the second moment is determined as the target growth value of the honey pomelo.

[0052] In this embodiment, a first gap value between the first growth change amount and the second growth change amount is obtained based on the first growth change amount and the second growth change amount. If the first gap value is less than the preset gap value, the growth value at the second moment corresponding to the first growth change amount is determined as the growth value of the honey pomelo. For example, the growth value of the honey pomelo at the second moment of the first growth change amount, 17 cm, is determined as the target growth value of the honey pomelo.

[0053] In this embodiment, the environmental information of the environment in which the honey pomelo is located is obtained through the honey pomelo growth environment acquisition unit, and the environmental information of the environment in which the honey pomelo is located is transmitted to the honey pomelo first growth change unit; the honey pomelo first growth change unit determines the first growth change amount of the honey pomelo in the target time period based on the environmental information, and transmits the first growth change amount of the honey pomelo in the target time period to the honey pomelo growth process processing unit, wherein the target time period includes a first moment and a second moment; the honey pomelo image information acquisition unit obtains the first image information and the second image information of the honey pomelo in the target time period, and transmits the first image information and the second image information to the honey pomelo growth process processing unit; the honey pomelo The growth process processing unit inputs the first image information and the second image information into the target convolutional neural network to obtain a first classification result of the first image information and a second classification result of the second image information, and determines the second growth change of the honey pomelo based on the first classification result and the second classification result, and determines the target growth value of the honey pomelo based on the first growth change and the second growth change of the honey pomelo, thereby solving the technical problem that the growth process monitoring method of the honey pomelo in the related art is single, resulting in inaccurate monitoring of the growth process of the honey pomelo, and achieves the technical effect of combining the image classification and sensor monitoring method to monitor the growth process of the honey pomelo and accurately monitoring the growth process of the honey pomelo.

[0054] Example 2

[0055] According to an embodiment of the present invention, a method for monitoring the growth process of honey pomelo based on machine vision is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system comprising at least one set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0056] Figure 3 FIG. 1 is a flow chart of a method for monitoring the growth process of honey pomelo based on machine vision according to an embodiment of the present invention. Figure 3 As shown, the method may include the following steps:

[0057] Step S301, obtaining environmental information of the environment in which the honey pomelo is located.

[0058] In the technical solution provided in the above step S301 of the present invention, the environmental information of the environment in which the honey pomelo is located is obtained through a soil sensor and a temperature sensor.

[0059] Step S302: determining a first growth change amount of honey pomelo in a target time period based on the environmental information.

[0060] In the technical solution provided in the above step S302 of the present invention, the first growth change of the honey pomelo in the target time period is obtained according to the environmental information of the environment in which the honey pomelo is located, and the first growth change of the target time period is obtained according to the growth of the honey pomelo at the first moment and the growth of the honey pomelo at the second moment, wherein the target time period includes the time period between the first moment and the second moment, wherein the first moment is a specific time point, and the second moment is a specific time point, for example, the first moment may be 15:20 on September 20, 2019, and the second moment may be 15:20 on September 28, 2019.

[0061] Step S303, obtaining first image information and second image information of honey pomelo in a target time period.

[0062] In the technical solution provided in the above step S303 of the present invention, the first image information and the second image information of the honey pomelo in the target time period are photographed in multiple directions by a drone, wherein the first image information is the image information photographed at the first moment, and the second image information is the image information photographed at the second moment.

[0063] Step S304: input the first image information and the second image information into the target convolutional neural network to obtain a first classification result of the first image information and a second classification result of the second image information.

[0064] In the technical solution provided in the above step S304 of the present invention, the first image information and the second image information are input into the target convolutional neural network, and the target convolutional neural network processes the first image information and the second image information respectively to obtain a first classification result of the first image information and a second classification result of the second image information.

[0065] Step S305: determining a second growth variation of the honey pomelo based on the first classification result and the second classification result.

[0066] In the technical solution provided in the above step S305 of the present invention, the first classification result and the second classification result are processed to obtain the second growth change amount of honey pomelo.

[0067] Step S306, determining a target growth value of the honey pomelo based on the first growth change amount and the second growth change amount of the honey pomelo.

[0068] In the technical solution provided in the above step S306 of the present invention, the target growth value of the honey pomelo is obtained according to the first growth change amount and the second growth change amount of the honey pomelo.

[0069] In an embodiment of the present invention, environmental information of the environment in which the honey pomelo is located is obtained; based on the environmental information, a first growth change amount of the honey pomelo in a target time period is determined; first image information and second image information of the honey pomelo in the target time period are obtained; the first image information and the second image information are input into a target convolutional neural network to obtain a first classification result of the first image information and a second classification result of the second image information; based on the first classification result and the second classification result, a second growth change amount of the honey pomelo is determined; based on the first growth change amount and the second growth change amount of the honey pomelo, a target growth value of the honey pomelo is determined, thereby solving the technical problem that the method for monitoring the growth process of the honey pomelo in the related art is single, resulting in inaccurate monitoring of the growth process of the honey pomelo, and achieving the technical effect of combining image classification and sensor monitoring to monitor the growth process of the honey pomelo and accurately monitoring the growth process of the honey pomelo.

[0070] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0071] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0072] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0073] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0074] In addition, each functional unit in each embodiment of the present invention may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0075] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc., which can store program code.

[0076] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A honey pomelo growth process monitoring system based on machine vision, characterized in that: include: A honey pomelo growth environment acquisition unit, a honey pomelo first growth change unit, a honey pomelo image information acquisition unit and a honey pomelo growth process processing unit; A honey pomelo growth environment acquisition unit, used to acquire environmental information of the environment in which the honey pomelo is located, and transmit the environmental information of the environment in which the honey pomelo is located to the honey pomelo first growth change unit; A honey pomelo first growth change unit, configured to determine a first growth change amount of the honey pomelo in a target time period based on the environmental information, and transmit the first growth change amount of the honey pomelo in the target time period to the honey pomelo growth process processing unit, wherein the target time period includes a first moment and a second moment; A honey pomelo image information acquisition unit, used to acquire first image information and second image information of the honey pomelo in the target time period, and transmit the first image information and the second image information to the honey pomelo growth process processing unit; A honey pomelo growth process processing unit is used to input the first image information and the second image information into a target convolutional neural network to obtain a first classification result of the first image information and a second classification result of the second image information, determine the second growth change of the honey pomelo based on the first classification result and the second classification result, and determine the target growth value of the honey pomelo based on the first growth change and the second growth change of the honey pomelo.

2. The machine vision-based honey pomelo growth process monitoring system according to claim 1, characterized in that: The determining, based on the environmental information, a first growth change amount of the honey pomelo in a target time period includes: Under the environmental information, measuring a first growth value of the honey pomelo at the first moment and a second growth value of the honey pomelo at the second moment; Based on the first growth value and the second growth value, a first growth change amount of the honey pomelo in the target time period is determined.

3. The machine vision-based honey pomelo growth process monitoring system according to claim 2 is characterized in that: The determining, based on the first growth value and the second growth value, a first growth change amount of the honey pomelo in the target time period includes: The difference between the second growth value and the first growth value is determined as the first growth change amount of the honey pomelo in the target time period.

4. The machine vision-based honey pomelo growth process monitoring system according to claim 1, characterized in that: Before inputting the first image information and the second image information into a target convolutional neural network to obtain a first classification result of the first image information and a second classification result of the second image information, the system further includes: A historical image acquisition unit, used to acquire historical image information of the honey pomelo; The classification unit is used to classify the historical image information of the honey pomelo according to the diameter of the honey pomelo to obtain a classification data set of the honey pomelo.

5. The machine vision-based honey pomelo growth process monitoring system according to claim 1, characterized in that: The target convolutional neural network includes: an input layer, a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a third convolutional layer, a third pooling layer, a fourth convolutional layer, a fourth pooling layer, a first cascade layer, a second cascade layer, a first flattening layer, a second flattening layer, a first fully connected layer, a second fully connected layer, a first classification layer, a second classification layer and an average classification layer.

6. The machine vision-based honey pomelo growth process monitoring system according to claim 1, characterized in that: The processing process of the target convolutional neural network is as follows: The first cascade layer cascades the features after the first pooling layer and the features after the third pooling layer to obtain a first fusion feature; The second cascade layer cascades the features after the second pooling layer and the features after the fourth pooling layer to obtain a second fusion feature; The first flattening layer flattens the first fused feature to obtain a first flattened feature; The second flattening layer flattens the second fused feature to obtain a second flattened feature; The first fully connected layer processes the first flattened feature to obtain a first processed feature; The second fully connected layer processes the second flattened feature to obtain a second processed feature; The first classification layer classifies the first processing feature to obtain a third classification result; The second classification layer classifies the second processing feature to obtain a fourth classification result; The averaging layer averages the third classification result and the fourth classification result to obtain the first classification result or the second classification result.

7. The machine vision-based honey pomelo growth process monitoring system according to claim 1, characterized in that: The determining, based on the first classification result and the second classification result, a second growth change amount of the honey pomelo comprises: Based on the type of the honey pomelo corresponding to the maximum value in the first classification result and the type of the honey pomelo corresponding to the maximum value in the first classification result, a second growth change amount of the honey pomelo is obtained.

8. The machine vision-based honey pomelo growth process monitoring system according to claim 1, characterized in that: The step of determining a target growth value of the honey pomelo based on the first growth change amount and the second growth change amount of the honey pomelo comprises: Determining a first gap value based on the first growth change amount and the second growth change amount; When the first gap value is smaller than a preset threshold, the growth value of the first growth change at the second moment is determined as the target growth value of the honey pomelo.

9. A method for monitoring the growth process of honey pomelo based on machine vision, characterized in that: include: Obtain the environmental information of the environment where the pomelo is located; Based on the environmental information, determining a first growth change amount of the honey pomelo in a target time period; Acquire first image information and second image information of the honey pomelo in the target time period; Inputting the first image information and the second image information into a target convolutional neural network to obtain a first classification result of the first image information and a second classification result of the second image information; Determining a second growth change amount of the honey pomelo based on the first classification result and the second classification result; A target growth value of the honey pomelo is determined based on the first growth change amount and the second growth change amount of the honey pomelo.

Images