Fruit nondestructive testing method, system and equipment based on hyperspectrum
By converting hyperspectral fruit images, fruit stem images and fruit leaf images into combined fruit feature maps and building a detection model with plant component location information, the problem of low detection accuracy caused by the single training data of the fruit sugar level detection model in the prior art is solved, and higher detection accuracy and diversity are achieved.
Patent Information
- Application Number
- CN202411700795.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-25
- Publication Date
- 2025-03-18
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the prior art, the training data of the fruit sugar detection model is single, resulting in low detection accuracy.
By obtaining hyperspectral fruit images, fruit stem images and fruit leaf images, it is converted into a combined fruit feature map, extracting the position information of the plant components, combining them into fruit coding sequence and position information, and constructing a fruit non-destructive detection model to detect the damage probability and position of the fruit.
It improves the accuracy of model recognition, provides a variety of model training data, enhances the accuracy of fruit detection, and reduces equipment costs.
Smart Images
Figure CN119649362A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a hyperspectral-based fruit nondestructive detection method, a hyperspectral-based fruit nondestructive detection system, a computer device and a computer-readable storage medium. Background Art
[0002] Hyperspectral imaging technology images the sample with high spatial resolution and collects spectral information at each pixel to obtain a hyperspectral image. By analyzing the hyperspectral image, information such as the sample's composition, structural characteristics, and surface morphology can be obtained.
[0003] In the prior art, hyperspectral data and sugar content data of fruits are collected, the hyperspectral data are preprocessed, and the hyperspectral data are used as training sample sugar content data as labels to train the initial fruit sugar content detection model to obtain the fruit sugar content detection model; the hyperspectral data to be detected is input into the fruit sugar content detection model to obtain the sugar content of the fruit to be detected. By collecting hyperspectral data of fruits and combining them with convolutional neural network models, non-destructive sugar content detection of fruits is achieved. However, the existing model training data is single and the detection accuracy is not high. Summary of the invention
[0004] In view of the above problems, embodiments of the present invention are proposed to provide a hyperspectral-based non-destructive fruit detection method, a hyperspectral-based non-destructive fruit detection system, a computer device and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.
[0005] In order to solve the above problems, the embodiment of the present invention discloses a non-destructive detection method of fruit based on hyperspectral, the method comprising:
[0006] Obtaining hyperspectral fruit images and corresponding fruit stem images and fruit leaf images;
[0007] Converting the hyperspectral fruit image and the corresponding fruit stalk image and fruit leaf image into a combined fruit feature map;
[0008] Extracting plant component position information of the fruit image, fruit stalk image, and fruit leaf image;
[0009] Converting the combined fruit characteristic map into a combined fruit coding sequence;
[0010] combining the combined fruit coding sequence and the plant component position information to obtain the combined fruit component position information;
[0011] Combining the combined fruit component position information and the pre-detection results into a fruit training sample;
[0012] Building a fruit nondestructive detection model through the training samples to obtain a trained fruit nondestructive detection model;
[0013] The new hyperspectral fruit images, fruit stem images and fruit leaf images are transmitted to the trained fruit nondestructive detection model to obtain the fruit nondestructive detection category.
[0014] Preferably, converting the hyperspectral fruit image and the corresponding fruit stalk image and fruit leaf image into a combined fruit feature map comprises:
[0015] A hyperspectral fruit image collected from the fruit image;
[0016] Acquire a fruit stalk image and a fruit leaf image within a preset horizontal distance and a preset vertical distance of the hyperspectral fruit image;
[0017] The hyperspectral fruit image and the fruit stalk image and fruit leaf image within a preset horizontal distance and a preset vertical distance are combined to form a fruit feature map.
[0018] Preferably, the extracting the plant component position information of the fruit image, the fruit stalk image, and the fruit leaf image includes:
[0019] Marking the position information of the first component of the hyperspectral fruit image;
[0020] Marking the second component position information of the fruit stalk image and the fruit leaf image within a preset horizontal distance and a preset vertical distance;
[0021] The first component position information and the second component position information are associated to obtain plant component position information.
[0022] Preferably, converting the combined fruit characteristic graph into a combined fruit coding sequence comprises:
[0023] Converting the hyperspectral fruit image into a fruit pixel feature coding sequence, wherein the fruit pixel feature coding sequence includes corresponding hyperspectral information;
[0024] Converting the fruit stalk image and the fruit leaf image into a fruit stalk and fruit leaf pixel coding sequence;
[0025] The fruit pixel feature coding sequence and the fruit stalk and leaf pixel coding sequence are associated to form a combined fruit coding sequence.
[0026] Preferably, the combining the combined fruit coding sequence and the plant component position information to obtain the combined fruit component position information comprises:
[0027] Adding the first component position information to the end of the sequence of the fruit pixel feature coding sequence in the combined fruit coding sequence;
[0028] The second component position information is added to the end of the sequence of the fruit stalk and leaf pixel coding sequence in the combined fruit coding sequence to obtain the combined fruit component position information.
[0029] Preferably, the fruit nondestructive detection category includes the fruit damage probability and the fruit damage location; the new hyperspectral fruit image, fruit stem image and fruit leaf image are transmitted to the trained fruit nondestructive detection model to obtain the fruit nondestructive detection category, including:
[0030] After transmitting the new hyperspectral fruit images, fruit stem images and fruit leaf images to the trained fruit nondestructive detection model, the output fruit damage probability and fruit damage location are obtained.
[0031] The embodiment of the present invention discloses a hyperspectral-based nondestructive fruit detection system, the system comprising:
[0032] The first acquisition module is used to acquire the hyperspectral fruit image and the corresponding fruit stem image and fruit leaf image;
[0033] A first conversion module is used to convert the hyperspectral fruit image and the corresponding fruit stalk image and fruit leaf image into a combined fruit feature image;
[0034] A first extraction module is used to extract the plant parts position information of the fruit image, the fruit stalk image and the fruit leaf image;
[0035] A conversion module, used for converting the combined fruit characteristic map into a combined fruit coding sequence;
[0036] A combination module, used for combining the combined fruit coding sequence and the plant component position information to obtain the combined fruit component position information;
[0037] A composition module, used for combining the combined fruit component position information and the pre-detection results into a fruit training sample;
[0038] A construction module, used to construct a fruit nondestructive detection model through the training samples to obtain a trained fruit nondestructive detection model;
[0039] The transmission module is used to transmit new hyperspectral fruit images, fruit stem images and fruit leaf images to the trained fruit non-destructive detection model to obtain the fruit non-destructive detection category.
[0040] Preferably, the first conversion module comprises:
[0041] A collection submodule, used for collecting a hyperspectral fruit image in the fruit image;
[0042] An acquisition submodule, used for acquiring a fruit stalk image and a fruit leaf image within a preset horizontal distance and a preset vertical distance of the hyperspectral fruit image;
[0043] The composition submodule is used to combine the hyperspectral fruit image and the fruit stalk image and fruit leaf image within a preset horizontal distance and a preset vertical distance into a fruit characteristic map.
[0044] The embodiment of the present invention discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned hyperspectral-based non-destructive fruit detection method when executing the computer program.
[0045] The embodiment of the present invention discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned hyperspectral-based non-destructive fruit detection method are implemented.
[0046] The embodiments of the present invention include the following advantages:
[0047] The hyperspectral-based nondestructive fruit detection method in the embodiment of the present invention comprises the following steps: obtaining a hyperspectral fruit image and a corresponding fruit stalk image and fruit leaf image; converting the hyperspectral fruit image and the corresponding fruit stalk image and fruit leaf image into a combined fruit feature map; extracting the plant component position information of the fruit image, fruit stalk image and fruit leaf image; converting the combined fruit feature map into a combined fruit coding sequence; combining the combined fruit coding sequence and the plant component position information to obtain combined fruit component position information; combining the combined fruit component position information and a pre-detection result into a fruit training sample; constructing a fruit nondestructive detection model through the training sample to obtain a trained fruit nondestructive detection model; transmitting new hyperspectral fruit images, fruit stalk images and fruit leaf images to the trained fruit nondestructive detection model to obtain a fruit nondestructive detection category. Improve the accuracy of model recognition. Specifically, the fruit image is a hyperspectral fruit image, while the images of the fruit stalks and leaves are ordinary images. This can effectively reduce equipment costs and improve recognition accuracy. The images of the fruit and the fruit stalks and leaves near the fruit are converted into hyperspectral and ordinary pixel sequence data. The location information of the grape fruit is superimposed on the hyperspectral and ordinary pixel sequence data as the training set of the artificial intelligence model to obtain the probability of fruit damage and the location of fruit damage, providing a variety of model training data and improving the accuracy of fruit detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0049] Figure 1Schematic diagram of a hyperspectral-based nondestructive fruit detection method according to an embodiment of the present invention;
[0050] Figure 2 is a schematic diagram of a fruit branch image according to an embodiment of the present invention;
[0051] Figure 3 is a schematic diagram of a fruit leaf image according to an embodiment of the present invention;
[0052] Figure 4 It is a structural block diagram of an embodiment of a fruit nondestructive detection system based on hyperspectral according to an embodiment of the present invention;
[0053] Figure 5 The diagram is a diagram of the internal structure of a computer device according to an embodiment. DETAILED DESCRIPTION
[0054] In order to make the technical problems, technical solutions and beneficial effects solved by the embodiments of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0055] In a core concept of an embodiment of the present invention, the images of fruits and fruit stalks and fruit leaves near the fruits are used as the original elements of the initial training set to improve the accuracy of model recognition. Specifically, the fruit images are hyperspectral fruit images, while the images of fruit stalks and fruit leaves are ordinary images, which can effectively reduce equipment costs and improve recognition accuracy. The images of fruits and fruit stalks and fruit leaves near the fruits are converted into hyperspectral and ordinary pixel sequence data, and the position information of grape fruits (such as 3-8 branches of fruiting branches, pixel distance information between fruit stalks and fruiting branches, and pixel distance information between fruit leaves and fruiting branches) are superimposed on the hyperspectral and ordinary pixel sequence data as the training set of the artificial intelligence model to obtain the probability of fruit damage and the location of fruit damage, thereby providing a variety of model training data and improving the accuracy of fruit detection.
[0056] Reference Figure 1 , shows a schematic diagram of an embodiment of a non-destructive detection method of fruit based on hyperspectral according to an embodiment of the present invention, which may specifically include the following steps:
[0057] Step S101, obtaining a hyperspectral fruit image and corresponding fruit stem image and fruit leaf image;
[0058] In an embodiment of the present invention, the hyperspectral-based nondestructive detection method for fruits is mainly applied to fruit sorting equipment, wherein the fruit sorting equipment may include a control system, a nondestructive detection equipment, a transmission equipment, a shunting equipment, a label pasting equipment, a packaging equipment, etc. The control system is respectively connected to the nondestructive detection equipment, the transmission equipment, the shunting equipment, the label pasting equipment, and the packaging equipment, and is used to control the operation of the nondestructive detection equipment, the transmission equipment, the shunting equipment, the label pasting equipment, and the packaging equipment; wherein the control system may run on a certain terminal hardware, and the terminal hardware is connected to various components of the laser marking machine. The terminal hardware may be various computers. The embodiment of the present invention does not limit the specific type of the terminal. The operating system of the terminal may include Android, Harmony OS, IOS, Windows Phone, Windows, etc., and the present invention does not impose too many restrictions on this.
[0059] Specifically, the nondestructive testing equipment in the embodiment of the present invention may include equipment for collecting and analyzing fruit images in two ways: machine vision and hyperspectral analysis; the hyperspectral analysis is based on visible light / near-infrared diffuse reflectance spectroscopy. By collecting the spectral information of the fruit and using the prediction model, nondestructive testing of various quality parameters of grapes can be performed. The nondestructive testing equipment mainly includes components such as spectral acquisition probes, spectral analysis devices, light source systems, controllers, and peripheral circuits. The embodiment of the present invention does not impose too many restrictions on the types and structures of the components of the nondestructive testing equipment.
[0060] Among them, the spectrum acquisition probe is mainly composed of a contoured plastic gasket, a base, a ring light source, a 74UV lens and an optical fiber. The probe base can be customized in size to ensure a close fit with the fruit and prevent interference from external natural light; the type of spectral analysis device can be a spectrometer with an effective wavelength range of 480 to 1100nm, such as an optical fiber spectrometer, etc., and the embodiment of the present invention does not impose too many restrictions on this, and parameter setting and data transmission are performed through a USB interface; the light source system can include a 50W halogen lamp cup, a collimator, a light output end, a lighting intensity adjustment module, a voltage stabilizing module, a lighting brightness adjustment knob and other components, and the embodiment of the present invention does not impose too many restrictions on this; the controller can include an ARM11 with S3C6410 as the core as the controller of the detector; the peripheral circuit can include a USB interface circuit, a touch screen, a speaker and a fan, etc.
[0061] The types of fruits may include apples, oranges, grapefruits, kiwis, grapes, watermelons, pears, bananas, tomatoes, mangoes, etc. The embodiments of the present invention do not impose excessive restrictions on the types of fruits.
[0062] In the embodiment of the present invention, the terminal hardware can obtain a preset hyperspectral fruit image and a corresponding fruit stem image and fruit leaf image; the hyperspectral fruit image may refer to the data of hyperspectral features and spatial features of each pixel in the image collected by the spectral acquisition probe and the spectral analysis device; and the fruit stem image refers to the image of the fruit branch stem within the preset range of the hyperspectral fruit image, such as Figure 2 As shown in the rectangular box in the figure, the fruit leaf image refers to the fruit leaf image within the predicted range of the hyperspectral fruit image, such as Figure 3 As shown in the three rectangular boxes.
[0063] Step S102, converting the hyperspectral fruit image and the corresponding fruit stalk image and fruit leaf image into a combined fruit feature map;
[0064] Further applied to the embodiment of the present invention, after obtaining the hyperspectral fruit image, the fruit stalk image, and the fruit leaf image, the above three are converted into a combined fruit feature map.
[0065] In an embodiment of the present invention, the converting of the hyperspectral fruit image and the corresponding fruit stalk image and fruit leaf image into a combined fruit feature map includes: collecting the hyperspectral fruit image in the fruit image; acquiring the fruit stalk image and fruit leaf image within a preset horizontal distance and a preset vertical distance of the hyperspectral fruit image; and combining the hyperspectral fruit image and the fruit stalk image and fruit leaf image within the preset horizontal distance and the preset vertical distance into a fruit feature map.
[0066] The hyperspectral fruit image may refer to the data of the hyperspectral characteristics and spatial characteristics of each pixel in the grape fruit image; the fruit stalk image refers to the image of the fruit branch stalk within a preset horizontal distance and a preset vertical distance in the grape fruit image, and the fruit leaf image refers to the image of the fruit leaf within a preset horizontal distance and a preset vertical distance in the grape fruit image.
[0067] For example, the fruit stalk image within the preset horizontal distance and the preset vertical distance may refer to a fruit stalk image within 5-30 pixels horizontally and 8-15 pixels vertically, and the fruit leaf image refers to a fruit leaf image within 2000 pixels horizontally and 3000 pixels vertically outside the grape fruit image. The three are combined to obtain a combined fruit feature map. In the embodiment of the present invention, the images of the fruit and the fruit stalks and fruit leaves near the fruit are used as the original elements of the initial training set to improve the accuracy of model recognition. Specifically, the fruit image is a hyperspectral fruit image, and the images of the fruit stalks and fruit leaves are ordinary images, which can effectively reduce equipment costs and improve recognition accuracy.
[0068] Step S103, extracting the plant component position information of the fruit image, the fruit stalk image, and the fruit leaf image;
[0069] In an embodiment of the present invention, the extraction of plant part position information of the fruit image, fruit stalk image, and fruit leaf image includes: marking the first part position information of the hyperspectral fruit image; marking the second part position information of the fruit stalk image and the fruit leaf image within a preset horizontal distance and a preset vertical distance; and associating the first part position information with the second part position information to obtain the plant part position information.
[0070] Specifically, the first component position information refers to multiple pixel position information of the hyperspectral fruit image; and the second component position information refers to multiple pixel position information of the fruit stalk image and the fruit leaf image. The two are associated to establish an association relationship between the two to obtain plant component position information. The plant component position information C T It can be expressed as: C T =(C1, C2), wherein C1 represents multiple pixel position information of the fruit image, i.e., the first component position information, and C2 represents multiple pixel position information of the fruit stalk image and the fruit leaf image, i.e., the second component position information.
[0071] Step S104, converting the combined fruit characteristic graph into a combined fruit coding sequence;
[0072] After obtaining the above-mentioned combined fruit feature map, the above-mentioned combined fruit feature map is converted into a combined fruit coding sequence. Specifically, the conversion of the combined fruit feature map into a combined fruit coding sequence includes: converting the hyperspectral fruit image into a fruit pixel feature coding sequence, wherein the fruit pixel feature coding sequence includes corresponding hyperspectral information; converting the fruit stalk image and the fruit leaf image into a fruit stalk and fruit leaf pixel coding sequence; and associating the fruit pixel feature coding sequence and the fruit stalk and fruit leaf pixel coding sequence into a combined fruit coding sequence.
[0073] The fruit pixel feature coding sequence refers to the pixel feature coding sequence converted from the pixel content of the hyperspectral fruit image. In a preferred example, the fruit pixel feature coding sequence can be expressed as
[0074] X b =(p d1 ,p d2 ,p d3 ......p dn ); n represents the number of pixels, p d1 Represents a pixel feature encoding sequence element of a fruit image.
[0075] Furthermore, the fruit stalk and fruit leaf pixel coding sequence refers to a pixel feature coding sequence converted from the pixel contents of the fruit stalk image and the fruit leaf image. The fruit stalk and fruit leaf pixel coding sequence can be expressed as X T =(p1,p2,p2......p n), n represents the number of pixels, and p1 represents a pixel coding sequence element of a fruit stalk or leaf.
[0076] On the other hand, the combined fruit coding sequence is a two-dimensional data composed of the two. The combined fruit coding sequence can be expressed as the combined fruit coding sequence Here, n represents the number of pixels.
[0077] Step S105, combining the combined fruit coding sequence and the plant component position information to obtain the combined fruit component position information;
[0078] Furthermore, after obtaining the combined fruit coding sequence and the plant part position information, the two can be combined to obtain the combined fruit part position information; the combining the combined fruit coding sequence and the plant part position information to obtain the combined fruit part position information includes: adding the first part position information to the end of the sequence of the fruit pixel feature coding sequence in the combined fruit coding sequence; adding the second part position information to the end of the sequence of the fruit stalk and leaf pixel coding sequence in the combined fruit coding sequence to obtain the combined fruit part position information.
[0079] In a specific application of the embodiment of the present invention, the first component position information can be added to the end of the sequence of the fruit pixel feature coding sequence in the combined fruit coding sequence, and further, the second component position information can be added to the end of the sequence of the stem leaf pixel coding sequence. The combined fruit component position information of the combination of the two is as follows:
[0080] For example, the combined fruit coding sequences The combined result coding sequence Xu with the added first component position information and the second component position information includes Where m represents the number of plant parts position information, c dm Represents the pixel position information of the fruit image; c m The pixel position information of the real image, m is a positive integer, ranging from 0, 1, 2...m;
[0081] Step S106, combining the combined fruit component position information and the pre-detection results into a fruit training sample;
[0082] For example, the pre-detection result may include pre-acquired fruit sugar content, fruit damage location, etc., which are not limited in this embodiment of the present invention. The combined fruit component location information is associated with the pre-detection result to obtain multiple sample data, which are composed of fruit training samples.
[0083] Step S107, constructing a fruit nondestructive detection model through the training samples to obtain a trained fruit nondestructive detection model;
[0084] In the embodiment of the present invention, after obtaining the training samples, a supervised fruit non-destructive detection model can be constructed to obtain a trained fruit non-destructive detection model.
[0085] First, Convert to multiple matrices The matrix and the pre-detection results are then converted into multi-dimensional vectors, and then processed through convolutional layers, pooling layers, fully connected layers, activation layers, output layers, etc. The error is calculated through the objective function, and the weight parameters are iteratively optimized to obtain the completed fruit non-destructive detection model.
[0086] In the embodiment of the present invention, the types of the fruit non-destructive detection model may include a linear regression model, a logistic regression model, a K-nearest neighbor model, a decision tree model, a support vector machine model, a naive Bayes model, a GBDT (CART regression tree-based learner Boosting algorithm) model, an XGBoost (Extreme Gradient Boosting, extreme gradient ascent) model, a LightGBM model, a CatBoost model, a random forest model, etc. The embodiment of the present invention does not impose too many restrictions on the types of fruit non-destructive detection models; further, for the structural composition of the fruit non-destructive detection model, it may include convolutional layers, pooling layers, fully connected layers, activation layers, output layers and other layer structures of different types and numbers, and the embodiment of the present invention also does not impose too many restrictions on this;
[0087] In a preferred embodiment of the present invention, the objective function of the fruit nondestructive detection model can be as follows:
[0088]
[0089] in, is the loss function of the model, the loss between each sample value and the measured value, is the spectral error function; sp i It represents the spectral error in the hyperspectral image. The spectral error refers to the difference between the hyperspectral value and the historical spectral value. In the embodiment of the present invention, the spectral error and the loss function are associated to obtain a comprehensive objective function to minimize the spectral error and the loss function, improve the relevance, robustness and prediction accuracy of the model, and prevent overfitting.
[0090] Step S108: transmitting the new hyperspectral fruit image, fruit stem image and fruit leaf image to the trained fruit nondestructive detection model to obtain the fruit nondestructive detection category.
[0091] In the embodiment of the present invention, the fruit nondestructive detection category includes the fruit damage probability and the fruit damage location; the new hyperspectral fruit image, fruit stem image and fruit leaf image are transmitted to the trained fruit nondestructive detection model to obtain the fruit nondestructive detection category, including:
[0092] After transmitting the new hyperspectral fruit images, fruit stem images and fruit leaf images to the trained fruit nondestructive detection model, the output fruit damage probability and fruit damage location are obtained.
[0093] The hyperspectral-based nondestructive fruit detection method in the embodiment of the present invention comprises the following steps: obtaining a hyperspectral fruit image and a corresponding fruit stalk image and fruit leaf image; converting the hyperspectral fruit image and the corresponding fruit stalk image and fruit leaf image into a combined fruit feature map; extracting the plant component position information of the fruit image, fruit stalk image and fruit leaf image; converting the combined fruit feature map into a combined fruit coding sequence; combining the combined fruit coding sequence and the plant component position information to obtain combined fruit component position information; combining the combined fruit component position information and a pre-detection result into a fruit training sample; constructing a fruit nondestructive detection model through the training sample to obtain a trained fruit nondestructive detection model; transmitting new hyperspectral fruit images, fruit stalk images and fruit leaf images to the trained fruit nondestructive detection model to obtain a fruit nondestructive detection category. Improve the accuracy of model recognition. Specifically, the fruit image is a hyperspectral fruit image, while the images of the fruit stalks and leaves are ordinary images. This can effectively reduce equipment costs and improve recognition accuracy. The images of the fruit and the fruit stalks and leaves near the fruit are converted into hyperspectral and ordinary pixel sequence data. The location information of the grape fruit is superimposed on the hyperspectral and ordinary pixel sequence data as the training set of the artificial intelligence model to obtain the probability of fruit damage and the location of fruit damage, providing a variety of model training data and improving the accuracy of fruit detection.
[0094] It should be noted that, for the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that this embodiment is not limited by the order of the actions described, because according to this embodiment, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by this embodiment.
[0095] Reference Figure 4 , shows a structural block diagram of an embodiment of a hyperspectral-based fruit nondestructive detection system of this embodiment, which may specifically include the following modules:
[0096] The first acquisition module 301 is used to acquire a hyperspectral fruit image and corresponding fruit stem image and fruit leaf image;
[0097] The first conversion module 302 is used to convert the hyperspectral fruit image and the corresponding fruit stem image and fruit leaf image into a combined fruit feature image;
[0098] The first extraction module 303 is used to extract the plant parts position information of the fruit image, the fruit stem image and the fruit leaf image;
[0099] A conversion module 304, used for converting the combined fruit characteristic graph into a combined fruit coding sequence;
[0100] A combination module 305 is used to combine the combined fruit coding sequence and the plant component position information to obtain the combined fruit component position information;
[0101] A composition module 306, for combining the combined fruit component position information and the pre-detection results into a fruit training sample;
[0102] A construction module 307 is used to construct a fruit nondestructive detection model through the training samples to obtain a trained fruit nondestructive detection model;
[0103] The transmission module 308 is used to transmit the new hyperspectral fruit image, fruit stem image and fruit leaf image to the trained fruit non-destructive detection model to obtain the fruit non-destructive detection category.
[0104] Preferably, the first conversion module comprises:
[0105] A collection submodule, used for collecting a hyperspectral fruit image in the fruit image;
[0106] An acquisition submodule, used for acquiring a fruit stalk image and a fruit leaf image within a preset horizontal distance and a preset vertical distance of the hyperspectral fruit image;
[0107] The composition submodule is used to combine the hyperspectral fruit image and the fruit stalk image and fruit leaf image within a preset horizontal distance and a preset vertical distance into a fruit characteristic map.
[0108] Preferably, the first extraction module comprises:
[0109] A first marking submodule, used for marking the position information of the first component of the hyperspectral fruit image;
[0110] The second marking submodule is used to mark the second component position information of the fruit stalk image and the fruit leaf image within a preset horizontal distance and a preset vertical distance;
[0111] The first associating submodule is used to associate the first component position information with the second component position information to obtain the plant component position information.
[0112] Preferably, the conversion module comprises:
[0113] A first conversion submodule is used to convert the hyperspectral fruit image into a fruit pixel feature coding sequence, wherein the fruit pixel feature coding sequence includes corresponding hyperspectral information;
[0114] The second conversion submodule is used to convert the fruit stalk image and the fruit leaf image into a fruit stalk and fruit leaf pixel coding sequence;
[0115] The second association submodule is used to associate the fruit pixel feature coding sequence and the fruit stalk and leaf pixel coding sequence into a combined fruit coding sequence.
[0116] Preferably, the combined module comprises:
[0117] A first adding submodule is used to add first component position information to the end of the sequence of the fruit pixel feature coding sequence in the combined fruit coding sequence;
[0118] The second adding submodule is used to add the second component position information to the end of the fruit stalk and leaf pixel coding sequence in the combined fruit coding sequence to obtain the combined fruit component position information.
[0119] Preferably, the fruit non-destructive detection category includes the fruit damage probability and the fruit damage location; and the transmission module includes:
[0120] The transmission module is used to transmit the new hyperspectral fruit image, fruit stem image and fruit leaf image to the trained fruit nondestructive detection model to obtain the output fruit damage probability and fruit damage location.
[0121] Each module in the hyperspectral-based nondestructive fruit detection system can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.
[0122] The hyperspectral-based nondestructive fruit detection system provided above can be used to execute the hyperspectral-based nondestructive fruit detection method provided in any of the above embodiments, and has corresponding functions and beneficial effects.
[0123] In one embodiment, a computer device is provided, which can be connected to a fruit sorting device, and its internal structure diagram can be as follows: Figure 5As shown. The computer device includes a processor, a memory, and a charging interface connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the computer program is executed by the processor, a hyperspectral-based fruit non-destructive detection system method is implemented.
[0124] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0125] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the above embodiment are implemented:
[0126] Obtaining hyperspectral fruit images and corresponding fruit stem images and fruit leaf images;
[0127] Converting the hyperspectral fruit image and the corresponding fruit stalk image and fruit leaf image into a combined fruit feature map;
[0128] Extracting plant component position information of the fruit image, fruit stalk image, and fruit leaf image;
[0129] Converting the combined fruit characteristic map into a combined fruit coding sequence;
[0130] combining the combined fruit coding sequence and the plant component position information to obtain the combined fruit component position information;
[0131] Combining the combined fruit component position information and the pre-detection results into a fruit training sample;
[0132] Building a fruit nondestructive detection model through the training samples to obtain a trained fruit nondestructive detection model;
[0133] The new hyperspectral fruit images, fruit stem images and fruit leaf images are transmitted to the trained fruit nondestructive detection model to obtain the fruit nondestructive detection category.
[0134] Preferably, converting the hyperspectral fruit image and the corresponding fruit stalk image and fruit leaf image into a combined fruit feature map comprises:
[0135] A hyperspectral fruit image collected from the fruit image;
[0136] Acquire a fruit stalk image and a fruit leaf image within a preset horizontal distance and a preset vertical distance of the hyperspectral fruit image;
[0137] The hyperspectral fruit image and the fruit stalk image and fruit leaf image within a preset horizontal distance and a preset vertical distance are combined to form a fruit feature map.
[0138] Preferably, the extracting the plant component position information of the fruit image, the fruit stalk image, and the fruit leaf image includes:
[0139] Marking the position information of the first component of the hyperspectral fruit image;
[0140] Marking the second component position information of the fruit stalk image and the fruit leaf image within a preset horizontal distance and a preset vertical distance;
[0141] The first component position information and the second component position information are associated to obtain plant component position information.
[0142] Preferably, converting the combined fruit characteristic graph into a combined fruit coding sequence comprises:
[0143] Converting the hyperspectral fruit image into a fruit pixel feature coding sequence, wherein the fruit pixel feature coding sequence includes corresponding hyperspectral information;
[0144] Converting the fruit stalk image and the fruit leaf image into a fruit stalk and fruit leaf pixel coding sequence;
[0145] The fruit pixel feature coding sequence and the fruit stalk and leaf pixel coding sequence are associated to form a combined fruit coding sequence.
[0146] Preferably, the combining the combined fruit coding sequence and the plant component position information to obtain the combined fruit component position information comprises:
[0147] Adding the first component position information to the end of the sequence of the fruit pixel feature coding sequence in the combined fruit coding sequence;
[0148] The second component position information is added to the end of the sequence of the fruit stalk and leaf pixel coding sequence in the combined fruit coding sequence to obtain the combined fruit component position information.
[0149] Preferably, the fruit nondestructive detection category includes the fruit damage probability and the fruit damage location; the new hyperspectral fruit image, fruit stem image and fruit leaf image are transmitted to the trained fruit nondestructive detection model to obtain the fruit nondestructive detection category, including:
[0150] After transmitting the new hyperspectral fruit images, fruit stem images and fruit leaf images to the trained fruit nondestructive detection model, the output fruit damage probability and fruit damage location are obtained.
[0151] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above embodiment are implemented:
[0152] Obtaining hyperspectral fruit images and corresponding fruit stem images and fruit leaf images;
[0153] Converting the hyperspectral fruit image and the corresponding fruit stalk image and fruit leaf image into a combined fruit feature map;
[0154] Extracting plant component position information of the fruit image, fruit stalk image, and fruit leaf image;
[0155] Converting the combined fruit characteristic map into a combined fruit coding sequence;
[0156] combining the combined fruit coding sequence and the plant component position information to obtain the combined fruit component position information;
[0157] Combining the combined fruit component position information and the pre-detection results into a fruit training sample;
[0158] Building a fruit nondestructive detection model through the training samples to obtain a trained fruit nondestructive detection model;
[0159] The new hyperspectral fruit images, fruit stem images and fruit leaf images are transmitted to the trained fruit nondestructive detection model to obtain the fruit nondestructive detection category.
[0160] Preferably, converting the hyperspectral fruit image and the corresponding fruit stalk image and fruit leaf image into a combined fruit feature map comprises:
[0161] A hyperspectral fruit image collected from the fruit image;
[0162] Acquire a fruit stalk image and a fruit leaf image within a preset horizontal distance and a preset vertical distance of the hyperspectral fruit image;
[0163] The hyperspectral fruit image and the fruit stalk image and fruit leaf image within a preset horizontal distance and a preset vertical distance are combined to form a fruit feature map.
[0164] Preferably, the extracting the plant component position information of the fruit image, the fruit stalk image, and the fruit leaf image includes:
[0165] Marking the position information of the first component of the hyperspectral fruit image;
[0166] Marking the second component position information of the fruit stalk image and the fruit leaf image within a preset horizontal distance and a preset vertical distance;
[0167] The first component position information and the second component position information are associated to obtain plant component position information.
[0168] Preferably, converting the combined fruit characteristic graph into a combined fruit coding sequence comprises:
[0169] Converting the hyperspectral fruit image into a fruit pixel feature coding sequence, wherein the fruit pixel feature coding sequence includes corresponding hyperspectral information;
[0170] Converting the fruit stalk image and the fruit leaf image into a fruit stalk and fruit leaf pixel coding sequence;
[0171] The fruit pixel feature coding sequence and the fruit stalk and leaf pixel coding sequence are associated to form a combined fruit coding sequence.
[0172] Preferably, the combining the combined fruit coding sequence and the plant component position information to obtain the combined fruit component position information comprises:
[0173] Adding the first component position information to the end of the sequence of the fruit pixel feature coding sequence in the combined fruit coding sequence;
[0174] The second component position information is added to the end of the sequence of the fruit stalk and leaf pixel coding sequence in the combined fruit coding sequence to obtain the combined fruit component position information.
[0175] Preferably, the fruit nondestructive detection category includes the fruit damage probability and the fruit damage location; the new hyperspectral fruit image, fruit stem image and fruit leaf image are transmitted to the trained fruit nondestructive detection model to obtain the fruit nondestructive detection category, including:
[0176] After transmitting the new hyperspectral fruit images, fruit stem images and fruit leaf images to the trained fruit nondestructive detection model, the output fruit damage probability and fruit damage location are obtained.
[0177] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0178] Those skilled in the art will appreciate that the embodiments of the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the embodiments of the present invention may take the form of complete hardware embodiments, complete software embodiments, or embodiments combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0179] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or boxes.
[0180] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0181] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0182] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.
[0183] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.
[0184] The above is a detailed introduction to a fruit non-destructive detection method based on hyperspectral, a fruit non-destructive detection system based on hyperspectral, a computer device and a computer-readable storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A hyperspectral-based nondestructive fruit detection method, characterized in that: The method comprises: Obtaining hyperspectral fruit images and corresponding fruit stem images and fruit leaf images; Converting the hyperspectral fruit image and the corresponding fruit stalk image and fruit leaf image into a combined fruit feature map; Extracting plant component position information of the fruit image, fruit stalk image, and fruit leaf image; Converting the combined fruit characteristic map into a combined fruit coding sequence; combining the combined fruit coding sequence and the plant component position information to obtain the combined fruit component position information; Combining the combined fruit component position information and the pre-detection results into a fruit training sample; Building a fruit nondestructive detection model through the training samples to obtain a trained fruit nondestructive detection model; The new hyperspectral fruit images, fruit stem images and fruit leaf images are transmitted to the trained fruit nondestructive detection model to obtain the fruit nondestructive detection category.
2. The hyperspectral-based nondestructive fruit detection method according to claim 1, characterized in that: The step of converting the hyperspectral fruit image and the corresponding fruit stalk image and fruit leaf image into a combined fruit feature map comprises: A hyperspectral fruit image collected from the fruit image; Acquire a fruit stalk image and a fruit leaf image within a preset horizontal distance and a preset vertical distance of the hyperspectral fruit image; The hyperspectral fruit image and the fruit stalk image and fruit leaf image within a preset horizontal distance and a preset vertical distance are combined to form a fruit feature map.
3. The hyperspectral-based nondestructive fruit detection method according to claim 2, characterized in that: The extracting of the plant component position information of the fruit image, the fruit stalk image, and the fruit leaf image includes: Marking the position information of the first component of the hyperspectral fruit image; Marking the second component position information of the fruit stalk image and the fruit leaf image within a preset horizontal distance and a preset vertical distance; The first component position information and the second component position information are associated to obtain plant component position information.
4. The hyperspectral-based nondestructive fruit detection method according to claim 3, characterized in that: The step of converting the combined fruit characteristic graph into a combined fruit coding sequence comprises: Converting the hyperspectral fruit image into a fruit pixel feature coding sequence, wherein the fruit pixel feature coding sequence includes corresponding hyperspectral information; Converting the fruit stalk image and the fruit leaf image into a fruit stalk and fruit leaf pixel coding sequence; The fruit pixel feature coding sequence and the fruit stalk and leaf pixel coding sequence are associated to form a combined fruit coding sequence.
5. The hyperspectral-based nondestructive fruit detection method according to claim 4, characterized in that: The step of combining the combined fruit coding sequence and the plant component position information to obtain the combined fruit component position information comprises: Adding the first component position information to the end of the sequence of the fruit pixel feature coding sequence in the combined fruit coding sequence; The second component position information is added to the end of the sequence of the fruit stalk and leaf pixel coding sequence in the combined fruit coding sequence to obtain the combined fruit component position information.
6. The hyperspectral-based nondestructive fruit detection method according to claim 4, characterized in that: The fruit nondestructive detection category includes the fruit damage probability and the fruit damage location; the new hyperspectral fruit image, fruit stem image and fruit leaf image are transmitted to the trained fruit nondestructive detection model to obtain the fruit nondestructive detection category, including: After transmitting the new hyperspectral fruit images, fruit stem images and fruit leaf images to the trained fruit nondestructive detection model, the output fruit damage probability and fruit damage location are obtained.
7. A fruit nondestructive detection system based on hyperspectral, characterized in that: The system comprises: The first acquisition module is used to acquire the hyperspectral fruit image and the corresponding fruit stem image and fruit leaf image; A first conversion module is used to convert the hyperspectral fruit image and the corresponding fruit stalk image and fruit leaf image into a combined fruit feature image; A first extraction module is used to extract the plant parts position information of the fruit image, the fruit stalk image and the fruit leaf image; A conversion module, used for converting the combined fruit characteristic map into a combined fruit coding sequence; A combination module, used for combining the combined fruit coding sequence and the plant component position information to obtain the combined fruit component position information; A composition module, used for combining the combined fruit component position information and the pre-detection results into a fruit training sample; A construction module, used to construct a fruit nondestructive detection model through the training samples to obtain a trained fruit nondestructive detection model; The transmission module is used to transmit new hyperspectral fruit images, fruit stem images and fruit leaf images to the trained fruit non-destructive detection model to obtain the fruit non-destructive detection category.
8. The hyperspectral-based nondestructive fruit detection system according to claim 7, characterized in that: The first conversion module comprises: A collection submodule, used for collecting a hyperspectral fruit image in the fruit image; An acquisition submodule, used for acquiring a fruit stalk image and a fruit leaf image within a preset horizontal distance and a preset vertical distance of the hyperspectral fruit image; The composition submodule is used to combine the hyperspectral fruit image and the fruit stalk image and fruit leaf image within a preset horizontal distance and a preset vertical distance into a fruit characteristic map.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the hyperspectral-based non-destructive fruit detection method according to any one of claims 1 to 6 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the hyperspectral-based non-destructive fruit detection method according to any one of claims 1 to 6 are implemented.