Adaptive recognition method and system for ripe fruits

Abstract

The embodiment of the present invention provides a method and system for adaptively identifying ripe fruits, which are applied to the adaptive identification of ripe fruits by a harvesting robot in a mixed tomato and strawberry mode, and belong to the field of image processing technology. The method comprises: collecting image information of crops, and performing content recognition on the image information; determining the crop type of the current crop based on the content recognition result; selecting a corresponding ripe fruit recognition algorithm based on the crop type of the current crop as a pre-execution algorithm; performing ripe fruit recognition in the image information of the crop according to the pre-execution algorithm, and outputting the corresponding recognition result. The scheme of the present invention solves the problems of the single application scenario and low recognition accuracy of the existing ripe fruit recognition method.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a method for adaptively identifying ripe fruits and a system for adaptively identifying ripe fruits. Background Art

[0002] In modern planting technology, mixed planting is used more and more frequently. Mixed planting refers to two different plants planted together, which will have a good influence on each other when growing. Mixed planting is a natural way to grow sustainable and healthy vegetable gardens and orchards while minimizing human input. Because the growing environment between tomatoes and strawberries is similar, and tomatoes grow vertically upwards, while strawberries grow horizontally along the ground, there is no conflict in the growing environment between the two, making the model of mixed planting of tomatoes and strawberries more and more popular.

[0003] With the development of smart agriculture, more and more crops can be harvested by intelligent robots. However, unlike conventional crop harvesting, the fruit maturity consistency of melon and fruit crops is not strong. The fruits on the same crop plant may have both mature fruits and immature fruits. This is the same as conventional crops. When harvesting indiscriminately, it is bound to cause waste of immature fruits. This also makes it necessary to distinguish between mature fruits and immature fruits when harvesting melon and fruit crops that cannot be harvested at the same time, so as to facilitate targeted harvesting of mature fruits and avoid waste of immature fruits. Existing fruit recognition methods mainly identify mature fruits by geometric shape and color, but fruits are easily blocked by branches and leaves and clustered fruits, resulting in changes in geometric parameters such as the shape and size of the fruit, resulting in reduced recognition accuracy. In addition, the existing methods have a single application scenario, and one algorithm can only be applied to the recognition of one crop fruit. For scenarios such as mixed planting of tomatoes and strawberries, it is impossible to achieve mature fruit recognition. In view of the problems existing in the existing methods, it is necessary to create a new adaptive recognition method for mature fruits. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a method and system for adaptively identifying ripe fruits, so as to at least solve the problems of low accuracy and single application scenario of existing methods for identifying ripe fruits.

[0005] In order to achieve the above-mentioned objectives, the first aspect of the present invention provides a method for adaptively identifying ripe fruits, which is applied to the adaptive identification of ripe fruits by a harvesting robot in a mixed tomato and strawberry mode. The method comprises: collecting image information of crops and performing content recognition on the image information; determining the crop type of the current crop based on the content recognition result; selecting a corresponding ripe fruit recognition algorithm as a pre-execution algorithm based on the crop type of the current crop; performing ripe fruit recognition in the image information of the crop according to the pre-execution algorithm, and outputting the corresponding recognition result.

[0006] Optionally, the collecting of crop image information includes: at a predetermined harvesting station, traversing and collecting images in a predetermined direction to obtain scene image information from multiple shooting angles; performing plant feature recognition on all scene image information to filter out scene image information in which the plants are completely located within the image; identifying the shooting angle of the filtered scene image information, and collecting image information of crops at the current harvesting station based on the identified shooting angle.

[0007] Optionally, performing content recognition includes: performing fruit outline recognition and performing ground film recognition.

[0008] Optionally, fruit contour recognition includes: binarizing the image information of the crop to obtain all contour information in the image; extracting features from all contour information, and analyzing the extracted contour features based on preset tomato contour features to determine whether there is a contour that meets the tomato contour features; ground film recognition includes: identifying black features in the image information of the crop; if a continuous black area is identified, it is determined that a ground film exists; otherwise, it is determined that a ground film does not exist.

[0009] Optionally, determining the crop type of the current crop based on the content recognition result includes: performing a first fruit type recognition based on the fruit contour recognition result; performing a second fruit type recognition based on the ground film recognition result; if the first fruit type recognition result is the same as the second fruit type recognition result, the crop type corresponding to the fruit types recognized by both is used as the currently identified crop type; otherwise, re-performing the first fruit type recognition and the second fruit type recognition until the two recognition results are the same.

[0010] Optionally, the first fruit type identification includes: if the fruit contour recognition result is the presence of tomato contour features, then the current crop is determined to be tomato; otherwise, it is strawberry; the second fruit type identification includes: if the ground film recognition result is the presence of ground film, then the current crop is determined to be strawberry; otherwise, it is tomato.

[0011] Optionally, ripe fruit identification is performed in the image information of the crop according to the pre-execution algorithm, including: ripe strawberry fruit identification, including: converting the image information of the crop into a LAB image; extracting an A channel image from the converted LAB image, and adjusting the A channel image based on a preset range; performing an opening operation and / or a closing operation on the adjusted A channel image, and extracting the strawberry fruit contour based on the Canny edge operator after the operation is completed; performing ellipse fitting on all strawberry fruit contours, and identifying ripe strawberries therein based on the fitting result.

[0012] Optionally, ripe fruit recognition is performed in the image information of the crop according to the pre-execution algorithm, including: ripe tomato fruit recognition, including: converting the image information of the crop into an HSV image, and converting the HSV image into a grayscale image; extracting texture features from the grayscale image based on a preset LBP algorithm, and identifying ripe tomatoes based on the texture feature extraction result; wherein, the texture feature extraction from the grayscale image based on the preset LBP algorithm includes: dividing the grayscale image into multiple areas in equal amounts, each area does not overlap; in each area, traversing each pixel point, and comparing the grayscale value between each pixel point and the other 8 adjacent pixels in a nine-square grid mode; marking each pixel point based on the comparison result; the marking rule is: when the grayscale values ​​of the other 8 adjacent pixels of a certain pixel point are greater than the grayscale value of the current pixel point, the pixel point is marked as 1, otherwise it is marked as 0; the histogram of each area is calculated based on the marking result, and each histogram is normalized; all normalized histograms are connected into a feature vector to obtain an LBP texture feature vector, and texture feature extraction is completed.

[0013] The second aspect of the present invention provides a ripe fruit adaptive recognition system, which is applied to the adaptive recognition of ripe fruits by a harvesting robot in a mixed tomato and strawberry mode. The system comprises: a collection unit, which is used to collect image information of crops and perform content recognition on the image information; a processing unit, which is used to: determine the crop type of the current crop based on the content recognition result; select a corresponding ripe fruit recognition algorithm based on the crop type of the current crop as a pre-execution algorithm; and a recognition unit, which is used to perform ripe fruit recognition in the image information of the crop according to the pre-execution algorithm and output the corresponding recognition result.

[0014] On the other hand, the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed on a computer, enables the computer to execute the above-mentioned method for adaptively identifying ripe fruits.

[0015] Through the above technical solution, tomatoes and strawberries are first distinguished through the corresponding fruit distinction algorithm, and then different ripe fruit recognition methods are enabled based on different crop types, solving the problem of single application scenarios of existing methods. In view of the characteristics of tomato and strawberry fruits, a method combining color judgment and surface texture judgment is developed to improve the accuracy of ripe fruit recognition.

[0016] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0018] Figure 1 It is a flowchart of the steps of a method for adaptively identifying ripe fruits provided by one embodiment of the present invention;

[0019] Figure 2 It is a system structure diagram of a ripe fruit adaptive identification system provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0020] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0021] With the development of smart agriculture, more and more crops can be harvested by smart robots. However, unlike conventional crop harvesting, the fruit maturity consistency of melon and fruit crops is not strong. The same crop plant may have both mature and immature fruits. This is the same as conventional crops. When harvesting indiscriminately, it is bound to cause waste of immature fruits. This means that when harvesting melon and fruit crops that cannot be harvested at the same time, it is necessary to distinguish between mature and immature fruits, so as to facilitate targeted harvesting of mature fruits and avoid waste of immature fruits.

[0022] Traditional digital image processing technology requires accurate target fruit feature information for ripe fruit recognition. However, this method cannot maintain high detection accuracy when lighting conditions change, when the fruit is blocked, or when the fruit and background colors are similar. Subsequently, data-based machine learning was used for target detection and recognition, but machine learning requires parameters to be set in advance, and the parameters have a great impact on the target classification results. In recent years, deep learning has been widely used in target detection and recognition of harvesting robots because of its strong ability to extract high-dimensional features of targets.

[0023] Fruits have significant and stable color features and are less dependent on image size. However, when using color features to detect fruits, the maturity, color, variety, uncertainty and differences of background features of fruits and changes in external lighting will affect the accuracy of detection. Geometric shapes provide another set of feature indicators for fruit detection. For example, apples and citrus fruits are rounder than branches and leaves. Geometric shape features are less affected by lighting conditions and are suitable for feature extraction in orchard environments. The main problem with using shape feature segmentation to detect fruits is that orchard environments are usually unstructured, and fruits are easily blocked by branches and leaves, clustered fruits, resulting in changes in geometric parameters such as the shape and size of the fruits, resulting in reduced recognition accuracy.

[0024] The present invention is directed to the identification of ripe fruits in the case of mixed planting of tomatoes and strawberries, because in modern planting technology, mixed planting is more and more frequently used. Mixed planting refers to two different plants planted together, which will have a good influence on each other when growing. Mixed planting is a natural way to plant sustainable and healthy vegetable gardens and orchards while minimizing human input. Because the growth environment between tomatoes and strawberries is similar, and tomatoes grow vertically upward, while strawberries grow horizontally along the ground, there is no conflict in the growth environment between the two, so that the mixed planting mode of tomatoes and strawberries is increasingly promoted. Of course, this mixed planting mode can be a mixed planting scheme of planting strawberries and tomatoes in a single greenhouse at the same time, or a mode of mixed planting of strawberries and tomatoes in different greenhouses in a section. No matter which mixed planting mode is used, the economic benefits of fruit farmers can be greatly improved, so that its actual application scenarios are more and more. In order to solve the automatic picking of ripe fruits in this case, the scheme of the present invention creates a method for adaptively identifying ripe fruits, so that the harvesting robot can adaptively identify strawberries and tomatoes, so as to execute the ripe fruit identification scheme applicable to the corresponding crops. It ensures that no human intervention is required on the farm. The harvesting robot can automatically circulate in the greenhouses and carry out appropriate ripe fruit identification schemes based on real-time harvesting stations. Under the premise of ensuring the accuracy of fruit identification, it reduces labor costs and workload and improves the intelligence of the farm.

[0025] Because the color difference between ripe strawberry and immature fruit is large, the ripe fruit can be well identified based on color features. However, because the color between ripe fruit and immature fruit of tomato is similar, the conventional color-based scheme is not suitable for ripe tomato fruit identification. In the existing methods, there is a tomato target identification method that combines an AdaBoost classifier with a color feature classifier. The AdaBoost classifier uses a classification model that combines Haar-like features and an AdaBoost self-learning algorithm. Haar-like features can be divided into three categories: boundary features, linear features, and radian features. The feature value is used to reflect the change of image grayscale. Although this scheme can meet the recognition requirements of ripe tomato fruits to a certain extent, its misrecognition rate is still relatively high. When the scheme of the present invention is used to identify ripe tomato fruits, when the AdaBoost classifier and the color feature classifier are combined to detect the ripe tomato target, the combination of the two classifiers can reduce the misrecognition rate and meet the performance requirements of the tomato picking robot in an unstructured environment. Using the cascade classifiers obtained from offline training, a global scan of the scene photos taken by the picking robot is performed. All classifiers are determined to be true, and the pixel domain objects in the current detection window are regarded as target tomatoes. The multi-scale detection results are merged using the connection domain merging method, and finally the target tomatoes in the image are identified.

[0026] When identifying ripe strawberry fruits, the A channel component of the strawberry image is extracted and binarized in the Lab color model space. Then, the center and radius of the ripe strawberry are calculated using the gradient Hough circle transform method to achieve recognition of the ripe strawberry.

[0027] Figure 2 FIG. 1 is a system structure diagram of a ripe fruit adaptive identification system provided by one embodiment of the present invention. Figure 2 As shown, an embodiment of the present invention provides a ripe fruit adaptive recognition system, the system comprising: a collection unit, used to collect image information of crops, and perform content recognition on the image information; a processing unit, used to: determine the crop type of the current crop based on the content recognition result; select a corresponding ripe fruit recognition algorithm based on the crop type of the current crop as a pre-execution algorithm; a recognition unit, used to recognize ripe fruits in the image information of the crop according to the pre-execution algorithm, and output a corresponding recognition result.

[0028] Figure 1 FIG. 1 is a flowchart of a method for adaptively identifying ripe fruits provided by one embodiment of the present invention. Figure 1 As shown, the embodiment of the present invention provides a method for adaptively identifying a ripe fruit, the method comprising:

[0029] Step S10: collecting image information of crops and performing content recognition on the image information.

[0030] Specifically, the harvesting robots analyze the collected crop images to identify the ripe fruits therein. The difference between the scheme of the present invention and the existing methods is that the traditional harvesting robots only need to be responsible for the fruit collection of one crop because of the single application scenario, so a set of recognition algorithms can be preset. Therefore, even without considering the recognition accuracy, the scenarios to be dealt with by the scheme of the present invention are more complicated, because in order to realize the automatic circulation of the harvesting robots, the harvesting robots need to have the ability to distinguish between strawberries and tomatoes, because the properties of the two fruits are different, and the corresponding ripe fruit harvesting schemes are also different. Therefore, the accuracy of crop type recognition is directly related to the implementation ability of the scheme of the present invention. Therefore, the technology that needs to be solved by the scheme of the present invention is to accurately identify the ripe fruits in the mixed planting scenario of strawberries and tomatoes, which requires accurate crop type recognition and corresponding back-end ripe fruit recognition.

[0031] It is common sense that strawberries grow horizontally along the ground, while tomatoes grow vertically upward, so it is foreseeable that there is a big difference between the fruit height of strawberries and the fruit height of tomatoes. Therefore, in order to focus on mature fruits, it is necessary to ensure that the target fruit is included in the captured image, and due to the height difference between strawberry plants and tomato plants, it is impossible to ensure that the target plant can be photographed indiscriminately under the same viewing angle. For example, if strawberries are photographed based on the tomato fruit height perspective, there must be no strawberry plants in the picture, and if tomatoes are photographed based on the strawberry height perspective, there must be only the rhizome part of the tomato plant in the picture. In order to avoid this problem, preferably, at a predetermined harvesting station, traverse to collect images in a predetermined direction to obtain scene image information at multiple shooting angles; perform plant feature recognition on all scene image information, and screen out the scene image information corresponding to the plant completely located in the image; identify the shooting angle of the proposed scene image information, and collect image information of the current harvesting station crop based on the shooting angle. That is, indifferent images are collected within the preset limit range of strawberry and tomato plants, and then plant feature recognition is performed based on the collection results. The preset plant features are of course strawberry plant features and tomato plant features.

[0032] In an embodiment of the present invention, after the plant type is identified, it is convenient to collect plant images at corresponding angles. It should be noted that this step can only obtain the specific position of the plant, and cannot accurately derive the type of the plant. Because the height of the plant is not fixed, it has dynamic differences within a range, and the plant type is only distinguished by plant characteristics, which has a high misrecognition rate. Therefore, in order to reduce the computing power required, this step only identifies whether there is a plant in the picture, and can compare and identify the area occupied by the plant in the picture, find the image information with a large proportion of the plant in the picture as the target image, and also facilitate the subsequent fruit positioning. The scheme of the present invention can also solve the problem of differences in captured images caused by uneven plant levels, and take images according to the specific conditions of plants at different harvesting stations to ensure good fruit focusing effect.

[0033] Step S20: determining the crop type of the current crop based on the content recognition result.

[0034] Specifically, after ensuring that there are plants in the picture and that the proportion of plants is large, specific crop types can be distinguished based on the image information. As is known above, there is a large distance difference between the fruit growth positions of strawberries and tomatoes. Based on this, strawberries and tomatoes can be distinguished in theory through fruit height information. However, in order to avoid the situation in which the height of tomatoes is close to that of strawberries in special scenarios and cannot be distinguished, in order to ensure the accuracy of the recognition results, in the present invention, a dual judgment standard is set, that is, in addition to distinguishing plant types based on height differences, plant types will also be distinguished based on fruit shape, because there is a large difference in appearance between strawberry fruits and tomato fruits, and the distinction between strawberry and tomato fruits can be achieved based on this difference in appearance. However, in order to avoid the situation of similar appearance caused by other reasons such as shooting angle relationship and genetic variation, the accuracy of the plant type obtained based on the dual judgment result can meet production needs.

[0035] Specifically, the image information of the crop is binarized to obtain all contour information in the image; all contour information is feature extracted, and these extracted contour features are analyzed based on the preset tomato contour features to determine whether there is a contour that meets the tomato contour features; the ground film recognition includes: black feature recognition in the image information of the crop; if a continuous black area is recognized, it is determined that there is a ground film; otherwise, it is determined that there is no ground film. The first fruit type recognition is performed based on the fruit contour recognition result; the second fruit type recognition is performed based on the ground film recognition result; when the first fruit type recognition result is the same as the second fruit type recognition result, the crop type corresponding to the fruit type recognized by the two is used as the current recognition type; otherwise, the first fruit type recognition and the second fruit type recognition are performed again until the recognition results of the two are the same. If the fruit contour recognition result is that there is a tomato contour feature, it is determined that the current crop is a tomato; otherwise, it is a strawberry; the second fruit type recognition includes: if the ground film recognition result is that there is a ground film, it is determined that the current crop is a strawberry; otherwise, it is a tomato.

[0036] In the embodiment of the present invention, the difference in shooting angles caused by the height difference of the two crops is used to analyze whether there is a ground film in the image. If there is a ground film in the image, it means that the shooting angle is downward, because strawberries grow horizontally along the ground film, so when shooting strawberry plants, the ground film will definitely be photographed. Tomatoes grow upward, and the shooting angle is higher, so there will be no ground film in the image. The judgment results obtained based on the two criteria ensure the accuracy of plant type recognition.

[0037] Step S30: Selecting a corresponding ripe fruit recognition algorithm based on the crop type of the current crop as a pre-execution algorithm.

[0038] Specifically, different types of crops require different ripe fruit recognition schemes. As mentioned above, the color of strawberries changes significantly after ripening and before ripening, so strawberry fruit recognition can be performed based on color distinction. Tomatoes may have similar colors before and after ripening, so the error recognition rate of ripe tomato fruit recognition based on color distinction algorithm must be very good. However, after ripening, the red color of tomatoes is evenly distributed, and there are fewer variegated textures on the surface. During the ripening process of tomatoes, the red color gradually covers the surface unevenly, and there are more variegated textures on the surface, so the accuracy of ripe tomato fruit recognition can be improved by identifying ripe tomatoes based on the texture of the tomato surface.

[0039] Based on this, the scheme of the present invention has two ripe fruit recognition algorithms, namely a strawberry ripe fruit recognition algorithm and a tomato ripe fruit recognition algorithm. After completing the plant type recognition, it is determined whether the target plant is a strawberry or a tomato based on the recognition result, and then the corresponding ripe fruit recognition algorithm is selected based on the corresponding plant type.

[0040] Step S40: identifying ripe fruits in the image information of the crop according to the pre-execution algorithm, and outputting corresponding identification results.

[0041] Specifically, to complete the determination of the ripe fruit recognition algorithm, it is necessary to call the corresponding ripe fruit recognition algorithm, analyze the previously collected image information, and identify ripe fruits from multiple fruits in the image.

[0042] Embodiment 1:

[0043] The image information of the crop is converted into a LAB image; an A channel image is extracted from the converted image, and the A channel image is adjusted based on a preset range; an opening operation and / or a closing operation is performed on the adjusted A channel image, and after the operation is completed, a strawberry fruit contour is extracted based on a Canny edge operator; an ellipse fitting is performed on all strawberry fruit contours, and ripe strawberries are identified based on the fitting results.

[0044] In an embodiment of the present invention, during the growth process, strawberry fruit will be evenly covered with red on the surface of the fruit, so that there is a large color difference between the fruit and the leaves of the plant, so it is necessary to extract the corresponding red features, and show the strawberry fruit in the image to ensure that the fruit located is ripe. Of course, in the process of fruit positioning, the ripe fruit screening is actually carried out at the same time, because the strawberry is not ripe, its color is also green or white, so it is necessary to distinguish between green and red in a targeted manner. And the LAB image can distinguish red and green colors very well. The colors included in A are from dark green (low brightness value) to gray (medium brightness value) to bright pink (high brightness value); B is from bright blue (low brightness value) to gray (medium brightness value) to yellow (high brightness value). It can be seen that by analyzing the A channel of the Canny edge operator with a preset threshold range, the green and red distinction can be completed well, and the preset threshold range is preferably 160-255. Through the information after screening, the ripe strawberry fruit can be directly located to complete the identification of ripe strawberries.

[0045] Embodiment 2:

[0046] The image information of the crop is converted into an HSV image, and the HSV image is converted into a grayscale image; texture features of the grayscale image are extracted based on a preset LBP algorithm, and ripe tomatoes are identified based on the texture feature extraction results; wherein, the texture feature extraction of the grayscale image based on the preset LBP algorithm includes: dividing the grayscale image into multiple areas of equal value, each area does not overlap; in each area, traversing each pixel point, and comparing the grayscale value between each pixel point and the other 8 adjacent pixels in a nine-square grid mode; marking each pixel point based on the comparison result; the marking rule is: when the grayscale value of the other 8 adjacent pixels of a certain pixel point is greater than the grayscale value of the current pixel point, the pixel point is marked as 1, otherwise it is marked as 0; the histogram of each area is calculated based on the marking result, and each histogram is normalized; all normalized histograms are connected into a feature vector to obtain an LBP texture feature vector, and the texture feature extraction is completed.

[0047] In the embodiment of the present invention, when the color of the tomato surface is uneven and there are other mottled textures, after being converted into a grayscale image, the grayscale values ​​corresponding to the regions with different colors are also different. Because the red color is evenly distributed on the ripe fruit, the difference between the grayscales is not large, and the surface histogram of the obtained image is highly uniform, and the proportion of the pixel value 0 is higher and evenly distributed. On the contrary, the immature fruit has a significant histogram with uneven distribution. The extracted texture features can well complete the identification of ripe tomatoes.

[0048] In an embodiment of the present invention, the scheme of the present invention can complete the identification and judgment of tomatoes and strawberries, obtain the type of fruit to be collected by analyzing the growth environment of the crop, use whether the image contains black to judge whether there is a ground film, and make a preliminary judgment, and then further determine whether the current working environment is to harvest strawberries or tomatoes by shape detection whether there is an elliptical shape, and then call the corresponding program. In the identification process of strawberries, the obvious color difference between ripe strawberries and unripe strawberries is used for identification, the RGB image is first converted to a LAB image, and then the A channel component is extracted from the LAB model, and the red feature is extracted by taking a value of 160-255, and then the opening and closing operations of corrosion and expansion are performed, and then the contour is extracted by the Canny edge operator with a threshold of 100 to 200, and then ellipse fitting is performed to complete the identification of ripe strawberry fruits, and multiple contours will not overlap in a single fruit. In the identification process of tomatoes, the RGB image is first converted to a grayscale image, and then the LBP algorithm with a radius of 3 is used to extract texture features to complete the identification of ripe tomatoes.

[0049] The embodiment of the present invention further provides a computer-readable storage medium, on which instructions are stored, which, when executed on a computer, enable the computer to execute the above-mentioned method for adaptively identifying ripe fruits.

[0050] Those skilled in the art will understand that all or part of the steps in the method for implementing the above-mentioned embodiments can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions for making a single-chip microcomputer, a chip or a processor (processor) perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0051] The optional embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the technical concept of the embodiments of the present invention, the technical scheme of the embodiments of the present invention can be subjected to a variety of simple modifications, and these simple modifications all belong to the protection scope of the embodiments of the present invention. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the embodiments of the present invention will not further describe various possible combinations.

[0052] In addition, various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the embodiments of the present invention, they should also be regarded as the contents disclosed in the embodiments of the present invention.

Claims

1. A method for adaptively identifying ripe fruits, applied to the adaptive identification of ripe fruits by a harvesting robot in a mixed tomato and strawberry mode, characterized in that: The method comprises: Collecting crop image information and performing content recognition on the image information; wherein, Content identification includes: Carry out fruit outline recognition and mulch film recognition; Fruit contour recognition includes: Binarize the crop image information to obtain all contour information in the image; Extract features from all contour information, and analyze the extracted contour features based on preset tomato contour features to determine whether there is a contour that meets the tomato contour features; Identification of ground film includes: Recognize black features in the image information of the crop; If a continuous black area is identified, it is determined that there is ground film; otherwise, it is determined that there is no ground film; Determine the crop type of the current crop based on the content recognition result; Selecting a corresponding ripe fruit recognition algorithm based on the crop type of the current crop as a pre-execution algorithm; According to the pre-execution algorithm, ripe fruits are identified in the image information of the crop, and corresponding identification results are output.

2. The method according to claim 1, characterized in that: The collecting of crop image information includes: At the predetermined collection station, traverse and collect images in the predetermined direction to obtain scene image information from multiple shooting angles; Perform plant feature recognition on all scene image information, and filter out scene image information where the plant is completely within the image; The shooting angle of the filtered scene image information is identified, and image information of the crops at the current harvesting station is collected based on the identified shooting angle.

3. The method according to claim 2, characterized in that Determining the crop type of the current crop based on the content recognition result includes: Perform the first fruit type recognition based on the fruit outline recognition results; A second fruit type identification is performed based on the film identification results; If the results of the first fruit type identification are the same as those of the second fruit type identification, the crop type corresponding to the fruit types identified by the two is used as the currently identified crop type; otherwise, the first fruit type identification and the second fruit type identification are performed again until the identification results of the two are the same.

4. The method according to claim 3, characterized in that: The first fruit type identification includes: If the fruit contour recognition result shows that there is a tomato contour feature, then the current crop is determined to be a tomato; otherwise, it is a strawberry; The second fruit type identification includes: If the ground film recognition result is that there is ground film, it is determined that the current crop is strawberry; otherwise, it is tomato.

5. The method according to claim 1, characterized in that Recognizing ripe fruits in the image information of the crop according to the pre-execution algorithm includes: Identify ripe strawberry fruits, including: Converting the image information of the crop into a LAB image; Extracting an A channel image from the converted LAB image, and adjusting the A channel image based on a preset range; Performing an opening operation and / or a closing operation on the adjusted A channel image, and extracting the outline of the strawberry fruit based on the Canny edge operator after the operation is completed; Ellipse fitting is performed on the contours of all strawberry fruits, and the ripe strawberries are identified based on the fitting results.

6. The method according to claim 1, characterized in that Recognizing ripe fruits in the image information of the crop according to the pre-execution algorithm includes: Identify ripe tomatoes, including: Converting the image information of the crop into an HSV image, and converting the HSV image into a grayscale image; The texture features of the grayscale image are extracted based on a preset LBP algorithm, and ripe tomatoes are identified based on the texture feature extraction results; wherein, The extracting texture features of the grayscale image based on a preset LBP algorithm includes: Dividing the grayscale image into a plurality of regions in equal amounts, each region not overlapping; In each area, traverse each pixel and compare the grayscale value of each pixel with the other 8 adjacent pixels in a nine-square grid mode; Mark each pixel based on the comparison results; the marking rules are: When the grayscale values ​​of the other eight pixels adjacent to a pixel are greater than the grayscale value of the current pixel, the pixel is marked as 1, otherwise it is marked as 0; Calculate the histogram of each area based on the marking results, and normalize each histogram; All normalized histograms are connected into a feature vector to obtain the LBP texture feature vector and complete the texture feature extraction.

7. A ripe fruit adaptive recognition system, applied to the adaptive recognition of ripe fruits by a harvesting robot in a mixed tomato and strawberry mode, characterized in that: The system comprises: A collection unit is used to collect image information of crops and perform content recognition on the image information; wherein, Content identification includes: Carry out fruit outline recognition and mulch film recognition; Fruit contour recognition includes: Binarize the crop image information to obtain all contour information in the image; Extract features from all contour information, and analyze the extracted contour features based on preset tomato contour features to determine whether there is a contour that meets the tomato contour features; Identification of ground film includes: Recognize black features in the image information of the crop; If a continuous black area is identified, it is determined that there is ground film; otherwise, it is determined that there is no ground film; A processing unit for: Determine the crop type of the current crop based on the content recognition result; Selecting a corresponding ripe fruit recognition algorithm based on the crop type of the current crop as a pre-execution algorithm; The recognition unit is used to recognize ripe fruits in the image information of the crop according to the pre-execution algorithm and output corresponding recognition results.

8. A computer-readable storage medium having instructions stored thereon, which, when executed on a computer, enables the computer to execute the method for adaptively identifying ripe fruits according to any one of claims 1 to 6.

Images