Pineapple harvesting intelligent internet-of-things system based on machine vision

Abstract

The invention relates to a machine vision-based pineapple harvesting intelligent internet-of-things system, which comprises a pineapple picking machine used for picking target pineapples and conveying the target pineapples through a conveyor belt; the Internet of Things module is arranged in the pineapple picking machine and is used for acquiring pineapple images at specific positions of a conveying belt through a camera, performing image recognition on the pineapple images to obtain the number of pineapples and the actual diameter of each target pineapple, and performing following judgment on any target pineapple, judging whether the actual diameter of the target pineapple is within a preset confidence interval, if so, judging that the diameter of the target pineapple is normal, and if not, judging that the diameter of the target pineapple is abnormal; and related data are uploaded to the cloud server through the NBIOT module. According to the invention, a plurality of pineapple picking machines can be managed, and the operation conditions of the machines can be monitored in real time, so that planting technicians can be helped to analyze field planting conditions based on optimized pineapple fruits to a certain extent, and pineapple breeding improvement can be promoted.

Description

Technical Field

[0001] The present invention relates to the technical field related to machine vision and the Internet of Things, and in particular to a pineapple harvesting smart Internet of Things system based on machine vision. Background Art

[0002] Pineapple is the world's third-largest tropical fruit, beloved by consumers for its delicious flavor and rich nutritional value. Currently, pineapples are primarily harvested manually, which results in low efficiency and high labor costs. Although some pineapple-picking machines have been designed for pineapple harvesting, these are primarily designed for single-machine operation and do not consider the coordinated operation of multiple machines. This makes it difficult to manage multiple machines for pineapple harvesting, resulting in low intelligence levels, limited collaborative operation, and weak integration with modern information technology. Therefore, it is crucial to fully utilize the advances of modern information technology, integrating advances in computer and network technologies, the Internet of Things, machine vision, and wireless communication technologies, to achieve intelligent management capabilities such as visual remote monitoring of pineapple harvesting, fruit quality evaluation, and intelligent screening, thereby promoting the development of agricultural modernization. Summary of the Invention

[0003] The purpose of the present invention is to address at least one of the deficiencies in the prior art and to provide a machine vision-based intelligent IoT system for pineapple harvesting.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions: Specifically, a machine vision-based smart IoT system for pineapple harvesting is proposed, including: A pineapple picking machine is used to pick target pineapples and transport the target pineapples through a conveyor belt; The Internet of Things module is arranged inside the pineapple picking machine and is used to obtain the image of pineapples at a specific position of the conveyor belt through a camera, perform image recognition on the pineapple image to obtain the number of pineapples and the actual diameter of each target pineapple, and perform the following judgment on any target pineapple: Determine whether the actual diameter of the target pineapple is within a preset confidence interval. If so, the target pineapple diameter is considered normal; otherwise, the target pineapple diameter is considered abnormal. If the diameter of the target pineapple is normal, the actual diameter of the target pineapple is counted as the data to be uploaded. If the diameter of the target pineapple is abnormal, the current real-time longitude and latitude information is obtained through the Beidou positioning device, and the pineapple image is associated with the real-time longitude and latitude information to obtain associated data as the data to be uploaded, and the data to be uploaded is uploaded to the cloud server through the NBIOT module.

[0005] Furthermore, the system further includes: There are multiple pineapple pickers, and the NBIOT module of each pineapple picker has a different built-in code. The built-in code is used to distinguish the operation information of different pineapple pickers when the NBIOT module transmits data. The built-in code and the data to be uploaded are uploaded to the cloud server through the NBIOT module. The cloud server identifies the built-in code and determines the pineapple picker to which the data to be uploaded belongs.

[0006] Further, specifically, performing image recognition on the pineapple image to obtain the number of pineapples and the actual diameter of each target pineapple includes: Performing grayscale processing on the pineapple image to obtain a grayscale image; Performing a reverse binarization process on the grayscale image to obtain a binarized image; Processing the binarized image through a Kalman filter algorithm to obtain a denoised image; Performing Gaussian blur processing on the noise reduction image, then extracting the contour features of the target pineapple through the Canny edge detection operator, and optimizing the contour features through opening and closing operations to obtain the target pineapple contour set; Then the number of target pineapple contours in the target pineapple contour set is the number of pineapples, and pixel counts of each target pineapple contour in the target pineapple contour set are performed to obtain the number of associated pixels of all target pineapple contours; The number of associated pixels of any target pineapple outline is matched with the number of pixels of pineapple outlines of different diameters in a pre-established template library, and the actual diameter of the target pineapple outline is determined based on the diameter of the pineapple outline corresponding to the matching result.

[0007] Furthermore, the cloud server is also used to view picking-related data in real time and monitor the operating conditions of multiple pineapple pickers; it can interact with mobile terminals, and after the mobile terminal logs in to the cloud server, it can view picking-related data, which includes the number of pineapples picked, the diameter of the pineapples, and the picking time. When the related data is received, it is marked on the map in the web page, and then the related data is visualized to assist staff in following up on agricultural production. At the same time, the cloud server can also obtain the machine status of the existing pineapple pickers, which includes the machine power and network status. After the user logs in to the cloud server, the machine status of the pineapple picker can be viewed.

[0008] Furthermore, specifically, the method for determining the confidence interval includes: For the actual diameter data of all pineapples picked for the first time, the confidence interval is initialized and pre-given. For the confidence interval of the subsequent pineapple picking, the confidence interval is determined by the statistical distribution of the actual diameter size data of all pineapples in the previous pineapple picking.

[0009] Furthermore, specifically, the camera is constructed based on a binocular camera.

[0010] Furthermore, the method further comprises: When the data to be uploaded is uploaded to the cloud server through the NBIOT module, the uploaded data will also be encrypted to obtain encrypted data, and the encrypted data will be uploaded to the cloud server through the NBIOT module. The cloud server parses the encrypted data through the corresponding decryption algorithm.

[0011] The beneficial effects of the present invention are: The present invention proposes a machine vision-based smart IoT system for pineapple harvesting, which combines an IoT module assembled from hardware devices, a Beidou positioning device, and a web management platform. By installing an IoT module containing data acquisition equipment and communication equipment on a pineapple picking machine and using machine vision recognition technology, data such as the number, weight, and outer diameter of pineapples picked are collected and statistically analyzed, and finally uploaded to the server through the communication equipment. After logging into the management platform, pineapple picking data and machine equipment status can be viewed, and the machine location can also be viewed in real time on a map within the webpage. With the help of this platform, not only can multiple picking machines be managed and their operating conditions monitored in real time, but it can also help planting technicians to analyze field planting conditions based on selected pineapple fruits to a certain extent, promoting pineapple breeding and improvement. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features of the present disclosure will become more apparent through a detailed description of the embodiments shown in conjunction with the accompanying drawings. The same reference numerals in the drawings of the present disclosure represent the same or similar elements. Obviously, the drawings described below are only some embodiments of the present disclosure. It is possible for a person skilled in the art to derive other drawings based on these drawings without inventive effort. In the drawings: Figure 1 The figure shows the operation flow chart of the pineapple harvesting smart IoT system based on machine vision of the present invention; Figure 2 The figure shows a flow chart of the pineapple execution judgment of the pineapple harvesting smart Internet of Things system based on machine vision of the present invention; Figure 3 The figure shows a display content diagram of the cloud server for the operation information of different pineapple picking machines when the present invention is applied in a specific manner. DETAILED DESCRIPTION

[0013] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of this application can be combined with each other unless there is a conflict. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0014] Example 1, with reference to Figure 1 as well as Figure 2 The present invention proposes a pineapple harvesting smart Internet of Things system based on machine vision, comprising: A pineapple picking machine is used to pick target pineapples and transport the target pineapples through a conveyor belt; The Internet of Things module is arranged inside the pineapple picking machine and is used to obtain the image of pineapples at a specific position of the conveyor belt through a camera, perform image recognition on the pineapple image to obtain the number of pineapples and the actual diameter of each target pineapple, and perform the following judgment on any target pineapple: Determine whether the actual diameter of the target pineapple is within a preset confidence interval. If so, the target pineapple diameter is considered normal; otherwise, the target pineapple diameter is considered abnormal. If the diameter of the target pineapple is normal, the actual diameter of the target pineapple is counted as the data to be uploaded. If the diameter of the target pineapple is abnormal, the current real-time longitude and latitude information is obtained through the Beidou positioning device, and the pineapple image is associated with the real-time longitude and latitude information to obtain associated data as the data to be uploaded, and the data to be uploaded is uploaded to the cloud server through the NBIOT module.

[0015] In this first embodiment, smart agriculture is applied to pineapple picking. Currently, the level of automation in pineapple picking is low, and pineapples are mostly picked manually, which is not only inefficient but also costly. Smart agriculture, a new direction and trend in agricultural development, applies modern information technologies such as cloud computing and the Internet of Things to traditional agriculture. This can not only significantly improve agricultural production and management efficiency but also accelerate the process of agricultural modernization. Promoting the development of pineapple picking towards smart agriculture will help improve pineapple picking efficiency, reduce picking costs, save human resources, and thus further promote the development of the pineapple industry. Furthermore, the Internet of Things module is independently designed so that it can be independently installed on a variety of pineapple picking machines. The Internet of Things module consists of hardware devices such as a binocular camera, an NBIOT communication module, and a Beidou positioning system. The camera is responsible for identifying the pineapple and checking its diameter. If the diameter deviates from the statistical distribution pattern, it will take a photo of the pineapple and obtain its longitude and latitude information using the Beidou positioning system. The photo and corresponding longitude and latitude information are then uploaded to a server using the NBIOT communication module. The NBIOT communication module has the advantages of wide coverage, fast speed and low power consumption. It can not only upload data quickly, but also has low power consumption and long battery life.

[0016] As a preferred embodiment of the present invention, the system further includes: There are multiple pineapple pickers, and the NBIOT module of each pineapple picker has a different built-in code. The built-in code is used to distinguish the operation information of different pineapple pickers when the NBIOT module transmits data. The built-in code and the data to be uploaded are uploaded to the cloud server through the NBIOT module. The cloud server identifies the built-in code and determines the pineapple picker to which the data to be uploaded belongs.

[0017] In this preferred embodiment, when multiple machines are working together, distinguishing the information uploaded by different machines and accurately locating the abnormal pineapples picked by different machines is a difficult problem. However, accurate positioning of abnormal pineapples can help fruit farmers understand the growth environment of high-quality pineapples and substandard pineapples, and based on this, carry out smart farmland management and breeding to improve pineapple yield and quality. How to realize that the NBIOT transmission modules on different machines have different built-in codes, by attaching coding information in the uploaded data, the picking operation information of different machines is distinguished. At the same time, the high-precision positioning of the Beidou positioning system is coordinated to determine the planting location of abnormal pineapples.

[0018] As a preferred embodiment of the present invention, specifically, performing image recognition on the pineapple image to obtain the number of pineapples and the actual diameter of each target pineapple includes: Performing grayscale processing on the pineapple image to obtain a grayscale image; Performing a reverse binarization process on the grayscale image to obtain a binarized image; Processing the binarized image through a Kalman filter algorithm to obtain a denoised image; Performing Gaussian blur processing on the noise reduction image, then extracting the contour features of the target pineapple through the Canny edge detection operator, and optimizing the contour features through opening and closing operations to obtain the target pineapple contour set; Then the number of target pineapple contours in the target pineapple contour set is the number of pineapples, and pixel counts of each target pineapple contour in the target pineapple contour set are performed to obtain the number of associated pixels of all target pineapple contours; The number of associated pixels in any target pineapple's outline is matched against the number of pixels in a pre-established template library for pineapples of varying diameters. The actual diameter of the target pineapple's outline is then determined based on the diameter of the pineapple outline corresponding to the matching result. (Since pineapples are nearly elliptical, only the number of pixels in the pineapple's outline is required. The similarity between the pixels and the number of pixels in the pre-established template library is used to match the pixels. Given the variability of practical applications, a match is considered successful when a predetermined level of matching, such as 95%, is achieved.) Once a match is successful, the actual diameter of the target pineapple's outline can be directly read, enabling rapid identification of the actual diameters of all target pineapple outlines.

[0019] In this preferred embodiment, a pineapple passes through the camera position in the IoT module via a conveyor belt. After the camera detects the pineapple, the pineapple image is first subjected to image grayscale processing through machine vision technology to simplify the image, reduce the data dimension of the pineapple image and provide better contrast and brightness information, making it easy to analyze and identify image features. The image is then binarized to separate the pineapple and the background image into significantly different pixel values, making it easier for the machine to extract pineapple-related features and facilitate subsequent target detection processing tasks. Afterwards, the Kalman filter algorithm is used to reduce noise on the pineapple image, improve the quality of the image, and thus increase the image recognition rate of the machine. Finally, a Gaussian mixture model is used to identify features such as the texture of the target. According to the pixel ratio of the extracted features, the actual diameter of the pineapple and information such as the number of pineapples are calculated and transmitted to the NBIOT module for uploading. The detected diameter is compared with the pineapple diameter statistics in the system to determine whether the diameter of the target pineapple is within the confidence interval of the statistical data in the system. If it deviates from the confidence interval, it is considered that the pineapple diameter is abnormal. At this point, real-time latitude and longitude information is obtained through the Beidou positioning device. The original pineapple image used for identification, along with the latitude and longitude information, is uploaded to the server via the NBIOT module. Relevant information such as harvesting data and machine status can be viewed on the smart agriculture management platform.

[0020] Reference Figure 3 As a preferred embodiment of the present invention, the cloud server is also used to view picking-related data in real time and monitor the operating conditions of multiple pineapple pickers; it can interact with mobile terminals. After the mobile terminal logs in to the cloud server, the picking-related data can be viewed. The picking-related data includes the number of pineapples picked, the diameter of the pineapples, and the picking time. When the related data is received, it is marked on the map in the web page, and then the related data is visualized to assist staff in following up on the agricultural production situation. At the same time, the cloud server can also obtain the machine status of the existing pineapple pickers, and the machine status includes the machine power and network status. After the user logs in to the cloud server, the machine status of the pineapple picker can be viewed.

[0021] In this preferred embodiment, binocular camera is fixed in the middle of Internet of Things module specific position, after machine completes picking, pineapple diameter size is detected by machine vision technology.The data that detection obtains and the existing pineapple diameter data statistics in the system are compared, judge whether this pineapple diameter is positioned at the confidence interval of existing statistical data.If diameter size is not in this confidence interval, then can think that this pineapple diameter size deviates from normal value, camera can take pictures of this pineapple, after obtaining longitude and latitude information by Beidou positioning device, use NBIOT communication module that photo and longitude and latitude are uploaded to server, and on the map in the wisdom agriculture management platform, this position is marked.At every turn, after picking, the pineapple diameter data statistics in the system can be updated, to judge the growth change situation of pineapple.

[0022] Furthermore, the Smart Agriculture IoT cloud platform not only allows real-time viewing of harvesting data but also monitors the performance of multiple machines. By logging in to the website using a mobile phone or computer, you can view relevant data collected during harvesting, such as the number of pineapples picked, their diameter, and the time of harvest. When a machine detects a pineapple with a diameter that deviates from the statistical distribution, a camera takes a photo of the pineapple and uploads the photo and corresponding longitude and latitude to the server, where it is marked on a map within the website. The cloud platform allows users to view the location of pineapples with abnormal diameters, along with photos of them, helping staff analyze agricultural production and identify related issues promptly. The cloud platform also allows users to monitor the status of existing machines, such as battery level and network connection status, enabling efficient management of multiple pineapple harvesting machines and realizing the development of smart agriculture.

[0023] As a preferred embodiment of the present invention, specifically, the method for determining the confidence interval includes: For the actual diameter data of all pineapples picked for the first time, the confidence interval is initialized and pre-given. For the confidence interval of the subsequent pineapple picking, the confidence interval is determined by the statistical distribution of the actual diameter size data of all pineapples in the previous pineapple picking.

[0024] In this preferred embodiment, the diameter of the pineapples picked this time is compared with the diameter statistical distribution of the pineapples picked last time to judge whether the pineapples planted this time are too large or too small compared with the pineapples planted last time. If the diameter is not within the confidence interval, the pineapple will be photographed and uploaded. After picking, the pineapple diameter statistics will be updated to help staff judge the pineapple growth situation and development trend.

[0025] As a preferred embodiment of the present invention, specifically, the camera is constructed based on a binocular camera.

[0026] As a preferred embodiment of the present invention, the method further comprises: When the data to be uploaded is uploaded to the cloud server through the NBIOT module, the uploaded data will also be encrypted to obtain encrypted data, and the encrypted data will be uploaded to the cloud server through the NBIOT module. The cloud server parses the encrypted data through the corresponding decryption algorithm.

[0027] Although the present invention has been described in considerable detail and with particularity with respect to several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but rather should be construed as providing a broad possible interpretation of these claims in view of the prior art by reference to the appended claims, thereby effectively encompassing the intended scope of the invention. In addition, the invention has been described above in terms of embodiments foreseen by the inventors for the purpose of providing a useful description, and those insubstantial modifications of the invention that are not currently foreseen may still represent equivalent modifications of the invention.

[0028] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. As long as the technical effects of the present invention are achieved by the same means, they shall fall within the scope of protection of the present invention. Within the scope of protection of the present invention, various modifications and variations of the technical solutions and / or implementation methods may be made.

Claims

1. The pineapple harvesting intelligent IoT system based on machine vision is characterized by: include: A pineapple picking machine is used to pick target pineapples and transport the target pineapples through a conveyor belt; The Internet of Things module is arranged inside the pineapple picking machine and is used to obtain the image of pineapples at a specific position of the conveyor belt through a camera, perform image recognition on the pineapple image to obtain the number of pineapples and the actual diameter of each target pineapple, and perform the following judgment on any target pineapple: Determine whether the actual diameter of the target pineapple is within a preset confidence interval. If so, the target pineapple diameter is considered normal; otherwise, the target pineapple diameter is considered abnormal. If the diameter of the target pineapple is normal, the actual diameter of the target pineapple is counted as the data to be uploaded. If the diameter of the target pineapple is abnormal, the current real-time longitude and latitude information is obtained through the Beidou positioning device, and the pineapple image is associated with the real-time longitude and latitude information to obtain associated data as the data to be uploaded, and the data to be uploaded is uploaded to the cloud server through the NBIOT module.

2. The pineapple harvesting wisdom Internet of Things system based on machine vision according to claim 1, is characterized in that, The system further includes, There are multiple pineapple pickers, and the NBIOT module of each pineapple picker has a different built-in code. The built-in code is used to distinguish the operation information of different pineapple pickers when the NBIOT module transmits data. The built-in code and the data to be uploaded are uploaded to the cloud server through the NBIOT module. The cloud server identifies the built-in code and determines the pineapple picker to which the data to be uploaded belongs.

3. The pineapple harvesting wisdom Internet of Things system based on machine vision according to claim 1, is characterized in that, Specifically, image recognition is performed on the pineapple image to obtain the number of pineapples and the actual diameter of each target pineapple, including: Performing grayscale processing on the pineapple image to obtain a grayscale image; Performing a reverse binarization process on the grayscale image to obtain a binarized image; Processing the binarized image through a Kalman filter algorithm to obtain a denoised image; Performing Gaussian blur processing on the noise reduction image, then extracting the contour features of the target pineapple through the Canny edge detection operator, and optimizing the contour features through opening and closing operations to obtain the target pineapple contour set; Then the number of target pineapple contours in the target pineapple contour set is the number of pineapples, and pixel counts of each target pineapple contour in the target pineapple contour set are performed to obtain the number of associated pixels of all target pineapple contours; The number of associated pixels of any target pineapple outline is matched with the number of pixels of pineapple outlines of different diameters in a pre-established template library, and the actual diameter of the target pineapple outline is determined based on the diameter of the pineapple outline corresponding to the matching result.

4. The pineapple harvesting wisdom Internet of Things system based on machine vision according to claim 2, is characterized in that, The cloud server is also used to view picking-related data in real time and monitor the operating conditions of multiple pineapple pickers; it can interact with mobile terminals. After the mobile terminal logs in to the cloud server, it can view picking-related data, which includes the number of pineapples picked, the diameter of the pineapples, and the picking time. When the related data is received, it is marked on the map in the web page, and then the related data is visualized to assist staff in following up on the agricultural production situation. At the same time, the cloud server can also obtain the machine status of the existing pineapple pickers, which includes the machine power and network status. After the user logs in to the cloud server, the machine status of the pineapple picker can be viewed.

5. The pineapple harvesting wisdom Internet of Things system based on machine vision according to claim 1, is characterized in that, Specifically, the method for determining the confidence interval includes: For the actual diameter data of all pineapples picked for the first time, the confidence interval is initialized and pre-given. For the confidence interval of the subsequent pineapple picking, the confidence interval is determined by the statistical distribution of the actual diameter size data of all pineapples in the previous pineapple picking.

6. The pineapple harvesting wisdom Internet of Things system based on machine vision according to claim 1, characterized in that Specifically, the camera is constructed based on a binocular camera.

7. The pineapple harvesting wisdom Internet of Things system based on machine vision according to claim 1 is characterized in that, The system further includes, When the data to be uploaded is uploaded to the cloud server through the NBIOT module, the uploaded data will also be encrypted to obtain encrypted data, and the encrypted data will be uploaded to the cloud server through the NBIOT module. The cloud server parses the encrypted data through the corresponding decryption algorithm.

Images