Tobacco leaf harvesting quality detection method and system based on visual identification

Abstract

The invention discloses a tobacco leaf harvesting quality detection method and system based on visual identification, and the system comprises a mobile platform which can move in a tobacco field; the data acquisition unit is arranged on the mobile platform and is used for acquiring images in the tobacco field and the position of the mobile platform; and the data processing and identifying unit is used for identifying the acquired image, obtaining tobacco harvesting operation quality and obtaining a spatial position, corresponding to the operation quality, in a tobacco field in combination with the position of the mobile platform. The tobacco leaf harvesting quality detection method and system based on visual identification have the advantages of being timely in detection, high in precision, comprehensive in coverage and the like, the manual inspection workload can be greatly reduced, and the tobacco leaf harvesting management efficiency is improved.

Description

Technical Field

[0001] The present invention relates to a tobacco leaf harvesting quality detection method and system based on visual recognition, belonging to the field of agricultural machinery and equipment. Background Art

[0002] Tobacco is an important cash crop, and its harvesting process has a direct impact on the quality and yield of tobacco leaves. Traditional tobacco harvesting typically uses a layered picking method, harvesting tobacco leaves at different layers in multiple stages according to their maturity. With the development of agricultural mechanization, the use of tobacco harvesters has improved harvesting efficiency. However, during the mechanical harvesting process, operational quality issues often occur, such as tobacco leaves falling to the ground, some leaves remaining unpicked on the tobacco plants, and damage to the tobacco stems (tobacco plant stems) caused by the harvester. If these problems are not discovered and addressed promptly, they will lead to losses in tobacco yield and quality, and may affect subsequent harvesting of the tobacco plants.

[0003] Currently, monitoring the quality of tobacco harvesting operations primarily relies on manual inspection. Farmers or technicians typically enter the tobacco fields after harvesting to check for fallen leaves, missed leaves on the plants, and broken or severely damaged stems. Manual inspections are not only labor-intensive and time-consuming, but also susceptible to subjective factors, leading to frequent missed inspections and misjudgments. Especially in large-scale cultivation environments, it's difficult for manual inspections to fully inspect all plants in a short period of time, potentially missing opportunities for timely remediation.

[0004] Therefore, it is necessary to conduct more in-depth research on the existing tobacco leaf harvesting operation methods and systems to solve the above problems. Summary of the Invention

[0005] In order to overcome the above problems, we conducted in-depth research and proposed a tobacco leaf harvest quality detection system based on visual recognition, which includes:

[0006] Mobile platform, capable of moving in tobacco fields;

[0007] A data acquisition unit, provided on the mobile platform, for acquiring images of the tobacco field and the position of the mobile platform;

[0008] The data processing and recognition unit recognizes the collected images, obtains the quality of tobacco harvesting operations, and combines the position of the mobile platform to obtain the spatial position in the tobacco field corresponding to the operation quality.

[0009] In a preferred embodiment, the mobile platform has tracks.

[0010] In a preferred embodiment, the data acquisition unit includes a front-view camera and a side-view camera, wherein:

[0011] The front-facing camera faces the front of the mobile platform to collect images of the ground in front and the front of the tobacco plants, so as to identify the damage of tobacco leaves and tobacco stems that have fallen on the ground;

[0012] The side-view camera faces the side of the mobile platform to collect images of the side faces of the tobacco plants in the tobacco ridge.

[0013] In a preferred embodiment, there are at least two side-view cameras, which are respectively arranged on both sides of the mobile platform.

[0014] In a preferred embodiment, the height or angle of the front-facing camera is adjustable;

[0015] The height or angle of the side-view camera is adjustable.

[0016] In a preferred embodiment, the data collection unit includes a GNSS receiver for acquiring the current position.

[0017] In a preferred embodiment, a neural network is provided in the data processing and identification unit, and the collected pictures are identified by the neural network, and the identification includes identifying whether the tobacco leaves are dropped, missed, or the tobacco stems are damaged.

[0018] In a preferred embodiment, the data processing and identification unit processes the images taken by different cameras separately, and outputs the target categories identified in the image and their positions in the image. The target categories include tobacco leaves fallen on the ground, tobacco leaves missed from the tobacco plants, and tobacco stems damaged by machinery.

[0019] In a preferred embodiment, the data processing and identification unit is arranged in the cloud and connected to the data acquisition unit via a wireless network.

[0020] In a preferred embodiment, a target tracking module is provided in the data processing and identification unit, which adopts the ByteTrack multi-target tracking method to perform inter-frame tracking and association judgment of targets in the image sequence, realize stable numbering and path recording of the same target in continuous images, avoid repeated statistics of abnormal targets such as missed tobacco leaves and fallen leaves, and improve the consistency and robustness of target recognition results.

[0021] In a preferred embodiment, the data processing and identification unit is further provided with a layer detection module for identifying the position of the tobacco plant skeleton. Any known neural network, such as a CNN network, can be used, combined with the current installation height of the camera to determine whether the vertical position of the unharvested tobacco leaves in the image belongs to the current layer to be harvested, so as to distinguish between missed tobacco leaves and leaves from non-layers, thereby improving the adaptability and accuracy of the detection results to the stratified harvesting of tobacco.

[0022] In a preferred embodiment, in order to reduce the interference of the jitter generated by the mobile platform during its movement in the field on image recognition, the system is also provided with an image motion compensation module, which stabilizes the video image based on the optical flow method and the IMU sensor posture fusion method, eliminates the image jitter and offset in real time, and enhances the image stability and robustness of the recognition process.

[0023] The present invention also discloses a tobacco leaf harvest quality detection method based on visual recognition, comprising the following steps:

[0024] S1. After the tobacco leaf harvester has finished operating, the mobile platform is placed between ridges in the tobacco field. The height or angle of the front-view camera and the side-view camera are adjusted so that the front-view camera's field of view covers the ground area in front of the mobile platform and the tobacco plants below the adjacent ridge in front, and the side-view camera's field of view covers the layer of tobacco leaves that have just been harvested.

[0025] S2, the mobile platform moves along the tobacco field ridges, captures images and obtains current location information;

[0026] S3. Recognize the image. When the image is identified to contain an object, output the object category and the corresponding position.

[0027] The beneficial effects of the present invention include:

[0028] (1) Timely detection, high accuracy and comprehensive coverage;

[0029] (2) It can significantly reduce the workload of manual inspections and improve the efficiency of tobacco harvesting management;

[0030] (3) Simple structure, low production cost and easy maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a schematic structural diagram of a tobacco leaf harvest quality detection system based on visual recognition according to a preferred embodiment of the present invention;

[0032] Figure 2 The figure is a flow chart of a tobacco leaf harvest quality detection method based on visual recognition according to a preferred embodiment of the present invention.

[0033] Description of Reference Numerals

[0034] 1-Mobile platform;

[0035] 11- Tracks;

[0036] 21-Face the camera;

[0037] 22-Side view cameras. DETAILED DESCRIPTION

[0038] The present invention will be described in further detail below with reference to the accompanying drawings and examples, through which the features and advantages of the present invention will become more clearly understood.

[0039] The word "exemplary" is used exclusively herein to mean "serving as an example, example, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0040] According to the present invention, a tobacco leaf harvest quality detection system based on visual recognition is provided. Figure 1 Shown, including:

[0041] Mobile platform 1, capable of moving in the tobacco field;

[0042] A data acquisition unit, provided on the mobile platform, for acquiring images of the tobacco field and the position of the mobile platform;

[0043] The data processing and recognition unit recognizes the collected images, obtains the quality of tobacco harvesting operations, and combines the position of the mobile platform to obtain the spatial position in the tobacco field corresponding to the operation quality.

[0044] In a preferred embodiment, the mobile platform 1 has tracks 11, and the platform is moved by the tracks. The track movement has good off-road driving ability and stability, and is suitable for traveling in a muddy tobacco field environment.

[0045] In a preferred embodiment, the data acquisition unit includes a front view camera 21 and a side view camera 22, wherein:

[0046] The front-facing camera 21 faces the front of the mobile platform to collect images of the ground in front and the front of the tobacco plants, so as to identify the damage of tobacco leaves and tobacco stems that have fallen on the ground;

[0047] The side-view camera 22 faces the side of the mobile platform to collect images of the side of the tobacco plants in the tobacco ridges for identifying unpicked tobacco leaves left on the tobacco plants.

[0048] In the present invention, the moving direction of the mobile platform is taken as the front of the mobile platform.

[0049] In a preferred embodiment, there are at least two side-view cameras 22, which are respectively arranged on both sides of the mobile platform to simultaneously capture images of the sides of the tobacco plants in the tobacco ridges on the left and right sides of the mobile platform.

[0050] In a preferred embodiment, the height or angle of the front-view camera 21 is adjustable so that the ground area in front of the mobile platform and the tobacco plants under the adjacent front tobacco ridge are within the field of view of the front-view camera.

[0051] In a preferred embodiment, the height or angle of the side-view camera 22 is adjustable to better adapt to the layered harvesting of tobacco leaves, so that the layer of tobacco leaves that have just been harvested is within the field of view of the side-view camera.

[0052] In a preferred embodiment, the data collection unit includes a GNSS receiver for acquiring the current position.

[0053] In a preferred embodiment, the data processing and recognition unit is provided with a neural network, and the collected images are recognized by the neural network. Preferably, the recognition includes recognizing whether the tobacco leaves are dropped, missed, or damaged.

[0054] In the present invention, there is no limitation on the specific structure of the neural network, and those skilled in the art may adopt any known neural network, for example, the neural network adopts YOLOv10.

[0055] Preferably, the data processing and recognition unit processes images captured by different cameras separately and outputs the categories and locations of identified objects in the images. Object categories include fallen tobacco leaves, leaves missed from tobacco plants, and tobacco stems damaged by machinery. If no objects are found in the images, it indicates that the tobacco harvesting operation is proceeding smoothly. Otherwise, the object categories are recorded for subsequent observation and processing.

[0056] In a preferred embodiment, a target tracking module is provided in the data processing and identification unit, which adopts the ByteTrack multi-target tracking method to perform inter-frame tracking and association judgment of targets in the image sequence, realize stable numbering and path recording of the same target in continuous images, avoid repeated statistics of abnormal targets such as missed tobacco leaves and fallen leaves, and improve the consistency and robustness of target recognition results.

[0057] In a preferred embodiment, the data processing and identification unit is further provided with a layer detection module for identifying the position of the tobacco plant skeleton. Any known neural network, such as a CNN network, can be used, combined with the current installation height of the camera to determine whether the vertical position of the unharvested tobacco leaves in the image belongs to the current layer to be harvested, so as to distinguish between missed tobacco leaves and leaves from non-layers, thereby improving the adaptability and accuracy of the detection results to the stratified harvesting of tobacco.

[0058] In a preferred embodiment, in order to reduce the interference of the jitter generated by the mobile platform during its movement in the field on image recognition, the system is also provided with an image motion compensation module, which stabilizes the video image based on the optical flow method and the IMU sensor posture fusion method, eliminates the image jitter and offset in real time, and enhances the image stability and robustness of the recognition process.

[0059] In a preferred embodiment, the data processing and identification unit is arranged in the cloud and connected to the data acquisition unit via a wireless network.

[0060] The wireless network can adopt a cellular mobile network, such as 4G or 5G network, or a short-range wireless communication network, such as Zigbee, LoRa, NB-IoT, etc.

[0061] Preferably, the data acquisition unit further includes a data preprocessor, which synchronously integrates the image and position information and sends the data to the data processing and identification unit via a wireless network.

[0062] In the present invention, the specific model of the data preprocessor is not limited, and those skilled in the art can adopt any known processor, such as an ARM64 processor, a Raspberry Pi microcomputer, etc.

[0063] The present invention also discloses a tobacco leaf harvest quality detection method based on visual recognition, such as Figure 2 As shown, the following steps are included:

[0064] S1. After the tobacco leaf harvester has finished operating, the mobile platform is placed between ridges in the tobacco field. The height or angle of the front-view camera and the side-view camera are adjusted so that the front-view camera's field of view covers the ground area in front of the mobile platform and the tobacco plants below the adjacent ridge in front, and the side-view camera's field of view covers the layer of tobacco leaves that have just been harvested.

[0065] S2, the mobile platform moves along the tobacco field ridges, captures images and obtains current location information;

[0066] S3. Recognize the image. When the image is identified to contain an object, output the object category and the corresponding position.

[0067] Preferably, in S2, the mobile platform is moved by tracks.

[0068] In S3, the target categories include tobacco leaves that have fallen on the ground, tobacco leaves that have not been picked from the tobacco plants, and tobacco stems damaged by machinery. Tobacco leaves that have fallen on the ground and tobacco stems damaged by machinery are identified using images captured by the front-facing camera, while tobacco leaves that have not been picked from the tobacco plants are identified using images captured by the side-facing camera.

[0069] Preferably, in order to improve the consistency and accuracy of the recognition results, the ByteTrack tracking method is used to track the recognition target between frames to avoid repeated counting.

[0070] Preferably, based on the characteristics of layered harvesting of tobacco leaves, a neural network is set to identify the position of tobacco plants in the image, and combined with the camera height, it is determined whether the unharvested tobacco leaves belong to the current harvesting layer, thereby improving the accuracy of missed harvesting judgment.

[0071] Preferably, in order to reduce the interference of movement jitter on image recognition, the optical flow method is combined with IMU posture compensation to achieve image stabilization processing and improve recognition robustness.

[0072] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0073] Example

[0074] Example 1

[0075] After a certain lower tobacco leaf harvesting operation, the harvest quality is tested by the following system, which includes:

[0076] Mobile platform, capable of moving in tobacco fields;

[0077] A data acquisition unit, provided on the mobile platform, for acquiring images of the tobacco field and the position of the mobile platform;

[0078] The data processing and recognition unit recognizes the collected images, obtains the quality of tobacco harvesting operations, and combines the position of the mobile platform to obtain the spatial position in the tobacco field corresponding to the operation quality.

[0079] The mobile platform has tracks, and the data acquisition unit includes a front-view camera and a side-view camera. The front-view camera faces the front of the mobile platform and captures images of the ground in front and the front of the tobacco plants. The side-view camera faces the side of the mobile platform and captures images of the sides of the tobacco plants in the tobacco ridge. There are at least two side-view cameras, one on each side of the mobile platform. The height or angle of the front-view camera and the side-view camera are adjustable.

[0080] The data acquisition unit includes a GNSS receiver for acquiring a current position.

[0081] The data processing and recognition unit is equipped with a YOLOv10 neural network, which processes images taken by different cameras separately and outputs the target categories identified in the image and their positions in the image. The target categories include tobacco leaves fallen on the ground, tobacco leaves missed from the tobacco plants, and tobacco stems damaged by machinery.

[0082] The data processing and identification unit is arranged in the cloud and is connected to the data acquisition unit via a wireless network.

[0083] The detection method comprises the following steps:

[0084] S1. After the tobacco leaf harvester has finished operating, the mobile platform is placed between ridges in the tobacco field. The height or angle of the front-view camera and the side-view camera are adjusted so that the front-view camera's field of view covers the ground area in front of the mobile platform and the tobacco plants below the adjacent ridge in front, and the side-view camera's field of view covers the layer of tobacco leaves that have just been harvested.

[0085] S2, the mobile platform moves along the tobacco field ridges, captures images and obtains current location information;

[0086] S3. Recognize the image. When the image is identified to contain an object, output the object category and the corresponding position.

[0087] Inspection revealed two fallen tobacco leaves in the fifth row of Plot A, one unharvested leaf on each of three plants in the eighth row, and several tobacco stems with varying degrees of scratches. Management dispatched workers to clear the ground and re-harvest the remaining leaves, while also recording the location of the damaged plants for subsequent observation and action. Compared to the traditional practice of proceeding directly to the next round of farming operations without inspection, this system allows managers to fully monitor the quality of machine operations in near real time, significantly reducing losses caused by missed harvests and dropped leaves, and providing data support for improving machine equipment and operating methods.

[0088] The present invention has been described above with reference to preferred embodiments, but these embodiments are merely exemplary and serve only as illustrations. On this basis, various replacements and improvements can be made to the present invention, all of which fall within the scope of protection of the present invention.

Claims

1. A tobacco leaf harvest quality detection system based on visual recognition, characterized in that: include: Mobile platform, capable of moving in tobacco fields; A data acquisition unit, provided on the mobile platform, for acquiring images of the tobacco field and the position of the mobile platform; The data processing and recognition unit recognizes the collected images, obtains the quality of tobacco harvesting operations, and combines the position of the mobile platform to obtain the spatial position in the tobacco field corresponding to the operation quality.

2. The tobacco leaf harvest quality detection system based on visual recognition according to claim 1 is characterized in that: The mobile platform has tracks.

3. The tobacco leaf harvest quality detection system based on visual recognition according to claim 1, characterized in that: The data acquisition unit includes a front-view camera and a side-view camera, wherein: The front-facing camera faces the front of the mobile platform to collect images of the ground in front and the front of the tobacco plant; The side-view camera faces the side of the mobile platform to collect images of the side faces of the tobacco plants in the tobacco ridge.

4. The tobacco leaf harvest quality detection system based on visual recognition according to claim 3 is characterized in that: There are at least two side-view cameras, which are respectively arranged on both sides of the mobile platform.

5. The tobacco leaf harvest quality detection system based on visual recognition according to claim 3 is characterized in that: The height or angle of the front-facing camera is adjustable; The height or angle of the side-view camera is adjustable.

6. The tobacco leaf harvest quality detection system based on visual recognition according to claim 1, characterized in that: The data acquisition unit includes a GNSS receiver for acquiring a current position.

7. The tobacco leaf harvest quality detection system based on visual recognition according to claim 1, characterized in that: The data processing and identification unit is provided with a neural network, and the collected pictures are identified by the neural network, and the identification includes identifying whether the tobacco leaves are dropped, missed, or the tobacco stems are damaged.

8. The tobacco leaf harvest quality detection system based on visual recognition according to claim 7, characterized in that: The data processing and identification unit processes the images taken by different cameras separately, and outputs the target categories identified in the image and their positions in the image. The target categories include tobacco leaves fallen on the ground, tobacco leaves missed from the tobacco plants, and tobacco stems damaged by machinery.

9. The tobacco leaf harvest quality detection system based on visual recognition according to claim 1, characterized in that: The data processing and identification unit is arranged in the cloud and is connected to the data acquisition unit via a wireless network.

10. A tobacco leaf harvest quality detection method based on visual recognition, characterized in that: The following steps are involved: S1. After the tobacco leaf harvester has finished operating, the mobile platform is placed between ridges in the tobacco field. The height or angle of the front-view camera and the side-view camera are adjusted so that the front-view camera's field of view covers the ground area in front of the mobile platform and the tobacco plants below the adjacent ridge in front, and the side-view camera's field of view covers the layer of tobacco leaves that have just been harvested. S2, the mobile platform moves along the tobacco field ridges, captures images and obtains current location information; S3. Recognize the image. When the image is identified to contain an object, output the object category and the corresponding position.

Images