Large-leaf crop producing area identification method and system and storage medium
The origin of large-leaf crops is judged through spectral data and covariance, combined with deep learning methods, the high cost problem caused by manual separation is solved, and efficient and accurate place of origin identification and hierarchical processing is achieved.
Patent Information
- Application Number
- CN202410099732.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-24
- Publication Date
- 2025-07-25
AI Technical Summary
In the prior art, the identification of large-leaf crops relies on manual separation, resulting in high labor and time costs and affecting assembly line efficiency.
By obtaining the spectral data set of large-leaf crops, the covariance with different origins is calculated, the maximum covariance value is used to determine the origin, and leaf stem detection and spectral data acquisition are combined with deep learning methods to achieve automated identification.
It improves the accuracy and efficiency of large-leaf crop origin identification, reduces costs, and realizes non-destructive testing and convenient origin grading processing.
Smart Images
Figure CN120374914A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of visual detection, and particularly to a method, a system and a storage medium for identifying the origin of large-leaf crops. Background Art
[0002] After large-leaf crops (such as tobacco leaves) are collected from the origin, the staff will compact these crops and wrap them in bags, which requires the large-leaf crops to be loosened and separated before the subsequent grading and sorting processes. The existing method is to manually separate the leaves, and then put the large-leaf crops on the production line for subsequent operations. This method wastes a lot of human and time costs, thus affecting the efficiency of the overall production line. Summary of the Invention
[0003] The technical problem to be solved by the present invention is to provide a method, a system and a storage medium for identifying the origin of large-leaf crops, so as to accurately identify the origin of large-leaf crops in view of the deficiencies of the prior art.
[0004] To solve the above technical problem, the technical solution adopted by the present invention is: A method for identifying the origin of large-leaf crops, the method comprising:
[0005] S1. Obtain a first spectral data set of large-leaf crops, where the first spectral data set includes spectral data of multiple different spectral bands;
[0006] S2. Calculate the covariance between the first spectral data set and the second spectral data sets of each origin, and use the origin corresponding to the second spectral data set with the maximum covariance as the judgment result;
[0007] Wherein, the covariance calculation formula between the first spectral data set and the second spectral data set of any origin is:
[0008] s is the number of spectral bands, X i is the data of the i-th spectral band in the first spectral data set, is the mean value of all data in the first spectral data set, X ti is the data of the i-th spectral band in the second spectral data set of the t-th origin, is the mean value of all data in the second spectral data set of the t-th origin;
[0009] The calculation formula for the data of the i-th spectral band in the second spectral data set is: n is the number of data acquisitions in the i-th spectral band, λ p is the spectral reflectance intensity of the p-th acquisition..
[0010] The present invention determines the origin of large-leaf crops based on spectral data and covariance. The implementation process of this method is simple, and the recognition accuracy is high. By using spectral data analysis, the physical and chemical information of crops can be directly obtained without a large amount of calculation, and the cost is low. The spectral data reflects the growth state of crops. Therefore, judging the origin through spectral data has a high recognition accuracy. If the deep learning method is used to identify the origin of large-leaf crops, when the overall quality of the leaves in a certain origin this year is quite different from that of last year, then deep learning may need to be relabeled, consuming manpower and material resources. Otherwise, the accuracy will be greatly reduced. Therefore, the applicable range of the deep learning method is relatively narrow. The spectral data only needs to be taken several times to complete. The implementation process is simple, and historical data is not required. Moreover, spectral information does not require model inference, and the recognition efficiency is higher.
[0011] The specific implementation process of obtaining the spectral data of large-leaf crops includes:
[0012] 1) Obtain pictures of large-leaf crops;
[0013] 2) Use the pictures of the large-leaf crops to detect leaf stalks, and obtain k rectangular regions Rect1, Rect2... Rect containing leaf stalks k ;
[0014] 3) Calculate the average depth of each rectangular region;
[0015] 4) Obtain the minimum value of all the average depths of the rectangles. The rectangular region corresponding to this minimum value is the target rectangular frame;
[0016] 5) Convert the center point of the target rectangular frame to the robot coordinate system, and use the robot to grab the large-leaf crop corresponding to the target rectangular frame under the spectrometer to obtain spectral data.
[0017] The present invention utilizes depth information, which helps to improve the accuracy of positioning, can automatically perform non-destructive detection of large-leaf crops, and the spectral data has high flexibility and is convenient to use.
[0018] In the present invention, the average depth D of the p-th rectangular region Rectp The calculation formula is: m is the width of the rectangular region, n is the length of the rectangular region, f(i, j) is the pixel gray value of the point (i, j) in the picture of the large-leaf crop, p = 1, 2,...... k, and k is the number of rectangular regions.
[0019] The coordinate calculation formula of the center point (x target , y target ) of the target rectangular frame is:
[0020] x target = Recttarget .x + 0.5Rect width ;
[0021] y target = Rect target .y + 0.5Rect height ;
[0022] Among them, Rect target .x is the abscissa of the upper - left corner of the target rectangular box, Rect target .y is the ordinate of the upper - left corner of the target rectangular box, Rect width is the width of the target rectangular box, Rect height is the height of the target rectangular box.
[0023] The calibration formula for converting the center point of the target rectangular box to the robot coordinate system is:
[0024]
[0025] Among them, R is the rotation matrix, (x, y) is the coordinate of the center point of the target rectangular box (x target , y target ), M is the displacement coefficient matrix, and (x′, y′) is the converted coordinate.
[0026] The specific implementation process of step 2) includes:
[0027] A) Use the camera to take more than M pictures of large - leaf crops at the same position, select rectangular boxes at the leaf - stalk positions of each large - leaf crop picture, and generate text annotation data;
[0028] B) Use the large - leaf crop pictures and text annotation data as the input of the neural network model, train the neural network model, and obtain the recognition model;
[0029] C) Collect pictures of large - leaf crops from different origins, and use the recognition model to identify the rectangular areas with leaf - stalks in the collected large - leaf crop pictures.
[0030] The neural network model is the YOLO model.
[0031] The test spectral range of the spectral data is 300nm - 1000nm.
[0032] As an inventive concept, the present invention also provides a large - leaf crop origin recognition system, including:
[0033] A memory;
[0034] One or more processors;
[0035] One or more programs are stored in a memory, and when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the steps of the above method of the present invention.
[0036] As an inventive concept, the present invention also provides a computer-readable storage medium, including a memory; one or more programs are stored in the memory, and when the one or more programs are executed by a processor, the steps of the above method of the present invention are implemented.
[0037] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0038] 1. The present invention identifies the origin of large-leaf crops according to spectral data, with high identification accuracy and low identification cost. After identifying the origin, the large-leaf crops can be graded according to the origin information, which is convenient for subsequent grading processing;
[0039] 2. The present invention can detect the position of leaf stalks using depth information, which helps to improve the accuracy of positioning, can automatically perform non-destructive detection of large-leaf crops, and the spectral data has high flexibility and is convenient to use. Description of the Drawings
[0040] Figure 1 It is a system structure diagram of an embodiment of the present invention. Detailed Embodiments
[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.
[0042] In this document, terms such as "first", "second" and other similar words do not imply any order, quantity or importance, but are only used to distinguish different elements. In this document, terms such as "a", "an" and other similar words do not mean that there is only one such thing, but mean that the relevant description only refers to one of such things, and such things may have one or more. In this document, terms such as "comprising", "including" and other similar words are intended to represent a logical relationship and should not be regarded as representing a spatial structure relationship. For example, "A includes B" is intended to mean that logically B belongs to A, rather than meaning that B is located inside A in terms of space. In addition, the meanings of terms such as "comprising", "including" and other similar words should be regarded as open rather than closed. For example, "A includes B" is intended to mean that B belongs to A, but B does not necessarily constitute all of A, and A may also include other elements such as C, D, E, etc.
[0043] Example 1
[0044] Figure 1 Shown is a separation system for large-leaf crops based on three-dimensional spectral vision. The separation system includes conveyor belt 1 and conveyor belt 4, outer wrapping 2 of large-leaf crops, large-leaf crops 3, robotic arm 6 with grippers, RGB-D camera 5 and micro-spectrometer 7. When conveyor belt 1 transports large-leaf crops 3 wrapped in 2 to a designated position, RGB-D camera 5 will identify the position and depth of the leaf stalks in real time, and robotic arm 6 will thus receive the feedback position data to grab the leaf stalk positions. After grabbing the leaf stalks to the top, micro-spectrometer 7 acquires the spectral data of the grabbed large-leaf crops and compares it with the data of different origins in the library. After determining the origin, robotic arm 6 places the large-leaf crops on conveyor belt 4 to prepare for subsequent processing of the crops.
[0045] Furthermore, in the above separation system for large-leaf crops: Before the system works, RGB-D camera 5 needs to calibrate the coordinate system with robotic arm 6 to determine that the pixel positions in RGB-D camera 5 correspond one by one to the coordinates of robotic arm 6.
[0046] Furthermore, in the above separation system for large-leaf crops: Before the system works, a large amount of data of large-leaf crops needs to be collected and the leaf stalk parts need to be labeled, so as to train with a target detection model.
[0047] Furthermore, in the above separation system for large-leaf crops: Before the system works, a hyperspectral camera or spectrometer needs to be used to collect data of this large-leaf crop from different origins and analyze the spectral characteristics of this variety.
[0048] Further, in the above-mentioned large-leaf crop separation system: the order in which the robotic arm grasps the leaf stalks is determined by the depth point cloud information in the RGB-D camera 5, and the detection of the leaf stalks is identified using the target detection model in the RGB channel.
[0049] The method of this embodiment includes the following steps:
[0050] Before the system works, it is necessary to use the camera 5 to take more than 1500 pieces of large-leaf crop data at the same position, and use the annotation tool to select a rectangular box at the position of the leaf stalk, so as to generate text annotation data. The data is sent to the YOLO model for training, and the accuracy of the model needs to reach more than 85%.
[0051] Before the system works, it is also necessary to use a micro-spectrometer to collect the spectral curves of large-leaf crops from different origins. This example contains large-leaf crops from 3 origins, and the average spectral data is obtained by collecting the crops from each origin multiple times using formula one.
[0052]
[0053] In the above formula, n is the number of times of data collection in the i-th spectral band, λ p is the spectral reflectance intensity of the p-th collection. The test spectral band range is 300nm - 1000nm, and the X ti data set is represented as X t and used as the stored multi-spectral data (average spectral data set).
[0054] Before the system works, calibrate the camera 5 and the robotic arm 6 in a nine-point coordinate system. The calibration formula is shown in formula two:
[0055]
[0056] Among them, R is the rotation matrix, (x, y) is the image coordinate, M is the displacement coefficient matrix, (x′, y′) is the robot coordinate, and R and M are solved using the nine-point method.
[0057] During work, the rising edge of the sensor at the end of the conveyor belt 1 is triggered. When the package 2 containing the large-leaf crop 3 reaches the working area, the system starts to trigger the working state. The camera 5 uses the trained YOLOv5 model to detect the leaf stalks in the RGB channel, and obtains multiple rectangular regions Rect1, Rect2... Rect k containing the leaf stalks. Then, the depth channel of the camera 5 uses formula three to calculate the average distance of the rectangular regions.
[0058]
[0059] In the above formula, m is the width of the rectangle, n is the length of the rectangle, f(i, j) is the pixel gray value of the corresponding point in the image, and D Rect is the average depth of the output rectangle.
[0060] Use formula four to obtain the minimum value of the average depth.
[0061] D target = min(D Rect1 , D Rect2 ... D Rectk ) (Formula Four)
[0062] The grasping target is Rect target , that is, the coordinate points to be grasped in camera 5 are:
[0063] x target = Rect target .x + 0.5Rect width (Formula Five)
[0064] y target = Rect target .y + 0.5Rect height (Formula Six)
[0065] In the above formula, Rect target .x is the abscissa of the upper left corner of the target rectangle frame, Rect target .y is the ordinate of the upper left corner of the target rectangle frame, Rect width is the width of the target rectangle frame, and Rect height is the height of the target rectangle frame.
[0066] Then, convert the center point of the grasping target Rect target to the robot coordinate system through formula two, and operate the manipulator to automatically plan the path (the manipulator moves in the X direction first, and then in the Y direction), grasp the target large-leaf crop under the micro spectrometer 7 of the robotic arm 6, and the micro spectrometer collects spectral data to obtain the spectral data set X. Use formula seven to perform covariance processing on the collected data and the multi-spectral data in the three production areas within the specified wavelength band.
[0067]
[0068] Among them, s is the number of spectral bands, X i is the data of the i-th spectral band in the first spectral data set collected by the micro spectrometer 7, is the average value of all data in the first spectral data set, X ti is the data of the i-th spectral band in the second spectral data set of the t-th production area, is the mean value of all data in the second spectral data set of the t-th production area. The covariance reflects the variation trend of the collected data X and the data X stored in different production areas. t The target with the largest covariance is used as the judgment result of the production area. At the same time, the robotic arm 6 transports the grabbed large-leaf crops to the conveyor belt 4, which is conducive to the subsequent processing of the large-leaf crops.
[0069] Embodiment 2
[0070] Embodiment 2 of the present invention provides an identification system corresponding to Embodiment 1 above, including a memory, a processor, and a computer program stored on the memory; the processor executes the computer program on the memory to implement the steps of the method in Embodiment 1 above.
[0071] In some implementations, the memory can be a high-speed random access memory (RAM: Random Access Memory), and may also include non-volatile memory, such as at least one disk memory.
[0072] In other implementations, the processor can be a general-purpose processor of various types such as a central processing unit (CPU) and a digital signal processor (DSP), which are not limited here.
[0073] Embodiment 3
[0074] Embodiment 3 of the present invention provides a computer-readable storage medium corresponding to Embodiment 1 above, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the method in Embodiment 1 above are implemented.
[0075] A computer-readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination of the above.
[0076] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code. The solutions in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming languages such as Java and interpreted scripting languages such as JavaScript.
[0077] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, as well as the combination of flows and / or blocks in the flowchart and / or block diagram. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a means for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or a means for implementing the functions specified in one or more of the blocks.
[0078] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or a means for implementing the functions specified in one or more of the blocks.
[0079] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they know the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0080] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.
Claims
1. A method for identifying the origin of large-leaf crops, characterized in that, The method includes: S1. Obtain a first spectral data set of large-leaf crops, where the first spectral data set includes spectral data of multiple different spectral bands; S2. Calculate the covariance between the first spectral data set and the second spectral data sets of each production area, and use the production area corresponding to the second spectral data set with the maximum covariance as the judgment result; Among them, the covariance calculation formula between the first spectral data set and the second spectral data set of any origin is as follows: s is the number of spectral bands, X i is the data of the i-th spectral band in the first spectral data set, is the mean value of all data in the first spectral data set, X ti is the data of the i-th spectral band in the second spectral data set of the t-th production area, is the mean value of all data in the second spectral data set of the t-th production area; The calculation formula for the data of the i-th spectral band in the second spectral data set is: n is the number of data acquisitions in the i-th spectral band, λ p is the spectral reflectance intensity of the p-th acquisition.
2. The method for identifying the origin of large-leaf crops according to claim 1, wherein The specific implementation process of obtaining the spectral data of large-leaf crops includes: 1) Obtain pictures of large-leaf crops; 2) Detect leaf stalks using the large-leaf crop pictures to obtain k rectangular regions Rect1, Rect2... Rect containing leaf stalks k ; 3) Calculate the rectangular average depth of each of the rectangular areas; 4) Obtain the minimum value of all the rectangular average depths, and the rectangular area corresponding to this minimum value is the target rectangular frame; 5) Convert the center point of the target rectangular frame to the robot coordinate system, and use the robot to grab the large-leaf crops corresponding to the target rectangular frame under the spectrometer to obtain spectral data.
3. The method for identifying the origin of large-leaf crops according to claim 2, wherein, The rectangular average depth D of the p-th rectangular region Rectp The calculation formula is as follows: m is the width of the rectangular region, n is the length of the rectangular region, f(i, j) is the pixel gray value of the point (i, j) in the large-leaf crop image, p = 1, 2,..., k, and k is the number of rectangular regions.
4. The method for identifying the origin of large-leaf crops according to claim 2, wherein The coordinate calculation formula for the center point (x target , y target ) of the target rectangular box is as follows: x target = Rect target .x + 0.5Rect width ; y target = Rect target .y + 0.5Rect height ; Among them, Rect target .x is the abscissa of the upper left corner of the target rectangular box, Rect target .y is the ordinate of the upper left corner of the target rectangular box, Rect width is the width of the target rectangular box, Rect height is the height of the target rectangular box.
5. The method for identifying the origin of large-leaf crops according to claim 2, characterized in that, The calibration formula for converting the center point of the target rectangular frame to the robot coordinate system is: Among them, R is the rotation matrix, and (x, y) is the center point coordinates of the target rectangular box (x target , y target ), M is the displacement coefficient matrix, and (x′, y′) are the converted coordinates.
6. The method for identifying the origin of large-leaf crops according to any one of claims 2 to 5, characterized in that The specific implementation process of step 2) includes: A) Use a camera to take more than M pictures of large-leaf crops at the same position, draw rectangular frames at the petiole positions of each picture of large-leaf crops, and generate text annotation data; B) Use the pictures of large-leaf crops and the text annotation data as the input of the YOLO network model, train the neural network model, and obtain an identification model; C) Collect pictures of large-leaf crops from different production areas, and use the identification model to identify the rectangular areas with petioles in the collected pictures of large-leaf crops.
7. The method for identifying the origin of large-leaf crops according to claim 6, characterized in that, The YOLO model is the YOLOv5 model.
8. The method for identifying the origin of large-leaf crops according to any one of claims 1 to 5, characterized in that, The test spectral band range of the spectral data is 300nm - 1000nm.
9. A large-leaf crop origin identification system, characterized in that, Including: A memory; One or more processors; One or more programs are stored on the memory. When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the method according to any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that, Including a memory; one or more programs are stored on the memory. When the one or more programs are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.