Apparatus and method for measuring specification of strawberry

By integrating 2D and 3D data analysis, the device accurately measures strawberry dimensions and predicts weight, overcoming limitations of conventional systems and enhancing grading efficiency.

KR103047714B1Active Publication Date: 2026-11-11KONKUK UNIV IND COOP CORP
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
KR1020230142028
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Filing Date
2023-10-23
Publication Date
2026-11-11
Estimated Expiration
2043-10-23

AI Technical Summary

Technical Problem

Conventional computer vision-based strawberry classification systems are limited by the need for strict environmental control and struggle to accurately measure dimensions and predict weight due to reliance on 2D image data, leading to inefficiencies and potential damage during manual sorting.

Method used

A device and method utilizing both 2D image data and 3D scanning data from a lidar sensor to identify landmarks on strawberries, enabling accurate measurement of dimensions and prediction of weight through comprehensive data mapping and regression analysis.

Benefits of technology

Enables precise measurement of strawberry dimensions and weight without environmental constraints, reducing damage and improving efficiency in grading and sorting processes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112023116093784-PAT00006_ABST
    Figure 112023116093784-PAT00006_ABST
Patent Text Reader

Abstract

A device and method for measuring dimensions for strawberries are disclosed, and a method for measuring dimensions for strawberries according to one embodiment of the present invention may include: (a) acquiring 2D image data and 3D scanning data measured for strawberries; (b) identifying a plurality of landmarks pre-set for the strawberries from the 2D image data; and (c) calculating size information of the strawberries using the result of mapping the 2D image data and the 3D scanning data based on the plurality of landmarks.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present invention relates to a device and method for measuring specifications for strawberries. For example, the present invention relates to an automated technique for predicting the size and weight of strawberries. Background Technology

[0002] Recently, automation technology for crop harvesting and sorting has been advancing in relation to smart farms. In particular, strawberries are crops that require rapid harvesting and sorting processes to prevent quality degradation caused by their generally weak flesh and rapid over-ripening.

[0003] In particular, for domestic production, sales, and export, strawberry producers must grade harvested strawberries before packaging; strawberry quality varies based on criteria such as ripeness, shape, and size, as well as flavor, and traditionally, this process was carried out manually by farm workers.

[0004] Such manual sorting based on visual inspection is a time- and energy-intensive process that has limitations in that it cannot guarantee consistent grading. Furthermore, going through multiple such processes can lead to damage to the strawberries and economic losses during the export process; therefore, computer vision technology can provide a means to rapidly automate the harvesting and sorting of strawberries.

[0005] Meanwhile, conventional computer vision-based strawberry classification work had the difficulty of having to establish a limited working environment where cameras were installed at strictly limited heights, and recognizing and classifying strawberries by applying geometric formulas to 2D images.

[0006] The technology forming the background of the present invention is disclosed in Korean Published Patent Application No. 10-2016-0124770. The problem to be solved

[0007] The present invention aims to solve the problems of the aforementioned conventional technology by providing a device and method for measuring specifications of strawberries that can automatically measure the width (equatorial axis length), length (height), etc. of strawberries by comprehensively utilizing 2D and 3D data obtained using an image sensor and a lidar sensor, and even predict the weight information of the strawberries by utilizing this.

[0008] However, the technical problems that the embodiments of the present invention aim to solve are not limited to the technical problems described above, and other technical problems may exist. means of solving the problem

[0009] As a technical means for achieving the above-mentioned technical problem, a method for measuring dimensions of a strawberry according to one embodiment of the present invention may include: (a) acquiring 2D image data and 3D scanning data measured for a strawberry; (b) identifying a plurality of landmarks pre-set for the strawberry from the 2D image data; and (c) calculating size information of the strawberry using the result of mapping the 2D image data and the 3D scanning data based on the plurality of landmarks.

[0010] Additionally, the plurality of landmarks may include a first horizontal landmark and a second horizontal landmark selected based on the horizontal direction among a plurality of pixels forming an object area that is distinguished from the background area on the 2D image data and corresponds to the strawberry, and a first vertical landmark and a second vertical landmark selected based on the vertical direction among the plurality of pixels.

[0011] Additionally, step (b) may determine the pixel most protruding outwardly with respect to the horizontal direction among the plurality of pixels as the first horizontal landmark and the second horizontal landmark, respectively, determine the first vertical landmark among the pixels corresponding to the calyx portion of the strawberry located at one end of the vertical direction in the object area, and determine the pixel most protruding with respect to the other direction of the vertical direction among the plurality of pixels as the second vertical landmark.

[0012] In addition, the 3D scanning data may include a point cloud comprising a plurality of points corresponding to the outer surface of the strawberry, and a depth map projecting the plurality of points into a 2D form using distance information for each of the plurality of points.

[0013] Additionally, the above step (c) may include: (c1) obtaining target coordinate information, which is position information in the depth map for each of the two selected target landmarks among the plurality of landmarks, and target distance information, which is distance information to the target landmarks; (c2) calculating angle information formed between the lidar sensor and the two target landmarks using reference coordinate information, which is information about the position of the lidar sensor that measured the 3D scanning data, and the target coordinate information; and (c3) estimating the actual distance between the two target landmarks using the angle information and the target distance information.

[0014] Additionally, in step (c), the actual distance between the first horizontal landmark and the second horizontal landmark, estimated by setting the first horizontal landmark and the second horizontal landmark as the two target landmarks, can be derived as the horizontal length information among the size information.

[0015] Additionally, in step (c), the actual distance between the first vertical landmark and the second vertical landmark, estimated by setting the first vertical landmark and the second vertical landmark as the two target landmarks, can be derived as the vertical length information among the size information.

[0016] In addition, a method for measuring the specifications of a strawberry according to one embodiment of the present invention may include (d) a step of estimating the weight information of the strawberry using the size information.

[0017] Additionally, step (a) may receive the 2D image data and the 3D scanning data from a measuring device equipped with an image sensor for capturing the 2D image data and a LiDAR sensor for measuring the 3D scanning data.

[0018] Meanwhile, a standard measurement device for a strawberry according to one embodiment of the present invention may include a collection unit that acquires 2D image data and 3D scanning data measured for a strawberry, a landmark detection unit that identifies a plurality of landmarks pre-set for the strawberry from the 2D image data, and a size analysis unit that calculates size information of the strawberry using the result of mapping the 2D image data and the 3D scanning data based on the plurality of landmarks.

[0019] In addition, the size analysis unit obtains target coordinate information, which is position information in the depth map for each of the two selected target landmarks among the plurality of landmarks, and target distance information, which is distance information to the target landmarks; calculates angle information formed between the lidar sensor and the two target landmarks using reference coordinate information, which is information about the position of the lidar sensor that measured the 3D scanning data, and the target coordinate information; and can estimate the actual distance between the two target landmarks using the angle information and the target distance information.

[0020] In addition, the size analysis unit can derive the actual distance between the first horizontal landmark and the second horizontal landmark, estimated by setting the first horizontal landmark and the second horizontal landmark as the two target landmarks, as the horizontal length information among the size information.

[0021] In addition, the size analysis unit can derive the actual distance between the estimated first vertical landmark and the second vertical landmark as vertical length information among the size information by setting the first vertical landmark and the second vertical landmark as the two target landmarks.

[0022] In addition, a standard measuring device for strawberries according to one embodiment of the present invention may include a weight analysis unit that estimates weight information of the strawberries using the size information.

[0023] In addition, the collection unit may receive the 2D image data and the 3D scanning data from a measurement device equipped with an image sensor for capturing the 2D image data and a LiDAR sensor for measuring the 3D scanning data.

[0024] The means for solving the problem described above are merely exemplary and should not be interpreted as intended to limit the present invention. In addition to the exemplary embodiments described above, additional embodiments may exist in the drawings and the detailed description of the invention. Effects of the invention

[0025] According to the means for solving the problem of the present invention described above, a device and method for measuring dimensions of strawberries can be provided that automatically measures the width (equatorial axis length), length (height), etc. of strawberries by comprehensively utilizing 2D and 3D data obtained using an image sensor and a lidar sensor, and can even predict the weight information of the strawberries by utilizing this.

[0026] According to the solution to the problem of the present invention described above, the specifications of strawberries can be accurately measured using a measuring device capable of integrally acquiring image data and point cloud data.

[0027] However, the effects obtainable from this invention are not limited to those described above, and other effects may exist. Brief explanation of the drawing

[0028] FIG. 1 is a schematic diagram of a measurement system including a standard measuring device for strawberries according to one embodiment of the present invention. FIGS. 2 and 3 are conceptual diagrams showing the schematic operation flow of a measurement system including a standard measuring device for strawberries according to one embodiment of the present invention. Figure 4 is a diagram illustrating a plurality of landmarks identified from 2D image data. Figure 5 is a conceptual diagram showing a manual measurement method for strawberry specifications and a strawberry classification method based on specification information. FIG. 6 is a graph showing a data set including 2D image data and 3D scanning data for individual strawberries, as an experimental example associated with a standard measuring device for strawberries according to one embodiment of the present invention. FIG. 7 is a graph that exemplarily shows the estimation error for strawberry specifications as an experimental example linked with a specification measuring device for strawberries according to one embodiment of the present invention. FIG. 8 is an experimental example linked with a standard measuring device for strawberries according to one embodiment of the present invention, and is a graph showing the error between the actual standard and the estimated standard for strawberries by standard type. FIG. 9 is a table showing a comparison between a strawberry size measurement method according to a standard measuring device for strawberries according to one embodiment of the present invention and a conventional strawberry size measurement method. FIG. 10 is a diagram showing the results of landmark recognition according to the state of the calyx, as an experimental example linked with a standard measuring device for strawberries according to one embodiment of the present invention. FIG. 11 is a schematic diagram of a standard measuring device for strawberries according to one embodiment of the present invention. FIG. 12 is a flowchart of an operation for a standard measurement method for strawberries according to one embodiment of the present invention. FIG. 13 is a detailed operation flowchart for a calculation process of strawberry size information according to one embodiment of the present invention. Specific details for implementing the invention

[0029] Embodiments of the present invention are described below with reference to the attached drawings to enable those skilled in the art to easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0030] Throughout this specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" or "indirectly connected" with other elements interposed between them.

[0031] Throughout the entire specification, when a component is described as being located "on," "on top," "on top," "under," "on bottom," or "on bottom" of another component, this includes not only cases where the component is in contact with the other component but also cases where another component exists between the two components.

[0032] Throughout this specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0033] The present invention relates to a device and method for measuring specifications for strawberries. For example, the present invention relates to an automated technique for predicting the size and weight of strawberries.

[0034] FIG. 1 is a schematic diagram of a measurement system including a standard measuring device for strawberries according to one embodiment of the present invention.

[0035] Referring to FIG. 1, a strawberry standard measurement system (10) according to one embodiment of the present invention may include a standard measurement device (100) (hereinafter referred to as the 'standard measurement device (100)') and a measurement device (200) for a strawberry according to one embodiment of the present invention.

[0036] The standard measuring device (100) and the measuring device (200) can communicate with each other through a network (20). The network (20) refers to a connection structure that enables information exchange between each node, such as terminals and servers. Examples of such a network (20) include, but are not limited to, a 3GPP (3rd Generation Partnership Project) network, an LTE (Long Term Evolution) network, a 5G network, a WIMAX (World Interoperability for Microwave Access) network, the Internet, a LAN (Local Area Network), a Wireless LAN (Wireless Local Area Network), a WAN (Wide Area Network), a PAN (Personal Area Network), a Wi-Fi network, a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, and a DMB (Digital Multimedia Broadcasting) network.

[0037] The measuring device (200) may be provided in the form of a user terminal (not shown) including all types of wireless communication devices such as a smartphone, smartpad, tablet PC, PCS (Personal Communication System), GSM (Global System for Mobile communication), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), and Wibro (Wireless Broadband Internet) terminal, but is not limited thereto.

[0038] Additionally, referring to FIG. 1, the measuring device (200) may be equipped with an image sensor (210) for capturing 2D image data (11) and a LiDAR sensor (220) for measuring 3D scanning data (12).

[0039] FIGS. 2 and 3 are conceptual diagrams showing the schematic operation flow of a measurement system including a standard measuring device for strawberries according to one embodiment of the present invention.

[0040] Referring to FIGS. 2 and 3, the standard measuring device (100) can obtain 2D (2-Dimensional) image data (11) and 3D (3-Dimensional) scanning data (12) of a strawberry (1) whose standard is to be measured from the measuring device (200) ('strawberry data collection' in FIG. 2).

[0041] Meanwhile, in the description of the embodiments of the present invention, the 3D scanning data (12) may include a point cloud comprising a plurality of points corresponding to the outer surface of the strawberry (1) and a depth map projecting the plurality of points into a 2D form using distance information for each of the plurality of points. In this regard, in the case of the 3D scanning data (12), the 3D scanning data (2D image data (11) of the strawberry (1)) may include data regarding the 2D coordinates of the pixels of the portion representing the strawberry (1). According to one embodiment of the present invention, the 2D image data (11) may include an RGB image, but is not limited thereto.

[0042] In other words, the 2D image data (11) of the strawberry (1) may include data regarding the 2D coordinates of the pixels of the part representing the strawberry (1). According to one embodiment of the present invention, the 2D image data (11) may include an RGB image, but is not limited thereto.

[0043] Additionally, 3D scanning data (12) regarding a strawberry (1) obtained using a lidar sensor (220) may include information regarding the coordinates of points on the strawberry (1) and information regarding distance values ​​from the lidar sensor (220) to each point on the strawberry (1).

[0044] Additionally, the standard measuring device (100) can recognize a pre-set landmark (key point) through 2D image data (11) ('key point detection' in FIG. 2). For example, the standard measuring device (100) can operate to identify four different landmarks (key points) by inputting the 2D image data (11) into a pre-built (trained) artificial intelligence-based detection model, but is not limited thereto, and according to an embodiment of the present invention, the number of landmarks identified from the 2D image data (11) can be set to a number of less than four or a number of four or more.

[0045] Additionally, the standard measuring device (100) can estimate size information of a strawberry (1), including the horizontal (equatorial axis) length and vertical (height) length, by connecting two points through identified landmarks. To this end, the standard measuring device (100) utilizes 3D scanning data (12), and more specifically, to estimate the horizontal (equatorial axis) length and vertical (height) length of the strawberry (1), the standard measuring device (100) calculates the angle formed by the LiDAR sensor (220) with two landmarks using 2D coordinates, identifies two distance values ​​associated with the two points of the strawberry (1) and the 2D coordinates, and can estimate the actual length of the measurement item using the derived angle information and the two distance values ​​(size calculation in FIG. 2).

[0046] In addition, the standard measuring device (100) can predict (estimate) the weight of the strawberry (1) through regression analysis based on the predicted horizontal (equatorial axis) length and vertical (height) length of the strawberry (1) ('Weight prediction' in FIG. 2).

[0047] Below, the specific functions and operations of the standard measuring device (100) will be described in detail.

[0048] The standard measuring device (100) can acquire 2D image data (11) and 3D scanning data (12) measured on a strawberry (1).

[0049] For example, the standard measuring device (100) can receive 2D image data (11) and 3D scanning data (12) from a measuring device (200) equipped with an image sensor (210) for capturing 2D image data (11) and a LiDAR sensor (220) for measuring 3D scanning data (12).

[0050] In addition, the standard measuring device (100) can identify a plurality of landmarks that are pre-set for the strawberry (1) from the acquired 2D image data (11).

[0051] Specifically, the standard measuring device (100) can identify a plurality of landmarks including a first horizontal landmark and a second horizontal landmark selected based on the horizontal direction among a plurality of pixels forming an object area that is distinguished from the background area on the 2D image data (11) and corresponds to the strawberry (1), and a first vertical landmark and a second vertical landmark selected based on the vertical direction among a plurality of pixels.

[0052] Figure 4 is a diagram illustrating a plurality of landmarks identified from 2D image data.

[0053] Referring to FIG. 4, the standard measuring device (100) identifies the pixel that protrudes most outwardly with respect to the horizontal direction among a plurality of pixels of an object area on 2D image data (11) as a first horizontal landmark ('P' in FIG. 4). B1 ') and the second horizontal landmark ('P in Fig. 4) B2 It can be decided as ')

[0054] Additionally, referring to FIG. 4, the standard measuring device (100) [describes] a first vertical landmark ('P in FIG. 4) among the pixels corresponding to the calyx portion of the strawberry (1) located at one end of the vertical direction in the object area. A1 Determine '), and among multiple pixels of the object area, the pixel most protruding based on the other vertical direction is the second vertical landmark ('P in FIG. 4). A2 It can be decided as ')

[0055] Meanwhile, according to one embodiment of the present invention, a standard measuring device (100) can operate to identify a plurality of landmarks in input 2D image data (11) using a pre-built (trained) artificial intelligence-based detection model.

[0056] In this regard, AI-based detection models may include, for example, HRNet-based models that provide overall excellent detection performance by connecting multiple resolution sub-networks while maintaining high-resolution representations around key points in the image, but are not limited to this.

[0057] In addition, the standard measuring device (100) can calculate size information of a strawberry (1) by using the result of mapping 2D image data (11) and 3D scanning data (12) based on a plurality of landmarks derived by the standard measuring device (100).

[0058] In this regard, the standard measuring device (100) can obtain target coordinate information, which is position information in the depth map of each of the two selected target landmarks among a plurality of landmarks, and target distance information, which is distance information to the target landmark. For example, referring to (c) and (d) of FIG. 3 described above, the target coordinate information is (cx a , cy a ) and (cx b , cy b It is derived as a 2D coordinate value in the form of ), and the target distance information is d a or d b It may be derived as a scalar value of the form.

[0059] In addition, the standard measuring device (100) uses reference coordinate information and target coordinate information, which are information about the position of the lidar sensor (220) that measured the 3D scanning data (12), to obtain angle information ('r' in FIG. 3) formed between the lidar sensor (220) and two target landmarks. ab Can perform operations on ')

[0060] In addition, the standard measuring device (100) has calculated angle information (r ab ) and target distance information (d a , d b Using ) the actual distance between two target landmarks ( ) can be estimated.

[0061] In summary, the standard measuring device (100) can calculate the actual distance between two target landmarks (in other words, horizontal length information or vertical length information) through the following Equation 1.

[0062] [Equation 1]

[0063]

[0064] Here, is an estimate (calculated value) of the actual distance between two selected target landmarks, and is angle information formed by the lidar sensor (220) and two target landmarks, and d a is target distance information from the lidar sensor (220) to the first landmark among two target landmarks, and d b is target distance information from the lidar sensor (220) to the second landmark among the two target landmarks.

[0065] More specifically, the standard measuring device (100) can derive the actual distance between the estimated first horizontal landmark and the second horizontal landmark as horizontal length information (equatorial axis length information) among the size information by setting the first horizontal landmark and the second horizontal landmark as two target landmarks.

[0066] In addition, the standard measuring device (100) can set the first vertical landmark and the second vertical landmark as two target landmarks and derive the actual distance between the estimated first vertical landmark and the second vertical landmark as vertical length information (height information) among the size information.

[0067] In addition, the standard measuring device (100) can estimate the weight information of the strawberry (1) using the derived size information.

[0068] In this regard, the weight of strawberries tends to be proportional to their size, density, and shape, and since the local density varies depending on sweetness and moisture content, it is difficult to infer from images alone. Meanwhile, utilizing vision technology enables accurate weight estimation without damaging individual strawberries, thereby minimizing the reclassification process and reducing the harvesting and packaging stages.

[0069] The standard measuring device (100) can estimate weight coefficients by applying the calculated width information and length information to regression analysis using 2D image data (11) and 3D scanning data (12) to estimate the weight of the strawberry (1), and can operate to estimate the weight of the strawberry (1) from the width information and length information using the same.

[0070] That is, the standard measuring device (100) holds actual weight information of each of the multiple strawberries as reference data for deriving a regression equation, calculates weights by evaluating how accurately the actual weight information can be estimated based on the width information and the length information, and derives a regression equation as shown in Equation 2 below.

[0071] [Equation 2]

[0072]

[0073] Here, width represents the horizontal length information among the size information derived by the standard measuring device (100), and height represents the vertical length information among the size information derived by the standard measuring device (100).

[0074] Figure 5 is a conceptual diagram showing a manual measurement method for strawberry specifications and a strawberry classification method based on specification information.

[0075] Referring to FIG. 5, the specifications (size and weight) of individual strawberries can be manually measured in various ways; the length can be measured by hand using a ruler (mm), and the weight can be measured individually using a scale (g). Meanwhile, individual strawberries can be classified into three categories based on weight. First, for category M, the weight ranges from 12g to 17g; for category L, the weight ranges from 17g to 25g; and for category Extra-L, the weight can satisfy 25g or more, but is not limited to these.

[0076] Below, an experimental example linked with a standard measuring device (100) will be described.

[0077] FIG. 6 is a graph showing a data set including 2D image data and 3D scanning data for individual strawberries, as an experimental example associated with a standard measuring device for strawberries according to one embodiment of the present invention.

[0078] Referring to FIG. 6, according to an experimental example associated with a standard measuring device (100) according to one embodiment of the present invention, a total of 885 individual strawberry data were collected for a strawberry size measurement experiment.

[0079] Meanwhile, the detection model was trained to recognize four key points of strawberries and used to predict width and height. In this experiment, considering that strawberries are generally classified based on their width and weight after harvest, individual strawberry data were classified into Extra, Class I, and Class II based on the maximum value of the width and weight.

[0080] FIG. 7 is a graph that exemplarily shows the estimation error for strawberry specifications as an experimental example linked with a specification measuring device for strawberries according to one embodiment of the present invention.

[0081] Referring to FIG. 7, 'Abs. Err', 'Rel. Err', and 'S. E' represent abbreviations for absolute error, relative error, and standard error, respectively. Referring to FIG. 7, it can be seen that the relative error according to the specification information estimated by the specification measuring device (100) disclosed herein is small, with the horizontal length information (width) at 3.84%, the vertical length information (height) at 5.20%, and the weight information at 10.7%.

[0082] FIG. 8 is an experimental example linked with a standard measuring device for strawberries according to one embodiment of the present invention, and is a graph showing the error between the actual standard and the estimated standard for strawberries by standard type.

[0083] Referring to Figure 8, 70% of the collected data was used to train a detection model for strawberry recognition and landmark (key point) detection, and subsequently, 30% of the data (i.e., 264 samples) from the entire dataset consisting of 885 individual strawberry data were randomly selected to predict (estimate) the specification information of the strawberries.

[0084] In addition, referring to Figure 8, it can be seen that the distribution of the measured values ​​and the predicted values ​​are almost identical, and outliers with relatively large errors were rarely observed with a probability of less than 1%.

[0085] FIG. 9 is a table showing a comparison between a strawberry size measurement method according to a standard measuring device for strawberries according to one embodiment of the present invention and a conventional strawberry size measurement method.

[0086] Referring to FIG. 9, in the case of a conventional strawberry size measurement method, the actual size was derived using only image data, so all environmental elements between the camera for capturing image data and the strawberry had to be strictly controlled. In contrast, in the case of the standard measurement device (100) disclosed herein, three-dimensional depth (distance) information can be utilized using a lidar sensor (220), so there is an advantage that it is not required to have a strict measurement environment like the conventional method.

[0087] FIG. 10 is a diagram showing the results of landmark recognition according to the state of the calyx, as an experimental example linked with a standard measuring device for strawberries according to one embodiment of the present invention.

[0088] Referring to FIG. 10, it can be seen that when using the artificial intelligence-based detection model provided by the standard device (100) disclosed herein, landmark recognition can be successfully performed regardless of the state (shape, standard, direction, etc.) of the strawberry calyx reflected on the image data (11).

[0089] FIG. 11 is a schematic diagram of a standard measuring device for strawberries according to one embodiment of the present invention.

[0090] Referring to FIG. 11, the standard measuring device (100) may include a collection unit (110), a landmark detection unit (120), a size analysis unit (130), and a weight analysis unit (140).

[0091] The collection unit (110) can acquire 2D image data (11) and 3D scanning data (12) measured on a strawberry (1).

[0092] For example, the collection unit (110) can receive 2D image data (11) and 3D scanning data (12) from a measurement device (200) equipped with an image sensor (210) for capturing 2D image data (11) and a LiDAR sensor (220) for measuring 3D scanning data (12).

[0093] The landmark detection unit (120) can identify a plurality of landmarks that are pre-set for the strawberry (1) from the 2D image data (11) acquired by the collection unit (110).

[0094] Specifically, the landmark detection unit (120) can identify a plurality of landmarks including a first horizontal landmark and a second horizontal landmark selected based on the horizontal direction among a plurality of pixels forming an object area that is distinguished from the background area on the 2D image data (11) and corresponds to the strawberry (1), and a first vertical landmark and a second vertical landmark selected based on the vertical direction among a plurality of pixels.

[0095] In addition, according to one embodiment of the present invention, the landmark detection unit (120) can determine the pixel that protrudes most outwardly with respect to the horizontal direction among a plurality of pixels of an object area on 2D image data (11) as the first horizontal landmark and the second horizontal landmark, respectively.

[0096] In addition, according to one embodiment of the present invention, the landmark detection unit (120) can determine a first vertical landmark among pixels corresponding to the calyx portion of a strawberry (1) located at one end of the vertical direction of the object area, and determine a second vertical landmark among a plurality of pixels of the object area, the pixel that protrudes the most based on the other direction of the vertical direction.

[0097] The size analysis unit (130) can calculate size information of the strawberry (1) using the result of mapping 2D image data (11) and 3D scanning data (12) based on multiple landmarks derived by the landmark detection unit (120).

[0098] In this regard, the size analysis unit (130) can obtain target coordinate information, which is location information in the depth map of each of the two selected target landmarks among the plurality of landmarks, and target distance information, which is distance information to the target landmark.

[0099] Additionally, the size analysis unit (130) can calculate the angle information formed by the lidar sensor (220) and two target landmarks using reference coordinate information and target coordinate information, which are information about the position of the lidar sensor (220) that measured the 3D scanning data (12).

[0100] In addition, the size analysis unit (130) can estimate the actual distance between two target landmarks using the calculated angle information and target distance information.

[0101] Specifically, the size analysis unit (130) can derive the actual distance between the estimated first horizontal landmark and the second horizontal landmark as horizontal length information among the size information by setting the first horizontal landmark and the second horizontal landmark as two target landmarks.

[0102] Additionally, the size analysis unit (130) can set the first vertical landmark and the second vertical landmark as two target landmarks and derive the actual distance between the estimated first vertical landmark and the second vertical landmark as vertical length information among the size information.

[0103] The weight analysis unit (140) can estimate the weight information of the strawberry (1) using the size information derived by the size analysis unit (130).

[0104] Below, based on the details described above, we will briefly examine the operation flow of the present invention.

[0105] FIG. 12 is a flowchart of an operation for a standard measurement method for strawberries according to one embodiment of the present invention.

[0106] The standard measurement method for strawberries illustrated in FIG. 12 can be performed by the standard measurement device (100) described above. Therefore, even if the details are omitted below, the description of the standard measurement device (100) can be applied equally to the description of the standard measurement method for strawberries.

[0107] Referring to FIG. 12, in step S11, the collection unit (110) can acquire (a) 2D image data (11) and 3D scanning data (12) measured on a strawberry (1).

[0108] Specifically, in step S11, the collection unit (110) can receive 2D image data (11) and 3D scanning data (12) from a measurement device (200) equipped with an image sensor (210) for capturing 2D image data (11) and a LiDAR sensor (220) for measuring 3D scanning data (12).

[0109] Next, in step S12, the landmark detection unit (120) can identify a plurality of landmarks that are pre-set for the strawberry (1) from the acquired 2D image data (11) (b).

[0110] Specifically, in step S12, the landmark detection unit (120) can identify a plurality of landmarks including a first horizontal landmark and a second horizontal landmark selected based on the horizontal direction among a plurality of pixels forming an object area that is distinguished from the background area on the 2D image data (11) and corresponds to the strawberry (1), and a first vertical landmark and a second vertical landmark selected based on the vertical direction among a plurality of pixels.

[0111] Additionally, according to one embodiment of the present invention, in step S12, the landmark detection unit (120) can determine the pixel that protrudes most outwardly with respect to the horizontal direction among a plurality of pixels of the object area on the 2D image data (11) as the first horizontal landmark and the second horizontal landmark, respectively.

[0112] Additionally, according to one embodiment of the present invention, in step S12, the landmark detection unit (120) determines a first vertical landmark among pixels corresponding to the calyx portion of a strawberry (1) located at one end of the vertical direction of the object area, and can determine a second vertical landmark among a plurality of pixels of the object area, the pixel that protrudes the most based on the other direction of the vertical direction.

[0113] Next, in step S13, the size analysis unit (130) can calculate the size information of the strawberry (1) using the result of mapping (c) 2D image data (11) and 3D scanning data (12) based on a plurality of landmarks derived through step S12.

[0114] Specifically, in step S13, the size analysis unit (130) can set the first horizontal landmark and the second horizontal landmark as two target landmarks and derive the actual distance between the estimated first horizontal landmark and the second horizontal landmark as horizontal length information among the size information.

[0115] Additionally, in step S13, the size analysis unit (130) can set the first vertical landmark and the second vertical landmark as two target landmarks and derive the actual distance between the estimated first vertical landmark and the second vertical landmark as vertical length information among the size information.

[0116] Next, in step S14, the weight analysis unit (140) can estimate the weight information of the strawberry (1) using the size information derived through step S13 (d).

[0117] In the description above, steps S11 through S14 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Additionally, some steps may be omitted as necessary, and the order of the steps may be changed.

[0118] FIG. 13 is a detailed operation flowchart for a calculation process of strawberry size information according to one embodiment of the present invention.

[0119] The calculation process of strawberry size information illustrated in FIG. 13 can be performed by the standard measuring device (100) described above. Therefore, even if the details are omitted below, the description of the standard measuring device (100) can be applied equally to the description of FIG. 13.

[0120] Referring to FIG. 13, in step S131, the size analysis unit (130) can obtain (c1) target coordinate information, which is location information in the depth map of each of the two selected target landmarks among the plurality of landmarks, and target distance information, which is distance information to the target landmark.

[0121] Next, in step S132, the size analysis unit (130) can calculate the angle information formed by the lidar sensor (220) and two target landmarks using reference coordinate information and target coordinate information, which are information about the position of the lidar sensor (220) that measured the (c2) 3D scanning data (12).

[0122] Next, in step S133, the size analysis unit (130) can estimate the actual distance between two target landmarks using (c3) angle information and target distance information.

[0123] In the description above, steps S131 to S133 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Additionally, some steps may be omitted as necessary, and the order of the steps may be changed.

[0124] A method for measuring the dimensions of strawberries according to one embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

[0125] In addition, the aforementioned method for measuring the specifications of strawberries can also be implemented in the form of a computer program or application executed by a computer stored on a recording medium.

[0126] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical concept or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0127] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and the concept of equivalents thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0128] 10: Strawberry Standard Measurement System 100: A standard measuring device for strawberries 110: Collection Department 120: Landmark Detector 130: Size Analysis Section 140: Weight Analysis Unit 200: Measurement device 210: Image sensor 220: LiDAR sensor 1: Strawberry 11: 2D image data 12: 3D Scanning Data 20: Network

Claims

Claim 1 A method for measuring the dimensions of a strawberry comprises: (a) acquiring 3D scanning data including 2D image data measured on the strawberry, a point cloud including a plurality of points corresponding to the outer surface of the strawberry, and a depth map projected in 2D form using distance information for each of the plurality of points; (b) identifying a plurality of landmarks pre-set for the strawberry from the 2D image data; and (c) a step of calculating size information of the strawberry using the result of mapping the 2D image data and the 3D scanning data based on the plurality of landmarks, wherein the plurality of landmarks include a first horizontal landmark and a second horizontal landmark selected based on the horizontal direction among a plurality of pixels forming an object area corresponding to the strawberry, which is distinguished from the background area on the 2D image data, and a first vertical landmark and a second vertical landmark selected based on the vertical direction among the plurality of pixels, and the step (c) comprises: (c1) a step of obtaining target coordinate information in the form of 2D coordinate values ​​as position information in the depth map for each of the two target landmarks selected among the plurality of landmarks, and target distance information in the form of scalar values ​​as distance information to the target landmarks; (c2) a step of calculating angle information formed by the lidar sensor and the two target landmarks using reference coordinate information, which is information about the position of the lidar sensor that measured the 3D scanning data, and the target coordinate information. A measurement method comprising the step of (c3) estimating the actual distance between the two target landmarks using the angle information and the target distance information. Claim 2 delete Claim 3 A measurement method according to claim 1, wherein step (b) determines the pixel most protruding outwardly with respect to the horizontal direction among the plurality of pixels as the first horizontal landmark and the second horizontal landmark, respectively, determines the first vertical landmark among the pixels corresponding to the calyx portion of the strawberry located at one end of the vertical direction in the object area, and determines the pixel most protruding with respect to the other direction of the vertical direction among the plurality of pixels as the second vertical landmark. Claim 4 delete Claim 5 delete Claim 6 A measurement method according to claim 1, wherein step (c) is to derive the actual distance between the first horizontal landmark and the second horizontal landmark, estimated by setting the first horizontal landmark and the second horizontal landmark as the two target landmarks, as the horizontal length information among the size information, and to derive the actual distance between the first vertical landmark and the second vertical landmark, estimated by setting the first vertical landmark and the second vertical landmark as the two target landmarks, as the vertical length information among the size information. Claim 7 A measurement method according to claim 1, further comprising: (d) a step of estimating weight information of the strawberry using the size information. Claim 8 A measurement method according to claim 1, wherein step (a) is to receive the 2D image data and the 3D scanning data from a measurement device having an image sensor for capturing the 2D image data and a LiDAR sensor for measuring the 3D scanning data. Claim 9 A standard measurement device for a strawberry, comprising: a collection unit for acquiring 3D scanning data including 2D image data measured for a strawberry, a point cloud including a plurality of points corresponding to the outer surface of the strawberry, and a depth map projected in a 2D form using distance information for each of the plurality of points; and a landmark detection unit for identifying a plurality of landmarks pre-set for the strawberry from the 2D image data. A measuring device comprising: a size analysis unit that calculates size information of the strawberry using the result of mapping the 2D image data and the 3D scanning data based on the plurality of landmarks; wherein the plurality of landmarks include a first horizontal landmark and a second horizontal landmark selected based on the horizontal direction among a plurality of pixels forming an object area corresponding to the strawberry, which is distinguished from the background area on the 2D image data, and a first vertical landmark and a second vertical landmark selected based on the vertical direction among the plurality of pixels; wherein the size analysis unit obtains target coordinate information in the form of 2D coordinate values ​​as position information in the depth map for each of the two target landmarks selected among the plurality of landmarks, and target distance information in the form of scalar values ​​as distance information to the target landmarks; calculates angle information formed between the lidar sensor and the two target landmarks using reference coordinate information, which is information about the position of the lidar sensor that measured the 3D scanning data, and the target coordinate information; and estimates the actual distance between the two target landmarks using the angle information and the target distance information. Claim 10 delete Claim 11 delete Claim 12 delete Claim 13 A measuring device according to claim 9, wherein the size analysis unit derives the actual distance between the first horizontal landmark and the second horizontal landmark, estimated by setting the first horizontal landmark and the second horizontal landmark as the two target landmarks, as horizontal length information among the size information, and derives the actual distance between the first vertical landmark and the second vertical landmark, estimated by setting the first vertical landmark and the second vertical landmark as the two target landmarks, as vertical length information among the size information. Claim 14 A measuring device according to claim 9, further comprising a weight analysis unit that estimates weight information of the strawberry using the size information. Claim 15 A measuring device according to claim 9, wherein the collecting unit receives the 2D image data and the 3D scanning data from a measuring device having an image sensor for capturing the 2D image data and a LiDAR sensor for measuring the 3D scanning data.

Citation Information

Patent Citations

  • Autonomous detection and treatment of agricultural objects via precision treatment delivery system

    US20220117215A1

  • Detecting multiple objects of interest in an agricultural environment

    US20220400596A1

  • Autonomous detection and control of vegetation

    WO2023069841A1