A method for measuring volume of a grapefruit based on three-view image contour fitting

Abstract

The application provides a method for measuring the volume of a pomelo based on three-view image contour fitting, which is used for volume measurement based on images, and comprises the following steps: a plurality of image acquisition devices acquire surface images of pomelos to be measured; the positions of the image acquisition devices used for acquiring top views of the pomelos to be measured are calibrated; the surface images are subjected to intensive extraction processing to obtain standard contour images; the shape and size of a slice are obtained based on the standard contour images, and a pomelo slice contour is fitted to obtain a B-spline curve; the volume of a single pomelo slice is calculated according to the B-spline curve, and the total volume of the pomelos to be measured is further summed up. The application is suitable for volume measurement of fruits in various shapes such as spherical, ellipsoidal, water-drop-shaped and cylindrical shapes. While ensuring high precision, the application has high detection efficiency and can be practically applied to a pomelo sorting assembly line. The application has a simple structure and low cost and is composed of only three fixed-position cameras and supports.

Description

Technical Field

[0001] This invention belongs to the field of image-based volume measurement technology, and particularly relates to a method for measuring the volume of grapefruit based on three-view image contour fitting. Background Technology

[0002] Grapefruit is a subtropical fruit, rich in nutrients. In recent years, some large-scale grapefruit production enterprises have gradually guided the grapefruit market, requiring rapid and non-destructive testing and sorting of grapefruit quality in their actual production. Volume is an important reference indicator in grapefruit testing and sorting. Accurate volume measurement can not only perform preliminary sorting of grapefruit by size, but also provide key data for the accuracy of internal indicators such as edibility and sugar content. Therefore, achieving high-precision and high-efficiency volume detection of grapefruit is crucial in grapefruit testing and sorting production lines. Traditional grapefruit volume measurement methods mainly rely on manual water displacement. However, manual measurement suffers from high labor intensity, low efficiency, and long processing time. With the rapid development of computer and image technology, machine vision has shown great potential in the field of non-destructive testing of grapefruit. Currently, various fruit volume measurement methods based on machine vision technology mainly include ellipsoidal fitting measurement, slice integration measurement, multi-contour measurement, depth camera measurement, machine learning measurement, and multi-view 3D reconstruction measurement. Ellipsoid Fitting Measurement Method: This method directly treats the fruit being measured as an ellipsoid to approximate its volume. It is suitable for fruits such as watermelons and cantaloupes, but has a larger error for non-ellipsoidal fruits. Slice Circle Fitting Integral Measurement Method: This method decomposes the fruit into several thin slices along a specific direction, approximating each slice as a circle, calculating the volume of each slice separately, and finally superimposing the volumes to obtain the total fruit volume. Multi-Contour Surface Approximation Measurement Method: This method builds a rotating platform, selects a rotation center, and takes images every 6-12 degrees, using a curve to approximate and fit a wireframe model. Depth Camera Measurement Method: This method uses a single depth camera to measure volume, but has poor accuracy. Machine Learning Measurement Method: This method uses a depth camera to extract geometric features from the depth images and trains a volume prediction model using these features. Multi-View 3D Reconstruction Method: This method acquires multi-angle images of the fruit being measured and uses a stereo vision 3D reconstruction algorithm to reconstruct the 3D model of the fruit, thereby calculating its volume.

[0003] Ellipsoid fitting, slice circle fitting integration, and depth camera measurement methods all suffer from low accuracy. Machine learning measurement methods require different training models for measuring different varieties and sizes of fruit, necessitating model retraining for each new type of fruit, thus limiting their effectiveness. Multi-contour surface approximation measurement methods are structurally complex, requiring dozens of images for a single grapefruit, resulting in low efficiency. Multi-view 3D reconstruction requires a rotating platform and is time-consuming per measurement, making it unsuitable for practical production.

[0004] As can be seen from the above, the current volume detection methods on the market cannot achieve both measurement accuracy and efficiency. There is an urgent need for a simple, accurate, and efficient method for measuring the volume of grapefruit. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method for measuring the volume of grapefruit based on three-view image contour fitting, so as to solve the technical problems existing in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for measuring grapefruit volume based on three-view image contour fitting includes:

[0008] Multiple image acquisition devices captured images of the surface of the grapefruit to be tested;

[0009] The position of the image acquisition device used to acquire a top view of the grapefruit to be tested is calibrated;

[0010] The surface image is enhanced and extracted to obtain a standard contour image;

[0011] The slice shape and size are obtained based on the standard contour image, and a B-spline curve is obtained by fitting the grapefruit slice contour.

[0012] The volume of a single grapefruit slice is calculated based on the B-spline curve, and then summed to obtain the total volume of the grapefruit to be tested.

[0013] Preferably, the position of the image acquisition device used to acquire a top view of the grapefruit to be tested is specifically determined as follows:

[0014] The calibration board is placed horizontally at the average height of the grapefruit. The intrinsic parameter matrix of camera A is... ;

[0015] The vertical distance from the camera to the calibration plate is measured as follows: ;

[0016] Camera A takes a top-down view of a grapefruit lying flat, positioned vertically downwards.

[0017] Preferably, contour extraction of the surface image specifically includes:

[0018] The surface image contains three channels: R, G, and B. The image is converted to the HSI color space, the red R channel and the hue H channel are extracted, thresholded and superimposed, and the contours in the superimposed image are extracted using an edge detection operator. The above processing is performed on the surface images of the grapefruit to be tested acquired by the three image acquisition devices respectively, resulting in three contour images.

[0019] Preferably, the surface image enhancement and extraction process to obtain a standard contour image specifically includes:

[0020] The surface image is sequentially subjected to contour extraction, contour image size adjustment, and determination of the vertical baseline of the contour to obtain a standard contour image.

[0021] Preferably, obtaining the slice shape and size based on the slice preprocessing result specifically includes:

[0022] Each row of pixels in the standard contour image is treated as an image slice, resulting in a total of H slices from the stem end downwards. Taking the i-th slice as an example, the number of black pixels in this image slice is counted from the left end of the image to the right, up to the white pixels, and the number of black pixels is recorded as . Flip the image horizontally and count the number of black pixels from left to right. If the width of the image is a numerical value, then... ;

[0023] In the i-th image slice, the left endpoint of the grapefruit To the vertical baseline distance : ;

[0024] In the i-th image slice, the right endpoint of the grapefruit To the vertical baseline distance : ;

[0025] Therefore, we can obtain two lists: a list of distances from the H left endpoints of the grapefruit outline in the image to the vertical baseline. for: ;

[0026] List of distances from the H right endpoints of the grapefruit outline in the image to the vertical baseline. for: ;

[0027] In the three images captured by cameras A, B, and C, the left and right endpoints of the pomelo are listed as follows: and The distances from the above endpoints to the vertical baseline M are listed below: and .

[0028] Among them, the angle between camera B and the horizontal plane is The camera is positioned so that it can be positioned to capture the left side of the pomelo; the angle between camera C and the horizontal plane is [angle not specified]. Take a picture of the right side of the pomelo.

[0029] Preferably, the B-spline curve is obtained by fitting the contour of a grapefruit slice, specifically including:

[0030] Fit the grapefruit slices along the horizontal direction in polar coordinates; using the vertical baseline M as the pole, determine the slice endpoints in polar coordinates based on the angle between the camera and the horizontal plane. and The position; convert the polar coordinate system to a rectangular coordinate system, and then use a cubic B-spline curve to... and Perform interpolation fitting to obtain a cubic B-spline curve. .

[0031] Preferably, calculating the volume of a single grapefruit slice based on the B-spline curve specifically includes:

[0032] Calculate the B-spline curve of the i-th slice. The number of surrounding pixels is The volume of a single slice Approximately: .

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] This invention provides a rapid and high-precision method for detecting the volume of pomelos, applicable to the volume measurement of fruits of various shapes such as spheres, ellipsoids, teardrops, and cylinders. While ensuring high precision, this invention also boasts high detection efficiency, making it practically applicable to pomelos sorting production lines. Furthermore, this invention features a simple structure and low cost, consisting only of three fixed-position cameras and a support frame. It also exhibits high precision; experiments showed an average error of 2.24% when measuring the volume of 120 pomelos. Moreover, this invention offers high detection efficiency, as the volume measurement can be completed simultaneously with the pomelo's transport on the conveyor belt, eliminating the need for rotating the pomelo. Attached Figure Description

[0035] Figure 1 This is a flowchart of the method of the present invention;

[0036] Figure 2 This is a schematic diagram showing the locations of multiple image acquisition devices;

[0037] Figure 3 (a)-(b) are schematic diagrams of image overlay during contour extraction;

[0038] Figure 4 (a)-(c) are schematic diagrams of the image alignment results;

[0039] Figure 5 (a)-(b) are schematic diagrams of obtaining the slice size and shape;

[0040] Figure 6 A schematic diagram showing the results of fitting a spline curve to the outline of a grapefruit slice. Detailed Implementation

[0041] The technical solutions provided by the present invention will be described in detail below with reference to the embodiments, but they should not be construed as limiting the scope of protection of the present invention.

[0042] Example 1:

[0043] This embodiment discloses a method for measuring the volume of grapefruit based on three-view image contour fitting, including:

[0044] Multiple image acquisition devices captured images of the surface of the grapefruit to be tested;

[0045] The position of the image acquisition device used to collect a top view of the grapefruit to be tested is calibrated;

[0046] The surface image is enhanced and extracted to obtain a standard contour image;

[0047] The shape and size of the slices are obtained based on the standard contour image, and the B-spline curve is obtained by fitting the contour of the grapefruit slices.

[0048] The volume of a single grapefruit slice is calculated based on the B-spline curve, and then summed to obtain the total volume of the grapefruit to be tested.

[0049] Specifically:

[0050] In practice, the pomelo to be tested is placed flat on a conveyor belt tray, with black light-absorbing material placed underneath. Three cameras, A, B, and C, are positioned along the horizontal direction of the tray. Camera A shoots vertically downwards, capturing the upper part of the pomelo; camera B makes an angle of [missing information] with the horizontal plane. Photograph the left side of the pomelo; the angle between C and the horizontal plane is... Take a picture of the right side of the pomelo.

[0051] In this embodiment, a camera is used as the image acquisition device. In actual implementation, the camera will be calibrated according to actual needs. Specifically, the location of the image acquisition device used to acquire a top-view image of the grapefruit to be tested will be calibrated as follows:

[0052] The calibration board is placed horizontally at the average height of the grapefruit. The intrinsic parameter matrix of camera A is... ; ;

[0053] The vertical distance from the camera to the calibration plate is measured as follows: ;

[0054] Camera A takes a top-down view of a grapefruit lying flat, positioned vertically downwards.

[0055] Furthermore, contour extraction is performed on the surface image, specifically including:

[0056] In actual implementation, each camera captures one image, and the three images are surface images of the pomelo from different perspectives. The surface image contains three channels: R, G, and B. The image is converted to the HSI color space, the red R channel and the hue H channel are extracted, thresholded and superimposed, and the contours in the superimposed image are extracted using an edge detection operator. The above processing is performed on the surface images of the pomelo to be tested captured by the three image acquisition devices respectively, and a total of three contour images are obtained.

[0057] The surface image enhancement and extraction process, which yields a standard contour image, specifically includes:

[0058] The surface image is sequentially processed by contour extraction, contour image size adjustment, and determination of the vertical baseline of the contour to obtain a standard contour image. Specifically: Image alignment: Based on the grapefruit contour segmentation image, the image size is adjusted using bilinear interpolation so that the number of pixels in the height direction of the three images is H.

[0059] Determine the vertical baseline of the outline After image alignment, the image height is H. All contour points, i.e., contour points of the fruit stalk end edge. Average of coordinates At point Draw a vertical line downwards as the vertical baseline of the outline. The x-coordinate of the vertical baseline is... .

[0060] The process of obtaining the slice shape and size based on the slice preprocessing results specifically includes:

[0061] Each row of pixels in the standard contour image is treated as an image slice, resulting in a total of H slices from the stem end downwards. Taking the i-th slice as an example, the number of black pixels in this image slice is counted from the left end of the image to the right, up to the white pixels, and the number of black pixels is denoted as H. Flip the image horizontally and count the number of black pixels from left to right. If the width of the image is a numerical value, then... ;

[0062] In the i-th image slice, the left endpoint of the grapefruit To the vertical baseline distance : ;

[0063] In the i-th image slice, the right endpoint of the grapefruit To the vertical baseline distance : ;

[0064] Therefore, we can obtain two lists: a list of distances from the H left endpoints of the grapefruit outline in the image to the vertical baseline. for: ;

[0065] List of distances from the H right endpoints of the grapefruit outline in the image to the vertical baseline. for: ;

[0066] In the three images captured by cameras A, B, and C, the left and right endpoints of the pomelo are listed as follows: and The distances from the above endpoints to the vertical baseline M are listed below: and ;

[0067] Among them, the angle between camera B and the horizontal plane is The camera is positioned so that it can be positioned to capture the left side of the pomelo; the angle between camera C and the horizontal plane is [angle not specified]. Take a picture of the right side of the pomelo.

[0068] B-spline curves were obtained by fitting the contours of grapefruit slices, specifically including:

[0069] Fit a grapefruit slice along the horizontal direction in polar coordinates. Using the vertical baseline M as the pole, determine the slice endpoints in polar coordinates based on the angle between the camera and the horizontal plane. and The position; convert the polar coordinate system to a rectangular coordinate system, and then use a cubic B-spline curve to... and Perform interpolation fitting to obtain a cubic B-spline curve. .

[0070] The volume of a single grapefruit slice is calculated based on the B-spline curve, specifically including:

[0071] Calculate the B-spline curve of the i-th slice. The number of surrounding pixels is The volume of a single slice Approximately: .

[0072] The total volume V of the pomelo is approximately: .

[0073] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for measuring the volume of grapefruit based on three-view image contour fitting, characterized in that, include: Multiple image acquisition devices captured images of the surface of the grapefruit to be tested; The position of the image acquisition device used to acquire a top view of the grapefruit to be tested is calibrated; The calibration board is placed horizontally at the average height of the grapefruit. The intrinsic parameter matrix of camera A is... ; The vertical distance from the camera to the calibration plate is measured as follows: ; Among them, camera A vertically downwards captures a top view of the pomelo lying flat; The angle between camera B and the horizontal plane is The camera is positioned so that it can be positioned to capture the left side of the pomelo; the angle between camera C and the horizontal plane is [angle not specified]. Take a picture of the right side of the pomelo; The surface image is enhanced and extracted to obtain a standard contour image; The surface image contains three channels: R, G, and B. The image is converted to the HSI color space, the red R channel and the hue H channel are extracted, thresholded and superimposed, and the contours in the superimposed image are extracted using an edge detection operator. The above processing is performed on the surface images of the grapefruit to be tested acquired by the three image acquisition devices respectively, and a total of three contour images are obtained. The surface image is sequentially subjected to contour extraction, contour image size adjustment and determination of the vertical baseline of the contour to obtain a standard contour image; The slice shape and size are obtained based on the standard contour image, and a B-spline curve is obtained by fitting the grapefruit slice contour. The volume of a single grapefruit slice is calculated based on the B-spline curve, and then summed to obtain the total volume of the grapefruit to be tested. Calculate the B-spline curve of the i-th slice. The number of surrounding pixels is The volume of a single slice Approximately: .

2. The method for measuring grapefruit volume based on three-view image contour fitting according to claim 1, characterized in that, The slice shape and size are obtained based on the slice preprocessing results, specifically including: Each row of pixels in the standard contour image is taken as an image slice, and a total of H slices are made from the stem end downwards. Taking the i-th slice as an example, the number of black pixels in the image slice is counted from the left end of the image to the right, up to the white pixels, and the number of black pixels is recorded as . Flip the image horizontally and count the number of black pixels from left to right. Let w be the width of the image. ; In the i-th image slice, the left endpoint of the grapefruit To the vertical baseline distance : ; In the i-th image slice, the right endpoint of the grapefruit To the vertical baseline distance : ; Therefore, we can obtain two lists: a list of distances from the H left endpoints of the grapefruit outline in the image to the vertical baseline. for: ; List of distances from the H right endpoints of the grapefruit outline in the image to the vertical baseline. for: ; In the three images captured by cameras A, B, and C, the left and right endpoints of the pomelo are listed as follows: and The distances from the above endpoints to the vertical baseline M are listed below: and .

3. The method for measuring grapefruit volume based on three-view image contour fitting according to claim 1, characterized in that, B-spline curves were obtained by fitting the contours of grapefruit slices, specifically including: Fit the grapefruit slices along the horizontal direction in polar coordinates; using the vertical baseline M as the pole, determine the slice endpoints in polar coordinates based on the angle between the camera and the horizontal plane. and The position; convert the polar coordinate system to a rectangular coordinate system, and then use a cubic B-spline curve to... and Perform interpolation fitting to obtain a cubic B-spline curve. .

Images