Pomelo fruit volume measurement method based on three-view image contour fitting

Abstract

Provided in the present invention is a pomelo fruit volume measurement method based on three-view image contour fitting, which is used for performing volume measurement on the basis of images and comprises: a plurality of image acquisition apparatuses acquiring surface images of a pomelo fruit to be measured; calibrating the position of an image acquisition apparatus used for acquiring the top view of said pomelo fruit; performing enhanced extraction processing on the surface images, so as to obtain standard contour images; on the basis of the standard contour images, obtaining shapes and dimensions of slices, and fitting contours of the pomelo fruit slices to obtain a B spline curve; according to the B spline curve, calculating the volume of a single pomelo fruit slice, and further summing to obtain the total volume of said pomelo fruit. The present invention is suitable for volume measurement of fruits of various shapes such as spherical, ellipsoidal, drop-shaped, cylindrical and the like, achieves relatively high measurement efficiency while ensuring high precision, and can be actually applied to a pomelo fruit sorting production line. The present invention has a simple structure and low cost, and only consists of three cameras and supports at fixed positions.

Description

A pomelo fruit volume measurement method based on three-view image contour fitting

[0001] This application claims priority to a Chinese patent application filed with the Patent Office of China on July 24, 2023, with application number 202310906194.0 and invention name “A method for measuring the volume of pomelo fruit based on three-view image contour fitting”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present invention relates to the technical field of volume measurement based on images, and in particular to a pomelo volume measurement method based on three-viewing angle image contour fitting. Background Art

[0003] Pomelo is a subtropical fruit rich in nutrients. China is a major producer of pomelo. In recent years, large-scale pomelo producers have gradually taken the lead in the Chinese pomelo market, requiring rapid, non-destructive testing and sorting of pomelo fruit quality in their production. Volume is a key indicator in pomelo inspection and sorting. Accurate volume measurement not only allows for preliminary size sorting but also provides critical data for accurate measurement of internal indicators such as edible rate and sugar content. Therefore, achieving high-precision and efficient volume measurement is crucial in pomelo inspection and sorting lines.

[0004] Traditional methods for measuring the volume of pomelo fruit are primarily manual, using the displacement method. However, manual measurement is labor-intensive, inefficient, and time-consuming. With the rapid development of computer and imaging technologies, machine vision has shown great potential in the non-destructive testing of pomelo fruit. Currently, various fruit volume measurement methods employing machine vision have been developed. Existing machine vision-based fruit volume measurement methods include ellipsoid fitting, slice integral measurement, multi-contour measurement, depth camera measurement, machine learning, and multi-view 3D reconstruction. The ellipsoid fitting method treats the fruit as an ellipsoid and approximates its volume. This method is suitable for fruits such as watermelon and cantaloupe, but suffers from high errors for non-ellipsoidal fruits. The slice circle fitting integral method breaks the fruit into thin slices along a specific direction, approximating each slice to a circle. The volume of each slice is calculated separately, and then the volume of the fruit is measured by superposition. The multi-contour surface approximation method uses a rotating platform with a selected rotation center. Images are captured at intervals of 6-12 degrees, and a curve approximation is used to fit a wireframe model. The depth camera method uses a single depth camera to measure volume, but suffers from poor accuracy. Machine learning measurement: A depth camera is used to extract geometric features from depth images and use these features to train a volume prediction model. Multi-view 3D reconstruction: Multi-angle images of the fruit are collected and a stereoscopic 3D reconstruction algorithm is used to restore the 3D model of the pomelo to calculate its volume.

[0005] Ellipsoid fitting, slice circle fitting integration, and depth camera measurement methods suffer from low accuracy. Machine learning measurement requires different training models for measuring different fruit varieties and sizes, and each new fruit requires retraining, which is limiting. Multi-contour surface approximation measurement is complex, requiring dozens of images of a single grapefruit, resulting in low efficiency. Multi-view 3D reconstruction requires a rotating platform and a long measurement cycle, making it unsuitable for production.

[0006] Based on the above, it can be seen that the current volume detection methods on the market have the problem of difficulty in balancing measurement accuracy and efficiency. There is an urgent need for a pomelo volume measurement method with simple structure, high accuracy and high efficiency.

[0007] Summary of the Invention

[0008] In view of this, the object of the present invention is to provide a pomelo volume measurement method based on three-view image contour fitting to solve the technical problems existing in the above-mentioned prior art.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] A pomelo fruit volume measurement method based on three-view image contour fitting, comprising:

[0011] Multiple image acquisition devices acquire surface images of the pomelo fruit to be tested;

[0012] Calibrate the position of the image acquisition device for acquiring the top view of the pomelo fruit to be tested;

[0013] Intensify and extract the surface image to obtain a standard contour image;

[0014] Obtaining the slice shape and size based on the standard contour image, and fitting the pomelo slice contour to obtain a B-spline curve;

[0015] The volume of a single pomelo fruit slice is calculated according to the B-spline curve, and the total volume of the pomelo fruit to be measured is obtained by summing up.

[0016] Optionally, the position of the image acquisition device for acquiring the top view of the pomelo fruit to be tested is calibrated as follows:

[0017] Place the calibration plate horizontally at the average height of the grapefruit. The intrinsic parameter matrix of camera A is:

[0018] Measure the vertical distance from the camera to the calibration plate as Z A ;

[0019] Among them, camera A shoots a top view of the pomelo lying flat vertically downward.

[0020] Optionally, performing contour extraction on the surface image specifically includes:

[0021] The surface image contains three channels: R, G, and B. The image is converted from a color space to an HSI color space, and the red R channel and the hue H channel are extracted. After threshold segmentation, they are superimposed. An edge detection operator is used to extract the contours in the superimposed image. The above processing is performed on the surface images of the pomelo fruit to be tested collected by three image acquisition devices, respectively, to obtain three contour images in total.

[0022] Optionally, performing enhanced extraction processing on the surface image to obtain a standard contour image specifically includes:

[0023] The surface image is sequentially subjected to contour extraction, contour image size adjustment, and contour vertical reference line determination to obtain a standard contour image.

[0024] Optionally, obtaining the slice shape and size based on the slice preprocessing result specifically includes:

[0025] Each row of pixels in the standard outline image is regarded as an image slice, and there are H slices from the fruit stem end downwards. Taking the i-th slice as an example, the black pixels of the image slice layer are counted from the left end of the image to the right, until the white pixels are reached. The number of black pixels is recorded as x. i1 , flip the image horizontally, and count the number of black pixels in the image from left to right as x i2 , w is the width of the image, then: u Li =x i1 ; u Ri =wx i2 ;

[0026] In the i-th image slice, the left endpoint of the grapefruit is A i To the vertical reference line M i Distance D Li : D Li =u M -u Li ;

[0027] In the i-th image slice, the right endpoint of the grapefruit is B i To the vertical reference line M i Distance D Ri : D Ri =u Ri -u M ;

[0028] Therefore, we can get two lists: the distance list D from the H left endpoints of the pomelo outline in the image to the vertical baseline L For: D L =[D L0 ,DL1 ,D L2 ,D L3 ,...,D LH ];

[0029] List D of the distances from the H right endpoints of the pomelo fruit outline in the image to the vertical reference line R For: D R =[D R0 ,D R1 ,D R2 ,D R3 ,...,D RH ];

[0030] In the three pictures captured by cameras A, B and C, the left and right endpoints of the grapefruit are A and C respectively. L 、A R 、B L 、B R 、C L and C R , the distances from the above endpoints to the vertical reference line M are: D LA 、D RA 、D LB 、D RB 、D LC and D RC ;

[0031] Among them, the angle between camera B and the horizontal plane is α, and the left side of the grapefruit is photographed; the angle between camera C and the horizontal plane is β, and the right side of the grapefruit is photographed.

[0032] Optionally, fitting the pomelo slice contour to obtain a B-spline curve specifically includes:

[0033] Fit the pomelo slice along the transverse direction in polar coordinates; take the vertical reference line M of the contour as the pole, and determine the slice endpoint A in polar coordinates according to the angle between the camera and the horizontal plane. L 、A R 、B L 、B R 、C L and C R The position of A is converted from polar coordinate system to rectangular coordinate system, and then the cubic B spline curve is used to calculate the L 、A R 、B L 、B R 、C L and C R Perform interpolation fitting to obtain the cubic B-spline curve P(t).

[0034] Optionally, calculating the volume of a single pomelo slice according to the B-spline curve specifically includes:

[0035] Calculate the B-spline curve P of the i-th slice i (t) The number of pixels surrounded is N i , then the volume of a single slice V i Approximately:

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention provides a rapid and high-precision method for detecting the volume of pomelo fruit, which is suitable for measuring the volume of fruits in various shapes, such as spheres, ellipsoids, teardrops, and columns. The present invention ensures high precision while having high detection efficiency, and can be practically applied to pomelo fruit sorting lines. The present invention also has a simple structure and low cost, consisting of only three fixed-position cameras and a bracket. The present invention also has high precision. Experiments show that the average error in measuring the volume of 120 pomelo fruits is 2.24%. Furthermore, the present invention has high detection efficiency, and the volume of the pomelo fruit can be measured while it is being transported on a conveyor belt, without the need for operations such as rotating the pomelo fruit.

[0038] Figures in the specification

[0039] The present invention will be further described below in conjunction with the accompanying drawings:

[0040] FIG1 is a flow chart of the method of the present invention;

[0041] FIG2 is a schematic diagram of the positions of multiple image acquisition devices;

[0042] FIG3 is a schematic diagram of image superposition during contour extraction;

[0043] FIG4 is a schematic diagram of image alignment results;

[0044] FIG5 is a schematic diagram of obtaining the slice size and shape;

[0045] Figure 6 is a schematic diagram of the spline curve fitting results of the pomelo slice contour. DETAILED DESCRIPTION

[0046] The following is a detailed description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work shall fall within the scope of protection of the present invention.

[0047] Example 1:

[0048] This embodiment discloses a pomelo volume measurement method based on three-view image contour fitting, as shown in FIG1 , comprising:

[0049] Step 101: multiple image acquisition devices acquire surface images of the pomelo fruit to be tested.

[0050] Step 102: calibrate the position of an image acquisition device for acquiring a top view of the pomelo fruit to be tested.

[0051] Step 103: Perform enhanced extraction processing on the surface image to obtain a standard contour image.

[0052] Step 104: Obtain the slice shape and size based on the standard contour image, and fit the pomelo slice contour to obtain a B-spline curve.

[0053] Step 105: Calculate the volume of a single pomelo fruit slice according to the B-spline curve, and further sum it up to obtain the total volume of the pomelo fruit to be measured.

[0054] Specifically:

[0055] During the specific implementation, the pomelo to be tested lies flat on a conveyor tray with a black light-absorbing material placed underneath. Three cameras A, B, and C are arranged in the transverse direction of the pomelo, as shown in Figure 2. Camera A shoots vertically downward, photographing the upper part of the pomelo; Camera B makes an angle α with the horizontal plane, photographing the left side of the pomelo; Camera C makes an angle β with the horizontal plane, photographing the right side of the pomelo.

[0056] A camera is used as the image acquisition device in this embodiment. In actual implementation, the camera is calibrated according to actual needs. Specifically, the position of the image acquisition device for acquiring the top view of the pomelo fruit to be tested is calibrated as follows:

[0057] Place the calibration plate horizontally at the average height of the grapefruit. The intrinsic parameter matrix of camera A is:

[0058] Measure the vertical distance from the camera to the calibration plate as Z A ;

[0059] Among them, camera A shoots a top view of the pomelo vertically downward.

[0060] Furthermore, contour extraction is performed on the surface image, including:

[0061] In actual implementation, each camera captures an image, and the three images represent the surface images of the pomelo fruit from different perspectives. The surface images contain three channels: R, G, and B. The images are converted to HSI color space, and the red R channel and hue H channel are extracted. After threshold segmentation and superposition, the images are then segmented and superimposed. An edge detection operator is used to extract the contours of the superimposed images. This processing is then repeated for the surface images of the pomelo fruit captured by the three image acquisition devices, resulting in three contour images. A schematic diagram of image superposition during contour extraction is shown in Figure 3, where (a) represents the H channel binary image and (b) represents the I channel binary image.

[0062] Among them, the surface image is enhanced and extracted to obtain a standard contour image, which specifically includes:

[0063] The surface image is sequentially subjected to contour extraction, contour image resizing, and vertical baseline determination to obtain a standard contour image. Specifically, image alignment involves segmenting the image according to the grapefruit's contours and resizing it using bilinear interpolation so that the height pixel count for all three images is H. A schematic diagram of the image alignment results is shown in Figure 4, where (a) represents the left-view contour image, (b) the right-view contour image, and (c) the top-view contour image.

[0064] Determine the vertical reference line u of the contour M : After image alignment, the image height is H, and all contour points with v < 10% H are taken, that is, the average value u of the contour points u at the edge of the fruit stem end M , at point (u M ,0) Draw a vertical line downward as the vertical reference line of the contour, and the horizontal coordinate of the vertical reference line is u M .

[0065] The slice shape and size are obtained based on the slice preprocessing results, specifically including:

[0066] Each row of pixels in the standard outline image is considered as an image slice, and there are H slices from the fruit stem end downward. Taking the i-th slice as an example, count the black pixels of the image slice from the left end to the right end until the white pixel, and record the number of black pixels as x i1 , flip the image horizontally, and count the number of black pixels in the image from left to right as x i2 , w is the width of the image, then: u Li =x i1 ; u Ri =wx i2 ;

[0067] In the i-th image slice, the left endpoint of the grapefruit is A i To the vertical reference line M i Distance D Li : D Li =u M -u Li ;

[0068] In the i-th image slice, the right endpoint of the grapefruit is B i To the vertical reference line M i Distance D Ri : D Ri =u Ri -u M ;

[0069] Therefore, we can get two lists: the distance list D from the H left endpoints of the pomelo outline in the image to the vertical baseline L For: D L =[D L0 ,D L1 ,D L2 ,D L3 ,...,D LH ];

[0070] List D of the distances from the H right endpoints of the pomelo fruit outline in the image to the vertical reference line R For: D R =[D R0 ,D R1 ,D R2 ,D R3 ,...,D RH ];

[0071] In the three pictures captured by cameras A, B and C, the left and right endpoints of the grapefruit are A and C respectively. L 、A R 、B L 、B R 、C L and C R , the distances from the above endpoints to the vertical reference line M are: D LA 、D RA 、D LB 、D RB 、D LC and D RC ;

[0072] Camera B, at an angle α to the horizontal plane, photographs the left side of the grapefruit. Camera C, at an angle β to the horizontal plane, photographs the right side of the grapefruit. A schematic diagram of obtaining the slice size and shape is shown in Figure 5. Part (a) of Figure 5 shows a schematic diagram of a grapefruit slice, and part (b) shows a schematic diagram of obtaining the outline size data.

[0073] Fitting the pomelo slice contour to obtain a B-spline curve includes:

[0074] Fit the pomelo slice along the transverse direction in polar coordinates. Take the vertical reference line M of the contour as the pole, and determine the slice endpoint A in polar coordinates according to the angle between the camera and the horizontal plane. L 、A R 、B L 、B R 、C L and C R The position of A is converted from polar coordinate system to rectangular coordinate system, and then the cubic B spline curve is used to calculate the L 、A R 、B L 、BR 、C L and C R Interpolation fitting is performed to obtain the cubic B-spline curve P(t). The schematic diagram of the spline curve fitting result of the pomelo slice contour is shown in Figure 6.

[0075] The volume of a single pomelo slice is calculated based on the B-spline curve, specifically including:

[0076] Calculate the B-spline curve P of the i-th slice i (t) The number of pixels surrounded is N i , then the volume of a single slice V i Approximately:

[0077] The total volume V of pomelo fruit is approximately:

[0078] The embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the scope of knowledge possessed by ordinary technicians in the relevant technical field without departing from the purpose of the present invention.

Claims

1. A method for measuring the volume of pomelo fruit based on three-view image contour fitting, characterized in that: include: A plurality of image acquisition devices acquire the surface image of the pomelo fruit to be tested; Calibrate the position of the image acquisition device used to acquire the top view of the pomelo fruit to be tested; Strengthening and extracting the surface image to obtain a standard contour image; Acquiring the slice shape and size based on the standard contour image, and fitting the pomelo slice contour to obtain a B-spline curve; The volume of a single pomelo fruit slice is calculated according to the B-spline curve, and further summed to obtain the total volume of the pomelo fruit to be measured.

2. The method for measuring the volume of pomelo fruit based on three-view image contour fitting according to claim 1, characterized in that: The position of the image acquisition device for acquiring the top view of the pomelo fruit to be tested is calibrated as follows: Place the calibration plate horizontally at the average height of the grapefruit, and the intrinsic parameter matrix of camera A is: Measure the vertical distance from the camera to the calibration plate as Z A ; Among them, camera A shoots a top view of the pomelo lying flat vertically downward.

3. The method for measuring the volume of pomelo fruit based on three-view image contour fitting according to claim 1, characterized in that: Performing contour extraction on the surface image specifically includes: The surface image includes three channels, R, G, and B. The image is converted from a color space into an HSI color space, the red R channel and the hue H channel are extracted, and the images are superimposed after threshold segmentation. The edge detection operator is used to extract the contour in the superimposed image. The above processing is performed on the surface images of the pomelo fruit to be tested collected by three image acquisition devices respectively, and three contour images are obtained in total.

4. The method for measuring the volume of pomelo fruit based on three-view image contour fitting according to claim 1, characterized in that: The surface image is enhanced and extracted to obtain a standard contour image, which specifically includes: The surface image is sequentially subjected to contour extraction, contour image size adjustment, and contour vertical reference line determination to obtain a standard contour image.

5. The method for measuring the volume of pomelo fruit based on three-view image contour fitting according to claim 1, characterized in that: Acquiring the slice shape and size based on the slice preprocessing result specifically includes: Each row of pixels in the standard contour image is regarded as an image slice, and there are H slices from the fruit stem end downward; taking the i-th slice as an example, the black pixels of the image slice layer are counted from the left end of the image to the right, until the white pixels are reached, and the number of black pixels is recorded as x i1 , flip the image horizontally, and count the number of black pixels in the image from left to right as x i2 , w is the width of the image, then: u Li =x i1 ; u Ri =w-x i2 ; In the i-th image slice, the left end point of the grapefruit is A i To the vertical reference line M i The distance D Li : D Li =u M -u Li ; In the i-th image slice, the right end point of the grapefruit is B i To the vertical reference line M i The distance D Ri : D Ri =u Ri -u M ; Therefore, we can get two lists: the distance list D from the H left endpoints of the pomelo fruit contour in the image to the vertical baseline L for: D L =[D L0 ,D L1 ,D L2 ,D L3 ,...,D LH ]; List D of the distances from the H right endpoints of the pomelo fruit outline in the image to the vertical reference line R for: D R =[D R0 ,D R1 ,D R2 ,D R3 ,...,D RH ]; In the three pictures captured by the three cameras A, B and C, the left and right endpoints of the grapefruit are A L , A R , B L , B R , C L and C R , the distances from the above endpoints to the vertical reference line M are: D LA , D RA , D LB , D RB , D LC and D RC ; Among them, the angle between camera B and the horizontal plane is α, and the left side of the grapefruit is photographed; the angle between camera C and the horizontal plane is β, and the right side of the grapefruit is photographed.

6. The method for measuring the volume of pomelo fruit based on three-view image contour fitting according to claim 1, characterized in that: Fitting the pomelo slice contour to obtain the B-spline curve includes: Fit the pomelo slice along the transverse direction in polar coordinates; take the vertical reference line M of the contour as the pole, and determine the slice endpoint A in polar coordinates according to the angle between the camera and the horizontal plane L , A R , B L , B R , C L and C R The position of the polar coordinate system is converted to a rectangular coordinate system, and then the cubic B-spline curve is used A L , A R , B L , B R , C L and C R Perform interpolation fitting to obtain the cubic B-spline curve P(t).

7. The method for measuring the volume of pomelo fruit based on three-view image contour fitting according to claim 1, characterized in that: Calculating the volume of a single pomelo fruit slice according to the B-spline curve specifically includes: Calculate the B-spline curve P of the i-th slice i (t) The number of pixels surrounded is N i , then the volume of a single slice V i Approximately:

Images