A non-contact kiwifruit ripeness detection system and method

Abstract

The application discloses a non-contact large-batch kiwi maturity detection system and method, which comprises a high-frame-rate video acquisition camera, a vibration platform and a processor, video images of kiwi micro-vibration are collected and transmitted to the processor for video analysis and calculation, and the position of the kiwi is located; the video micro-motion amplification technology is used to amplify the tiny displacement of the kiwi skin when the kiwi is vibrated to be observable; the centroid position of the kiwi is detected through binarization processing and edge filtering of the amplified image, the displacement change of the kiwi is recognized through video multi-frame image, the maturity grade of the kiwi is set through threshold setting of the amplified displacement image, and the maturity of the kiwi is calibrated through the calculated displacement change. Compared with the present kiwi pressing through a fruit hardness tester, the application realizes non-destructive detection and processing of large-batch and rapid grading of kiwi through visual detection of the hardness of the kiwi, improves the market competitiveness of kiwi fruits in China, and has important significance for continuous development and growth of kiwi fruits in the future.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of intelligent agriculture, and particularly relates to a non-contact large-batch kiwi maturity detection system and method. BACKGROUND

[0002] As a fruit rich in various vitamins and other nutrients, kiwi is sweet and sour, and is deeply loved by the public. However, the quality of kiwi cannot be accurately classified due to the inability to well identify the maturity, so it is difficult for consumers to determine whether the kiwi is mature by appearance when purchasing, resulting in that the quality of kiwi cannot well correspond to its value. The maturity of kiwi is usually determined by the hardness of the fruit, and whether the kiwi is mature can be determined by the hardness of the fruit. Unripe kiwi fruit is hard, has no aroma, and tastes bitter, indicating that it is not suitable for eating; ripe kiwi fruit is relatively soft, has fruit aroma, and tastes sweet, indicating that it is suitable for eating; overripe kiwi fruit is very soft and has an unusual smell, indicating that it is not suitable for eating.

[0003] Today's kiwi detection technology is not perfect. During detection, a fruit hardness tester is used to press the kiwi so that it cannot be eaten, and a sweetness tester is used to detect the sweetness of the fruit. These methods are not widely used in production and life, and more artificial sorting is used to determine the maturity of kiwi. This method not only requires a lot of manpower, but also easily causes damage to the fruit, and the detection efficiency is very low.

[0004] Considering the above factors, the present application provides a non-contact large-batch kiwi maturity detection system and method. The quality indicators inside the kiwi are identified by visual detection of the hardness of the kiwi, realizing non-contact rapid classification and non-destructive detection of a large number of kiwis, improving the market competitiveness of kiwi fruits in China, improving the overall value and export quantity of commodities, and having important significance for the continuous development and growth of kiwi fruits in the future. SUMMARY

[0005] The present application provides a non-contact large-batch kiwi maturity detection system and method, which uses a high-frame-rate video acquisition camera to accurately access the vibration image of kiwi, relies on the mutation recognition of adjacent pixel gray values of image processing to locate the skin boundary point position of kiwi. When the kiwi is shot by a visual sensor, the surface deformation of kiwis of different maturity is different under the same small vibration. The small displacement X-t image is enlarged by using micro-motion amplification technology, realizing the amplification processing of small displacement. Taking mature kiwi as a standard value, the small displacement under the same condition is set as a threshold value to distinguish whether the kiwi is mature. The equipment is reasonably adjusted to realize the rapid communication of the high-frame-rate video acquisition camera and the processor, and a large amount of pixel point information is synchronously transmitted, so that the whole system can orderly judge the maturity grade of a large number of kiwis in operation, realizing non-destructive detection.

[0006] 10. The technical solution adopted by the present application to solve its technical problems is: a non-contact large-batch kiwi maturity detection system, comprising a high-frame-rate video acquisition camera, a vibration platform and a processor;

[0007] The vibration platform comprises a vibration module; the vibration module is used to realize micro-vibration processing of kiwi through the vibration platform;

[0008] The high-frame-rate video acquisition camera comprises a video acquisition module; the video acquisition module is used to acquire video images of kiwi micro-vibration and transmit them to the processor for video analysis and calculation;

[0009] The processor combines video image information transmitted by the high-frame-rate video acquisition camera, including a kiwi recognition module, a video micro-motion amplification module and a maturity calculation module;

[0010] The kiwi recognition module is used to analyze the images acquired by the camera to identify the surface boundary of kiwi through the sudden change of the gray value of adjacent pixels in the image, and to locate the position of kiwi;

[0011] The video micro-motion amplification module is used to amplify the tiny displacement of the kiwi skin when it is vibrated to be observable through video image micro-motion amplification technology, including video data decomposition processing, video data acquisition processing, video signal denoising processing, video sequence amplification processing and video amplification construction processing;

[0012] The maturity calculation module comprises a maturity detection module and a maturity calibration module;

[0013] The maturity detection module is used to identify the position of the kiwi platform through the amplified gray image, to detect the centroid position of the kiwi through edge filtering, and to identify the displacement change of the kiwi through multiple frames of video images;

[0014] The maturity calibration module is used to set the maturity grade of kiwi as unripe, ripe and overripe through threshold setting of the amplified displacement image, and to calibrate the maturity of kiwi through the calculated displacement change.

[0015] A control method of a non-contact large-batch kiwi maturity detection system, characterized in that it comprises the following steps:

[0016] Collecting kiwi micro-vibration video data: the vibration module is used to realize micro-vibration processing of kiwi through the vibration platform; the video acquisition module is used to acquire video images of kiwi micro-vibration and transmit them to the processor for video analysis and calculation;

[0017] Kiwifruit recognition positioning: image analysis by camera collection, identifying the surface boundary of kiwifruit by the mutation of adjacent pixel gray value of the image, and marking the position of kiwifruit with a rectangular frame, and then image processing;

[0018] Video data decomposition processing: using complex-valued steerable pyramid to perform spatial decomposition on the collected kiwifruit image data frame by frame, obtaining images of different directions and different scales, and separating the amplitude and phase of the kiwifruit video image by Fourier transform method.

[0019] Video data acquisition processing: using time domain band pass filter to process the phase signal to obtain the data information of kiwifruit micro-motion amplification.

[0020] Video signal denoising processing: noise processing by setting appropriate Gaussian filter to improve signal noise ratio.

[0021] Video sequence amplification processing: motion amplification processing is performed on the displacement change signal δ(t) of the kiwifruit micro-motion after band pass filtering, and the motion amplified sequence f(x+(1+α)δ(t)) is obtained.

[0022] Video amplification construction processing: construction analysis is performed on the displacement video of the kiwifruit after amplification processing, and the video micro-motion data information after motion amplification is obtained.

[0023] Maturity detection calculation: the position of the kiwifruit on the platform is identified by recognizing the amplified gray image, the centroid position of the kiwifruit is detected by edge filtering, and the displacement change is identified by video multi-frame image.

[0024] Maturity calibration: the maturity grade of kiwifruit is set as unripe, mature and overripe by setting threshold value for amplified displacement image, and the maturity of kiwifruit is calibrated by calculated displacement change.

[0025] In the above scheme, the specific steps of collecting kiwifruit micro-vibration video data are: realizing the micro-vibration of kiwifruit by vibration platform, so that the kiwifruit realizes the micro-vibration; using high frame rate video acquisition camera to collect video image of kiwifruit micro-vibration and transmit to processor for video analysis and calculation;

[0026] In the above scheme, the specific steps of kiwifruit recognition positioning are:

[0027] Step S1: accurately accessing kiwifruit micro-vibration image by using high frame rate video acquisition camera;

[0028] Step S2: distinguishing kiwifruit from background color block by pixel threshold value 90 of image pixel gray value, and performing image binary processing, and identifying the position of kiwifruit by the mutation of edge pixel value 0 and 1;

[0029] Step S3: Position the boundary point position of the kiwi fruit, calibrate the kiwi fruit position with a rectangular frame icon, and intercept its image for calculation processing.

[0030] In the above scheme, the specific steps of the video data decomposition processing are: using complex manipulable pyramids with Fourier transform to perform spatial domain decomposition on the intercepted kiwi fruit image data frame by frame, obtaining images of different directions and different scales, separating the amplitude and phase analysis of the kiwi fruit video image, using global Fourier transform, one-dimensional kiwi fruit image profile f under global translation f(x+δ(t)), for the magnification factor α, using Fourier series decomposition, taking the displacement image profile f(x+δ(t)) as a sum of complex sines, where each frequency band corresponds to a single frequency ω.

[0031]

[0032] The frequency ω band is a complex sine wave

[0033] S ω (x,t)=A ω e iω(x+δ(t))

[0034] Since S ω is a sine wave, its phase ω(x+δ(t) contains motion information, and motion can be manipulated by modifying the phase. S ω (x,y) is a complex sine wave, and its motion is exactly 1+α times the input. The motion amplified video can be reconstructed by folding the pyramid, and the motion amplified sequence f(x+(1+α)δ(t)) can be obtained by summing all subbands.

[0035] In the above scheme, the specific steps of the video data acquisition processing are: using a suitable band-pass filter to filter the acquired phase signal, and limiting the displacement αδ(t) to make the amplified displacement well approximate to the real displacement signal. A standard deviation of a Gaussian window is used as a boundary, so that the data information acquisition of the kiwi fruit micro-motion amplification is obtained.

[0036] In the above scheme, the specific steps of the video signal denoising processing are: the noise in the input sequence will cause noise in the phase signal itself, thereby causing incorrect motion to be amplified. An amplitude weighted and Gaussian blurred phase is used, and a suitable Gaussian filter is set for noise processing, thereby improving the signal-to-noise ratio of the signal.

[0037] In the above scheme, the specific steps of the video sequence amplification processing are: collecting the phase of the effective kiwi data through a band-pass filter, and the time band-pass phase corresponds to the motion of different spatial scales and directions. In order to synthesize the amplified motion, multiply the band-pass phase by the amplification factor a. Then use these amplified phase differences to amplify the kiwi micro-motion in the sequence, through the phase modification of each coefficient of each frame. The motion amplification processing is performed on the displacement change signal δ (t) of the kiwi micro-motion after band-pass filtering, and the result of the action amplified by a times is obtained,

[0038]

[0039] In order to separate the changing part, a first-order Taylor series expansion is used to approximate the action expressed by the formula:

[0040]

[0041] Let the result of the band-pass filtering in the last step be B (x, t), and assume that the frequency range of all the change signals δ (t) is exactly within the frequency band range of the band-pass filtering, then

[0042]

[0043] Amplify the action approximated by the formula by multiplying the changing part by an amplification multiple a and adding it back to the original signal. That is:

[0044]

[0045] Solving the formula together, we can get:

[0046]

[0047] In this ideal case, The motion amplification sequence f (x + (1 + a) δ (t)) is obtained.

[0048] In the above scheme, the specific steps of the video amplification construction processing are: when the kiwi photographed by the visual sensor is subjected to the same slight vibration, the surface deformation of different maturity is different, and the slight deformation f (t) - t slice is amplified by the video slight displacement amplification to realize the video recombination of the amplified image, thereby realizing the slight displacement amplification processing.

[0049] In the above scheme, the specific steps of the maturity detection calculation are: as a typical respiratory burst type of fruit, the kiwi gradually softens from hard to edible during its growth period. Therefore, it shows the continuous softening of the kiwi fruit. In order to detect the maturity of the kiwi and without damaging it, the slight vibration can be used to show the hardness of its surface layer.

[0050] Step M1: sobei edge detection of kiwifruit is carried out;

[0051] Step M2: the regionprops function processing is carried out on the intercepted image, the region characteristics and the centroid position are detected;

[0052] Step M3: the centroid position of each frame image of the video is saved, and the small displacement is calculated, and the array is taken as an example, the displacement change is calculated, and is saved in the array.

[0053]

[0054] In the scheme, the specific steps of the maturity calibration are as follows: the maturity of kiwifruit is usually represented by its hardness, the maturity levels of kiwifruit are set as unripe, mature and overripe through threshold setting of the amplified displacement image, and the maturity of kiwifruit is calibrated according to the calculated displacement change;

[0055] When the kiwifruit is unripe, the surface hardness is too high, the friction with the vibration table is small, the displacement amplitude x<ω1 when subjected to slight vibration, and the unripe is indicated.

[0056] When the kiwifruit is mature, the surface hardness is moderate, the friction with the vibration table is moderate, the displacement amplitude ω1<x<ω2 when subjected to slight vibration is low, and the mature is indicated.

[0057] When the kiwifruit is overripe, the surface hardness is too low, the friction with the vibration table is large, the displacement amplitude x>ω2 when subjected to slight vibration is large, and the overripe is indicated.

[0058] Compared with the prior art, the beneficial effects of the present application are:

[0059] A non-contact kiwifruit maturity detection system and method are provided. The present kiwifruit detection technology is imperfect, and the fruit hardness meter is pressed during detection, so that the kiwifruit cannot be eaten again, the sweetness of the fruit is detected by using a sweetness meter, and the method is not widely used in production and life. More artificial sorting is the main method for distinguishing the maturity of kiwifruit, which not only has high demand for manpower, but also is easy to cause damage to the fruit, and the detection efficiency is very low.

[0060] The present application identifies the quality index inside the kiwifruit by visual detection of the hardness of the kiwifruit, realizes non-contact rapid grading of a large number of kiwifruits, achieves nondestructive detection processing, improves the market competitiveness of kiwifruit in China, improves the overall value and export quantity of commodities, and has important significance for the continuous development and growth of kiwifruit in the future. BRIEF DESCRIPTION OF DRAWINGS

[0061] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description, taken in conjunction with the following drawings of which:

[0062] Figure 1 is a hardware build chart of the present application.

[0063] Figure 2 is a workflow chart of the present application.

[0064] Figure 3 is a kiwi contrast image based on video micro-motion amplification of the present application.

[0065] Figure 4 is a kiwi micro-motion processing image based on video micro-motion amplification of the present application.

[0066] Figure 5 is a kiwi displacement collection image based on video micro-motion amplification of the present application. DETAILED DESCRIPTION

[0067] Embodiments of the present application are described in detail below with reference to the attached drawing figures, wherein the same or like reference numerals and characters throughout the figures denote the same elements or features.

[0068] In the description of the present application, it is to be understood that the orientations or positional relationships indicated by the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "axial", "radial", "vertical", "horizontal", "inner", "outer", and the like are based on the orientations or positional relationships shown in the drawings, and are merely for the purpose of facilitating the description of the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features referred to. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0069] In the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting", "fixing" and the like should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above-mentioned terms in the present application can be understood according to the specific circumstances.

[0070] The present application realizes a non-contact large-batch kiwi maturity detection system and method, which accurately accesses kiwi vibration images by using a high-frame-rate video acquisition camera, and relies on image processing to identify and locate the boundary point position of the kiwi skin. When the kiwi is subjected to the same slight vibration, the visual sensor captures the deformation of the kiwi surface, which is different for different maturity. The slight displacement X-t image is amplified by using the micro-motion amplification technology, and the amplification processing of the slight displacement is realized. Taking the mature kiwi as the standard value, the slight displacement under the same condition is set as the threshold value, and the maturity of the kiwi is distinguished. The equipment is reasonably adjusted, the high-frame-rate video acquisition camera and the processor are quickly communicated, and a large amount of pixel point information is synchronously transmitted, so that the whole system can orderly judge the maturity grade of the large-batch kiwi in operation, and non-destructive detection is realized.

[0071] Figure 1 The hardware building diagram of the non-contact large-batch kiwi maturity detection system comprises a high-frame-rate video acquisition camera, a vibration platform and a processor.

[0072] The vibration platform comprises a vibration module; the vibration module is used for realizing the slight vibration processing of the kiwi through the vibration platform;

[0073] The high-frame-rate video acquisition camera comprises a video acquisition module; the video acquisition module is used for acquiring the video image of the kiwi in slight vibration and transmitting the video image to the processor for video analysis and calculation;

[0074] The processor combines the video image information transmitted by the high-frame-rate video acquisition camera, and comprises a kiwi recognition module, a video micro-motion amplification module and a maturity calculation module;

[0075] The kiwi recognition module is used for analyzing the image acquired by the camera, identifying the surface boundary of the kiwi through the sudden change of the gray value of the adjacent pixel points of the image, and positioning the position of the kiwi;

[0076] The video micro-motion amplification module is used for amplifying the slight displacement of the kiwi skin to be observable through the video image micro-motion amplification technology when the kiwi is subjected to vibration, and comprises video data decomposition processing, video data acquisition processing, video signal denoising processing, video sequence amplification processing and video amplification construction processing.

[0077] The maturity detection module is used for identifying and calibrating the platform position where the kiwi fruit is located by recognizing the enlarged gray image, detecting the centroid position of the kiwi fruit by edge filtering, and recognizing the displacement change of the kiwi fruit by video multi-frame image.

[0078] The maturity calibration module is used for calibrating the maturity of the kiwi fruit by setting the threshold value of the enlarged displacement image, and setting the maturity level of the kiwi fruit as unripe, ripe and overripe, and calibrating the maturity of the kiwi fruit by the calculated displacement change.

[0079] Figure 2 The working flow chart of the control method of the kiwi fruit maturity detection system based on video micro-motion amplification of the application, characterized by comprising the following steps:

[0080] Collecting kiwi fruit micro-vibration video data: the vibration module is used for realizing the micro-vibration processing of the kiwi fruit through the vibration platform; the video acquisition module is used for collecting the video image of the kiwi fruit during micro-vibration and transmitting the video image to the processor for video analysis and calculation;

[0081] Kiwi fruit recognition and positioning: the image collected by the camera is analyzed, the surface boundary of the kiwi fruit is recognized according to the sudden change of the gray value of the adjacent pixel points of the image, the position where the kiwi fruit is located is calibrated by a rectangular frame, and the image is intercepted for processing;

[0082] Video data decomposition processing: the complex-valued controllable pyramid is used to perform spatial decomposition on the collected and intercepted kiwi fruit image data frame by frame, different direction and different scale images are obtained, and the amplitude and phase analysis of the kiwi fruit video image is separated by the Fourier transform method.

[0083] Video data acquisition processing: the time domain band-pass filter is used to process the phase signal to obtain the data information of the kiwi fruit micro-motion amplification.

[0084] Video signal denoising processing: the appropriate Gaussian filter is set to process the noise, so as to improve the signal noise ratio.

[0085] Video sequence amplification processing: the motion amplification processing is performed on the displacement change signal δ(t) of the kiwi fruit micro-motion after the band-pass filtering, and the motion amplification sequence f(x+(1+α)δ(t)) is obtained.

[0086] Video amplification construction processing: the displacement video of the kiwi fruit after the amplification processing is analyzed, and the video micro-motion data information after the motion amplification is obtained.

[0087] Maturity detection calculation: the platform position where the kiwi fruit is located is calibrated by recognizing the enlarged gray image, the centroid position of the kiwi fruit is detected by edge filtering, and the displacement change of the kiwi fruit is recognized by video multi-frame image.

[0088] The maturity of the kiwi is calibrated by threshold setting of the amplified displacement image to be unripe, ripe and overripe, and the displacement change is used to calibrate the maturity of the kiwi.

[0089] In the scheme, the specific steps of collecting the kiwi micro-vibration video data are as follows: the micro-vibration of the kiwi is realized by the vibration platform, and the high-frame-rate video acquisition camera is used to collect the video image of the kiwi micro-vibration and transmit it to the processor for video analysis and calculation.

[0090] In the scheme, the specific steps of kiwi recognition and positioning are as follows:

[0091] Step S1: accurately access the kiwi micro-vibration image by using the high-frame-rate video acquisition camera;

[0092] Step S2: distinguish the kiwi from the background color block by pixel point threshold 90 based on the pixel point gray value of the image, and perform image binarization processing to identify the position of the kiwi by the sudden change of the value 0 and 1 of the edge pixel point;

[0093] Step S3: position the boundary point of the kiwi, calibrate the position of the kiwi by a rectangular frame, and cut off the image for calculation and processing.

[0094] In the scheme, the specific steps of video data decomposition processing are as follows: the collected and cut-off kiwi image data is frame by frame spatially decomposed by using complex value controllable pyramid based on Fourier transform, different direction and different scale images are obtained, the amplitude and phase analysis of the kiwi video image are separated, the global Fourier transform is used, the profile f of one-dimensional kiwi image under global translation f(x+δ(t)) is used, for the amplification factor α, the displacement image profile f(x+δ(t)) is used as the sum of complex sine, and each frequency band corresponds to a single frequency ω.

[0095]

[0096] The frequency ω is a complex sine wave,

[0097] S ω (x,t)=A ω e iω(x+δ(t))

[0098] Since S ω is a sine wave, its phase ω(x+δ(t) contains motion information, and the motion can be controlled by modifying the phase. S ω(x,y) is a complex sinusoid whose motion is exactly (1+ a) times the input. The motion amplified video can be reconstructed by folding the pyramid and summing all the subbands to get the motion amplified sequence f(x+(1+ a) d(t)).

[0099] In the above scheme, the specific steps of the video data acquisition processing are: filtering the collected phase signal by using a suitable band-pass filter, and limiting the displacement a d(t) to make the amplified displacement well approximate to the real displacement signal. A standard deviation of a Gaussian window is used as a boundary, so as to obtain the kiwifruit micro-motion amplified data information acquisition.

[0100] In the above scheme, the specific steps of the video signal denoising processing are: the noise in the input sequence will cause the noise of the phase signal itself, thereby causing incorrect motion to be amplified, and the amplitude weighting and Gaussian blur are used on the phase, and a suitable Gaussian filter is set for noise processing, so as to improve the signal-to-noise ratio.

[0101] In the above scheme, the specific steps of the video sequence amplification processing are: collecting the phase of the effective kiwifruit data by using a band-pass filter, and the time band-pass phase corresponds to the motion of different spatial scales and directions. In order to synthesize the amplified motion, the band-pass phase is multiplied by the amplification factor a. Then, the amplified phase difference is used to amplify the kiwifruit micro-motion in the sequence, by modifying the phase of each coefficient of each frame. The displacement change signal d(t) of the kiwifruit micro-motion after band-pass filtering is subjected to motion amplification processing, and the result of amplifying this motion by a times is obtained,

[0102]

[0103] In order to separate the changing part, a first-order Taylor series expansion is used to approximate the motion represented by the formula:

[0104]

[0105] Let the result of the band-pass filtering in the last step be B(x,t), and assume that the frequency range of all the change signals d(t) is exactly within the frequency band range of the band-pass filtering, then

[0106]

[0107] The motion represented by the formula is amplified by multiplying the changing part by an amplification factor a and adding it back to the original signal. That is, the following formula is obtained:

[0108]

[0109] By combining the formulas, the following formula can be obtained:

[0110]

[0111] In this ideal situation, The motion amplification sequence f(x+(1+α)δ(t)) is obtained.

[0112] In the above scheme, the specific steps of the video magnification construction process are as follows: When the kiwi fruit captured by the visual sensor is subjected to the same small vibration, the deformation of the surface is different at different ripeness. By magnifying the small displacement of the video, the small deformation f(t)-t slice magnified image is reconstructed to realize the magnification of the small displacement.

[0113] In the above scheme, the specific steps for calculating maturity detection are as follows: As a typical climacteric fruit, kiwifruit gradually softens from hard to edible during its growth period. This is manifested as the continuous softening of the kiwifruit fruit. To detect the maturity of kiwifruit without damaging it, slight vibrations can be used to represent the hardness of its surface.

[0114] Step M1: Perform edge detection on the kiwi fruit;

[0115] Step M2: Process the cropped image using the regionprops function to detect its regional features and centroid location. Figure 4 This invention relates to a micro-motion processed image of a kiwi fruit based on video micro-motion magnification;

[0116] Step M3: Save the centroid position and calculate small displacements for each frame of the video image, and use an array... For example, calculate the displacement change and store it in an array. Figure 5 This invention is based on the acquisition of kiwi fruit displacement images using video micro-motion magnification.

[0117]

[0118] In the above scheme, the specific steps for calibrating the maturity are as follows: the maturity of kiwifruit is mostly characterized by its hardness. By setting the maturity level of kiwifruit as immature, mature, and overripe by setting the threshold of the magnified displacement image, the maturity of kiwifruit is calibrated by the calculated displacement change.

[0119] The calibration was performed using a large number of immature and ripe kiwifruit. The thresholds for immature and ripe kiwifruit were set as ω1 and the thresholds for ripe and overripe kiwifruit were set as ω2. The mean displacement change of the detected kiwifruit was x.

[0120] When a kiwifruit is unripe, its surface hardness is too high, the friction when it comes into contact with the vibration table is small, and the displacement amplitude x < ω1 when subjected to slight vibration indicates that it is unripe.

[0121] When the kiwi has matured, its surface hardness is moderate, the frictional force with the vibration table is moderate, and the displacement amplitude x is lower when it is subjected to slight vibration, indicating that it has matured.

[0122] When the kiwi is over-mature, its surface hardness is too low, the frictional force with the vibration table is larger, and the displacement amplitude x is larger than ω2 when it is subjected to slight vibration, indicating that it is over-mature.

[0123] It should be understood that although the present specification is described in terms of various embodiments, not every embodiment contains only one independent technical solution, and the specification is described in this way only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that those skilled in the art can understand.

[0124] The series of detailed descriptions listed above are only specific descriptions of the feasible embodiments of the present application, and are not intended to limit the protection scope of the present application, and any equivalent embodiments or changes made without departing from the spirit of the present application should be included in the protection scope of the present application.

Claims

1. A control method of a non-contact kiwifruit ripeness detection system, characterized by, The detection system comprises a high-frame-rate video acquisition camera, a vibration platform and a processor; The vibration platform comprises a vibration module; the vibration module is used to realize micro-vibration processing of the kiwifruit through the vibration platform; The high-frame-rate video acquisition camera comprises a video acquisition module; the video acquisition module is used to acquire video images of the kiwifruit in micro-vibration and transmit the video images to the processor for video analysis and calculation; The processor combines video image information transmitted by the high-frame-rate video acquisition camera, and comprises a kiwifruit recognition module, a video micro-motion amplification module and a maturity calculation module; The kiwifruit recognition module is used to recognize the surface boundary of the kiwifruit through image analysis of the camera and locate the position of the kiwifruit by sudden change of the gray value of adjacent pixel points in the image; The video micro-motion amplification module is used to amplify the micro-displacement of the kiwifruit surface when the kiwifruit is subjected to vibration to be observable through video image micro-motion amplification technology, and comprises video data decomposition processing, video data acquisition processing, video signal denoising processing, video sequence amplification processing and video amplification construction processing; The maturity calculation module comprises a maturity detection module and a maturity calibration module; The maturity detection module is used to recognize and calibrate the platform position of the kiwifruit through the amplified gray image, detect the centroid position of the kiwifruit through edge filtering and recognize the displacement change of the kiwifruit through multiple video frames; The maturity calibration module is used to set the maturity grade of the kiwifruit as unripe, ripe and overripe through threshold setting of the amplified displacement image, and calibrate the maturity of the kiwifruit through the calculated displacement change; The control method comprises the following steps: Acquiring kiwifruit micro-vibration video data: realizing micro-vibration processing of the kiwifruit through the vibration module; acquiring video images of the kiwifruit in micro-vibration through the video acquisition module and transmitting the video images to the processor for video analysis and calculation; Kiwifruit recognition and positioning: recognizing the surface boundary of the kiwifruit through image analysis of the camera and locating the position of the kiwifruit through sudden change of the gray value of adjacent pixel points in the image, and processing the image through rectangular frame calibration; Video data decomposition processing: using a complex-valued steerable pyramid to perform spatial decomposition on the acquired and intercepted kiwifruit image data frame by frame, obtaining images in different directions and different scales, and separating the amplitude and phase of the kiwifruit video image through Fourier transform method; Video data acquisition processing: acquiring data information of the kiwifruit micro-motion amplification through time domain band pass filtering of the phase signal; Video signal denoising processing: improving the signal noise ratio by setting a suitable Gaussian filter for noise processing; Video sequence amplification processing: performing motion amplification processing on the displacement change signal δ(t) of the kiwifruit micro-motion after band pass filtering, to obtain a motion amplification sequence f(x+(1+α)δ(t)); Video amplification construction processing: performing construction analysis on the kiwifruit displacement video after amplification processing, to obtain video micro-motion data information after motion amplification; Maturity detection and calculation: recognizing and calibrating the platform position of the kiwifruit through the amplified gray image, detecting the centroid position of the kiwifruit through edge filtering, and recognizing the displacement change of the kiwifruit through multiple video frames. The maturity of the kiwifruit is calibrated by threshold setting of the amplified displacement image, and the maturity of the kiwifruit is calibrated as unripe, ripe and overripe according to the calculated displacement change.

2. The control method of the non-contact mass kiwifruit ripeness detection system according to claim 1, characterized in that, The specific steps of the kiwifruit recognition and positioning are as follows: Step S1: accurately access the kiwifruit micro-vibration image by using a high frame rate video acquisition camera; Step S2: distinguish the kiwifruit from the background color block by using the pixel point gray value of the image and the pixel point threshold 90, and perform image binarization processing, so as to recognize the position of the kiwifruit by the sudden change of the values 0 and 1 of the edge pixel points; Step S3: position the boundary point of the kiwifruit, calibrate the position of the kiwifruit by a rectangular frame, and intercept the image of the kiwifruit for calculation and processing.

3. The control method of the non-contact mass kiwifruit ripeness detection system according to claim 1, characterized in that, The specific steps of the video data acquisition and processing are as follows: a suitable band-pass filter is used to filter the acquired phase signal, the displacement αδ(t) needs to be limited, the amplified displacement is well approximated to the real displacement signal, a standard deviation of a Gaussian window is used as a boundary, and thus the amplified data information of the kiwifruit micro-motion is acquired.

4. The control method of the non-contact mass kiwifruit ripeness detection system according to claim 1, characterized in that, The specific steps of the video signal denoising processing are as follows: the noise in the input sequence will cause noise in the phase signal itself, thus leading to incorrect motion being amplified, an amplitude weighting and a Gaussian blur are used on the phase, a suitable Gaussian filter is set for noise processing, and thus the signal noise ratio is improved.

5. The control method of the non-contact mass kiwifruit ripeness detection system according to claim 1, characterized in that, The specific steps of the video sequence amplification processing are as follows: the band-pass effective kiwifruit data phase, the time band-pass phase corresponds to different spatial scales and directions of motion, in order to synthesize the amplified motion, the band-pass phase is multiplied by an amplification factor α, then the amplified phase difference is used to amplify the kiwifruit micro-motion in the sequence, the displacement change signal δ(t) of the kiwifruit micro-motion after the band-pass filtering is processed by motion amplification, and a motion amplified sequence f(x+(1+α)δ(t)) is obtained.

6. The control method of the non-contact mass kiwifruit ripeness detection system according to claim 1, characterized in that, The specific steps of the video amplification construction processing are as follows: when the kiwifruit photographed by the visual sensor is subjected to the same micro-vibration, the surface deformation of the kiwifruit with different maturity is different, the micro-deformation f(t)-t slice is amplified by video micro-displacement amplification, and the amplified image is recombined to realize the amplification processing of the small displacement.

7. The control method of the non-contact mass kiwifruit ripeness detection system according to claim 1, characterized in that, The specific steps of the maturity detection and calculation are as follows: as a typical respiration jump type fruit, the kiwifruit gradually softens from hard to edible during its growth, and thus the kiwifruit continuously softens, the hardness of the surface layer of the kiwifruit is represented by micro-vibration, and the maturity of the kiwifruit is detected without damaging the kiwifruit; Step M1: perform sobei edge detection of the kiwifruit; Step M2: perform regionprops function processing on the intercepted image to detect the region features and the position of the centroid. Step M3: Centroid position saving and small displacement calculation for each frame of video image, to array Calculate displacement change and save in array 8. The control method of the non-contact mass kiwifruit ripeness detection system according to claim 1, characterized in that, The specific steps of the maturity calibration are as follows: the maturity of the kiwifruit is represented by its hardness, the maturity grade of the kiwifruit is set as unripe, ripe and overripe by setting the threshold value of the amplified displacement image, and the maturity of the kiwifruit is calibrated by the calculated displacement change; the threshold value of the unripe kiwifruit and the ripe kiwifruit is set as ω1, the threshold value of the ripe kiwifruit and the overripe kiwifruit is set as ω2, and the average displacement change of the kiwifruit is x; When the kiwifruit is unripe, the surface hardness is too high, the friction with the vibration table is small, the displacement amplitude x<ω1 when the kiwifruit is subjected to slight vibration, and the unripe kiwifruit is indicated; When the kiwifruit is ripe, the surface hardness is moderate, the friction with the vibration table is moderate, the displacement amplitude ω1<x<ω2 when the kiwifruit is subjected to slight vibration is low, and the ripe kiwifruit is indicated; When the kiwifruit is overripe, the surface hardness is too low, the friction with the vibration table is large, the displacement amplitude x>ω2 when the kiwifruit is subjected to slight vibration is large, and the overripe kiwifruit is indicated.

Images