Initial damage detection method based on stepwise standard deviation of temperature difference sequence on apple surface

Abstract

This invention discloses a method for detecting initial damage in apples based on the progressive standard deviation of the apple surface temperature difference sequence. The invention includes the following steps: First, temperature change information within the damage detection range is acquired using passive thermal imaging technology; then, the optimal image segmentation threshold between the damaged and undetected areas in the apple thermal imaging image is determined by combining the progressive standard deviation of the apple surface temperature difference sequence; next, the thermal imaging image of the apple to be tested is converted into an AUC grayscale image; finally, the AUC grayscale image is segmented using the optimal image segmentation threshold to obtain the damage detection result of the apple to be tested. This invention combines passive thermal imaging technology with the progressive standard deviation of the apple surface temperature difference sequence, achieving non-destructive detection of initial damage in apples.

Description

Technical Field

[0001] This invention relates to a method for detecting damage in apples in the field of non-destructive testing of fruit quality, specifically a method for detecting initial damage based on the stepwise standard deviation of the temperature difference sequence on the apple surface. Background Technology

[0002] Mechanical damage, caused by drops, collisions, compression, vibration, and other factors, is the main form of fruit damage. Losses due to mechanical damage account for 30%-40% of total fruit production. Especially in the early stages of damage, because most damaged areas are difficult to detect, the damaged parts further rot and deteriorate during subsequent storage and transportation, forming rotten fruit and spreading to surrounding fruits, causing serious losses to producers and consumers. If damage can be detected in its early stages and rotten fruit can be removed promptly, such losses can be effectively reduced and the quality of marketable fruit improved.

[0003] In the early stages of fruit damage, the damaged area resembles the surrounding healthy tissue, with indistinct surface features. This limits the application of visible light-based detection methods, as existing visible light imaging techniques cannot be used for early-stage fruit damage detection. Recent studies have demonstrated the potential of infrared thermal imaging technology for detecting early-stage damage. Thermal imaging technologies are mainly divided into active and passive types.

[0004] Active thermal imaging technology requires equipment to heat or cool the object being detected. Varith et al., using a thermal imager to detect defects in apples, found that during heating or cooling, the surface temperature of the defective area was 1-2°C lower than that of the normal area, with the temperature difference lasting 30-180 seconds (Varith J, Hyde GM, Baritelle A, et al. Non-contact bruise detection in apples by thermal imaging[J]. Innovative Food Science and Emerging Technologies, 2003, 4: 211-218.). Kim et al. used sinusoidal thermal energy to stimulate the object and measured the thermal emission information of pears using a thermal imager to quantitatively identify the size and depth of damage (Kim G, Kim G, Park J, et al. Application of infrared lock-in thermography for the quantitative evaluation of bruises on pears[J]. Infrared Physics and Technology, 2014, 63: 133-139.).

[0005] Active thermal imaging technology boasts advantages such as high sensitivity and strong controllability; however, improper operation during temperature control can lead to heat damage or chilling injury to fruits, and the energy consumption generated by temperature control significantly increases detection costs. Passive thermal imaging technology, on the other hand, measures the temperature difference between the object being tested and its surrounding environment, thus largely avoiding these problems. Summary of the Invention

[0006] The purpose of this invention is to overcome the problem that the small temperature difference between the damaged area and the surrounding area makes it difficult to obtain obvious characteristics by utilizing the temperature change characteristics of the damaged area in the early stage of mechanical damage to the fruit. This invention proposes a non-destructive detection method for early damage to apples based on the evolution of temperature difference characteristic parameters between the damaged area and the undamaged area.

[0007] The technical solution adopted by this invention to solve its technical problem is:

[0008] 1) Collect multiple thermal images M of consecutive apples to form a thermal image sequence;

[0009] 2) Extract the regions of interest from multiple thermal imaging images M in the thermal imaging image sequence, and obtain the ROI0 of the entire fruit body, the damaged region ROI1 and the undamaged region ROI2 corresponding to each thermal imaging image M.

[0010] 3) Based on the temperature data of the entire fruit area ROI0, damaged area ROI1 and undamaged area ROI2 corresponding to each thermal imaging image M, and combined with the stepwise standard deviation of the apple surface temperature difference sequence, the optimal image segmentation threshold is calculated.

[0011] 4) After performing image conversion on the thermal imaging image sequence of the apple to be tested, the corresponding AUC grayscale image is obtained. After segmenting the AUC grayscale image using the optimal image segmentation threshold, the damage detection result of the current apple to be tested is obtained.

[0012] Specifically, 3) refers to:

[0013] 3.1) Average the temperature data of each pixel within the ROI0 of the entire fruit body corresponding to each thermal imaging image M to obtain the average fruit body temperature corresponding to the current thermal imaging image M.

[0014] 3.2) Subtract the average fruit temperature from the temperature data of each pixel in the damaged region ROI1 and the undamaged region ROI2 corresponding to the current thermal imaging image M. Then, the temperature difference of each pixel in the damaged area ROI1 and the undamaged area ROI2 is obtained respectively, thereby obtaining the temperature difference of all pixels in the damaged area ROI1 and the undamaged area ROI2.

[0015] 3.3) Repeat steps 3.1)-3.2) to calculate the temperature difference of all pixels in the damaged region ROI1 and the undamaged region ROI2 in the remaining thermal imaging image M;

[0016] 3.4) Based on the temperature difference of all pixels in the damaged region ROI1 and the undamaged region ROI2 in each thermal imaging image M, the temperature standard deviation of each pixel in different thermal imaging images M is obtained by calculating the stepwise standard deviation of the temperature difference of each pixel in the time dimension. Then, the temperature standard deviation of each pixel is plotted as the vertical axis and the sampling time i of the thermal imaging image M is plotted as the horizontal axis, and the area under the curve (AUC) of each curve is calculated. After normalizing the area under the curve (AUC) of each pixel, the normalized area under the curve (AUC) data is obtained.

[0017] 3.5) Determine the optimal image segmentation threshold based on the normalized lower area AUC data.

[0018] In step 4), after image conversion of the thermal imaging image sequence of the apple to be tested, the corresponding AUC grayscale image is obtained, specifically as follows:

[0019] S1: Extract the region of interest from each thermal imaging image in the thermal imaging image sequence of the apple to be tested, and obtain the entire area of ​​the fruit corresponding to the current thermal imaging image.

[0020] S2: After averaging the temperature data of each pixel in the entire area of ​​the fruit corresponding to the current thermal imaging image, the average fruit temperature of the current thermal imaging image is obtained.

[0021] S3: Subtract the average fruit temperature from the temperature data of each pixel in the entire area of ​​the fruit corresponding to the current thermal imaging image to obtain the temperature difference of each pixel in the current thermal imaging image.

[0022] S4: Repeat S1-S3 to calculate the temperature difference of each pixel in the remaining thermal imaging images in the thermal imaging image sequence of the apple to be tested;

[0023] S5: After calculating the stepwise standard deviation of the temperature difference of each pixel in the thermal imaging image sequence of the apple under test, the temperature standard deviation of each pixel in different thermal imaging images is obtained. Then, the temperature standard deviation of each pixel in different thermal imaging images is used as the vertical axis and the sampling time of different thermal imaging images is used as the horizontal axis to plot the curve of each pixel and calculate the area under the curve (AUC) of each curve. After normalizing the area under the curve (AUC) of each pixel, the AUC grayscale image is obtained.

[0024] In step 4), after segmenting the AUC grayscale image using the optimal image segmentation threshold, the damage detection result of the current apple to be tested is obtained, specifically as follows:

[0025] After segmenting the slope grayscale image using the optimal image segmentation threshold, an initial segmented image is obtained. Then, the damage region is accurately segmented in the initial segmented image to obtain a secondary segmented image. Finally, the damage is located in the secondary segmented image using the minimum bounding rectangle to obtain the damage detection result of the apple to be tested.

[0026] The precise segmentation of damaged regions in the initial segmented image specifically involves using image dilation, erosion, maximum connected component extraction, and hole filling to precisely segment damaged regions in the initial segmented image.

[0027] The present invention, by adopting the above technical solution, has the following beneficial effects:

[0028] First, this invention utilizes the temperature change over time at the site of mechanical damage in apples during the initial stages of injury, providing a theoretical basis for non-destructive testing of early-stage mechanical damage in apples. Second, this invention obtains the temperature standard deviation σ at different image sampling times. i Furthermore, the temperature standard deviation data were plotted as a curve to obtain the area under the curve (AUC), thereby enhancing the damage characteristics while simultaneously reducing data dimensionality. Finally, this invention utilizes passive thermal imaging technology and simple image processing techniques to achieve non-destructive detection of early-stage damage in apples. Attached Figure Description

[0029] Figure 1 The flowchart shows a method for detecting initial damage based on the stepwise standard deviation of the temperature difference sequence on the apple surface.

[0030] Figure 2 Thermal imaging images of early damage to an apple;

[0031] Figure 3 A schematic diagram showing the extraction of ROI0, ROI1, and ROI2;

[0032] Figure 4 A graph showing the standard deviation of temperature in the damaged area versus time;

[0033] Figure 5 A graph showing the relationship between the standard deviation of temperature in the non-damaged area and time;

[0034] Figure 6 Grayscale image of AUC in the early stages of damage to an apple;

[0035] Figure 7 This is a segmentation diagram of the initial damage to the apple.

[0036] Figure 8 This is a schematic diagram of an initial damage detection system based on the stepwise standard deviation of the temperature difference sequence on the apple surface.

[0037] In the diagram: 1-Thermal imaging system, 2-Rotating shooting platform, 3-Placement platform. Detailed Implementation

[0038] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0039] like Figure 1 As shown, the present invention includes the following steps:

[0040] 1) Multiple thermal images M of consecutive apples are acquired at a certain frequency using a thermal imaging system, thus forming a thermal image sequence; each thermal image M contains damaged and undamaged areas. For example... Figure 8 As shown, the thermal imaging system includes a thermal imaging camera, a rotating shooting platform, and a placement platform. The rotating shooting platform 2 (50mm high, 700mm diameter) rotates at an angular velocity of 1° / s around its center. Multiple placement platforms 3 (100mm diameter) are distributed circumferentially on the upper surface of the rotating shooting platform 2 for placing apples. The center of each placement platform 3 is 283mm from the center of the rotating shooting platform 2. The surface of each placement platform 3 is covered with black high-density sponge. The thermal imaging camera 1 (500mm high, 400mm wide, with a top and bottom corner radius of 50mm) is placed on the rotating shooting platform 2, with its center aligned with the center of each placement platform 3.

[0041] In this embodiment, the same batch of 'Red Fuji' apples purchased from the local fruit wholesale market were used as the experimental material. Before the experiment, the apples were inspected, and cracked, rotten, and misshapen fruits were removed to ensure they were undamaged. The apples were placed at room temperature for 24 hours before the experiment to ensure their temperature matched the room temperature.

[0042] Apple damage simulation was conducted using a MY-A721 impact testing bench (Dongguan Mingyu Intelligent Technology Co., Ltd.). The impact hammer had a diameter of 80mm and a weight of 500g. The impact heights were 100mm, 150mm, and 200mm. Based on the impact height, the damaged samples were divided into three groups: H1, H2, and H3, with 20 damaged apple samples prepared at each height. Simultaneously, 20 undamaged samples were designated as group H0.

[0043] The thermal imaging system is a DS-2TD2636-10 thermal imaging dual-spectrum network tube camera with a spectral range of 800–1400 nm, a noise equivalent temperature difference of less than 50 mk, a focal length of 10 mm, a pixel resolution of 384 × 288 pixels, and a frame rate of 50 fps.

[0044] The temperature of the damaged area of ​​a 'Red Fuji' apple changes within 0-1.5 hours after mechanical damage. Therefore, the number and frequency of thermal imaging image acquisitions need to be evaluated based on the apple's harvest time or the expected damage time. In Example 1, thermal imaging images were acquired immediately after the 'Red Fuji' apple was damaged, with a time interval of 6 minutes, for a total of 16 acquisitions over 90 minutes.

[0045] 2) Extract the regions of interest from multiple thermal imaging images M in the thermal imaging image sequence, and obtain the ROI0 of the entire fruit body, the damaged region ROI1 and the undamaged region ROI2 corresponding to each thermal imaging image M.

[0046] 3) Based on the temperature data of the entire fruit area ROI0, damaged area ROI1 and undamaged area ROI2 corresponding to each thermal imaging image M, and combined with the stepwise standard deviation of the apple surface temperature difference sequence, the optimal image segmentation threshold is calculated.

[0047] 3) Specifically:

[0048] 3.1) Average the temperature data of each pixel within the ROI0 of the entire fruit body corresponding to each thermal imaging image M to obtain the average fruit body temperature corresponding to the current thermal imaging image M.

[0049] 3.2) Subtract the average fruit temperature from the temperature data of each pixel in the damaged region ROI1 and the undamaged region ROI2 corresponding to the current thermal imaging image M. Then, the temperature difference of each pixel in the damaged area ROI1 and the undamaged area ROI2 is obtained respectively, thereby obtaining the temperature difference of all pixels in the damaged area ROI1 and the undamaged area ROI2.

[0050] 3.3) Repeat steps 3.1)-3.2) to calculate the temperature difference of all pixels in the damaged region ROI1 and the undamaged region ROI2 in the remaining thermal imaging image M;

[0051] 3.4) Based on the temperature difference of all pixels in the damaged region ROI1 and the undamaged region ROI2 in each thermal imaging image M, the temperature standard deviation of each pixel in different thermal imaging images M is obtained by calculating the stepwise standard deviation of the temperature difference of each pixel in the time dimension. Then, the temperature standard deviation of each pixel is plotted as the vertical axis and the sampling time i of the thermal imaging image M is plotted as the horizontal axis, and the area under the curve (AUC) of each curve is calculated. After normalizing the area under the curve (AUC) of each pixel, the normalized area under the curve (AUC) data is obtained.

[0052] Wherein, the temperature standard deviation σ of each pixel at sampling time i i The calculation formula is as follows:

[0053]

[0054]

[0055] Where, ΔT x The temperature difference at sampling time x; Let be the average temperature difference at sampling time i.

[0056] 3.5) Based on the normalized lower area AUC data, the optimal image segmentation threshold is determined using the Otsu thresholding method.

[0057] 4) After performing image conversion on the thermal imaging image sequence of the apple to be tested, the corresponding AUC grayscale image is obtained. After segmenting the AUC grayscale image using the optimal image segmentation threshold, the damage detection result of the current apple to be tested is obtained.

[0058] In step 4), after image conversion of the thermal imaging image sequence of the apple to be tested, the corresponding AUC grayscale image is obtained, specifically as follows:

[0059] S1: Extract the region of interest from each thermal imaging image in the thermal imaging image sequence of the apple to be tested, and obtain the entire area of ​​the fruit corresponding to the current thermal imaging image.

[0060] S2: After averaging the temperature data of each pixel in the entire area of ​​the fruit corresponding to the current thermal imaging image, the average fruit temperature of the current thermal imaging image is obtained.

[0061] S3: Subtract the average fruit temperature from the temperature data of each pixel in the entire area of ​​the fruit corresponding to the current thermal imaging image to obtain the temperature difference of each pixel in the current thermal imaging image.

[0062] S4: Repeat S1-S3 to calculate the temperature difference of each pixel in the remaining thermal imaging images in the thermal imaging image sequence of the apple to be tested;

[0063] S5: After calculating the stepwise standard deviation of the temperature difference of each pixel in the thermal imaging image sequence of the apple under test, the temperature standard deviation of each pixel in different thermal imaging images is obtained. Then, the temperature standard deviation of each pixel in different thermal imaging images is used as the vertical axis and the sampling time of different thermal imaging images is used as the horizontal axis to plot the curve of each pixel and calculate the area under the curve (AUC) of each curve. After normalizing the area under the curve (AUC) of each pixel, the AUC grayscale image is obtained.

[0064] In step 4), after segmenting the AUC grayscale image using the optimal image segmentation threshold, the damage detection result of the apple to be tested is obtained, specifically as follows:

[0065] After segmenting the slope grayscale image using the optimal image segmentation threshold, an initial segmented image is obtained. Then, the damage region is accurately segmented in the initial segmented image to obtain a secondary segmented image. Finally, the damage is located in the secondary segmented image using the minimum bounding rectangle to obtain the damage detection result of the apple to be tested.

[0066] The initial segmented image is used to accurately segment the damaged regions. Specifically, this is achieved by using image dilation, erosion, maximum connected component extraction, and hole filling techniques.

[0067] In this embodiment, the damage detection results are shown in Table 1:

[0068] Table 1 shows the non-destructive testing results of initial damage to apples from impacts at different heights (100mm, 150mm, 200mm).

[0069]

[0070] Finally, it should be noted that the above embodiments and descriptions are only used to illustrate the technical solutions of the present invention and not to limit it. Those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the disclosure of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for detecting initial damage based on the stepwise standard deviation of an apple surface temperature difference sequence, characterized in that, Includes the following steps: 1) Collect multiple thermal images M of consecutive apples to form a thermal image sequence; 2) Extract the regions of interest from multiple thermal imaging images M in the thermal imaging image sequence, and obtain the ROI0 of the entire fruit body, the damaged region ROI1 and the undamaged region ROI2 corresponding to each thermal imaging image M. 3) Based on the temperature data of all regions ROI0, damaged regions ROI1 and undamaged regions ROI2 of the fruit corresponding to each thermal imaging image M, generate the temperature difference of all pixels in damaged regions ROI1 and undamaged regions ROI2 in each thermal imaging image M. Combine the stepwise standard deviation of the apple surface temperature difference sequence to calculate the optimal image segmentation threshold. The third) includes: Based on the temperature difference of all pixels in the damaged region ROI1 and the undamaged region ROI2 in each thermal image M, the stepwise standard deviation of the temperature difference of each pixel in the time dimension is calculated to obtain the temperature standard deviation of each pixel in different thermal images M. Then, with the temperature standard deviation of each pixel as the ordinate and the sampling time i of the thermal image M as the abscissa, a curve corresponding to each pixel is plotted and the area under the curve (AUC) corresponding to each curve is calculated. After normalizing the AUC corresponding to each pixel, the normalized AUC data is obtained. The optimal image segmentation threshold is determined based on the normalized AUC data. 4) After performing image conversion on the thermal imaging image sequence of the apple to be tested, the corresponding AUC grayscale image is obtained. After segmenting the AUC grayscale image using the optimal image segmentation threshold, the damage detection result of the current apple to be tested is obtained. After performing image conversion on the thermal imaging image sequence of the apple to be tested, the corresponding AUC grayscale image is obtained, including: The temperature difference of each pixel in the thermal imaging image sequence of the apple to be tested is generated. After calculating the stepwise standard deviation of the temperature difference of each pixel in the thermal imaging image sequence of the apple to be tested over time, the temperature standard deviation of each pixel in different thermal imaging images is obtained. Then, the temperature standard deviation of each pixel in different thermal imaging images is used as the ordinate and the sampling time of different thermal imaging images is used as the abscissa to plot the curve of each pixel and calculate the area under the curve (AUC) of each curve. After normalizing the area under the curve (AUC) of each pixel, the AUC grayscale image is obtained.

2. The method for detecting initial damage based on the stepwise standard deviation of apple surface temperature difference sequence according to claim 1, characterized in that, The third) also includes: 3.1) Average the temperature data of each pixel within the ROI0 of the entire fruit body corresponding to each thermal imaging image M to obtain the average fruit body temperature corresponding to the current thermal imaging image M. ; 3.2) Subtract the average fruit temperature from the temperature data of each pixel in the damaged region ROI1 and the undamaged region ROI2 corresponding to the current thermal imaging image M. Then, the temperature difference of each pixel in the damaged area ROI1 and the undamaged area ROI2 is obtained respectively, thereby obtaining the temperature difference of all pixels in the damaged area ROI1 and the undamaged area ROI2. 3.3) Repeat steps 3.1)-3.2) to calculate the temperature difference of all pixels in the damaged region ROI1 and the undamaged region ROI2 in the remaining thermal imaging image M.

3. The method for detecting initial damage based on the stepwise standard deviation of apple surface temperature difference sequence according to claim 1, characterized in that, In step 4), the temperature difference of each pixel in the thermal imaging image sequence of the apple to be tested is generated as follows: S1: Extract the region of interest from each thermal imaging image in the thermal imaging image sequence of the apple to be tested, and obtain the entire area of ​​the fruit corresponding to the current thermal imaging image. S2: After averaging the temperature data of each pixel in the entire area of ​​the fruit corresponding to the current thermal imaging image, the average fruit temperature of the current thermal imaging image is obtained. S3: Subtract the average fruit temperature from the temperature data of each pixel in the entire area of ​​the fruit corresponding to the current thermal imaging image to obtain the temperature difference of each pixel in the current thermal imaging image. S4: Repeat S1-S3 to calculate the temperature difference of each pixel in the remaining thermal imaging images in the thermal imaging image sequence of the apple to be tested.

4. The method for detecting initial damage based on the stepwise standard deviation of apple surface temperature difference sequence according to claim 1, characterized in that, In step 4), after segmenting the AUC grayscale image using the optimal image segmentation threshold, the damage detection result of the current apple to be tested is obtained, specifically as follows: After segmenting the slope grayscale image using the optimal image segmentation threshold, an initial segmented image is obtained. Then, the damage region is accurately segmented in the initial segmented image to obtain a secondary segmented image. Finally, the damage is located in the secondary segmented image using the minimum bounding rectangle to obtain the damage detection result of the apple to be tested.

5. The method for detecting initial damage based on the stepwise standard deviation of apple surface temperature difference sequence according to claim 4, characterized in that, The precise segmentation of damaged regions in the initial segmented image specifically involves using image dilation, erosion, maximum connected component extraction, and hole filling to precisely segment damaged regions in the initial segmented image.

Images