A method for detecting initial damage based on time-varying characteristics of temperature difference on apple surface

Abstract

The application discloses a kind of based on apple surface temperature difference time-varying characteristics initial injury detection method.The application includes the following steps: first, based on passive thermal imaging technology obtains temperature variation information in damage detection range;Combined with the time-varying characteristics of apple surface temperature difference, the best image segmentation threshold between damage area and undetected area in apple thermal image is determined;Then the thermal image of the apple to be tested is converted into a range of gray scale images;Finally, the best image segmentation threshold is used to segment the range of gray scale images, and the damage detection result of the apple to be tested is obtained.The application combines passive thermal imaging technology and the time-varying characteristics of apple surface temperature difference, and realizes nondestructive testing of initial injury of apple.

Description

Technical Field

[0001] This invention relates to a method for detecting early damage to apples in the field of non-destructive testing of fruit quality, specifically a method for detecting early damage based on the time-varying characteristics of temperature difference on the apple surface. Background Technology

[0002] Mechanical damage, caused by drops, collisions, compression, vibration, and other factors, is the main form of fruit damage. Losses due to mechanical damage account for 30%-40% of total fruit production. Especially in the early stages of damage, because most damaged areas are difficult to detect, the damaged parts further rot and deteriorate during subsequent storage and transportation, forming rotten fruit and spreading to surrounding fruits, causing serious losses to producers and consumers. If damage can be detected in its early stages and rotten fruit can be removed promptly, such losses can be effectively reduced and the quality of marketable fruit improved.

[0003] In the early stages of fruit damage, the damaged area resembles the surrounding healthy tissue, with indistinct surface features. This limits the application of visible light-based detection methods, as existing visible light imaging techniques cannot be used for early-stage fruit damage detection. Recent studies have demonstrated the potential of infrared thermal imaging technology for detecting early-stage damage. Thermal imaging technologies are mainly divided into active and passive types.

[0004] Active thermal imaging technology requires equipment to heat or cool the object being detected. Varith et al., using a thermal imager to detect defects in apples, found that during heating or cooling, the surface temperature of the defective area was 1-2°C lower than that of the normal area, with the temperature difference lasting 30-180 seconds (Varith J, Hyde GM, Baritelle A, et al. Non-contact bruise detection in apples by thermal imaging[J]. Innovative Food Science and Emerging Technologies, 2003, 4: 211-218.). Kim et al. used sinusoidal thermal energy to stimulate the object and measured the thermal emission information of pears using a thermal imager to quantitatively identify the size and depth of damage (Kim G, Kim G, Park J, et al. Application of infrared lock-in thermography for the quantitative evaluation of bruises on pears[J]. Infrared Physics and Technology, 2014, 63: 133-139.).

[0005] Active thermal imaging technology boasts advantages such as high sensitivity and strong controllability; however, improper operation during temperature control can lead to heat damage or chilling injury to fruits, and the energy consumption generated by temperature control significantly increases detection costs. Passive thermal imaging technology, on the other hand, measures the temperature difference between the object being tested and its surrounding environment, thus largely avoiding these problems. Summary of the Invention

[0006] The purpose of this invention is to overcome the problem that the small temperature difference between the damaged area and the surrounding area makes it difficult to obtain obvious characteristics by utilizing the temperature change characteristics of the fruit damage site in the early stage of mechanical damage. This invention provides a non-destructive detection method for apple damage in the early stage based on the evolution of temperature difference characteristic parameters between the damaged area and the undamaged area of ​​the apple.

[0007] The technical solution adopted by this invention to solve its technical problem is:

[0008] 1) Collect multiple thermal images M of consecutive apples to form a thermal image sequence;

[0009] 2) Extract the regions of interest from multiple thermal imaging images M in the thermal imaging image sequence, and obtain the ROI0 of the entire fruit body, the damaged region ROI1 and the undamaged region ROI2 corresponding to each thermal imaging image M.

[0010] 3) Based on the temperature data of the entire fruit area ROI0, damaged area ROI1 and undamaged area ROI2 corresponding to each thermal imaging image M, and combined with the time-varying characteristics of the apple surface temperature difference, the optimal image segmentation threshold is calculated.

[0011] 4) After performing image conversion on the thermal imaging image sequence of the apple to be tested, the corresponding range grayscale image is obtained. After segmenting the range grayscale image using the optimal image segmentation threshold, the damage detection result of the current apple to be tested is obtained.

[0012] Specifically, 3) refers to:

[0013] 3.1) Average the temperature data of each pixel within the ROI0 of the entire fruit body corresponding to each thermal imaging image M to obtain the average fruit body temperature corresponding to the current thermal imaging image M.

[0014] 3.2) Subtract the average fruit temperature from the temperature data of each pixel in the damaged region ROI1 and the undamaged region ROI2 corresponding to the current thermal imaging image M. Then, the temperature difference of each pixel in the damaged area ROI1 and the undamaged area ROI2 is obtained respectively, thereby obtaining the temperature difference of all pixels in the damaged area ROI1 and the undamaged area ROI2.

[0015] 3.3) Repeat steps 3.1)-3.2) to calculate the temperature difference of all pixels in the damaged region ROI1 and the undamaged region ROI2 in the remaining thermal imaging image M;

[0016] 3.4) The Gauss function is used to fit the sampling time of the thermal imaging image M with the temperature difference of each pixel in the damaged area ROI1 and the undamaged area ROI2 to obtain the fitting curves corresponding to each pixel in the damaged area ROI1 and the undamaged area ROI2.

[0017] 3.5) Calculate the range of each pixel based on the fitted curve of each pixel; after normalizing the range of each pixel, obtain the range normalized data.

[0018] 3.6) Determine the optimal image segmentation threshold based on the range normalized data.

[0019] In step 4), after image conversion of the thermal imaging image sequence of the apple to be tested, the corresponding range grayscale image is obtained, specifically as follows:

[0020] S1: Extract the region of interest from each thermal imaging image in the sequence of thermal imaging images of the apple to be tested, and obtain the entire area of ​​the fruit corresponding to the current thermal imaging image.

[0021] S2: After averaging the temperature data of each pixel in the entire area of ​​the fruit corresponding to the current thermal imaging image, the average fruit temperature of the current thermal imaging image is obtained.

[0022] S3: Subtract the average fruit temperature from the temperature data of each pixel in the entire area of ​​the fruit corresponding to the current thermal imaging image to obtain the temperature difference of each pixel in the current thermal imaging image.

[0023] S4: Repeat S1-S3 to calculate the temperature difference of each pixel in the remaining thermal imaging images in the thermal imaging image sequence of the apple to be tested;

[0024] S5: The Gauss function is used to fit the sampling time of the thermal imaging image with the temperature difference of each pixel to obtain the fitting curve corresponding to each pixel; then the range of each pixel is calculated based on the fitting curve of each pixel; after normalizing the range of each pixel, the range grayscale image is obtained.

[0025] In step 4), after segmenting the range grayscale image using the optimal image segmentation threshold, the damage detection result of the current apple to be tested is obtained, specifically as follows:

[0026] After segmenting the range grayscale image using the optimal image segmentation threshold, an initial segmented image is obtained. Then, the damage region of the initial segmented image is accurately segmented to obtain a secondary segmented image. Finally, the damage is located in the secondary segmented image using the minimum bounding rectangle to obtain the damage detection result of the apple to be tested.

[0027] The precise segmentation of damaged regions in the initial segmented image specifically involves using image dilation, erosion, maximum connected component extraction, and hole filling to precisely segment damaged regions in the initial segmented image.

[0028] The present invention, by adopting the above technical solution, has the following beneficial effects:

[0029] First, this invention utilizes the temperature change over time at the site of mechanical damage in apples during the initial stages of injury, providing a theoretical basis for non-destructive testing of early-stage mechanical damage in apples. Second, this invention utilizes the Gaussian function to analyze the temperature difference data ΔT. t The relationship between the temperature difference and the image sampling time t is fitted, and the maximum temperature difference T is obtained through the fitted curve. max and minimum value T min And calculate the range T R While enhancing damage features, this invention also achieves data dimensionality reduction. Finally, by utilizing passive thermal imaging technology and simple image processing techniques, this invention enables non-destructive detection of early-stage damage in apples. Attached Figure Description

[0030] Figure 1 This is a flowchart of an initial damage detection method based on the time-varying characteristics of temperature difference on the apple surface;

[0031] Figure 2 Thermal imaging images of early damage to an apple;

[0032] Figure 3 A schematic diagram showing the extraction of ROI0, ROI1, and ROI2;

[0033] Figure 4 The fitted curve shows the relationship between temperature difference in the damaged area and time.

[0034] Figure 5 The fitted curve shows the relationship between temperature difference and time in the non-damaged area;

[0035] Figure 6 This is a grayscale image showing the extreme damage to an apple in its early stages.

[0036] Figure 7 This is a segmentation diagram of the initial damage to the apple.

[0037] Figure 8 This is a schematic diagram of an early damage detection system based on the time-varying temperature difference characteristics of an apple surface.

[0038] In the picture: 1-thermal imaging camera, 2-rotating shooting platform, 3-placement platform. Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0040] like Figure 1 As shown, the present invention includes the following steps:

[0041] 1) Use a thermal imaging system to acquire multiple consecutive thermal images M of apples at a certain frequency, such as... Figure 2 As shown, this forms a sequence of thermal imaging images; each thermal imaging image M contains damaged and undamaged areas. Figure 8 As shown, the thermal imaging system includes a thermal imaging camera 1, a rotating shooting platform 2, and a placement platform 3. The rotating shooting platform 2 (50mm high, 700mm diameter) rotates at an angular velocity of 1° / s around its center. Multiple placement platforms 3 (100mm diameter) are distributed circumferentially on the upper surface of the rotating shooting platform 2 for placing apples. The center of each placement platform 3 is 283mm from the center of the rotating shooting platform 2. The surface of each placement platform 3 is covered with black high-density sponge. The thermal imaging camera 1 (500mm high, 400mm wide, with a top and bottom corner radius of 50mm) is placed on the rotating shooting platform 2, with the center of the thermal imaging camera 1 aligned with the center of each placement platform 3.

[0042] In this embodiment, the same batch of 'Red Fuji' apples purchased from the local fruit wholesale market were used as the experimental material. Before the experiment, the apples were inspected, and cracked, rotten, and misshapen fruits were removed to ensure they were undamaged. The apples were placed at room temperature for 24 hours before the experiment to ensure their temperature matched the room temperature.

[0043] Apple damage simulation was conducted using a MY-A721 impact testing bench (Dongguan Mingyu Intelligent Technology Co., Ltd.). The impact hammer had a diameter of 80mm and a weight of 500g. The impact heights were 100mm, 150mm, and 200mm. Based on the impact height, the damaged samples were divided into three groups: H1, H2, and H3, with 20 damaged apple samples prepared at each height. Simultaneously, 20 undamaged samples were designated as group H0.

[0044] The thermal imaging system is a DS-2TD2636-10 thermal imaging dual-spectrum network tube camera with a spectral range of 800–1400 nm, a noise equivalent temperature difference of less than 50 mk, a focal length of 10 mm, a pixel resolution of 384 × 288 pixels, and a frame rate of 50 fps.

[0045] The temperature of the damaged area of ​​a 'Red Fuji' apple changes within 0-1.5 hours after mechanical damage. Therefore, the number and frequency of thermal imaging image acquisitions need to be evaluated based on the apple's harvest time or the expected damage time. In Example 1, thermal imaging images were acquired immediately after the 'Red Fuji' apple was damaged, with a time interval of 6 minutes, for a total of 16 acquisitions over 90 minutes.

[0046] 2) Extract the regions of interest (ROIs) from multiple thermal images M in the thermal imaging image sequence, and obtain the ROI0 of the entire fruit body, the damaged region ROI1, and the undamaged region ROI2 corresponding to each thermal image M, such as... Figure 3 As shown;

[0047] 3) Based on the temperature data of the entire fruit area ROI0, damaged area ROI1 and undamaged area ROI2 corresponding to each thermal imaging image M, and combined with the time-varying characteristics of the apple surface temperature difference, the optimal image segmentation threshold is calculated.

[0048] 3) Specifically:

[0049] 3.1) Average the temperature data of each pixel within the ROI0 of the entire fruit body corresponding to each thermal imaging image M to obtain the average fruit body temperature corresponding to the current thermal imaging image M.

[0050] 3.2) Subtract the average fruit temperature from the temperature data of each pixel in the damaged region ROI1 and the undamaged region ROI2 corresponding to the current thermal imaging image M. Then, the temperature difference of each pixel in the damaged area ROI1 and the undamaged area ROI2 is obtained respectively, thereby obtaining the temperature difference of all pixels in the damaged area ROI1 and the undamaged area ROI2.

[0051] 3.3) Repeat steps 3.1)-3.2) to calculate the temperature difference of all pixels in the damaged region ROI1 and the undamaged region ROI2 in the remaining thermal imaging image M;

[0052] 3.4) The Gaussian function is used to fit the sampling time of the thermal imaging image M with the temperature difference of each pixel in the damaged region ROI1 and the undamaged region ROI2, obtaining the fitting curves corresponding to each pixel in the damaged region ROI1 and the undamaged region ROI2. The fitting curve of the damaged region ROI1 is shown in Figure 3. Figure 4 As shown, the fitted curve of the undamaged region ROI1 is as follows: Figure 5 As shown; the formula for the fitted curve of each pixel is as follows:

[0053]

[0054] Where f(t) is the fitted temperature difference value at image sampling time t; a is the maximum temperature difference value of the fitted curve; b is the time corresponding to the maximum temperature difference value of the fitted curve; and c is the standard deviation of the temperature difference of the current pixel.

[0055] 3.5) Calculate the range of each pixel based on the fitted curve of each pixel; after normalizing the range of each pixel, obtain the range normalized data.

[0056] The formula for calculating the range is as follows:

[0057]

[0058] Among them, T R The range of each pixel, T max The maximum temperature difference, T min This is the minimum temperature difference value.

[0059] 3.6) Based on the range normalized data, the optimal image segmentation threshold is determined using the Otsu thresholding method.

[0060] 4) After image conversion of the thermal imaging image sequence of the apple to be tested, the corresponding range grayscale image is obtained, such as... Figure 6 As shown, after segmenting the range grayscale image using the optimal image segmentation threshold, the damage detection result of the apple to be tested is obtained, as follows. Figure 7 As shown.

[0061] In step 4), after image conversion of the thermal imaging image sequence of the apple to be tested, the corresponding range grayscale image is obtained, specifically as follows:

[0062] S1: Extract the region of interest from each thermal imaging image in the sequence of thermal imaging images of the apple to be tested, and obtain the entire area of ​​the fruit corresponding to the current thermal imaging image.

[0063] S2: After averaging the temperature data of each pixel in the entire area of ​​the fruit corresponding to the current thermal imaging image, the average fruit temperature of the current thermal imaging image is obtained.

[0064] S3: Subtract the average fruit temperature from the temperature data of each pixel in the entire area of ​​the fruit corresponding to the current thermal imaging image to obtain the temperature difference of each pixel in the current thermal imaging image.

[0065] S4: Repeat S1-S3 to calculate the temperature difference of each pixel in the remaining thermal imaging images in the thermal imaging image sequence of the apple to be tested;

[0066] S5: The Gauss function is used to fit the sampling time of the thermal imaging image with the temperature difference of each pixel to obtain the fitting curve corresponding to each pixel; then the range of each pixel is calculated based on the fitting curve of each pixel; after normalizing the range of each pixel, the range grayscale image is obtained.

[0067] In step 4), after segmenting the range grayscale image using the optimal image segmentation threshold, the damage detection result of the apple to be tested is obtained, specifically:

[0068] After segmenting the range grayscale image using the optimal image segmentation threshold, an initial segmented image is obtained. Then, the damage region of the initial segmented image is accurately segmented to obtain a secondary segmented image. Finally, the damage is located in the secondary segmented image using the minimum bounding rectangle to obtain the damage detection result of the apple to be tested.

[0069] The initial segmented image is used to accurately segment the damaged regions. Specifically, this is achieved by using image dilation, erosion, maximum connected component extraction, and hole filling techniques.

[0070] In this embodiment, the damage detection results are shown in Table 1:

[0071] Table 1 shows the non-destructive testing results of initial damage to apples from impacts at different heights (100mm, 150mm, 200mm).

[0072]

[0073] Finally, it should be noted that the above embodiments and descriptions are only used to illustrate the technical solutions of the present invention and not to limit it. Those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the disclosure of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for early damage detection based on time-varying characteristics of surface temperature difference of apples, characterized in that, Includes the following steps: 1) Collect multiple thermal images M of consecutive apples to form a thermal image sequence; 2) Extract the regions of interest from multiple thermal imaging images M in the thermal imaging image sequence, and obtain the ROI0 of the entire fruit body, the damaged region ROI1 and the undamaged region ROI2 corresponding to each thermal imaging image M. 3) Based on the temperature data of the entire fruit body ROI0, damaged area ROI1 and undamaged area ROI2 corresponding to each thermal imaging image M, generate fitting curves for each pixel in damaged area ROI1 and undamaged area ROI2. Combined with the time-varying characteristics of apple surface temperature difference, calculate the optimal image segmentation threshold. The third) includes: The range of each pixel is calculated based on the fitted curve of each pixel; the range of each pixel is normalized to obtain the normalized range data; the optimal image segmentation threshold is determined based on the normalized range data. 4) After performing image conversion on the thermal imaging image sequence of the apple to be tested, the corresponding range grayscale image is obtained. After segmenting the range grayscale image using the optimal image segmentation threshold, the damage detection result of the current apple to be tested is obtained. The process of converting the thermal imaging image sequence of the apple under test to obtain the corresponding range grayscale image includes: The temperature difference of each pixel in the thermal imaging image sequence of the apple to be tested is generated; the Gauss function is used to fit the sampling time of the thermal imaging image with the temperature difference of each pixel to obtain the fitting curve corresponding to each pixel; then the range of each pixel is calculated based on the fitting curve of each pixel; after normalizing the range of each pixel, the range grayscale image is obtained.

2. The method for detecting initial damage based on the time-varying characteristics of temperature difference on the surface of an apple, as described in claim 1, is characterized in that... The third) also includes: 3.1) Average the temperature data of each pixel within the ROI0 of the entire fruit body corresponding to each thermal imaging image M to obtain the average fruit body temperature corresponding to the current thermal imaging image M. ; 3.2) Subtract the average fruit temperature from the temperature data of each pixel in the damaged region ROI1 and the undamaged region ROI2 corresponding to the current thermal imaging image M. Then, the temperature difference of each pixel in the damaged area ROI1 and the undamaged area ROI2 is obtained respectively, thereby obtaining the temperature difference of all pixels in the damaged area ROI1 and the undamaged area ROI2. 3.3) Repeat steps 3.1)-3.2) to calculate the temperature difference of all pixels in the damaged region ROI1 and the undamaged region ROI2 in the remaining thermal imaging image M; 3.4) The Gauss function is used to fit the sampling time of the thermal image M with the temperature difference of each pixel in the damaged area ROI1 and the undamaged area ROI2 to obtain the fitting curves corresponding to each pixel in the damaged area ROI1 and the undamaged area ROI2.

3. The method for detecting initial damage based on the time-varying characteristics of temperature difference on the surface of an apple, as described in claim 1, is characterized in that... In step 4), the temperature difference of each pixel in the thermal imaging image sequence of the apple to be tested is generated as follows: S1: Extract the region of interest from each thermal imaging image in the sequence of thermal imaging images of the apple to be tested, and obtain the entire area of ​​the fruit corresponding to the current thermal imaging image. S2: After averaging the temperature data of each pixel in the entire area of ​​the fruit corresponding to the current thermal imaging image, the average fruit temperature of the current thermal imaging image is obtained. S3: Subtract the average fruit temperature from the temperature data of each pixel in the entire area of ​​the fruit corresponding to the current thermal imaging image to obtain the temperature difference of each pixel in the current thermal imaging image. S4: Repeat S1-S3 to calculate the temperature difference of each pixel in the remaining thermal imaging images in the thermal imaging image sequence of the apple to be tested.

4. The method for detecting initial damage based on the time-varying characteristics of temperature difference on the surface of an apple, as described in claim 1, is characterized in that... In step 4), after segmenting the range grayscale image using the optimal image segmentation threshold, the damage detection result of the current apple to be tested is obtained, specifically as follows: After segmenting the range grayscale image using the optimal image segmentation threshold, an initial segmented image is obtained. Then, the damage region of the initial segmented image is accurately segmented to obtain a secondary segmented image. Finally, the damage is located in the secondary segmented image using the minimum bounding rectangle to obtain the damage detection result of the apple to be tested.

5. The method for detecting initial damage based on the time-varying characteristics of temperature difference on the surface of an apple, as described in claim 4, is characterized in that... The precise segmentation of damaged regions in the initial segmented image specifically involves using image dilation, erosion, maximum connected component extraction, and hole filling to precisely segment damaged regions in the initial segmented image.

Images