Fruit recessive bruise detection system and method based on near-infrared imaging

Abstract

The invention provides a fruit recessive bruise detection system and method based on near-infrared imaging, and relates to the technical field of agricultural nondestructive testing, and the method comprises a handheld device and a table-type device; a handheld device, an annular LED light source, a frosted light scattering plate, a camera obscura body and an objective table are arranged in the table type device; the annular LED light source is positioned below the handheld device; the frosted light scattering plate is arranged below the annular LED light source; the handheld device comprises a handheld shell, a CMOS (complementary metal oxide semiconductor) near-infrared camera, a near-infrared band-pass filter, an embedded processing module, a power module, a light source module and a display screen module; the power supply module is used for adopting a rechargeable lithium battery to movably charge the handheld device; and the display screen module is used for display of detection results and man-machine interaction of users. The portability of the fruit recessive bruise detection system is improved, and the accuracy of fruit recessive bruise detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural non-destructive testing, and in particular to a fruit hidden bruise detection system and method based on near-infrared imaging. Background Art

[0002] The fruit hidden bruise detection system based on near-infrared imaging is a system that uses near-infrared spectroscopy technology to detect hidden damage or bruises inside fruits. The system irradiates the fruit with near-infrared light and uses the absorption and scattering characteristics of near-infrared light by different tissues of the fruit to obtain spectral information reflecting its internal structure. Since the tissue structure of the fruit will change after being injured, near-infrared imaging can identify these changes and accurately detect hidden bruises inside the fruit.

[0003] With the intensive development of the global fruit industry chain, post-harvest quality control has become a core link in ensuring the added value of agricultural products. Among them, fruit bruises are the main source of loss during transportation and warehousing, and the demand for early bruise detection is becoming increasingly urgent.

[0004] However, existing fruit hidden bruise detection systems typically use a discrete module design, resulting in an overall heavy system weight, limiting the portability and mobility of the equipment, and making it difficult to popularize in large-scale agricultural production. When designing the optical path, traditional ring light sources are usually used, resulting in uneven illumination in the edge area, affecting the accuracy of detection. When processing data, more traditional image processing algorithms are used, resulting in high image processing and classification delays, which cannot meet the needs of real-time detection. Summary of the Invention

[0005] In order to solve the technical problems that existing fruit hidden bruise detection systems generally use a discrete module design, resulting in an excessively heavy overall system weight, limiting the portability and mobility of the equipment, and difficult to popularize in large-scale agricultural production; when designing the optical path, traditional ring light sources are generally used, resulting in uneven illumination in the edge area, affecting the accuracy of detection; when processing data, more traditional image processing algorithms are used, resulting in high image processing and classification delays, and unable to meet the needs of real-time detection, the present invention provides a fruit hidden bruise detection system and method based on near-infrared imaging.

[0006] The technical solutions provided by the embodiments of the present invention are as follows:

[0007] First aspect:

[0008] An embodiment of the present invention provides a fruit hidden bruise detection system based on near-infrared imaging, comprising: a handheld device and a desktop device;

[0009] The desktop device is equipped with the handheld device, an annular LED light source, a frosted light diffusion plate, a darkroom body, and a stage;

[0010] The annular LED light source is located below the handheld device, and is used to emit infrared light and evenly irradiate the infrared light on the surface of the fruit to be tested;

[0011] The frosted light diffuser is placed below the annular LED light source, and the frosted light diffuser is used to reduce the bright spot phenomenon on the fruit surface caused by direct light source;

[0012] The handheld device includes a handheld housing, a CMOS near-infrared camera, a near-infrared bandpass filter, an embedded processing module, a power module, a light source module and a display module;

[0013] The near-infrared bandpass filter is used to transmit external scene information into the sensor through the lens of the CMOS near-infrared camera;

[0014] The embedded processing module is used to process and output image data through a lightweight deep learning method, and transmit the image information to a mobile phone application based on the Android platform;

[0015] The power module is used to use a rechargeable lithium battery to charge the handheld device on the go;

[0016] The display screen module is used for displaying the test results and for user-computer interaction.

[0017] Second aspect:

[0018] An embodiment of the present invention provides a method for detecting hidden bruises on fruits based on near-infrared imaging, comprising:

[0019] S1: Obtain a sample image of bruises on the fruit to be tested;

[0020] S2: constructing a dataset of early hidden bruises on fruits based on the bruise sample images;

[0021] S3: Build a fruit bruise detection model based on the improved YOLOv8 network framework;

[0022] S4: inputting the fruit early hidden bruise dataset into the fruit bruise detection model for training;

[0023] S5: Acquire a real-time image of the fruit to be detected;

[0024] S6: inputting the real-time image into the trained fruit bruise detection model, and outputting fruit hidden bruise area data;

[0025] S7: Uploading the fruit hidden bruise area data to a cloud database through the MQTT protocol, and displaying the fruit hidden bruise area data through a mobile application.

[0026] The third aspect:

[0027] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for detecting hidden bruises on fruits based on near-infrared imaging as described in the first aspect is implemented.

[0028] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0029] In the embodiment of the present invention, the combined design of the handheld device and the desktop device makes the system portable and multifunctional. The handheld device can be used in situations that require fast movement and convenient use, and the desktop device provides a more stable detection environment. The use of a CMOS near-infrared camera reduces system costs, making the system more cost-effective and more suitable for agricultural production and small-scale applications. The combination of the CMOS near-infrared camera and the near-infrared bandpass filter can effectively extract image information of hidden bruises on the surface of fruits, enhance the contrast of the bruised area in the image, and enable the camera to more accurately capture the details of the hidden bruise area. The use of a ring-shaped LED light source and a frosted diffuser plate design solves the problem of uneven lighting, ensures image clarity and accuracy, avoids false detections caused by light spots, and thus improves the uniformity and clarity of the image, allowing the system to adapt to different working environments and have stronger environmental adaptability. The display module provides real-time display of images and test results, allowing users to clearly see the test process and results, facilitating operation and monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 A schematic diagram of the structure of a fruit hidden bruise detection system based on near-infrared imaging provided by an embodiment of the present invention;

[0032] Figure 2 A schematic diagram of the structure of a handheld device provided in an embodiment of the present invention;

[0033] Figure 3 A schematic structural diagram of a ring-shaped LED light source provided by an embodiment of the present invention;

[0034] Figure 4 A circuit diagram of a ring-shaped LED light source provided by an embodiment of the present invention;

[0035] Figure 5 A schematic diagram of a process for detecting hidden bruises on fruits based on near-infrared imaging provided by an embodiment of the present invention;

[0036] Figure 6 This is a schematic diagram of the structure of a fruit bruise detection model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0038] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0039] In the embodiments of the present invention, the terms "image" and "picture" may be used interchangeably. It should be noted that, when the distinction between them is not emphasized, their intended meanings are the same. The terms "of," "corresponding," and "corresponding" may be used interchangeably. It should be noted that, when the distinction between them is not emphasized, their intended meanings are the same.

[0040] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0041] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0042] Reference Manual Figure 1 , shows a structural schematic diagram of a fruit hidden bruise detection system based on near-infrared imaging provided by an embodiment of the present invention.

[0043] Reference Manual Figure 2 , which shows a schematic structural diagram of a handheld device provided in an embodiment of the present invention.

[0044] like Figure 2 As shown, 1 represents the display module, 2 represents the battery module, 3 represents the Raspberry Pi 4B+ processor, 4 represents the near-infrared sensor, 5 represents the housing, and 6 represents the near-infrared lens.

[0045] Reference Manual Figure 3, shows a structural schematic diagram of a ring-shaped LED light source provided in an embodiment of the present invention.

[0046] like Figure 3 As shown, the light source consists of eight 1.4V LEDs arranged in a circular array within the ring structure. All LEDs are connected in parallel, with a 24Ω protection resistor in the main circuit to ensure current stability. The ring-shaped LED light source is connected via a cable to a 5V, 2000mAh rechargeable battery and controlled by a switch. This ring-shaped LED light source emits 940nm near-infrared light, evenly illuminating the surface of the fruit under test. This ensures uniform illumination across the fruit, preventing light spots from affecting test results. It is suitable for detecting hidden bruises on fruit.

[0047] Reference Manual Figure 4 , shows a circuit schematic diagram of a ring-shaped LED light source provided in an embodiment of the present invention.

[0048] like Figure 4 The following is a schematic diagram of the ring LED light source, showing the specific connection structure of the light board. The core of the circuit is eight LED beads (XL-1608IRC940), which are connected in parallel in a circular array to form a ring LED light source. Each LED bead is connected through the same current path, and the overall circuit uses a 24Ω resistor (R1) to limit the current and ensure safe operation of the LED. Power is provided via switch (SW1) and connector (H1), and the supply voltage is 5V.

[0049] An embodiment of the present invention provides a fruit hidden bruise detection system based on near-infrared imaging, comprising a handheld device and a desktop device.

[0050] The desktop device contains a handheld device, a ring LED light source, a frosted diffuser, a darkroom body, and a stage.

[0051] Among them, the ring LED light source is a ring light source composed of multiple light-emitting diodes (LEDs), which is used to reduce local lighting differences and ensure that the light source is evenly distributed. The frosted diffuser is an optical element used to scatter light, usually made of frosted glass or similar materials. It can effectively reduce the high light spots or local overbrightness caused by direct light sources, and help the light to be evenly illuminated on the surface of the target object. The darkroom body is a closed box or container used to provide an environment without external interference. The darkroom body is sprayed with diffuse reflective paint with a reflectivity greater than 95%, which can avoid external light interference, ensure that the light source is evenly illuminated and maintain stable measurement conditions. The stage is a support platform for placing objects to be inspected (such as fruits). The height and angle can be adjusted to ensure that the objects can be properly illuminated and accurately aligned for effective inspection.

[0052] The annular LED light source is located below the handheld device and is used to emit infrared light and evenly irradiate the infrared light on the surface of the fruit to be tested.

[0053] The frosted diffuser is placed under the annular LED light source. The frosted diffuser is used to reduce the bright spots on the surface of the fruit caused by direct light source.

[0054] The handheld device includes a handheld housing, a CMOS near-infrared camera, a near-infrared bandpass filter, an embedded processing module, a power module, a light source module and a display module.

[0055] In the present invention, the handheld housing is made of nylon material and measures 100mm×76mm×74mm. It integrates a 700-1100nm CMOS near-infrared camera (Sony IMX sensor, resolution 8.3 million pixels), a near-infrared bandpass filter (coated band 900-1100nm, transmittance ≥95%), an embedded processing module (Raspberry Pi 4B+, 4GB memory), four 940nm infrared LED light sources (single power 0.5W, connected in parallel, equipped with a 24Ω current-limiting resistor), a 3.7V / 3000mAh lithium battery (battery life ≥4 hours) and a 3.5-inch capacitive touch screen (resolution 480×320, support for touch operation). The four LED light sources are distributed in a ring around the camera lens, with a light source spacing of 40mm. The frosted light diffusion plate (made of polycarbonate, thickness 2mm) is used to evenly illuminate the fruit surface to avoid light spot interference. The Raspberry Pi is connected to the camera via a USB 3.0 port, communicates with the mobile application via a Wi-Fi module, and supports the MQTT protocol to upload data to the cloud (Alibaba Cloud IoT platform).

[0056] The near-infrared bandpass filter is used to transmit external scene information into the sensor through the lens in the CMOS near-infrared camera.

[0057] The embedded processing module is used to process and output image data through lightweight deep learning methods, and transmit image information to mobile applications based on the Android platform.

[0058] The power module is used to use rechargeable lithium batteries to charge handheld devices on the go.

[0059] The display screen module is used to display the test results and for user-computer interaction.

[0060] In the present invention, a near-infrared bandpass filter transmits external scene information into the sensor through the lens of the near-infrared camera. The embedded processing module Raspberry 4B processes and outputs the captured image data using a lightweight deep learning method, and transmits the image information to a mobile application based on the Android platform. The app can take real-time photos, view detection results, and interact with the cloud database through the MQTT protocol. The power module uses a 3.7V, 3000mAh rechargeable lithium battery to provide mobile charging for the handheld device. The display module is used to display detection results and facilitate user interaction.

[0061] In one possible implementation, the CMOS near-infrared camera, the near-infrared bandpass filter, the embedded processing module, the power module, the light source module, and the display module are integrated into the interior of the handheld housing.

[0062] It should be noted that by combining a desktop device with a handheld device, a ring-shaped LED light source and a frosted diffuser ensure uniform illumination of the fruit surface, reduce bright spots, and improve detection accuracy. The handheld device integrates a CMOS near-infrared camera, a near-infrared bandpass filter, and a deep learning algorithm to provide efficient image processing and real-time output of detection results. The embedded processing module achieves a lightweight design, ensuring the system is easy to move and use for a long time, making it suitable for a variety of environments.

[0063] In a possible implementation, the handheld device is embedded in a groove on the top of the darkroom body.

[0064] The handheld device is designed for standalone use or connected to a desktop device.

[0065] The annular LED light source includes a plurality of light emitting diodes, and each light emitting diode is distributed in a circular array to form a parallel circuit.

[0066] The dark box body is used to support the handheld device, the ring LED light source and the frosted diffuser.

[0067] In a possible implementation, a near-infrared anti-reflection film is coated on the surface of the near-infrared bandpass filter.

[0068] In this invention, a near-infrared bandpass filter is coated with a near-infrared antireflection coating that provides high transmittance in the 900-1100nm near-infrared wavelength range. The near-infrared camera utilizes the absorption characteristics of the OH bond of water molecules at 960nm, combining the bandpass filter with a near-infrared light source to enhance the contrast of the bruised area. The mobile application supports the following functions: real-time photography and image preview, displaying and saving test results, and interacting with a cloud database via the MQTT protocol, enabling remote storage and access of test results.

[0069] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0070] In the embodiment of the present invention, the combined design of the handheld device and the desktop device makes the system portable and multifunctional. The handheld device can be used in situations that require fast movement and convenient use, and the desktop device provides a more stable detection environment. The use of a CMOS near-infrared camera reduces system costs, making the system more cost-effective and more suitable for agricultural production and small-scale applications. The combination of the CMOS near-infrared camera and the near-infrared bandpass filter can effectively extract image information of hidden bruises on the surface of fruits, enhance the contrast of the bruised area in the image, and enable the camera to more accurately capture the details of the hidden bruise area. The use of a ring-shaped LED light source and a frosted diffuser plate design solves the problem of uneven lighting, ensures image clarity and accuracy, avoids false detections caused by light spots, and thus improves the uniformity and clarity of the image, allowing the system to adapt to different working environments and have stronger environmental adaptability. The display module provides real-time display of images and test results, allowing users to clearly see the test process and results, facilitating operation and monitoring.

[0071] Reference Manual Figure 5 , shows a flow chart of a method for detecting hidden bruises on fruits based on near-infrared imaging provided by an embodiment of the present invention.

[0072] The embodiment of the present invention provides a method for detecting hidden bruises on fruits based on near-infrared imaging, which is applied to the above-mentioned embodiment of the present invention. The embodiment of the present invention provides a system for detecting hidden bruises on fruits based on near-infrared imaging, including:

[0073] S1: Obtain a sample image of a bruise on the fruit to be tested.

[0074] In a possible implementation, S1 specifically includes:

[0075] Using a portable near-infrared imaging device, images of bruise samples were obtained.

[0076] It should be noted that by obtaining bruise sample images of the fruit to be tested, clear and quantifiable data support is provided. The detection system can identify and analyze hidden bruises on the surface of the fruit, thereby improving the accuracy and effectiveness of the detection.

[0077] In the present invention, fruit bruises are simulated by steel ball impact or free fall to produce fruit samples bruised within half an hour. A portable near-infrared imaging device is used to capture 900-1100 nm images of bruised samples within half an hour. A dataset of early hidden bruises on fruits is constructed, and data enhancement and annotation are performed on the dataset.

[0078] S2: Construct a dataset of early hidden bruises on fruits based on bruise sample images.

[0079] It should be noted that by constructing a dataset of early hidden bruises on fruits based on bruise sample images, the model can adapt to different types and degrees of fruit bruises, improving the accuracy and breadth of detection.

[0080] In a possible implementation manner, after S2, the method further includes:

[0081] Image enhancement including gamma correction and contrast enhancement was performed on the fruit early hidden bruise dataset.

[0082] It should be noted that by performing gamma correction and contrast enhancement on the dataset of early hidden bruises on fruits, the image quality can be effectively improved and the recognizability of hidden bruises can be enhanced. Gamma correction can adjust the image brightness to make details more obvious, while contrast enhancement can enhance the contrast between the bruised area on the surface of the fruit and the surrounding area, so that the model can more accurately identify and locate bruises during training and actual detection.

[0083] Gamma correction is specifically:

[0084]

[0085] Among them, I input () represents the pixel value of the fruit early hidden bruise dataset, (x, y) represents the pixel position, I output () represents the pixel value of the fruit early hidden bruise dataset after gamma correction, and γ represents the gamma value.

[0086] Contrast enhancement is specifically:

[0087] I′ output (x,y)=α·I input (x,y)+β

[0088] Among them, I′ output (x, y) represents the pixel value of the fruit early hidden bruise dataset after contrast enhancement, α represents the contrast gain, and β represents the brightness correction term.

[0089] In the present invention, α=1.0, β=0.

[0090] Reference Manual Figure 6 , shows a structural schematic diagram of a fruit bruise detection model provided by an embodiment of the present invention.

[0091] like Figure 6As shown in the figure, a structural diagram of the fruit bruise detection model provided by an embodiment of the present invention is shown. The model is divided into three main parts: Backbone (backbone network), Neck (neck network) and Head (output module). Backbone includes multiple stages (Stage1, Stage2, Stage3, Stage4). Each stage merges features through the Merging module to extract the underlying features of the fruit image, and fuses features through multiple C2f modules to make the features richer and more recognizable. Neck is mainly responsible for further integrating and optimizing the features transferred from Backbone. The Concat operation and Upsample module are used to splice and upsample the feature maps to further improve the recognition ability of the target. The CBS module realizes feature extraction and nonlinear transformation of the input feature map by combining convolutional layers, batch normalization layers and activation functions. The SPPF module is used to process multi-scale information and enhance the network's detection ability for targets of different sizes. Head is the final output part, which includes the Detect module, which is used to perform target detection and output the detection results of the fruit bruise area, including category and bounding box information. Maxpooling is used for downsampling, reducing the spatial size of the feature map while retaining the most significant features. Three max pooling operations are used in the figure. Batch Normalization (BN) stands for batch normalization. After each convolution operation (Conv), BN is used to normalize the output of the convolutional layer.

[0092] S3: Build a fruit bruise detection model based on the improved YOLOv8 network framework.

[0093] It should be noted that by building a fruit bruise detection model based on the improved YOLOv8 network framework, we can utilize YOLOv8's powerful real-time target detection capabilities to improve detection speed while maintaining high detection accuracy. The improved YOLOv8 framework can better handle the detection of hidden bruises on fruits in complex backgrounds, reduce false detections and missed detections, and improve the model's operating efficiency through lightweight design, enabling fast inference on embedded devices to meet real-time detection needs in practical applications.

[0094] In one possible implementation, the fruit bruise detection model includes: a trunk module, a neck module, and an output module.

[0095] The backbone module includes a FasterNet unit and a SPPF unit connected in sequence.

[0096] The FasterNet unit includes an Embedding subunit, a Stage1 subunit, a first Merging subunit, a Stage2 subunit, a second Merging subunit, a Stage3 subunit, a third Merging subunit, and a Stage4 subunit, which are connected in sequence.

[0097] The neck module includes a first row structure and a second row structure.

[0098] The first column structure includes a first upsampling unit, a first splicing operation unit, a first C2f unit, a second upsampling unit, and a second splicing operation unit, which are connected in sequence.

[0099] The second column structure includes a second C2f unit, a first CBS unit, a third splicing operation unit and a third C2f unit connected in sequence.

[0100] The output module includes a small target detection head unit and a medium target detection head unit.

[0101] The SPPF unit is connected to the first up-sampling unit.

[0102] The Stage2 unit is connected to the second splicing operation unit.

[0103] The Stage3 unit is connected to the first splicing operation unit.

[0104] The second C2f unit is connected to the small target detection head unit.

[0105] The third C2f unit is connected to the central target detection head unit.

[0106] In the present invention, the backbone is composed of a head FasterNet and a tail SPPF module in series from top to bottom, wherein the head includes a serial structure consisting of four stages of partial convolution PConv and point-by-point convolution, the convolution kernel size of the partial convolution layer PConv is 3, the first stage is preceded by an embedding layer, and the second, third, and fourth stages are preceded by an integration layer. The neck is composed of two columns of bottom-up structures, the first column consists of upsample, Concat splicing operation, and C2f module, and the upsample and Concat splicing operation are connected in sequence from bottom to top, and the second column consists of C2f module, CBS module, Concat splicing operation, and C2f module, which are connected in sequence from top to bottom. The output of the SPPF module of the backbone is connected to the upsample at the bottom of the first column of the neck, and the second stage of the backbone is connected to the second Concat splicing operation at the top of the first column of the neck. The third stage is connected to the first concat operation at the bottom of the neck. In the output part, the C2f at the top of the neck is connected to the small target detection head of the output part, and the C2f at the bottom of the second column is connected to the medium target detection head of the output part. The PConv module includes a 3×3 partial convolution followed by two 1×1 point-by-point convolutions. The two point-by-point convolutions are subjected to batch normalization and ReLu activation function.

[0107] The backbone network replaces the original CSPDarknet53 backbone with the FasterNet architecture. By introducing a new partial convolution (PConv) and an efficient activation function design, it significantly reduces computational complexity (FLOPs reduced by 36.6%) while maintaining feature extraction capabilities. The feature extraction module is mainly based on the FasterNet architecture. This network generates multi-scale feature maps through a four-stage hierarchical structure with strides of 4, 8, 16, and 32 in each stage. The first layer is preceded by an embedding layer (a regular 4×4 convolution with a stride of 4), and the second, third, and fourth layers are preceded by a merging layer (a regular 2×2 convolution with a stride of 2). These embedding and merging layers are used for spatial downsampling and expanding the number of channels. Each layer has a FasterNet Block, which consists of a 3×3 partial convolution followed by two 1×1 point-by-point convolutions. Including more blocks in the last two layers can reduce computational complexity. Because the pixel size of bruised fruit is small, the large object detection head is redundant. Therefore, to address the need for small and medium-sized object detection, the original YOLOv8's 20×20 large object detection head was removed, retaining the two medium-resolution detection heads of 40×40 and 80×80, reducing the model parameters by one-third. Through the coordinated optimization of the Feature Pyramid Network (FPN) and the Path Aggregation Network (PAN), the cross-scale fusion of shallow high-resolution features and deep semantic features is strengthened, allowing the network to focus on the fine localization of small and medium-sized objects.

[0108] S4: Input the early hidden bruise dataset of fruits into the fruit bruise detection model for training.

[0109] Specifically, inputting the early hidden bruise dataset of fruits into the detection model for training can enable the model to learn the characteristics of hidden bruises on the surface of different types of fruits. By training on a large amount of labeled data, the model can effectively improve the recognition accuracy of hidden bruises, optimize detection performance, and reduce errors and missed detections.

[0110] In a possible implementation, S4 is specifically:

[0111] The early hidden bruise dataset of fruits is input into the fruit bruise detection model for training until the loss function value is less than the preset loss function value.

[0112] It should be noted that those skilled in the art can set the size of the preset loss function value according to actual needs, and the present invention is not limited here.

[0113] It should be noted that inputting the fruit early hidden bruise dataset into the model and training it until the loss function value is less than the preset threshold can ensure that the model is continuously optimized during the learning process, gradually reducing prediction errors, improving the model's accuracy, avoiding overfitting or underfitting, and ensuring that the model can accurately detect hidden bruises on fruits in real scenarios.

[0114] The loss function value is calculated as follows:

[0115]

[0116] Among them, L MPDIoU represents the loss function, IoU represents the intersection-over-union ratio, and λ represents the balance weight coefficient. and Represents the pixel coordinates of the i-th corner point of the prediction box, and Represents the pixel coordinates of the i-th corner point of the ground-truth box, N represents the total number of corner points, and n represents the n-th corner point (n≤N).

[0117] S5: Acquire a real-time image of the fruit to be inspected.

[0118] S6: Input the real-time image into the trained fruit bruise detection model and output the fruit hidden bruise area data.

[0119] It's important to note that by feeding real-time images into a trained fruit bruise detection model, test results can be quickly obtained, outputting data on hidden bruise areas on the fruit. This allows for rapid response to on-site testing needs, improves efficiency, and avoids long wait times. Furthermore, the use of the trained model ensures detection precision and accuracy, enabling timely detection of hidden bruises on the fruit surface, helping fruit farmers or warehouse managers make quick decisions and reduce losses.

[0120] In the present invention, the total detection time of the fruit bruise detection model on the Raspberry Pi is ≤300ms, the detection accuracy (mAP@0.5) is ≥96.3%, the recall rate (Recall) is ≥98.5%, and the precision is ≥99.4%.

[0121] The accuracy is calculated as follows:

[0122]

[0123] Among them, Precision represents the accuracy, TP represents the correctly predicted positive class, and FP represents the negative class that is incorrectly predicted as the positive class.

[0124] The recall rate is calculated as follows:

[0125]

[0126] Among them, Recall represents the recall rate, and FN represents the positive class that is mistakenly detected as the negative class.

[0127] The calculation method of detection accuracy is as follows:

[0128]

[0129] Among them, mAP represents the detection accuracy of all categories, AP(i) represents the detection accuracy of the i-th category, i = 0, 1, ..., n, and n represents the total number of categories.

[0130] AP refers to the area enclosed by the PR curve. The AP value range is between 0 and 1. The higher the AP, the better the model detection effect.

[0131] S7: The data of hidden bruise areas of fruits are uploaded to the cloud database through the MQTT protocol, and the data of hidden bruise areas of fruits are displayed through the mobile application.

[0132] It should be noted that uploading fruit hidden bruise area data to a cloud database via the MQTT protocol enables real-time data storage and management, ensuring long-term data preservation and remote access. Cloud storage allows multiple users to share data, facilitating data analysis and decision-making. Displaying hidden bruise area data via a mobile app allows users to view test results anytime, anywhere, greatly improving the system's convenience and responsiveness.

[0133] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0134] In the embodiment of the present invention, the combined design of the handheld device and the desktop device makes the system portable and multifunctional. The handheld device can be used in situations that require fast movement and convenient use, and the desktop device provides a more stable detection environment. The use of a CMOS near-infrared camera reduces system costs, making the system more cost-effective and more suitable for agricultural production and small-scale applications. The combination of the CMOS near-infrared camera and the near-infrared bandpass filter can effectively extract image information of hidden bruises on the surface of fruits, enhance the contrast of the bruised area in the image, and enable the camera to more accurately capture the details of the hidden bruise area. The use of a ring-shaped LED light source and a frosted diffuser plate design solves the problem of uneven lighting, ensures image clarity and accuracy, avoids false detections caused by light spots, and thus improves the uniformity and clarity of the image, allowing the system to adapt to different working environments and have stronger environmental adaptability. The display module provides real-time display of images and test results, allowing users to clearly see the test process and results, facilitating operation and monitoring.

[0135] It should be understood that the processor in the embodiments of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0136] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0137] The above embodiments can be implemented in whole or in part through software, hardware (such as circuits), firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired method (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, or magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0138] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0139] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0140] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0141] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0142] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0143] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0144] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0145] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0146] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0147] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for detecting hidden bruises on fruits based on near-infrared imaging as described in the method embodiment is implemented.

[0148] The computer-readable storage medium provided by the present invention can implement the steps and effects of the fruit hidden bruise detection method based on near-infrared imaging of the above-mentioned method embodiment. To avoid repetition, the present invention will not go into details.

[0149] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0150] In the embodiment of the present invention, the combined design of the handheld device and the desktop device makes the system portable and multifunctional. The handheld device can be used in situations that require fast movement and convenient use, and the desktop device provides a more stable detection environment. The use of a CMOS near-infrared camera reduces system costs, making the system more cost-effective and more suitable for agricultural production and small-scale applications. The combination of the CMOS near-infrared camera and the near-infrared bandpass filter can effectively extract image information of hidden bruises on the surface of fruits, enhance the contrast of the bruised area in the image, and enable the camera to more accurately capture the details of the hidden bruise area. The use of a ring-shaped LED light source and a frosted diffuser plate design solves the problem of uneven lighting, ensures image clarity and accuracy, avoids false detections caused by light spots, and thus improves the uniformity and clarity of the image, allowing the system to adapt to different working environments and have stronger environmental adaptability. The display module provides real-time display of images and test results, allowing users to clearly see the test process and results, facilitating operation and monitoring.

[0151] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

[0152] There are a few points to note:

[0153] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention. Other structures may refer to conventional designs.

[0154] (2) For the sake of clarity, the thickness of layers or regions in the drawings used to describe the embodiments of the present invention are exaggerated or reduced, that is, these drawings are not drawn to scale. It is understood that when an element such as a layer, film, region, or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or intervening elements may be present.

[0155] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to form new embodiments.

[0156] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A fruit hidden bruise detection system based on near-infrared imaging, characterized in that: include: Handheld and desktop devices; The desktop device is equipped with the handheld device, an annular LED light source, a frosted light diffusion plate, a darkroom body, and a stage; The annular LED light source is located below the handheld device, and is used to emit infrared light and evenly irradiate the infrared light on the surface of the fruit to be tested; The frosted light diffuser is placed below the annular LED light source, and the frosted light diffuser is used to reduce the bright spot phenomenon on the fruit surface caused by direct light source; The handheld device includes a handheld housing, a CMOS near-infrared camera, a near-infrared bandpass filter, an embedded processing module, a power module, a light source module and a display module; The near-infrared bandpass filter is used to transmit external scene information into the sensor through the lens of the CMOS near-infrared camera; The embedded processing module is used to process and output image data through a lightweight deep learning method, and transmit the image information to a mobile phone application based on the Android platform; The power module is used to use a rechargeable lithium battery to charge the handheld device on the go; The display screen module is used for displaying the test results and for user-computer interaction.

2. The fruit hidden bruise detection system based on near infrared imaging according to claim 1, characterized in that: The CMOS near-infrared camera, the near-infrared bandpass filter, the embedded processing module, the power module, the light source module and the display screen module are integrated inside the handheld housing.

3. The fruit hidden bruise detection system based on near infrared imaging according to claim 1, characterized in that: The handheld device is embedded in the groove on the top of the dark box body; The handheld device is used alone or in connection with the desktop device; The annular LED light source includes a plurality of light emitting diodes, each of which is distributed in a circular array to form a parallel circuit; The dark box body is used to support the handheld device, the annular LED light source and the frosted light diffusion plate.

4. The fruit hidden bruise detection system based on near infrared imaging according to claim 1, characterized in that: The surface of the near-infrared bandpass filter is coated with a near-infrared anti-reflection film.

5. A method for detecting hidden bruises on fruits based on near-infrared imaging, applied to the fruit hidden bruise detection system based on near-infrared imaging according to any one of claims 1 to 4, characterized in that: include: S1: Obtain a sample image of bruises on the fruit to be tested; S2: constructing a dataset of early hidden bruises on fruits based on the bruise sample images; S3: Build a fruit bruise detection model based on the improved YOLOv8 network framework; S4: inputting the fruit early hidden bruise dataset into the fruit bruise detection model for training; S5: Acquire a real-time image of the fruit to be detected; S6: inputting the real-time image into the trained fruit bruise detection model, and outputting fruit hidden bruise area data; S7: Uploading the fruit hidden bruise area data to a cloud database through the MQTT protocol, and displaying the fruit hidden bruise area data through a mobile application.

6. The method for detecting hidden bruises on fruits based on near-infrared imaging according to claim 5, wherein: The S1 is specifically: The bruise sample image is acquired using a portable near-infrared imaging device.

7. The method for detecting hidden bruises on fruits based on near-infrared imaging according to claim 5, wherein: After S2, the method further includes: Performing image enhancement including gamma correction and contrast enhancement on the fruit early hidden bruise dataset; The gamma correction is specifically: Among them, I input () represents the pixel value of the fruit early hidden bruise dataset, (x, y) represents the pixel position, I output () represents the pixel value of the fruit early hidden bruise dataset after gamma correction, γ represents the gamma value; The contrast enhancement is specifically as follows: I′ output (x,y)=α·I input (x,y)+β; Among them, I′ output (x, y) represents the pixel value of the fruit early hidden bruise dataset after contrast enhancement, α represents the contrast gain, and β represents the brightness correction term.

8. The method for detecting hidden bruises on fruits based on near-infrared imaging according to claim 5, wherein: The fruit bruise detection model includes: a trunk module, a neck module and an output module; The backbone module includes a FasterNet unit and an SPPF unit connected in sequence; The FasterNet unit includes an Embedding subunit, a Stage1 subunit, a first Merging subunit, a Stage2 subunit, a second Merging subunit, a Stage3 subunit, a third Merging subunit, and a Stage4 subunit connected in sequence; The neck module includes a first row structure and a second row structure; The first column structure includes a first upsampling unit, a first splicing operation unit, a first C2f unit, a second upsampling unit, and a second splicing operation unit connected in sequence; The second column structure includes a second C2f unit, a first CBS unit, a third splicing operation unit and a third C2f unit connected in sequence; The output module includes a small target detection head unit and a medium target detection head unit; The SPPF unit is connected to the first upsampling unit; The Stage 2 unit is connected to the second splicing operation unit; The Stage 3 unit is connected to the first splicing operation unit; The second C2f unit is connected to the small target detection head unit; The third C2f unit is connected to the mid-target detection head unit.

9. The method for detecting hidden bruises on fruits based on near-infrared imaging according to claim 5, wherein: The S4 is specifically: Inputting the fruit early hidden bruise dataset into the fruit bruise detection model for training until the loss function value is less than a preset loss function value; The loss function value is calculated as follows: Among them, L MPDIoU represents the loss function, IoU represents the intersection-over-union ratio, and λ represents the balance weight coefficient. and Represents the pixel coordinates of the i-th corner point of the prediction box, and Represents the pixel coordinates of the i-th corner point of the ground-truth box, N represents the total number of corner points, and n represents the n-th corner point (n≤N).

10. A fruit hidden bruise detection system based on near-infrared imaging, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method for detecting hidden bruises on fruits based on near-infrared imaging according to any one of claims 1 to 9 is implemented.

Images