A wearable method and device for dynamic detection and display of bayberry quality

Abstract

This invention discloses a wearable method and device for dynamic detection and display of bayberry quality. The method includes: inputting spectral images acquired by three spectral image sensors into a target recognition model to identify the location of bayberries in the images; performing stereo matching on a first feature spectral image and a second feature spectral image to extract the spatial location and diameter of each bayberry; extracting the grayscale value of each bayberry location from the first feature spectral image and the second feature spectral image, respectively, as two feature reflectance values; using the R channel pixel value of each bayberry location from the RGB spectral image as a reference brightness; and using the ratio of the two feature reflectance values ​​to the reference brightness as the corrected relative reflectance of the two features to obtain the sugar content and acidity of the bayberry. This invention provides users with real-time bayberry quality identification and guidance for bayberry picking.

Description

Technical Field

[0001] This invention belongs to the field of agricultural fruit quality visualization, specifically relating to a wearable method and device for dynamic detection and display of bayberry quality. Background Technology

[0002] As a specialty economic fruit of China, especially Zhejiang Province, the waxberry is widely favored in the market for its unique flavor and rich nutrition. However, the waxberry fruit naturally lacks a protective peel, making it highly susceptible to physical damage and microbial contamination throughout the entire industry chain, including harvesting, sorting, and transportation, leading to quality decline and economic losses. Currently, the quality assessment of waxberries mainly relies on manual screening. This method is not only inefficient and unable to meet the needs of large-scale production, but also highly dependent on worker experience, resulting in strong subjectivity, inconsistent standards, and poor accuracy. More importantly, the repeated touching and fruit piling during manual screening can exacerbate secondary damage to the waxberries. Therefore, developing a rapid, non-destructive, accurate, intuitive, and portable intelligent quality detection technology is crucial to improving the overall efficiency of the waxberry industry.

[0003] While algorithms for identifying round fruits using traditional image processing and for obtaining the sugar and acid content of bayberries using spectral analysis have been developed in the field, their applications are significantly disconnected. Furthermore, traditional image recognition algorithms struggle to detect bayberries due to factors such as leaf occlusion in real-world scenarios. This invention employs artificial intelligence visual analysis methods to process images containing bayberries, automatically detecting and locating their positions within the image, thus overcoming the shortcomings of traditional methods in terms of positioning efficiency and accuracy. Augmented reality (AR) visualization, an advanced technology, enables hands-free, intuitive information overlay, significantly reducing physical damage and microbial contamination of bayberries while conveniently detecting their quality indicators. However, AI's powerful analytical capabilities are largely confined to industrial production lines or smartphone applications requiring manual operation, lacking real-time, seamless interaction with frontline workers. While AR glasses can achieve hands-free, intuitive information overlay, they have not yet been applied to real-time, quantitative analysis of agricultural product quality. Therefore, how to deeply integrate AI's "intelligent analysis" with AR's "intuitive visualization" on wearable devices is a current technological challenge.

[0004] Compared to existing technologies, such as the invention patent with application number "2021115391332" which discloses a portable detection device that uses an LED light source and a light intensity sensor, requiring the light-shielding wall to be physically attached to the bayberry fruit for measurement, this solution, while achieving portability, still belongs to contact or close-range fixed single-fruit detection. The operation process is relatively cumbersome and cannot achieve long-distance, non-contact, rapid scanning detection. More importantly, they only display digital results on a screen, lacking the intuitiveness of integrating quality information with the real-world view, and do not apply artificial intelligence technology to achieve intelligent target recognition in complex environments.

[0005] In summary, there are clear technological gaps in the current field of non-destructive testing of bayberry quality: on the one hand, traditional non-destructive testing equipment is expensive, inconvenient, and provides unintuitive results; on the other hand, although powerful AI visual analysis algorithms exist, their application has not been efficiently integrated with the actual work processes of operators; simultaneously, although mature AR visualization and interaction technology exists, it has not yet been applied to the real-time, quantitative analysis of agricultural product quality. Most existing technical solutions suffer from problems in one or more aspects, such as cost, accuracy, portability, environmental adaptability, degree of result visualization, and real-time human-computer interaction. Therefore, this invention aims to fill this gap by creatively integrating advanced AI visual analysis models, spectral sensing systems, and portable AR smart glasses to propose a visualized, intelligent, accurate, portable, and low-cost non-destructive testing solution for bayberry quality. Summary of the Invention

[0006] This invention aims to solve the problems of existing bayberry quality testing methods, such as invisible indicators, low efficiency, great influence from ambient light, inaccurate results, lack of portability, and high cost. It provides a bayberry AI glasses and its sugar and acidity quality non-destructive testing device and method to achieve intuitive, intelligent, accurate, portable, and low-cost bayberry quality non-destructive testing.

[0007] In a first aspect, the present invention provides a wearable dynamic detection and display method for bayberry quality, the display device of which includes an eyeglass frame, and a spectral acquisition module and a display module fixed on the eyeglass frame; the spectral acquisition module includes a first spectral image sensor, a second spectral image sensor and a reference spectral image sensor arranged in parallel; filters corresponding to the sugar content and acidity of bayberries are respectively installed on the lenses of the first spectral image sensor and the second spectral image sensor.

[0008] The wearable dynamic detection and display method for bayberry quality includes:

[0009] The first feature spectral image, the second feature spectral image, and the RGB spectral image acquired by three spectral image sensors are input into the target recognition model to identify the location of the bayberry in the image;

[0010] Binocular stereo matching was performed on the first feature spectral image and the second feature spectral image to extract the spatial position and diameter of each bayberry;

[0011] The grayscale values ​​of each bayberry location are extracted from the first feature spectral image and the second feature spectral image, respectively, as two feature reflectance brightnesses; the R channel pixel values ​​of each bayberry location in the RGB spectral image are used as the reference brightness.

[0012] The ratio of the reflectance of the two features to the reference reflectance is used as the corrected relative reflectance of the two features;

[0013] The relative reflectance of the two features is input into the bayberry sugar and acidity extraction model to obtain the sugar and acidity of bayberries;

[0014] The display module shows the quality data for each bayberry; the quality data includes diameter, sugar content, and acidity.

[0015] Preferably, the first feature spectral image, the second feature spectral image, and the RGB spectral image are input into three independent target recognition models, and the recognition results corresponding to the RGB spectral image are used as a benchmark to perform association matching of the recognition results of the three spectral images.

[0016] Preferably, the display module projects the quality data displayed on the screen onto the user's eyes through a reflective prism; the projection position of the quality data is the side of the user's eye corresponding to the location of the bayberry.

[0017] Preferably, the characteristic reflectance brightness is obtained by calculating the average gray value of the area occupied by the bayberry in the first characteristic spectral image or the second characteristic spectral image; the reference brightness is obtained by calculating the average pixel value of the R channel of the area occupied by the bayberry in the reference characteristic spectral image; the area occupied by the bayberry is a circular area with the bayberry recognition center as the center.

[0018] Secondly, the present invention provides a wearable dynamic detection and display device for bayberry quality, which is used to execute the aforementioned dynamic detection and display method; the wearable dynamic detection and display device for bayberry quality includes an eyeglass frame, a control module, and a spectral acquisition module and a display module fixed on the eyeglass frame; the spectral acquisition module includes a first spectral image sensor, a second spectral image sensor, and a reference spectral image sensor fixed on the top front of the eyeglass frame; filters corresponding to the sugar content and acidity of bayberries are respectively installed on the lenses of the first spectral image sensor and the second spectral image sensor; the display module includes a display screen and a reflecting prism aligned with each other; the position of the reflecting prism corresponds to the position of the user's eyes.

[0019] Preferably, the optical axes of the three spectral image sensors are parallel to each other; the reference spectral image sensor is located between the first spectral image sensor and the second spectral image sensor.

[0020] Preferably, the front of the eyeglass frame is provided with a mounting beam corresponding to the position above the user's eyes; the top of the mounting beam is integrally provided with three L-shaped mounting brackets; the three L-shaped mounting brackets are respectively fixed to the spectral image sensor through reserved connection holes.

[0021] Preferably, a mounting base is fixed to one side of the eyeglass frame; the display screen and the reflecting prism are fixed to the mounting base.

[0022] Preferably, the control module is set independently of the eyeglass frame and communicates with the spectral acquisition module and the display module via wired or wireless communication.

[0023] Preferably, the control module includes a data extraction module, an automatic identification module, and a sugar-acidity extraction module. The data extraction module acquires real-time spectral image data streams from three spectral image sensors and controls the display module to visualize the quality data. The automatic identification module is used to identify the planar position of the bayberry from the spectral image data from the three spectral image sensors and perform binocular stereo matching using three target detection models based on convolutional neural networks. The sugar-acidity extraction module is used to perform spectral analysis on the spectral image data from the three spectral image sensors using a bayberry sugar-acidity extraction model to calculate the sugar and acidity of the target bayberry.

[0024] The beneficial effects of this invention are as follows:

[0025] 1. Accuracy of Target Recognition and Quality Analysis: This invention uses spectral images as a benchmark, performing target recognition on three spectral images separately. Then, it utilizes the parallax of two characteristic spectral images to achieve accurate spatial positioning and diameter detection of outdoor bayberry fruits. Simultaneously, it uses the R channel of the RGB image to correct the spectral image and obtain relative reflectance, aiming to eliminate the influence of outdoor ambient light on the detection of sugar content and acidity of bayberry fruits. This allows the device provided by this invention to be portable and worn as eyeglasses, displaying real-time quality data of multiple bayberry targets to the user.

[0026] 2. Intuitive Display of Results: This invention projects the display results onto the user's eyes via a prism, enabling augmented reality display of quality data. Specifically, a small display module digitally overlays internal quality data, such as sugar content and acidity, invisible to the human eye, onto the actual fruit in the user's field of vision in real time. This "what you see is what you get" visualization transforms abstract data into intuitive perception, significantly improving user efficiency and decision-making clarity during harvesting and sorting. It solves the problems of data separation from the actual fruit and unintuitive results in traditional equipment.

[0027] 3. Portability of the Device: This invention integrates image sensing, data display, and other functional modules into a lightweight, eyeglass-shaped shell. A data cable connects the eyeglasses to a control and computing module that can be held in the hand or placed in a pocket. This portable design allows high-precision quality inspection capabilities to be easily taken to the field for on-site, anytime, anywhere testing, rather than being limited to a laboratory or fixed workstation.

[0028] 4. Low-cost solution: This invention cleverly employs a hardware solution of "general-purpose image sensor + customized filter" for bayberry quality detection, simulating the core functions of expensive hyperspectral instruments at an extremely low cost. This solution successfully reduces the core cost of the equipment from hundreds of thousands of yuan in traditional hyperspectral solutions to several thousand yuan, greatly lowering the application threshold of precision agriculture technology and making it affordable for small-scale fruit farmers and cooperatives. It has extremely high promotional value and economic applicability. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the AI ​​glasses device for quality detection of bayberries provided in this invention.

[0030] Figure 2 This is a system block diagram of the AI ​​glasses device for quality detection of bayberries provided by the present invention.

[0031] Reference numerals: 1. First spectral image sensor; 2. Second spectral image sensor; 3. Reference spectral image sensor; 4. Control module; 5. Mounting base; 6. Reflecting prism. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0033] Example 1

[0034] like Figure 1 and Figure 2As shown, a wearable dynamic detection and display device for bayberry quality includes an eyeglass frame, a control and computing module 4, a power supply module, and a spectral acquisition module and a display module fixed on the eyeglass frame.

[0035] The eyeglass frame is designed as a wearable, face-mounted structure, and is integrally molded from lightweight, high-strength materials to minimize wearing weight while ensuring structural stability. The outer surface and interior of the eyeglass frame are spatially designed to provide a compact and stable mounting base for the display module and spectral acquisition module. This eyeglass frame-based design allows this embodiment to be used as a personal wearable device, easily carried by operators to any testing site, extending testing capabilities from the laboratory to the field.

[0036] In some embodiments, the eyeglass frame is integrally molded from lightweight, high-strength ABS engineering plastic using high-precision 3D printing technology. The main structure of the eyeglass frame is an arc-shaped frame, the curvature of which is optimized to fit the head contours of most users, forming a head-mounted structure.

[0037] The display module includes a display screen and a reflective prism 6. The reflective prism 6 is positioned corresponding to the user's eye position. The display screen is located on the side of the eyeglass frame and aligned with the reflective prism 6. The reflective prism 6 is used to reflect the display results from the display screen to the user's eyes, so that the user can easily view the display results, making the invisible quality of bayberries (sugar content, acidity, and diameter) digitized and visualized.

[0038] The spectral acquisition module includes three spectral image sensors mounted on the top front of the eyeglass frame: a first spectral image sensor 1, a second spectral image sensor 2, and a reference spectral image sensor 3.

[0039] To achieve low-cost spectral sensing of specific chemical indicators (sugar content and acidity) of bayberry quality, the first spectral image sensor 1 and the second spectral image sensor 2 are composed of general-purpose image sensors and filters for corresponding wavelengths. Specifically, the lenses of the first spectral image sensor 1 and the second spectral image sensor 2 are respectively equipped with red and green filters to acquire narrow-band spectral information in two wavelengths strongly correlated with the acidity and sugar content of bayberries. The lens of the second spectral image sensor 2 is equipped with a green filter to acquire narrow-band spectral information strongly correlated with the acidity and sugar content of bayberries. The reference spectral image sensor 3 has no filter on its lens; its function is to acquire visible light color spectral images of bayberries to capture and eliminate the influence of ambient light, and to serve as the reference spectral image source for the stereo vision system.

[0040] In some further embodiments, to maximize the signal-to-noise ratio and accurately lock the characteristic band, the center wavelength of the red filter is 610 nm and the bandwidth is 30 nm; the center wavelength of the green filter is 570 nm and the bandwidth is 30 nm.

[0041] The eyeglass frame features a mounting beam positioned above the user's eyes on its front. Three L-shaped mounting brackets are integrated into the top of the mounting beam. Each L-shaped bracket has pre-drilled screw holes that match the housing of the spectral image sensors, ensuring that the optical axes of the three spectral image sensors remain strictly parallel after installation. The three spectral image sensors are respectively fixed to the three L-shaped mounting brackets, with their lenses all facing forward to synchronously capture the field of view. This integrated structural design ensures that the relative spatial relationships of all optical components remain constant after leaving the factory, providing crucial physical assurance for the calibration accuracy and subsequent computational stability of the binocular vision system.

[0042] The display module also includes a mounting base. An integral mounting base for fixing the display screen and the reflective prism 6 extends from the front edge of one side (left side in this embodiment). This mounting base has slots and holes for fixing the display screen and the reflective prism 6. The display screen and the reflective prism 6 are fixed in the corresponding slots and holes, maintaining a fixed relative position to provide a stable display for the user's eyes.

[0043] The control module 4 is integrated with the power module and is set independently of the eyeglass frame to reduce the burden on the user's face. The control module 4 is connected to an external control switch via a wire to allow the user to turn the device on or off. The power module includes a battery. The device, with the control module 4 and power module packaged together, can be held in the hand or placed in a pocket.

[0044] Each spectral image sensor is connected via an independent data line ( Figure 1 (Not shown in the image) The data cables are led out from the rear and converged before being connected to the control module 4. The display screen and the control module 4 are connected via a video transmission line to establish communication. To ensure neat wiring and signal stability, all data cables are bundled and managed.

[0045] The control module 4 is the automation control center and data processing core of this embodiment. Internally, it contains a dynamic detection model for bayberry quality, which executes the entire workflow from data acquisition to result output. This dynamic detection model includes a data extraction module, an automatic identification module, and a sugar-acidity extraction module.

[0046] The data extraction module receives and synchronously manages real-time spectral image data streams from three spectral image sensors, and transmits the final obtained bayberry quality data and RGB spectral image data to the display module for visualization. This enables the entire data "input-processing-output" chain to operate automatically without human intervention. The bayberry quality data includes the sugar content, acidity, and diameter of the bayberries.

[0047] The automatic identification module is used for target identification of bayberries. This module loads three pre-trained target detection models based on convolutional neural networks (CNNs), each analyzing spectral image data from video streams from three different spectral image sensors. Utilizing the powerful pattern recognition capabilities of AI, the module can quickly and accurately identify and box-select bayberry fruits, a crucial prerequisite for all subsequent precise analysis.

[0048] The sugar and acidity extraction module is used to calculate the spatial location and diameter of the target bayberry by using a binocular stereo matching algorithm based on the target location extracted by the automatic identification module, and to perform spectral analysis on the video stream spectral image data of three spectral image sensors by using the bayberry sugar and acidity extraction model to calculate the sugar and acidity of the target bayberry.

[0049] In this embodiment, the DN value of the tested bayberry fruit measured by the first spectral image sensor 1 is used to express the 610nm wavelength radiation energy reflected by the bayberry surface; the DN value of the tested bayberry fruit measured by the second spectral image sensor 2 is used to express the 570nm wavelength radiation energy reflected by the bayberry surface. The DN value of the tested bayberry fruit measured by the red channel of the reference spectral image sensor 3 is used as the radiation energy incident on the bayberry surface under 610nm and 570nm wavelength light sources.

[0050] The ratio of the dendritic density (DN) value of the tested bayberry fruit measured by the first spectral image sensor 1 to the DN value of the tested bayberry fruit measured by the red channel of the reference spectral image sensor 3 is used as the relative spectral reflectance for sugar content detection. The ratio of the DN value of the tested bayberry fruit measured by the second spectral image sensor 2 to the DN value of the tested bayberry fruit measured by the red channel of the reference spectral image sensor 3 is used as the relative spectral reflectance for acidity detection. Using relative spectral reflectance at wavelengths of 570 nm and 610 nm to calculate sugar content and acidity helps to eliminate interference from ambient light in complex outdoor optical environments and when eyeglass frames are far from the user.

[0051] During operation, the three spectral image sensors acquire spectral image data and transmit it to the control module 4 as a video stream. The control module 4 uses a dynamic quality detection model to identify the location and diameter of the bayberries and calculates the sugar content M of the tested bayberry fruit using the image data from the three spectral image sensors as input signals. BX and acidity M pH as follows:

[0052]

[0053]

[0054] The sugar content is expressed as a percentage, while the acidity value is a unitless value.

[0055] DN A The DN value of the tested bayberry fruit measured by the first spectral image sensor 1; DN red The DN value of the tested bayberry fruit measured by the R channel of the reference spectral image sensor 3; DN B The DN value of the tested bayberry fruit is measured by the second spectral image sensor 2; A1, B1, C1, A2, B2, and C2 are the detection coefficients obtained through pre-calibration.

[0056] In this embodiment, the reference spectral image sensor 3 measures the R channel DN value of the spectral image. red The primary wavelength corresponds to the central red band around 700 nm. In this band, the reflectivity of the bayberry surface is independent of its acidity and sugar content; therefore, the reflected light intensity in this band can be used as a representative of ambient light intensity. In contrast, the DN values ​​of the spectral images acquired by the second and third spectral image sensors mainly correspond to the 570 nm and 610 nm bands, respectively, and are related to the acidity and sugar content of the bayberry. However, the reflected light in these bands is affected by ambient light intensity. To eliminate this effect, the DN values ​​of the 570 nm and 610 nm bands are divided by the DN value of the 700 nm band to obtain the relative reflectivity. Using the relative reflectivity to calculate acidity and sugar content effectively eliminates interference from changes in ambient light.

[0057] In this embodiment, A1 is 0.01087; B1 is -0.3669712; C1 is 12.185397; A2 is 1.404; B2 is -0.3151; and C2 is 2.029.

[0058] In some embodiments, the target detection model in the automatic identification module is built based on the YOLOv8 model.

[0059] This embodiment utilizes three YOLOv8 deep learning models, one for analyzing RGB spectral images, the other for analyzing 610nm narrowband spectral images, and the other for analyzing 570nm narrowband spectral images, to be locally deployed in the control module. This enables real-time processing of data streams simultaneously acquired from the three spectral image sensors, achieving rapid and accurate identification and targeting of the same bayberry fruit even in field environments with varying lighting, cluttered backgrounds, and partial fruit occlusion. This design overcomes the shortcomings of traditional machine vision algorithms in real agricultural scenarios, such as poor adaptability and susceptibility to interference. It also solves the network latency and environmental limitations caused by the reliance on cloud computing in existing technologies. This provides a stable and reliable prerequisite for subsequent high-precision size calculation and quality analysis, significantly improving the automation level and environmental adaptability of the detection process.

[0060] In some embodiments, in the bayberry sugar-acidity extraction model, sugar content is correlated with the relative reflectance at 610 nm, and acidity (pH value) is correlated with the relative reflectance at 570 nm.

[0061] In some embodiments, the reflective mirrors in the reflective prism 6 are placed vertically and at a 45° angle to the front of the user's line of sight, so that the processed RGB bayberry spectrum image is reflected by the reflective prism 6 and the display content is reflected into the user's eyes, transforming the physicochemical indicators that cannot be directly observed by traditional senses into intuitive digital information displayed in the user's field of vision.

[0062] In some embodiments, the spectral image sensor is an industrial-grade CMOS spectral image sensor. To ensure real-time data processing, all three spectral image sensors are configured to acquire spectral images at a resolution of 2592×1944 pixels, thereby achieving high-speed spectral image capture at 30 frames per second (fps).

[0063] In some embodiments, the underlying data access of the spectral image sensor is implemented through the V4L2 (Video4Linux2) interface, and its hardware buffer size is set to 1. The purpose of this configuration is to discard old cached frames and ensure that each read operation obtains the latest spectral image captured by the camera at the current instant.

[0064] In some embodiments, the specific configuration and functions of the three spectral image sensors are as follows:

[0065] The lens of the first spectral image sensor 1 is fitted with a circular narrow-bandpass optical filter with a center wavelength of 610 nm and a full width at half maximum (FWHM) of 30 nm via a precision threaded interface. The sensor's function is to acquire monochromatic spectral image information of the bayberry in the central region of the 610 nm spectrum. This spectral image data will be input as the left view into the binocular stereo matching algorithm and used in subsequent spectral analysis to calculate the relative reflectance R.610 .

[0066] The second spectral image sensor 2 has a circular narrow-bandpass optical filter with a center wavelength of 570 nm and a full width at half maximum (FWHM) of 30 nm mounted on its lens front end via a precision threaded interface. This sensor's function is to acquire monochromatic spectral image information of the bayberry in the central region of the 570 nm spectrum. This spectral image data will be input as a right view into the binocular stereo matching algorithm and used in subsequent spectral analysis to calculate the relative reflectance R. 570 .

[0067] The reference spectral image sensor 3, acting as a reference spectral image sensor, has no additional filters installed in front of its lens. Its function is to acquire a full-color visible light (RGB) spectral image of the bayberry. This color spectral image provides two key data points for subsequent processing: firstly, it provides rich visual features such as color and texture for the target detection model; secondly, it acquires ambient light intensity information in its R channel, providing a real-time reference for the synchronous correction algorithm and reducing the impact of ambient light on the data.

[0068] The second spectral image sensor 2 and the first spectral image sensor are fixedly mounted on the eyeglass frame. The distance between their optical centers is constant at 33 mm, and their optical axes are kept strictly parallel, together forming a binocular stereo vision system.

[0069] In some embodiments, the hardware of the control module 4 is a customized circuit board equipped with an NVIDIA Jetson Orin Nano core module. In the control module 4, the data extraction module adopts a multi-threaded architecture, starting independent acquisition threads for each of the three spectral image sensors. Each thread is responsible for initializing its corresponding camera (configured to 2592×1944 resolution) and continuously reading the spectral image sensor at a high frame rate (30 frames / second). The latest frame is stored in a frame queue with a capacity of 1 to ensure that the main processing program can acquire the latest spectral image data at any time. The main program runs in a loop at a target rate of 15 frames / second, coordinating the entire data processing and control flow.

[0070] The automatic recognition module loads three independent, specially trained deep learning-based object detection models, which are then run on a GPU to accelerate processing. These three models are the baseline model, the red channel model, and the green channel model, respectively.

[0071] The baseline model specifically processes RGB images from the baseline spectral image sensor 31-3 to accurately identify bayberry fruits in the RGB images. The red channel model specifically processes red-filtered spectral images from the first spectral image sensor 11-1 to accurately identify bayberry fruits in the red-filtered spectral images. The green channel model specifically processes green-filtered spectral images from the second spectral image sensor 21-2 to accurately identify bayberry fruits in the green-filtered spectral images.

[0072] During target recognition, the recognition results of the baseline model are used as the benchmark for bayberry recognition. These results are matched with the recognition results of the red channel model and the green channel model, ensuring a one-to-one correspondence between the recognition results on the RGB spectral image, the red filter spectral image, and the green filter spectral image. For each identified bayberry target, the spatial coordinates and physical diameter of the bayberry fruit are calculated using a stereo matching algorithm on both the red filter spectral image and the green filter spectral image. Quality data is then detected using a dynamic bayberry quality detection model.

[0073] This embodiment utilizes augmented reality technology to create a "what you see is what you get" visualization method that integrates quality data with physical objects. In some embodiments, the display module is the human-computer interaction terminal of this embodiment, and the display screen used is a 0.96-inch TFT display screen, which is installed inside the front end of the temple of the eyeglass frame. The displayed content is optically coupled through a freeform prism, which reflects the spectral image emitted by the TFT display screen to form a clear virtual image in front of the user's eyes, thus not affecting the user's primary field of vision for observing the real world. After receiving data from the control module 4, the module will clearly display the quality information (diameter, sugar content, acidity) of each tracked bayberry in text form.

[0074] This embodiment transforms abstract physicochemical indicators invisible to the human eye into intuitive visual information, achieving spatial integration of test results and physical objects. This fundamentally solves the problems of data separation from physical objects and unintuitive results in traditional equipment. Users do not need to shift their gaze to look at the external screen, allowing them to seamlessly perform physical operations such as harvesting and sorting while obtaining quality information.

[0075] This embodiment introduces an innovative technology combining a spectral image recognition model with a binocular stereo matching algorithm for high-precision positioning of the target bayberry fruit in three-dimensional space. The spectral image recognition model can accurately identify the target bayberry in the field of view and define its two-dimensional spectral image region. Subsequently, the first spectral image sensor 1 and the second spectral image sensor 2 capture the left and right spectral images, and the stereo matching algorithm is used to calculate the spatial coordinates of the target bayberry. This technological innovation fundamentally solves the problems of inaccurate target spatial positioning and inability to accurately overlay AR information in traditional detection equipment in two-dimensional image data. It ensures that the detection results such as sugar content and acidity can be accurately projected and locked onto the real fruit in the user's field of view, greatly improving the user experience and operational efficiency.

[0076] This embodiment combines artificial intelligence (AI) technology and algorithms for target recognition and localization in complex environments. By deploying deep learning models and binocular stereo matching algorithms, and introducing algorithms to eliminate the influence of ambient light differences, the device can quickly and accurately locate bayberries in field environments with changing lighting, cluttered backgrounds, and partial fruit occlusion, just like a human. This intelligent recognition capability effectively overcomes the shortcomings of traditional machine vision algorithms in real agricultural scenarios, such as poor adaptability and susceptibility to interference, as well as the poor spatial localization effect of artificial intelligence (AI) technology, significantly improving the stability and automation level of detection.

[0077] This embodiment achieves precise capture of specific spectral information strongly correlated with sugar and acidity indicators through a combination of narrowband filters and spectral analysis algorithms. By performing targeted analysis on characteristic wavelengths such as 610nm and 570nm, and combining this with synchronous correction technology to eliminate the influence of ambient light, the device can achieve high-precision non-destructive testing of core quality indicators such as sugar and acidity in bayberries, with a testing accuracy comparable to expensive laboratory equipment.

[0078] This embodiment uses R (red) band image data from the RGB spectral image as the calibration benchmark for ambient light differences. Since the wearable bayberry quality dynamic detection display device needs to operate in uncontrolled natural light environments, fluctuations in ambient light intensity and spectral distribution can significantly interfere with the accuracy of the spectral detection results. By using the reference spectral image sensor 3 to acquire the R-band DN value of the target bayberry in real time and using it as a reference for the total radiant energy incident on the bayberry surface, the ratio of the reflected energy of the measured spectral band to the DN value of the R band is normalized to calculate the relative reflectivity. This R-band-based normalization mechanism effectively eliminates systematic errors caused by changes in ambient light, significantly improves the robustness and accuracy of the detection model, and ensures that this embodiment can provide stable and reliable quality detection data both outdoors and under different lighting conditions.

[0079] Example 2

[0080] A method for dynamic detection of bayberry quality, using a wearable bayberry quality dynamic detection display device provided in Example 1. The method includes the following steps:

[0081] Step 1: Perform distortion correction and line alignment on the first spectral image sensor 1 (right view) and the second spectral image sensor 2 (left view) using pre-calibrated camera parameters. The user wears the glasses frame on their face, facing the area where the bayberries are located.

[0082] Step 2: Device Initialization. The wearer turns on the control switch, and the control module 4 completes self-test and program loading. The first spectral image sensor 1, the second spectral image sensor 2, and the reference spectral image sensor 3 respectively acquire the first characteristic spectral image, the second characteristic spectral image, and the RGB spectral image, and transmit them to the control module 4 in the form of video streams.

[0083] Step 3: Input the first feature spectral image, the second feature spectral image, and the RGB spectral image into the baseline model, red channel model, and green channel model used for bayberry target recognition, respectively, to obtain the identified bayberry target locations on the three spectral images. Perform position association matching on the same bayberry target in the three spectral images to obtain the positional differences of each bayberry on the three spectral images.

[0084] Step 4: Process the first and second feature spectral images using the Semi-Global Block Matching (SGBM) algorithm to obtain a disparity map. The disparity map is then smoothed using a Weighted Least Squares (WLS) filter to optimize accuracy. For each identified bayberry target, the spatial coordinates and physical diameter are calculated using the disparity map, based on disparity, focal length, and baseline, and through triangulation principles.

[0085] Step 5: For each bayberry target in the first feature spectral image, the second feature spectral image, and the RGB spectral image, perform spectral analysis to obtain the sugar content and acidity of each bayberry.

[0086] 5-1. Feature extraction.

[0087] For each bayberry target, a circular region with a diameter of 15 pixels is extracted from the center of the target in both the first and second feature spectral images as the target recognition region. The grayscale value of the target recognition region is then extracted as the feature reflectance DN from both the first and second feature spectral images. A DN BFor RGB spectral images, the red channel pixel values ​​of the target recognition region are extracted as the reference brightness DN of the incident relative sugar acidity spectral characteristics. red .

[0088] 5-2. Data Conversion.

[0089] The gray values ​​corresponding to the first and second feature spectral images obtained in step 5-1 are converted into physically meaningful relative reflectance values ​​R using a synchronous correction model. A R B The specific process is as follows: the first feature is the relative reflectance. Second characteristic: relative reflectivity .

[0090] 5-3. Quality Calculation.

[0091] Substituting the converted relative reflectance into the bayberry sugar-acidity extraction model, the sugar content M of the bayberry fruit is obtained. BX and acidity M pH as follows:

[0092]

[0093]

[0094] Step Six: Determine the diameter and sugar content (M) of each bayberry target. BX and acidity M pH The corresponding recognition boxes are marked on the sides of the RGB spectral image to obtain the bayberry quality identification map. The bayberry quality identification map is then displayed to the user's eyes using a display module.

[0095] To verify the accuracy of the bayberry quality detection in this embodiment, the following detection accuracy verification experiment was conducted:

[0096] The method provided in this embodiment was used to detect the sugar-acidity and diameter of 24 bayberries with different ripeness. After removing outliers and missing values, the detection results of the front and back sides of each bayberry sample were averaged to reduce detection error. The averaged measurement data was used as the predicted data of the sample. The actual values ​​of diameter, pH value and reducing sugar content of bayberry samples were collected by vernier calipers, pH meter and saccharimeter as the measured diameter and sugar-acidity values ​​of the sample. By comparing the predicted and actual sugar-acidity values ​​of the sample, the standard deviation and accuracy of the data were calculated. The accuracy of the instrument was analyzed as shown in equations (1) and (2), and the results are shown in Table 1.

[0097] Absolute deviation = |Predicted value - Actual value| Equation (1)

[0098] Equation (2)

[0099] Table 1. Verification of the accuracy of bayberry sugar acidity

[0100]

[0101] As shown in Table 1, the AI ​​glasses for detecting bayberry quality in this invention have detection accuracies of 85.2583%, 96.3583%, and 93.1105% for pH value, sugar content, and acidity, respectively, which are very high.

[0102] Experiments have verified that the sugar content prediction model can be constructed based on a combination of the relative reflectance of the 610nm band and the relative reflectance of the 570nm band; the acidity (pH value) prediction model can be constructed based on the relative reflectance of the 570nm band.

Claims

1. A wearable method for dynamic detection and display of bayberry quality, characterized in that: The display device used includes an eyeglass frame, and a spectral acquisition module and a display module fixed on the eyeglass frame; the spectral acquisition module includes a first spectral image sensor (1), a second spectral image sensor (2), and a reference spectral image sensor (3) arranged in parallel; filters corresponding to the sugar content and acidity of bayberries are respectively installed on the lenses of the first spectral image sensor (1) and the second spectral image sensor (2); the center wavelength of the filter on the first spectral image sensor (1) is 610nm and the bandwidth is 30nm; the center wavelength of the filter on the second spectral image sensor (2) is 570nm and the bandwidth is 30nm; The wearable dynamic detection and display method for bayberry quality includes: The first feature spectral image, the second feature spectral image, and the RGB spectral image acquired by three spectral image sensors are input into the target recognition model to identify the location of the bayberry in the image; Binocular stereo matching was performed on the first feature spectral image and the second feature spectral image to extract the spatial position and diameter of each bayberry; The grayscale values ​​of each bayberry location are extracted from the first feature spectral image and the second feature spectral image, respectively, as two feature reflectance brightnesses; the R channel pixel values ​​of each bayberry location in the RGB spectral image are used as the reference brightness. The ratio of the reflectance of the two features to the reference reflectance is used as the corrected relative reflectance of the two features; The relative reflectance of the two features is input into the bayberry sugar-acidity extraction model to obtain the sugar content and acidity of the bayberry; the sugar content of the bayberry M BX and acidity M pH The method to obtain it is as follows: in, DN A The DN value of the bayberry measured by the first spectral image sensor (1); DN red The DN value of bayberry was measured by the R channel of the reference spectral image sensor (3); DN B The DN value of the bayberry is measured by the second spectral image sensor (2); A1, B1, C1, A2, B2, and C2 are the detection coefficients obtained by pre-calibration. The display module shows the quality data for each bayberry; the quality data includes diameter, sugar content, and acidity.

2. The wearable dynamic detection and display method for bayberry quality according to claim 1, characterized in that: The first feature spectral image, the second feature spectral image, and the RGB spectral image are respectively input into three independent target recognition models. The recognition results corresponding to the RGB spectral image are used as a benchmark to perform correlation matching of the recognition results of the three spectral images.

3. The wearable dynamic detection and display method for bayberry quality according to claim 1, characterized in that: The display module projects the quality data displayed on the screen onto the user's eyes through a reflective prism; the projection position of the quality data is the side of the user's eye corresponding to the location of the bayberry.

4. The wearable dynamic detection and display method for bayberry quality according to claim 1, characterized in that: The characteristic reflectance brightness is obtained by calculating the average gray value of the area occupied by the bayberry in the first characteristic spectral image or the second characteristic spectral image; the reference brightness is obtained by calculating the average pixel value of the R channel of the area occupied by the bayberry in the reference characteristic spectral image; the area occupied by the bayberry is a circular area with the bayberry recognition center as the center.

5. A wearable dynamic quality detection and display device for bayberries, characterized in that: The wearable bayberry quality dynamic detection and display device is used to perform the dynamic detection and display method as described in claim 1. The device includes an eyeglass frame, a control module (4), and a spectral acquisition module and a display module fixed on the eyeglass frame. The spectral acquisition module includes a first spectral image sensor (1), a second spectral image sensor (2), and a reference spectral image sensor (3) fixed on the top front of the eyeglass frame. The lenses of the first spectral image sensor (1) and the second spectral image sensor (2) are respectively equipped with filters corresponding to the sugar content and acidity of bayberries. The display module includes a display screen and a reflecting prism (6) aligned with each other. The position of the reflecting prism (6) corresponds to the position of the user's eyes. The control module (4) includes a data extraction module, an automatic identification module, and a sugar and acidity extraction module. The data extraction module collects real-time spectral image data streams from three spectral image sensors and controls the display module to visualize the quality data. The automatic identification module is used to identify the planar position of the bayberry from the spectral image data of the three spectral image sensors and perform binocular stereo matching using three target detection models based on convolutional neural networks. The sugar and acidity extraction module is used to perform spectral analysis on the spectral image data of the three spectral image sensors using the bayberry sugar and acidity extraction model to calculate the sugar and acidity of the target bayberry.

6. The wearable dynamic detection and display device for bayberry quality according to claim 5, characterized in that: The optical axes of the three spectral image sensors are parallel to each other; the reference spectral image sensor (3) is located between the first spectral image sensor (1) and the second spectral image sensor (2).

7. The wearable dynamic detection and display device for bayberry quality according to claim 5, characterized in that: The front of the eyeglass frame is provided with a mounting beam corresponding to the position above the user's eyes; the top of the mounting beam is integrally provided with three L-shaped mounting brackets; the three L-shaped mounting brackets are respectively fixed to the spectral image sensor through reserved connection holes.

8. The wearable dynamic detection and display device for bayberry quality according to claim 5, characterized in that: A mounting base is fixed to one side of the eyeglass frame; the display screen and the reflective prism (6) are fixed to the mounting base.

9. A wearable dynamic detection and display device for bayberry quality according to claim 5, characterized in that: The control module (4) is set independently of the eyeglass frame and communicates with the spectrum acquisition module and the display module via wired or wireless communication.

Images