Fishbone detection method and device based on acousto-optic dual-mode fusion

Abstract

The invention relates to a nondestructive testing technology, and discloses a fishbone detection method and device based on acousto-optic dual-mode fusion, and the method comprises the steps: collecting a dual-mode sensing signal of a fishbone to be detected, carrying out the preprocessing and feature extraction of an ultrasonic echo signal, and carrying out the feature extraction of the ultrasonic echo signal; the method comprises the following steps: generating an acoustic candidate queue containing at least one candidate target of an ultrasonic echo signal and a first confidence coefficient, performing preprocessing and feature extraction on a fluorescence image signal, generating an optical candidate queue containing at least one candidate target of the fluorescence image signal and a second confidence coefficient, and generating an optical candidate queue based on the acoustic candidate queue and the optical candidate queue. And carrying out data fusion and decision making through weighted Bayesian, and outputting a fishbone detection result. Through innovative bimodal sensor fusion and embedded intelligent analysis, fishbone detection can be rapidly, accurately, easily and safely carried out.

Description

Technical Field

[0001] This invention relates to nondestructive testing technology, and more particularly to a method and apparatus for detecting fish bones based on acoustic-optical dual-modal fusion. Background Technology

[0002] Accidental ingestion of fish bones is a common food safety risk, posing a significant threat, especially to the elderly. Due to declining vision, reduced oral tactile sensitivity, and weakened swallowing reflexes, older adults often struggle to effectively identify and remove fish bones through visual inspection or chewing, leading to a significantly increased risk of accidental ingestion and subsequent esophageal injury, perforation, or even more serious medical consequences. Therefore, developing a technology that can quickly, accurately, and safely detect tiny fish bones in fish fillets is of significant social importance and practical value in ensuring the food safety of the elderly and other high-risk groups.

[0003] Currently, the detection of foreign objects in food mainly relies on the following technical approaches, but they all have insurmountable limitations when applied to home-based and portable fishbone detection scenarios: Manual visual inspection: This method relies entirely on the operator's experience and vigilance. It is not only inefficient and time-consuming, but also prone to missing semi-transparent fish bones that are embedded deep in the fish flesh or are very small (such as fish bones with a diameter of less than 0.5 mm), making it unreliable.

[0004] X-ray or industrial computed tomography (CT) technology: Although such technologies can penetrate the interior of objects to form images, the equipment is usually large and expensive. More importantly, it involves ionizing radiation and must be operated by professionally trained personnel under special protective conditions. It is completely unsuitable for widespread use in scenarios such as home kitchens, nursing home canteens, or ordinary catering establishments.

[0005] For example, existing technology CN201910694235.8. Non-destructive testing technology based on a single physical mode: Pure ultrasonic testing technology: This technology utilizes the principle that ultrasonic waves propagate in a medium and are reflected at interfaces with differences in acoustic impedance to form images. However, the axial resolution of conventional medical or industrial ultrasonic frequencies (typically 5-20 MHz) is limited, lacking sufficient resolution for extremely fine calcified fish bones (e.g., target diameter ≤ 0.3 mm), making it difficult to form clear and discernible image features, resulting in insufficient detection sensitivity.

[0006] Pure optical or ultraviolet fluorescence detection techniques: These techniques utilize the characteristic fluorescence emitted by specific fluorophores in fish bones under ultraviolet light excitation for identification. While this method is sensitive to surface contaminants or near-surface calcification structures, its light penetration is extremely weak, making it ineffective in detecting fish bones completely covered by fish tissue or buried at depths of several millimeters, resulting in a serious limitation in detection depth. Summary of the Invention

[0007] This invention addresses the problems of unreliable and time-consuming fishbone detection methods in existing technologies, which easily miss fishbones hidden deep in the fish meat or semi-transparent fishbones; high cost, large size, and radiation-related issues, making them unsuitable for home or catering settings; conventional frequencies (5–20 MHz) often lack the ability to distinguish extremely fine fishbones (<0.5 mm); and the inability to detect fishbones deeply embedded in tissues. It provides a fishbone detection method and device based on acousto-optic dual-modal fusion.

[0008] To solve the above-mentioned technical problems, the present invention provides the following technical solution: A fishbone detection method based on acousto-optic dual-modal fusion includes a high-frequency ultrasonic transducer, an ultraviolet excitation source, and an optical sensor, comprising: The system acquires dual-modal sensing signals from the fishbone under test. These signals include an ultrasonic echo signal obtained by a high-frequency ultrasonic transducer and a fluorescence image signal excited by an ultraviolet light source and captured by an optical sensor. The ultrasonic echo signal is preprocessed and its features are extracted to generate an acoustic candidate queue containing at least one ultrasonic echo signal and a first confidence level. Feature extraction of the ultrasonic echo signal includes peak detection based on dynamic thresholds and acoustic shadow feature verification. The fluorescence image signal is also preprocessed and its features are extracted to generate an optical candidate queue containing at least one fluorescence image signal and a second confidence level. Feature extraction of the fluorescence image signal includes calcification structure identification based on spectral features. Based on the acoustic and optical candidate queues, data fusion and decision-making are performed using weighted Bayesian methods, and the detection result of the fishbone is output.

[0009] As a preferred embodiment, data fusion based on acoustic and optical candidate queues using weighted Bayesian methods includes: The prior probability is set, specifically the probability P that the ultrasound detects a fishbone. US The probability P of detecting a fishbone with ultraviolet light UV ; The dynamic weights are allocated based on a set prior probability; the dynamic weights include the dynamic weight w for ultrasonic detection of fish bones. US The dynamic weight w of the fishbone was detected by ultraviolet light. UV ; The final output fused data P final The fused data P is obtained by combining the dynamically assigned weights and prior probabilities. final .

[0010] As a preferred option, the final output fused data P final The acquisition also includes the fused data P output with the assistance of a rules engine. final .

[0011] Preferably, the preprocessing and feature extraction of the ultrasonic echo signal includes: The constant false alarm rate (CFAR) algorithm is adopted to dynamically set the detection threshold by calculating the statistical characteristics of the local background noise of the signal, and the signal peaks exceeding the threshold are marked as preliminary candidate targets. For each preliminary candidate target, check whether its signal energy in a specific distance range in the direction of echo propagation shows significant attenuation; if so, confirm the preliminary candidate target as an acoustic candidate target and enter the acoustic candidate queue.

[0012] As a preferred option, the constant false alarm rate (CFAR) algorithm is used for the ultrasonic echo signal, and the linear structure of the fishbone image is also identified through Hough transform or skeletonization analysis.

[0013] Preferably, identifying the linear structure of a fishbone image using Hough transform includes: Edge detection is performed on the fishbone image of the ultrasonic signal. Create a Hough space in the parameter space to record the number of votes for each parameter combination; Iterate through every non-zero pixel in the edge image and transform it into all possible curves or surfaces in the parameter space. For each point on a curve, increment the count in the corresponding cell of the accumulator array. Find the location of the local maximum in the accumulator array to obtain the linear structure of the fishbone image in the image.

[0014] To address the aforementioned technical problems, the present invention also provides a fishbone detection device based on acoustic-optical dual-modal fusion, comprising a high-frequency ultrasonic transducer, an ultraviolet excitation light source, and an optical sensor, which is used to implement the aforementioned fishbone detection method based on acoustic-optical dual-modal fusion.

[0015] Preferably, the high-frequency ultrasonic transducer is a high-frequency ultrasonic transducer with a frequency greater than 20 MHz.

[0016] Preferably, the wavelength of the ultraviolet excitation source is between 320 nm and 380 nm.

[0017] This invention, by adopting the above technical solutions, has significant technical effects: This invention effectively overcomes the inherent limitations of single detection technologies by integrating two sensing modalities: high-frequency ultrasound and ultraviolet fluorescence. High-frequency ultrasound provides excellent axial resolution of approximately 40 μm, enabling reliable identification of extremely fine fish bones with a diameter ≥0.2 mm, and possesses a tissue penetration depth of 0-15 mm, allowing detection of hidden dangers deeply embedded within the fish flesh. Simultaneously, the ultraviolet fluorescence modality utilizes the specific fluorescence spectrum of fish bones, with an ultraviolet fluorescence excitation spectrum peak of 365 nm and fish bone emission fluorescence of approximately 515 nm, providing high-contrast recognition of surface and near-surface calcification structures. By fusing the information from both modalities through an algorithm, the system achieves comprehensive and highly sensitive screening of fish bones, with detection capabilities significantly superior to traditional manual visual inspection or single-modal devices.

[0018] This invention, through innovative dual-modal sensor fusion and embedded intelligent analysis, successfully realizes a fast, accurate, easy-to-use, and safe handheld fishbone detection solution, effectively solving the problems of insufficient reliability, complex operation, or environmental limitations in the detection of small-sized, deep-position fishbones in existing technologies. Attached Figure Description

[0019] Figure 1 This is a flowchart of the present invention.

[0020] Figure 2 A schematic diagram of the device of the present invention.

[0021] Figure 3 This is an ultraviolet fluorescence image of a fishbone from the present invention.

[0022] Figure 4 This is the ultrasound image of the present invention.

[0023] Figure 5 This is an image showing the results of fishbone edge detection according to the present invention. Detailed Implementation

[0024] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0025] Example 1 A fishbone detection method based on acousto-optic dual-modal fusion includes a high-frequency ultrasonic transducer, an ultraviolet excitation source, and an optical sensor, comprising: The dual-modal sensing signals of the fishbone to be tested are collected. The dual-modal sensing signals include ultrasonic echo signals acquired by a high-frequency ultrasonic transducer and fluorescence image signals excited by an ultraviolet light source and captured by an optical sensor. The ultrasonic echo signal is preprocessed and its features are extracted to generate an acoustic candidate queue containing at least one ultrasonic echo signal and a first confidence level. The feature extraction of the ultrasonic echo signal includes peak detection and acoustic shadow feature verification based on dynamic threshold. The fluorescence image signal is preprocessed and its features are extracted to generate an optical candidate queue containing at least one candidate target of the fluorescence image signal and a second confidence level, wherein the feature extraction of the fluorescence image signal includes calcification structure identification based on spectral features; Based on acoustic and optical candidate queues, data fusion and decision-making are performed using weighted Bayesian methods, and the detection results of fish bones are output.

[0026] Example 2 Based on Example 1, this embodiment is a fishbone detection method based on acousto-optic dual-modal fusion, the method comprising: The system starts up and performs a self-test. The user presses and holds the power button to turn on the device. The device executes a self-test program to check battery power, ultrasonic pulse emission, UV LED illumination, and sensor response. If everything is normal, the LED indicator will remain solid green, indicating that the system is ready.

[0027] For data acquisition, the user places the probe coupling pad firmly against the surface of a fish fillet no more than 15 mm thick and briefly presses the button. The device simultaneously triggers two modes: Ultrasonic scanning: emitting a single ultrasonic pulse and acquiring the echo signal sequence (A-line data) via an ADC. Ultraviolet fluorescence imaging: pulsatingly illuminating a UV LED (5% duty cycle) while simultaneously capturing a fluorescence image frame by a CMOS sensor; the entire acquisition process is completed within 100 ms.

[0028] Ultrasonic signal preprocessing and feature extraction. Preprocessing: Bandpass filtering (center frequency 35MHz) was applied to the echo signal, TGC curve was applied (with an attenuation rate of 1.5 dB / cm / MHz to compensate for depth signal attenuation), and envelope detection (Hilbert transform) was performed.

[0029] Feature extraction and CFAR detection: Within a depth range of 0.5 mm to 15 mm, the mean and standard deviation of local noise are calculated using a sliding window (1 mm wide). Points with signal amplitudes exceeding (mean + 4 times the standard deviation) are marked as preliminary candidate points. Acoustic shadow verification: For each preliminary candidate point, it is checked whether the average signal amplitude within the subsequent 0.5-1.5 mm depth range drops below 30% of the average amplitude of the region before the candidate point. If so, the candidate point is determined to correspond to a hard foreign object with acoustic shadow characteristics, confirmed as an acoustic candidate target, and its confidence level is calculated. Based on signal-to-noise ratio and acoustic-to-shadow contrast, the data is mapped to the 0-1 range. Geometric screening: Connected-domain analysis is performed on acoustic candidate targets. If multiple candidate points exhibit a continuous linear distribution in depth and lateral position, and the fitted length is greater than 3 mm, they are merged into one target, and their confidence level is increased. UV fluorescence signal preprocessing and feature extraction: The dark current reference frame is subtracted from the captured image, and then a 515 nm bandpass filter is applied to obtain the fluorescence intensity map. Spectral ratio calculation: The fluorescence intensity ratio (FIR) of the 515 nm channel and the 580 nm channel (obtained through spectral analysis or calculation) is calculated in a local region (e.g., a 3x3 pixel window). Calcification structure identification: If the FIR of a region is >2.0, and its 515 nm fluorescence intensity exceeds the global background mean of the image by more than 3 standard deviations, the region is marked as an optical candidate target. Morphological screening: Binarization and connected-domain analysis are performed on optical candidate targets. Noise points with an area less than 0.05 mm² are removed, and elongated regions (aspect ratio greater than 4:1) are retained. Calculate the confidence level for each optical candidate target. (Based on FIR value and morphological regularity).

[0030] Dual-modal data fusion and decision-making. For each spatial location (or associated acoustic-optical target pair), dynamic weights are assigned based on the target depth d: like ,but ; like ,but ; like d> 8 mm ,but ; The probability of a fishbone at that location after fusion is calculated using the weighted Bayesian formula. : ; like Furthermore, if the sound and shadow characteristics are extremely obvious, then a forced setting will be implemented. If only If the value is high (>0.6) but there is no corresponding ultrasound support, and the target area is very small, then... Lowered by 0.2.

[0031] The processor determines the final probability. Output control signal: If all positions If the location is deemed safe, the green light will illuminate. If any location exists... If the location is deemed suspicious, a yellow light will illuminate, and a gentle vibration at a frequency of 200Hz and a short beep will be triggered. If any location is present... If the system is deemed dangerous, a red light will illuminate, and a strong, continuous vibration at a frequency of 800Hz and a persistent high-frequency beeping sound will be triggered.

[0032] If two consecutive short-interval scans of the same area yield "suspicious" (yellow light) results but do not trigger the danger red light, the system will automatically ignore the static interference characteristics of the area during the third scan and directly output a safe green light to avoid continuous false alarms caused by fixed impurities.

[0033] Example 3 Based on the above embodiments, this embodiment presents a fishbone detection device based on acousto-optic dual-modal fusion. A focusing transducer made of single-crystal PMN-PT piezoelectric material is used, with a center frequency of 35 MHz, a -6 dB bandwidth greater than 70%, a fixed focal length of 7.5 mm, and a theoretical axial resolution of approximately 40 μm. The transducer is used to emit ultrasonic pulses to the fish fillet under test and receive the echoes. The ultraviolet optics include a narrowband ultraviolet LED array with a peak wavelength of 365 nm (emission band 320-380 nm) and a dual-channel CMOS image sensor. A bandpass filter (center wavelength 515 nm, bandwidth ±10 nm) is installed in front of the sensor for selectively capturing fishbone fluorescence. This device advances the reliable detection limit to a diameter of 0.15 mm while maintaining a penetration depth of 15 mm and real-time performance, truly meeting the safety needs of high-risk groups.

Claims

1. A fishbone detection method based on acousto-optic dual-mode fusion, comprising a high-frequency ultrasonic transducer, an ultraviolet excitation light source and an optical sensor, characterized in that, include: The system acquires dual-modal sensing signals from the fishbone under test. These signals include an ultrasonic echo signal obtained from a high-frequency ultrasonic transducer and a fluorescence image signal excited by an ultraviolet light source and captured by an optical sensor. The ultrasonic echo signal is preprocessed and its features are extracted to generate an acoustic candidate queue containing at least one ultrasonic echo signal and a first confidence level. Feature extraction of the ultrasonic echo signal includes peak detection based on dynamic thresholds and acoustic shadow feature verification. The fluorescence image signal is also preprocessed and its features are extracted to generate an optical candidate queue containing at least one fluorescence image signal and a second confidence level. Feature extraction of the fluorescence image signal includes calcification structure identification based on spectral features. Based on the acoustic and optical candidate queues, data fusion and decision-making are performed using weighted Bayesian methods, and the detection result of the fishbone is output.

2. The fishbone detection method based on acousto-optic dual-modal fusion according to claim 1, characterized in that: The implementation of data fusion based on acoustic and optical candidate queues using weighted Bayesian methods includes: Setting of the prior probability, setting the probability P that the ultrasound detects a fishbone US and the probability P that the ultraviolet light detects a fishbone UV ; The dynamic weight is allocated by setting a prior probability, and the dynamic weight includes a dynamic weight w of ultrasonic detection of fish bones US , a dynamic weight w of ultraviolet detection of fish bones UV ​ The final output fusion data P final The final output fusion data P is obtained by combining the dynamic weight assigned and the prior probability final .

3. The fishbone detection method based on acousto-optic dual-modal fusion according to claim 2, characterized in that: The final output fusion data P final The acquisition further includes the fusion data P final output by the rule engine.

4. The fishbone detection method based on acousto-optic dual-modal fusion according to claim 1, characterized in that: Preprocessing and feature extraction of ultrasonic echo signals include: The constant false alarm rate (CFAR) algorithm is adopted to dynamically set the detection threshold by calculating the statistical characteristics of the local background noise of the signal, and the signal peaks exceeding the threshold are marked as preliminary candidate targets. For each preliminary candidate target, check whether its signal energy in a specific distance range in the direction of echo propagation shows significant attenuation; if so, confirm the preliminary candidate target as an acoustic candidate target and enter the acoustic candidate queue.

5. The fishbone detection method based on acousto-optic dual-modal fusion according to claim 1, characterized in that: The constant false alarm rate (CFAR) algorithm is used for ultrasonic echo signals, and the linear structure of fishbone images is also identified through Hough transform or skeletonization analysis.

6. The fishbone detection method based on acousto-optic dual-modal fusion according to claim 5, characterized in that: Identifying the linear structure of fishbone images using Hough transform includes: Edge detection is performed on the fishbone image of the ultrasonic signal. Create a Hough space in the parameter space to record the number of votes for each parameter combination; Iterate through every non-zero pixel in the edge image and transform it into all possible curves or surfaces in the parameter space. For each point on a curve, increment the count in the corresponding cell of the accumulator array. Find the location of the local maximum in the accumulator array to obtain the linear structure of the fishbone image in the image.

7. A fishbone detection device based on acousto-optic dual-modal fusion, comprising a high-frequency ultrasonic transducer and an ultraviolet excitation source, characterized in that, Used to implement the fishbone detection method based on acousto-optic dual-modal fusion as described in any one of claims 1-6.

8. A fishbone detection device based on acousto-optic dual-modal fusion according to claim 7, characterized in that: The high-frequency ultrasonic transducer is a high-frequency ultrasonic transducer with a frequency greater than 20 MHz.

9. A fishbone detection device based on acousto-optic dual-modal fusion according to claim 7, characterized in that: The wavelength of the ultraviolet excitation source is 320nm to 380nm.

Images