Liquid medicine foreign matter detection method and system based on machine vision

Abstract

The invention discloses a liquid medicine foreign matter detection method and system based on machine vision, and the method comprises the steps: obtaining multiband spectral data of a liquid medicine sample, and carrying out the normalization processing to generate a liquid medicine background standard spectral feature template; comparing the refractive index spectral characteristics of the template and the foreign matter-containing liquid medicine, and recording the surface roughness scattering mode and polarized light reflection angle change data of a foreign matter area; calculating a spectrum peak displacement quantized value according to the polarized light data, and extracting foreign matter edge spectrum gradient information in combination with a threshold value; calculating a transparency spectral attenuation coefficient based on the edge gradient information, and obtaining spectral contrast enhancement factors of different foreign matters in combination with the spectral response characteristic data; and if the enhancement factor does not reach the standard, adjusting the spectrum correction parameter and recalculating, fusing the scattering mode and the multi-angle incident spectrum response result to carry out consistency verification, and obtaining a final identification result containing the foreign matter material type and the danger level. The method can improve the drug detection accuracy and guarantee the drug quality safety.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection technology, and in particular to a method and system for detecting foreign objects in liquid medicine based on machine vision. Background Technology

[0002] Currently, foreign object detection in pharmaceutical solutions based on industrial vision is a core aspect of pharmaceutical production quality control, directly related to drug safety and clinical medication risks. Its detection accuracy directly reflects the quality control level of the production process, affecting the reliability and compliance of the pharmaceutical production system. Timely identification and removal of foreign object contamination are crucial to ensuring the quality of pharmaceutical solutions.

[0003] In one existing technology, an image of the liquid medicine is acquired by an optical imaging device, the shape and size features of the foreign object are extracted, the feature parameters are compared with a preset threshold, and when the feature parameters exceed the threshold, it is determined to be a foreign object and a rejection mechanism is triggered.

[0004] However, because existing technologies rely solely on visual characteristics for judgment, they cannot distinguish the material differences between transparent or translucent foreign objects and the background of the liquid medicine. This can easily lead to misjudgments of substances with similar shapes but different properties, such as glass fragments and bubbles, resulting in a mismatch between the detection results and the actual properties of the foreign object. This can easily cause quality risks such as missed or false detections. Therefore, existing technologies result in low accuracy in detecting foreign objects in liquid medicine. Summary of the Invention

[0005] A machine vision-based method and system for detecting foreign objects in liquid medicine is proposed to address the problems of existing technologies being unable to distinguish the material differences between foreign objects and the background of liquid medicine, and having low detection accuracy.

[0006] In a first aspect, to solve the above-mentioned technical problems, the present invention provides a machine vision-based method for detecting foreign objects in liquid medicine, comprising: Multi-band spectral data of the drug solution sample were acquired and normalized to obtain the standard spectral feature template of the drug solution background. The refractive index spectral characteristics of the drug solution background standard spectral feature template are compared with those of the drug solution sample containing foreign matter, and the surface roughness scattering mode and polarized light reflection angle change data of the foreign matter region are recorded. Based on the polarized light reflection angle change data, the spectral peak displacement quantization value between the foreign object and the drug liquid background is calculated, and the spectral gradient information of the foreign object edge is extracted by combining the preset displacement threshold. The transparency spectral attenuation coefficient is calculated based on the spectral gradient information of the foreign object edge, and the spectral contrast enhancement factor of different foreign objects is calculated by combining the spectral response characteristic data of different foreign objects obtained in advance. If the spectral contrast enhancement factor is lower than the preset recognition threshold, the spectral correction parameters are dynamically adjusted, and the spectral peak shift quantization value is recalculated to obtain the multi-angle incident spectral response result. The surface roughness scattering mode and the multi-angle incident spectrum response results are fused and consistency verification is performed to obtain the final identification result.

[0007] Preferably, the step of acquiring multi-band spectral data of the drug solution sample and performing normalization processing to obtain a standard spectral feature template for the drug solution background includes: Obtain the raw spectral data of the drug sample in the preset ultraviolet to visible light band; The original spectral data was corrected by spectral baseline shift to obtain multi-band spectral data of the drug sample. Based on the solution concentration spectral response characteristics in the multi-band spectral data, the near-infrared reflectance intensity values ​​of different batches of drug solutions are normalized using a pre-established temperature-compensated spectral coefficient to obtain the background standard spectral feature template of the drug solution.

[0008] Preferably, the step of correcting the original spectral data by spectral baseline shift to obtain multi-band spectral data of the drug sample includes: If the ambient temperature deviates from the preset temperature threshold, a polynomial fitting process is performed on the original spectral data to obtain corrected spectral data. Based on the corrected spectral data, the absorption peak position of the ultraviolet spectrum is identified by the derivative method in the preset ultraviolet band, and the visible light transmittance curve is calculated in the preset visible light band. Based on the visible light transmittance curve and the ultraviolet spectral absorption peak position, a preset number of characteristic sub-bands are divided, the average light intensity is extracted, and the multi-band spectral data is generated through weighted fusion processing.

[0009] Preferably, the step of comparing the background standard spectral feature template of the drug solution with the refractive index spectral features of the drug solution sample containing foreign matter, and recording the surface roughness scattering mode and polarized light reflection angle change data of the foreign matter region, includes: Multi-angle spectral scanning was performed on the drug sample containing foreign matter to obtain reflectance spectral data; If the refractive index of the material in the reflection spectrum data deviates from the refractive index of the standard spectral feature template of the drug solution background by more than a preset refractive index deviation threshold, it is marked as a foreign object area. Based on the reflectance spectrum data of the foreign object region, surface roughness features are extracted and scattering modes are calculated to obtain a scattering intensity distribution map. Obtain polarized light reflection angle data, calculate the rate of change of polarized light reflection angle based on the scattering intensity distribution map, and record the surface roughness scattering mode and the polarized light reflection angle change data.

[0010] Preferably, the step of extracting surface roughness features and calculating scattering modes based on the reflectance spectral data of the foreign object region to obtain a scattering intensity distribution map includes: Based on the reflectance spectrum data of the foreign object region, surface roughness characteristic parameters are extracted; The gray-level co-occurrence matrix of the surface roughness feature parameters is calculated to obtain the spatial distribution relationship of pixel pairs with different gray levels; Based on the spatial distribution relationship, the scattering mode parameters are determined, and a scattering intensity distribution map is generated.

[0011] Preferably, the step of calculating the spectral peak displacement quantization value between the foreign object and the drug liquid background based on the polarized light reflection angle change data, and extracting the spectral gradient information of the foreign object edge in combination with a preset displacement threshold, includes: The polarized light reflection angle change data is filtered to obtain denoised polarized light reflection data; The denoised polarized light reflection data is subjected to frequency domain transformation processing to calculate the quantized value of the spectral peak shift between the foreign object and the drug liquid background. If the spectral peak displacement quantization value exceeds the preset displacement threshold, the region where the foreign object exists is determined, and gradient calculation is performed on the edge of the region where the foreign object exists to extract the spectral gradient information of the foreign object edge.

[0012] Preferably, the step of calculating the transparency spectral attenuation coefficient based on the spectral gradient information of the foreign object edge, and calculating the spectral contrast enhancement factor of different foreign objects by combining the pre-acquired spectral response characteristic data of different foreign objects, includes: Based on the spectral gradient information of the foreign object edge, the incident light intensity and transmitted light intensity are obtained, the absorbance is calculated, and the transparency spectral attenuation coefficient is obtained by combining the pre-acquired optical path length parameter with the analysis. By fusing the spectral gradient information of the foreign object's edge and the spectral attenuation coefficient of transparency, and combining it with the spectral response characteristic data of different foreign objects obtained in advance, the spectral contrast enhancement factor of different foreign objects is calculated.

[0013] Preferably, if the spectral contrast enhancement factor is lower than a preset recognition threshold, the spectral correction parameters are dynamically adjusted, and the spectral peak shift quantization value is recalculated to obtain the multi-angle incident spectral response result, including: If the spectral contrast enhancement factor is lower than the preset recognition threshold, the spectral correction parameters are dynamically adjusted to obtain the adjusted spectral correction parameters. Based on the adjusted spectral correction parameters, the spectral peak shift quantization value is recalculated; Acquire surface roughness data of foreign objects at multiple different wavelengths, record the corresponding scattering mode parameters, and obtain a basic dataset containing the correspondence between wavelength features and scattering modes; The incident angle of the light source is adjusted based on the basic dataset, and a multi-angle spectral response test is performed in combination with the recalculated spectral peak displacement quantization value to obtain the multi-angle incident spectral response results.

[0014] Preferably, the process of fusing the surface roughness scattering mode and the multi-angle incident spectral response results, and performing consistency verification to obtain the final identification result, includes: The data of the surface roughness scattering mode at different wavelengths and the multi-angle incident spectral response results are weighted and fused to generate a fused feature vector. Calculate the fusion feature consistency coefficient of the fused feature vector. If the fusion feature consistency coefficient is greater than the preset fusion feature consistency threshold, the detection result is determined to be reliable, and a reliable detection result is output. Based on the reliable detection results and combined with the preset mapping relationship of foreign object material hazard level, the final identification result including foreign object material type, size estimation, and hazard level is output.

[0015] Secondly, the present invention provides a machine vision-based foreign object detection system for liquid medicine, comprising: The template generation module is used to acquire multi-band spectral data of drug liquid samples and perform normalization processing to obtain the standard spectral feature template of drug liquid background. The foreign matter feature comparison and recording module is used to compare the standard spectral feature template of the drug solution background with the refractive index spectral feature of the drug solution sample containing foreign matter, and record the surface roughness scattering mode and polarized light reflection angle change data of the foreign matter area. The spectral peak calculation and edge extraction module is used to calculate the spectral peak displacement quantization value between the foreign object and the drug liquid background based on the polarized light reflection angle change data, and extract the spectral gradient information of the foreign object edge by combining the preset displacement threshold. The material parameter calculation module is used to calculate the transparency spectral attenuation coefficient based on the spectral gradient information of the foreign object edge, and to calculate the spectral contrast enhancement factor of different foreign objects by combining the spectral response characteristic data of different foreign objects obtained in advance. The parameter adjustment and multi-angle response module is used to dynamically adjust the spectral correction parameters and recalculate the spectral peak shift quantization value if the spectral contrast enhancement factor is lower than the preset recognition threshold, so as to obtain the multi-angle incident spectral response result. The result fusion and verification module is used to fuse the surface roughness scattering mode and the multi-angle incident spectrum response results, and perform consistency verification to obtain the final identification result.

[0016] Thirdly, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the machine vision-based foreign object detection method for liquid medicine as described in any one of the above.

[0017] Fourthly, the present invention also provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the machine vision-based foreign object detection method for pharmaceutical liquid described in any one of the above.

[0018] Compared with the prior art, the present invention has the following beneficial effects: (1) By obtaining the ultraviolet absorption peak position and visible light transmittance curve of the drug sample, and after spectral baseline drift correction and temperature compensation, the present invention can effectively eliminate the interference of ambient temperature and differences between different batches of drug, provide a stable and reliable reference standard for foreign matter detection, and solve the problem of insufficient accuracy caused by background interference in traditional detection.

[0019] (2) By comparing the differences in refractive index spectral characteristics between foreign objects and the background of medicinal liquid, this invention achieves in-depth analysis of the material properties of foreign objects, effectively distinguishing between glass fragments and bubbles, which are similar in shape but different in material, effectively reducing the false judgment and false detection rate, and improving the accuracy of detection.

[0020] (3) This invention calculates the spectral peak displacement quantization value, the spectral gradient information of the foreign object edge, the transparency spectral attenuation coefficient and the spectral contrast enhancement factor, and integrates the multi-wavelength scattering mode and multi-angle incident spectral response results for consistency verification, and finally realizes the identification of foreign object material type and hazard level assessment, providing effective technical protection for drug quality and safety. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the machine vision-based foreign object detection method for liquid medicine provided in the first embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of the machine vision-based foreign object detection system for liquid medicine provided in the second embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Reference Figure 1 The first embodiment of the present invention provides a machine vision-based method for detecting foreign objects in liquid medicine, comprising the following steps: S1. Acquire multi-band spectral data of the drug liquid sample and perform normalization processing to obtain the standard spectral feature template of the drug liquid background. S2, compare the background standard spectral feature template of the drug solution with the refractive index spectral feature of the drug solution sample containing foreign matter, and record the surface roughness scattering mode and polarized light reflection angle change data of the foreign matter area; S3. Based on the polarized light reflection angle change data, calculate the spectral peak displacement quantization value between the foreign object and the drug liquid background, and extract the spectral gradient information of the foreign object edge by combining the preset displacement threshold. S4. Calculate the transparency spectral attenuation coefficient based on the spectral gradient information of the foreign object edge, and calculate the spectral contrast enhancement factor of different foreign objects by combining the spectral response characteristic data of different foreign objects obtained in advance. S5. If the spectral contrast enhancement factor is lower than the preset recognition threshold, the spectral correction parameters are dynamically adjusted and the spectral peak shift quantization value is recalculated to obtain the multi-angle incident spectral response result. S6. The surface roughness scattering mode and the multi-angle incident spectrum response results are fused and consistency verification is performed to obtain the final identification result.

[0024] In step S1, multi-band spectral data of the drug solution sample are acquired and normalized to obtain a standard spectral feature template of the drug solution background, including: S11, Obtain the original spectral data of the drug sample in the preset ultraviolet to visible light band; S12, the original spectral data is corrected by spectral baseline shift to obtain multi-band spectral data of the drug sample; S13. Based on the solution concentration spectral response characteristics in the multi-band spectral data, the near-infrared reflectance intensity values ​​of different batches of drug solutions are normalized using a pre-established temperature compensation spectral coefficient to obtain the background standard spectral feature template of the drug solution.

[0025] In step S11, the original spectral data of the drug sample in the preset ultraviolet to visible light band is obtained.

[0026] It should be noted that the preset ultraviolet to visible light band range is 380 nm to 780 nm. A spectrophotometer is used to perform spectral scanning on the drug sample to obtain the raw spectral data. This data includes the light intensity values ​​of each band. At the same time, ambient temperature sensor data needs to be collected simultaneously. The acquisition frequency is consistent with the spectral data acquisition frequency to ensure the accuracy of subsequent temperature correction.

[0027] For example, when testing a certain antibiotic solution, a spectrophotometer collects a light intensity data point every 1 nanometer, obtaining a total of 401 data points in the 380-780 nanometer wavelength range. A temperature sensor records changes in ambient temperature at the same frequency, forming a raw dataset containing light intensity and temperature.

[0028] In step S12, the original spectral data is corrected by spectral baseline shift to obtain multi-band spectral data of the drug sample, including: S121, If ​​the ambient temperature deviates from the preset temperature threshold, then perform polynomial fitting processing on the original spectral data to obtain corrected spectral data; S122, Based on the corrected spectral data, the absorption peak position of the ultraviolet spectrum is identified in the preset ultraviolet band using the derivative method, and the visible light transmittance curve is calculated in the preset visible light band. S123, based on the visible light transmittance curve and the ultraviolet spectral absorption peak position, a preset number of characteristic sub-bands are divided, the average light intensity is extracted, and the multi-band spectral data is generated through weighted fusion processing.

[0029] In step S121, if the ambient temperature deviates from the preset temperature threshold, polynomial fitting processing is performed on the original spectral data to obtain corrected spectral data.

[0030] It should be noted that the preset temperature threshold is usually set to 25 degrees Celsius, with an allowable fluctuation range of ±2 degrees Celsius. When the ambient temperature sensor detects that the temperature exceeds this range, a polynomial fitting (usually a cubic polynomial) is used to fit and correct the original spectral data. By establishing a mapping relationship between temperature and spectral baseline drift, the spectral data at non-standard temperatures is converted into corrected spectral data under standard temperature (25 degrees Celsius) conditions to eliminate the systematic shift of the spectral baseline caused by temperature.

[0031] For example, when the ambient temperature is 28 degrees Celsius (3 degrees Celsius away from the preset threshold), the system calls a cubic polynomial fitting function to fit the light intensity values ​​in the 380-780 nanometer band of the original spectral data to obtain the corrected spectral data, so that the baseline level of the data is consistent with the standard baseline at 25 degrees Celsius.

[0032] In step S122, based on the corrected spectral data, the absorption peak position of the ultraviolet spectrum is identified using the derivative method in the preset ultraviolet band, and the visible light transmittance curve is calculated in the preset visible light band.

[0033] It should be noted that the preset ultraviolet band is 380 nm to 580 nm. Within this band, the derivative method (first or second derivative) is used to process the calibration spectral data. By calculating the rate of change of the spectral curve, the broad absorption peaks are transformed into sharp peaks to improve the accuracy of peak position identification. The preset visible light band is 580 nm to 780 nm. Within this band, the ratio of transmitted light intensity to incident light intensity is calculated to obtain the visible light transmittance curve, which reflects the ability of the drug solution to transmit visible light of different wavelengths.

[0034] For example, the corrected spectral data of vitamin C injection solution were processed using the second derivative method in the 380-580 nm ultraviolet band to identify the characteristic absorption peak at 265 nm; in the 580-780 nm visible light band, the transmittance at 580 nm was calculated to be 0.82, at 650 nm to be 0.75, and at 780 nm to be 0.68, forming a visible light transmittance curve.

[0035] In step S123, based on the visible light transmittance curve and the ultraviolet spectral absorption peak position, a preset number of characteristic sub-bands are divided, the average light intensity is extracted, and the multi-band spectral data is generated through weighted fusion processing.

[0036] It should be noted that the preset number is usually 6. The visible light band from 580 nm to 780 nm is divided into 6 characteristic sub-bands (each sub-band is about 33 nm wide). Combined with the absorption peak position distribution of the ultraviolet band, the average light intensity of the calibrated spectral data in each sub-band is extracted. Weights are assigned according to the signal-to-noise ratio and stability (fluctuation coefficient) of each sub-band (the higher the signal-to-noise ratio and the better the stability, the larger the weight). Multi-band spectral data is generated by weighted summation (i.e., the product of the average light intensity of each sub-band and the weight is obtained and then summed).

[0037] For example, the 580-780 nm band is divided into six sub-bands: 580-613 nm, 614-647 nm, 648-681 nm, 682-715 nm, 716-749 nm, and 750-780 nm. The average light intensity values ​​extracted are 0.85, 0.78, 0.72, 0.68, 0.65, and 0.61, respectively. The weights calculated based on the signal-to-noise ratio of each sub-band are 0.2, 0.18, 0.18, 0.16, 0.14, and 0.14, respectively. After weighted fusion, the multi-band spectral data is obtained as 0.85×0.2+0.78×0.18+0.72×0.18+0.68×0.16+0.65×0.14+0.61×0.14=0.73.

[0038] In step S13, based on the solution concentration spectral response characteristics in the multi-band spectral data, the near-infrared reflectance intensity values ​​of different batches of drug solutions are normalized using a pre-established temperature compensation spectral coefficient to obtain the background standard spectral feature template of the drug solution.

[0039] The temperature-compensated spectral coefficient is a correction parameter reflecting the deviation between temperature and the near-infrared reflectance intensity of the drug solution. It is established by experimentally collecting spectral data of standard drug solution samples under different temperature conditions, creating a database that includes: a temperature range typically covering the temperature range that may occur during drug solution production and testing (e.g., 15℃ to 35℃), with temperature points set at 1℃ intervals; wavelength coverage, measuring key wavelengths in the near-infrared band (e.g., 850 nm, 950 nm, 1050 nm, corresponding to the characteristic absorption peaks of the main components of the drug solution); and coefficient determination, measuring the ratio of the reflectance intensity of the standard drug solution to the reflectance intensity at a standard temperature (e.g., 25℃) at each temperature and key wavelength. This ratio is the temperature-compensated spectral coefficient corresponding to that temperature and wavelength. For example, if the reflectance intensity at 850 nm wavelength is 1.03 times that at 25℃, the compensation coefficient at that temperature and wavelength is 1.03; at 26℃, the compensation coefficient for that wavelength is 0.97 (because the reflectance intensity decreases with increasing temperature).

[0040] In actual testing, raw near-infrared reflectance intensity data for different batches of the drug solution is required. First, the current ambient temperature is acquired (synchronously collected via a temperature sensor). If the current temperature is not directly stored in the database (e.g., 24℃), an interpolation method (e.g., linear interpolation) is used to calculate the compensation coefficient at that temperature (e.g., based on 1.03 at 22℃ and 0.97 at 26℃, the interpolation coefficient for 24℃ is 1.00). The calculated compensation coefficient is then used to correct the raw reflectance intensity (raw intensity × compensation coefficient) to obtain corrected reflectance intensity data that eliminates the influence of temperature, which is then subjected to subsequent normalization processing.

[0041] For example, if the original reflection intensity of a batch of medicine solution at 24℃ and a wavelength of 850 nm is 0.68, and the compensation coefficient at this temperature is 1.00 by interpolation, then the corrected reflection intensity is 0.68 × 1.00 = 0.68, ensuring that it is consistent with the spectral data at the standard temperature.

[0042] It should be noted that a linear regression algorithm is used to establish a quantitative relationship curve between solution concentration and spectral response intensity based on the light intensity values ​​of each band in the multi-band spectral data. If the solution concentration exceeds the preset range, segmentation processing is performed (e.g., linear fitting for low concentration segments and quadratic polynomial fitting for high concentration segments) to obtain concentration response characteristic parameters. By querying the temperature compensation coefficient database, the spectral correction coefficient at the current temperature is calculated using interpolation. Temperature compensation is applied to the raw near-infrared reflectance intensity data of different batches of drug solutions. Then, the maximum and minimum value normalization method is used to unify the data to the range of 0 to 1, and abnormal data deviating from the normal range are removed. Finally, the spectral feature values ​​of key wavelength points (e.g., 850 nm, 950 nm, 1050 nm) in the near-infrared band are extracted, and the data from multiple batches are weighted and fused to generate a standard spectral feature template of the drug solution background containing concentration response characteristic parameters and temperature compensation parameters.

[0043] For example, for a certain antibacterial drug solution, a linear relationship was established between concentrations of 10 mg / ml, 20 mg / ml, and 30 mg / ml and spectral response intensity (equation: y = 0.0365x + 0.055). For the high concentration range of 30 mg / ml to 50 mg / ml, a quadratic polynomial was used for fitting (coefficients: quadratic term -0.002, linear term 0.089). At a temperature of 24 degrees Celsius, a correction coefficient of 1.00 was applied at 850 nm through linear interpolation, compensating for the original reflectance intensity of 0.68 to 0.68. After normalization, the data from a certain batch ranged from 0 to 1. Outliers of 1.89 were removed, and five batches of data (weights 0.25, 0.22, 0.18, 0.18, and 0.17) were fused to generate a standard template.

[0044] In step S2, the refractive index spectral characteristics of the drug solution background standard spectral feature template are compared with those of the drug solution sample containing foreign matter, and the surface roughness scattering mode and polarized light reflection angle change data of the foreign matter region are recorded, including: S21, Perform multi-angle spectral scanning on the drug sample containing foreign matter to obtain reflectance spectral data; S22, if the material refractive index in the reflection spectrum data deviates from the refractive index of the standard spectral feature template of the drug liquid background by more than a preset refractive index deviation threshold, it is marked as a foreign object area; S23, Based on the reflection spectrum data of the foreign object region, extract the surface roughness features and calculate the scattering mode to obtain the scattering intensity distribution map; S24, acquire polarized light reflection angle data, calculate the rate of change of polarized light reflection angle in conjunction with the scattering intensity distribution map, and record the surface roughness scattering mode and the polarized light reflection angle change data.

[0045] In step S21, the drug sample containing foreign matter is subjected to multi-angle spectral scanning to obtain reflectance spectral data.

[0046] It should be noted that a high-precision spectrometer was used to perform a 360-degree rotational scan on the drug sample containing foreign matter, with a scanning angle interval of 15 degrees. Reflectance spectral data was collected at each angle to comprehensively obtain the spectral characteristics of the sample in different directions.

[0047] For example, for an infusion sample containing suspected foreign matter, a high-precision spectrometer collected reflectance spectral data every 15 degrees, starting from 0 degrees, until 360 degrees, obtaining a total of 24 sets of reflectance spectral data at different angles.

[0048] In step S22, if the material refractive index in the reflection spectral data deviates from the refractive index of the standard spectral feature template of the drug solution background by more than a preset refractive index deviation threshold, it is marked as a foreign object region.

[0049] It should be noted that the material refractive index parameter is extracted from the reflectance spectrum data and compared with the standard refractive index in the standard spectral feature template of the drug liquid background. The deviation value between the two is calculated. The preset refractive index deviation threshold is usually 0.03. When the deviation value exceeds this threshold, the corresponding area is marked as a foreign object area.

[0050] For example, if the background refractive index of a certain medicine solution is 1.38, and the material refractive index of a certain area is detected to be 1.42, the deviation value is 0.04, which exceeds the preset threshold of 0.03. The system will automatically mark this area as a foreign object area.

[0051] In step S23, based on the reflection spectrum data of the foreign object region, surface roughness features are extracted and scattering modes are calculated to obtain a scattering intensity distribution map, including: S231, extract surface roughness characteristic parameters based on the reflectance spectrum data of the foreign object region; S232, perform gray-level co-occurrence matrix calculation on the surface roughness feature parameters to obtain the spatial distribution relationship of pixel pairs with different gray levels; S233, Based on the spatial distribution relationship, determine the scattering mode parameters and generate a scattering intensity distribution map.

[0052] In step S231, surface roughness characteristic parameters are extracted based on the reflectance spectrum data of the foreign object region.

[0053] It should be noted that the surface roughness characteristic parameters are mainly obtained by analyzing the light intensity changes of adjacent wavelengths in the reflectance spectrum data of the foreign object region, including the reflectance intensity value of a specific wavelength (such as 850 nm) and the light intensity change amplitude within an adjacent preset wavelength range (such as 5 nm). These parameters reflect the roughness of the foreign object surface—the rougher the surface, the more significant the light intensity change of adjacent wavelengths.

[0054] For example, the reflection spectrum data of a foreign object region was analyzed, and the reflection intensity at 850 nm was extracted to be 0.65. The light intensity variation range within the adjacent 5 nm wavelength range (845-855 nm) reached 0.12. These values ​​were recorded as surface roughness characteristic parameters.

[0055] In step S232, the gray-level co-occurrence matrix is ​​calculated for the surface roughness feature parameters to obtain the spatial distribution relationship of pixel pairs with different gray levels.

[0056] It should be noted that the gray-level co-occurrence matrix (GLCM) is a matrix constructed by statistically analyzing the frequency of occurrence of pixels of different gray levels in an image within specific spatial relationships (such as horizontal, vertical, 45-degree, and 135-degree directions). It is used to describe the texture features of an image. For surface roughness feature parameters, after converting them to grayscale image format, fixed distance and direction parameters (such as a distance of 1 pixel and the horizontal direction) are set to calculate the GLCM. The matrix element values ​​represent the probability of occurrence of corresponding gray-level pixel pairs, thus obtaining the spatial distribution relationship of different gray-level pixel pairs.

[0057] For example, the extracted surface roughness feature parameters are converted into an 8-bit grayscale image (grayscale level 0-255). The grayscale co-occurrence matrix is ​​calculated with a distance of 1 pixel in the horizontal direction. The probability of occurrence of pixel pairs with grayscale level (120, 140) in the matrix is ​​0.08, and the probability of occurrence of (140, 160) is 0.12, thereby reflecting the spatial distribution of pixels with different grayscale levels.

[0058] In step S233, scattering mode parameters are determined based on the spatial distribution relationship, and a scattering intensity distribution map is generated.

[0059] It should be noted that scattering mode parameters include contrast (reflecting the degree of difference in gray levels) and uniformity (reflecting the uniformity of gray level distribution), which are calculated using the gray-level co-occurrence matrix: contrast is the sum of the products of the matrix element values ​​and the squares of the gray level differences, and uniformity is the sum of the squares of the matrix element values. Based on these parameters, combined with the spatial location information of the foreign object region, a scattering intensity distribution map is generated using a color coding method (e.g., higher intensity means brighter color), visually displaying the scattering characteristics of the foreign object.

[0060] For example, the contrast parameter of a foreign object is calculated to be 8.5 and the uniformity parameter is 0.23 by the gray-level co-occurrence matrix. Based on these parameters and the coordinate position of the foreign object in the image, a scattering intensity distribution map is generated. The map shows that the foreign object area presents a strip-shaped high scattering area along the 45-degree direction, which is consistent with the scattering characteristics of fibrous foreign objects.

[0061] In step S24, polarized light reflection angle data is obtained, and the rate of change of polarized light reflection angle is calculated in conjunction with the scattering intensity distribution map. The surface roughness scattering mode and the data on the change of polarized light reflection angle are recorded.

[0062] It should be noted that the polarized light reflection angle data is collected by a polarized light sensor, including the polarized light reflection angle values ​​of the foreign object region under different incident angles (such as 15 degrees, 30 degrees, 45 degrees, etc.); combined with the generated scattering intensity distribution map (containing spatial scattering distribution information of the foreign object region), the polarized light reflection angle change rate (in degrees / degree) is obtained by calculating the ratio of the change in reflection angle to the change in incident angle at adjacent incident angles; the recorded surface roughness scattering mode includes key parameters (such as contrast and uniformity) in the scattering intensity distribution map, and the polarized light reflection angle change data includes the reflection angle value corresponding to each incident angle and the calculated change rate.

[0063] For example, the polarization light sensor collected a reflection angle of 29 degrees at an incident angle of 30 degrees, a reflection angle of 43 degrees at 45 degrees, and a reflection angle of 58 degrees at 60 degrees. Combined with the scattering intensity distribution map (showing that the foreign object is fibrous scattering), the angle change rate from 30 degrees to 45 degrees was calculated to be (43-29) / (45-30)=14 / 15≈0.93, and the change rate from 45 degrees to 60 degrees was (58-43) / (60-45)=15 / 15=1.0. The contrast of the surface roughness scattering mode was recorded as 8.5, the uniformity as 0.23, and the polarization light reflection angle change data (30 degrees incident corresponds to 29 degrees reflection, 45 degrees incident corresponds to 43 degrees reflection, change rate 0.93 and 1.0).

[0064] In step S3, based on the polarized light reflection angle change data, the spectral peak displacement quantization value between the foreign object and the drug background is calculated. Combined with a preset displacement threshold, the spectral gradient information of the foreign object edge is extracted, including: S31, The polarized light reflection angle change data is filtered to obtain the denoised polarized light reflection data; S32, perform frequency domain conversion processing on the denoised polarized light reflection data, and calculate the quantized value of the spectral peak shift between the foreign object and the drug liquid background. S33, if the spectral peak displacement quantization value exceeds the preset displacement threshold, then the region where the foreign object exists is determined and the gradient calculation is performed on the edge of the region where the foreign object exists to extract the spectral gradient information of the foreign object edge.

[0065] In step S31, the polarized light reflection angle change data is filtered to obtain denoised polarized light reflection data.

[0066] It should be noted that a Gaussian filter is used to filter the background noise of the polarized light reflection angle variation data. The standard deviation of the filter is adjusted (usually set to 0.8) to balance the denoising effect and data feature preservation, removing random noise caused by optical device vibration, ambient light interference, etc., to obtain the denoised polarized light reflection data.

[0067] For example, the original polarized light reflection angle variation data is processed with a Gaussian filter with a standard deviation of 0.8 to filter out high-frequency noise and obtain denoised data with a signal-to-noise ratio improved by more than 3 times.

[0068] In step S32, the denoised polarized light reflection data is subjected to frequency domain transformation processing to calculate the quantized value of the spectral peak shift between the foreign object and the drug liquid background.

[0069] It should be noted that the denoised polarized light reflection data is converted from the time domain to the frequency domain using Fast Fourier Transform. The spectral peak frequencies of the foreign object region and the drug liquid background region are determined by spectral analysis, and the difference between the two is calculated. This difference is the spectral peak shift quantization value.

[0070] For example, after performing a fast Fourier transform on the denoised polarized light reflection data, the analysis showed that the main spectral peaks of the drug liquid background region were in the wavelength range of 280 nm to 320 nm, and the spectral peaks of the glass fragments and foreign objects were in the range of 350 nm to 380 nm. The calculated spectral peak shift quantization value was 70 nm.

[0071] In step S33, if the spectral peak displacement quantization value exceeds the preset displacement threshold, the region where the foreign object exists is determined and the gradient of the edge of the region where the foreign object exists is calculated to extract the spectral gradient information of the foreign object edge.

[0072] It should be noted that the preset displacement threshold is adjusted according to the type of drug solution. For example, it is set to 45 nm for highly transparent saline solutions and 65 nm for suspensions containing suspended particles. When the spectral peak displacement quantization value exceeds this threshold, an adaptive threshold binarization algorithm is used to separate the foreign object region from the drug solution background to obtain the foreign object contour boundary. The gradient values ​​of the contour boundary pixels in the horizontal and vertical directions are calculated using the Sobel operator to obtain the spectral gradient information of the foreign object edge, including the gradient magnitude and direction.

[0073] For example, when detecting a suspension, the preset displacement threshold is 65 nanometers. The calculated spectral peak displacement quantization value is 70 nanometers, which exceeds the threshold. The foreign object contour is obtained through adaptive threshold binarization. The Sobel operator is used to calculate the gradient information of the fibrous foreign object edge, which shows a slow gradient change in the long axis direction and a sharp change in the vertical direction.

[0074] In step S4, the transparency spectral attenuation coefficient is calculated based on the spectral gradient information of the foreign object edge. Combined with pre-acquired spectral response characteristic data of different foreign objects, the spectral contrast enhancement factor of different foreign objects is calculated, including: S41, based on the spectral gradient information of the foreign object edge, obtain the incident light intensity and transmitted light intensity, calculate the absorbance, and analyze it in conjunction with the pre-obtained optical path length parameter to obtain the transparency spectral attenuation coefficient; S42, the spectral gradient information of the foreign object edge and the transparency spectral attenuation coefficient are fused together, and the spectral response characteristic data of different foreign objects obtained in advance are combined to calculate the spectral contrast enhancement factor of different foreign objects.

[0075] In step S41, the incident light intensity and transmitted light intensity are obtained based on the spectral gradient information of the foreign object edge, the absorbance is calculated, and the transparency spectral attenuation coefficient is obtained by combining the pre-obtained optical path length parameter.

[0076] It should be noted that the optical path length parameter refers to the path length of light propagating in the drug sample. It is a key parameter when calculating the transparency spectral attenuation coefficient, i.e., the distance light travels from its incident point to penetrating the drug sample. It is usually determined by the thickness of the drug container (such as ampoules, infusion bags, etc.), and the unit is generally centimeters (cm). This parameter is a pre-obtained fixed value, obtained by directly measuring the thickness of the drug container using physical measurement methods (such as calipers). Because the specifications of the drug container are fixed, the optical path length remains unchanged during the detection process.

[0077] It is worth noting that, according to Beer-Lambert's law A=lg(I0 / I), where A is absorbance, dimensionless, used to quantify the degree of light absorption by a substance; I0 is the incident light intensity, in lux (lx), referring to the initial light intensity irradiating the surface of the drug sample; and I is the transmitted light intensity, in lux (lx), referring to the light intensity emitted after passing through the drug sample (including foreign matter). The absorbance is calculated, and combined with the pre-measured optical path length (i.e., the thickness of the drug sample in the detection optical path), the transparency spectral attenuation coefficient is obtained by analyzing the relationship between absorbance and optical path length. This coefficient reflects the absorption and scattering ability of the foreign matter to light.

[0078] The method of obtaining the transparency spectral attenuation coefficient by analyzing the relationship between absorbance and optical path length is specifically based on the quantitative correlation of Beer-Lambert's law. The steps include: determining the absorbance (A), locking the detection wavelength position based on the spectral gradient information of the foreign object edge, and collecting the incident light intensity (IO, in lux) and transmitted light intensity (I, in lux) at that wavelength using a spectrometer, and calculating the absorbance according to the formula A=lg(I0 / I) (A is a dimensionless parameter reflecting the degree of light absorption by the foreign object); obtaining the optical path length (L), which is a fixed value obtained in advance through physical measurement (such as calipers), that is, the path length occupied by the drug sample in the detection optical path (usually equal to the thickness of the drug container, in centimeters); establishing a quantitative relationship to calculate the attenuation coefficient, based on a simplified application of Beer-Lambert's law (ignoring secondary factors such as foreign object concentration and focusing on the direct correlation between optical path and absorption), the absorbance (A) is directly proportional to the optical path length (L), and the proportionality coefficient is the transparency spectral attenuation coefficient (ε), which is calculated as ε=A / L. The attenuation coefficient can be directly obtained from absorbance and optical path length using this formula. The unit of this coefficient is centimeters (cm⁻¹), and its value reflects the foreign object's ability to absorb and scatter light (the larger the value, the stronger the absorption and scattering).

[0079] For example, in the wavelength range of 500 nm to 540 nm, where the spectral gradient changes significantly at the edge of the fiber foreign object, the incident light intensity I0 = 1000 lx and the transmitted light intensity I = 398 lx were measured. The absorbance A = lg(1000 / 398) = 0.5 was calculated. The optical path length L = 2 cm was measured beforehand. Therefore, the transparency spectral attenuation coefficient ε = 0.5 / 2 = 0.25 can be calculated using the formula. This value can be used to calculate the subsequent spectral contrast enhancement factor. Considering the characteristics of this type of foreign object in the corresponding wavelength range, the final range of its transparency spectral attenuation coefficient was determined to be 0.12 to 0.18 cm⁻¹.

[0080] In step S42, the spectral gradient information of the foreign object edge and the transparency spectral attenuation coefficient are fused together, and the spectral contrast enhancement factor of different foreign objects is calculated by combining the spectral response characteristic data of different foreign objects obtained in advance.

[0081] It should be noted that the core parameter of the spectral gradient information at the edge of the foreign object is the gradient change rate, which is measured in nanometers and reflects the rate at which the spectral intensity at the edge of the foreign object changes with wavelength.

[0082] During data fusion, the basic feature value is calculated by adding the value of the gradient change rate and the value of the transparency spectral attenuation coefficient, and then dividing by 2 (since the units of the two are different, only the numerical value is used in the calculation). The resulting basic feature value is a dimensionless parameter.

[0083] The pre-acquired spectral response characteristic data of different foreign objects refers to the reflectance (dimensionless, range 0-1) of foreign objects such as glass fragments, fibers, and metal particles at different wavelengths (unit: nanometers) and incident angles (unit: degrees). This data is stored in a pre-defined database. During calculation, the reflectance value needs to be magnified by 100 times (only to adjust the numerical magnitude, without changing its physical meaning).

[0084] It is worth noting that the spectral contrast enhancement factor is calculated by multiplying the fused basic feature value by the amplified reflectance value. The result is the spectral contrast enhancement factor of the foreign object, which is a dimensionless parameter used to quantify the degree of spectral difference between the foreign object and the drug solution background.

[0085] For example, the gradient change rate of the glass fragment is 0.15 (per nanometer), the transparency spectral attenuation coefficient is 0.05 (per centimeter), and the basic characteristic value is calculated as (0.15 plus 0.05) divided by 2, resulting in 0.10 (dimensionless). Combined with its average reflectance of 0.08 (dimensionless) near an incident angle of 45 degrees, the reflectance value is magnified 100 times to 8, and the spectral contrast enhancement factor is calculated as 0.10 multiplied by 18 (the adjusted reflectance magnification value in the original example), resulting in 1.8 (dimensionless).

[0086] In step S5, if the spectral contrast enhancement factor is lower than a preset recognition threshold, the spectral correction parameters are dynamically adjusted, and the spectral peak shift quantization value is recalculated to obtain the multi-angle incident spectral response result, including: S51, if the spectral contrast enhancement factor is lower than the preset recognition threshold, the spectral correction parameters are dynamically adjusted to obtain the adjusted spectral correction parameters; S52, based on the adjusted spectral correction parameters, recalculate the spectral peak shift quantization value; S53, acquire surface roughness data of foreign objects at multiple different wavelengths, record the corresponding scattering mode parameters, and obtain a basic dataset containing the correspondence between wavelength features and scattering modes; S54, adjust the incident angle of the light source according to the basic dataset, and perform multi-angle spectral response test in combination with the recalculated spectral peak displacement quantization value to obtain the multi-angle incident spectral response result.

[0087] In step S51, if the spectral contrast enhancement factor is lower than the preset recognition threshold, the spectral correction parameters are dynamically adjusted to obtain the adjusted spectral correction parameters.

[0088] It should be noted that spectral correction parameters are key parameters used to eliminate environmental interference (such as temperature changes and baseline drift) to ensure the accuracy of spectral data. They mainly include spectral baseline drift correction parameters and temperature compensation spectral coefficients.

[0089] The spectral baseline drift correction parameter is used to eliminate systematic shifts in the spectral baseline caused by factors such as ambient temperature fluctuations and light source aging. When the ambient temperature deviates from a preset threshold (e.g., 25℃±2℃), the original spectral data can be fitted using polynomial fitting (e.g., cubic polynomial), or a Kalman filter can be used to track the baseline drift trend in real time to generate corrected parameters. This corrects the spectral data under non-standard conditions to the standard baseline level, thus obtaining the spectral baseline drift correction parameter. The temperature-compensated spectral coefficient is used to normalize the near-infrared reflectance intensity of the drug solution at different temperatures, eliminating the influence of temperature on the spectral response. A pre-established temperature compensation coefficient database can be used to establish a nonlinear mapping relationship between temperature and spectral coefficients within the range of 15℃-35℃ using interpolation or polynomial fitting (e.g., cubic polynomial), generating a temperature compensation coefficient matrix (containing correction coefficients at different temperatures and wavelengths), thus obtaining the temperature-compensated spectral coefficient.

[0090] It is worth noting that the preset recognition threshold is usually set to 0.85. When the spectral contrast enhancement factor is lower than this value, the Kalman filter is activated to track and compensate for the spectral baseline drift in real time. The baseline drift is estimated by updating the state prediction and measurement, and the corrected spectral baseline drift correction parameters are generated. Based on these parameters, polynomial fitting (such as cubic polynomial) is used to recalibrate the temperature-compensated spectral coefficients in the temperature range of 15℃ to 35℃, and a nonlinear mapping relationship between temperature and spectral coefficients is established to obtain the updated temperature-compensated spectral coefficient matrix, which is the adjusted spectral correction parameters.

[0091] For example, when the spectral contrast enhancement factor of a foreign object is 0.8, which is lower than the threshold of 0.85, the Kalman filter tracks and compensates for the baseline drift of 0.05 absorbance units to generate a corrected baseline correction parameter; data is collected in the range of 15°C to 35°C, and a temperature compensation coefficient matrix is ​​obtained by fitting a cubic polynomial. For example, the spectral coefficient at a wavelength of 550 nm changes by 0.003 when the temperature is between 25°C and 30°C, and the correction value is recorded in the matrix.

[0092] In step S52, the spectral peak shift quantization value is recalculated based on the adjusted spectral correction parameters.

[0093] It should be noted that the original spectral data is reprocessed using the adjusted spectral correction parameters (including the corrected baseline drift correction parameters and the temperature compensation spectral coefficient matrix), including baseline correction and temperature compensation. Then, the frequency domain transformation and peak analysis process in step S32 is repeated to recalculate the quantized value of the spectral peak shift between the foreign matter and the drug background.

[0094] For example, the original spectral data of a certain metal particle foreign object was recalibrated by applying the adjusted spectral correction parameters. After fast Fourier transform and spectral analysis, the spectral peak shift quantization value was recalculated to be 68 nanometers.

[0095] In step S53, surface roughness data of foreign objects at multiple different wavelengths are acquired, and the corresponding scattering mode parameters are recorded to obtain a basic dataset containing the correspondence between wavelength characteristics and scattering modes.

[0096] It should be noted that a laser interferometer (such as emitting a 632.8 nm helium-neon laser) is used to irradiate the surface of the foreign object. The surface roughness value of the foreign object is measured at multiple wavelengths in the 450 nm to 850 nm band (such as every 20 nm). When the roughness value exceeds a preset threshold (such as 0.5 μm), the corresponding scattering mode parameters (such as scattering angle range and scattering intensity attenuation coefficient) are recorded. The correspondence between wavelength and scattering mode parameters is compiled into a basic dataset.

[0097] For example, a laser interferometer measures a plastic particle foreign object at wavelengths of 450 nm, 470 nm...850 nm. When the roughness value reaches 1.2 micrometers, parameters such as the forward scattering angle of 15-45 degrees and the backscattering intensity attenuation coefficient are recorded to form a basic dataset containing 200 wavelength points and corresponding scattering modes.

[0098] In step S54, the incident angle of the light source is adjusted according to the basic dataset, and a multi-angle spectral response test is performed in combination with the recalculated spectral peak displacement quantization value to obtain the multi-angle incident spectral response result.

[0099] It should be noted that, based on the scattering characteristics of different wavelengths in the basic dataset, the incident angle of the light source was adjusted from 10 degrees to 80 degrees, and a test angle was set every 10 degrees. At each angle, the spectral intensity value was synchronously collected using a photodetector array (such as 64 detector units) in conjunction with the recalculated spectral peak displacement quantization value. The spectral response data at different angles and wavelengths were recorded and the results of the multi-angle incident spectral response were obtained.

[0100] For example, the incident angle is set to 10 degrees, 20 degrees...80 degrees according to the basic dataset. At an incident angle of 30 degrees, combined with the recalculated 68-nanometer peak displacement quantization value, the photodetector array measures the peak spectral intensity of the metal particle at 480 nanometers. At an incident angle of 45 degrees, the peak shifts to 520 nanometers. These data are recorded to form multi-angle incident spectral response results.

[0101] In step S6, the surface roughness scattering mode and the multi-angle incident spectral response results are fused and consistency verification is performed to obtain the final identification result, including: S61, perform weighted fusion processing on the data of the surface roughness scattering mode at different wavelengths and the multi-angle incident spectrum response results to generate a fused feature vector; S62, calculate the fusion feature consistency coefficient of the fusion feature vector. If the fusion feature consistency coefficient is greater than the preset fusion feature consistency threshold, the detection result is determined to be reliable, and a reliable detection result is output. S63, based on the reliable detection results and combined with the preset foreign object material hazard level mapping relationship, output the final identification result including foreign object material type, size estimation, and hazard level.

[0102] In step S61, the data of the surface roughness scattering mode at different wavelengths and the multi-angle incident spectrum response results are weighted and fused to generate a fused feature vector.

[0103] It should be noted that, based on the signal-to-noise ratio of surface roughness scattering mode data (such as scattering intensity and scattering angle) and multi-angle incident spectral response results (such as spectral intensity peak and reflectivity) at different wavelengths, a weighted average method is used to fuse the two types of data, extract key parameters such as roughness value, main scattering wavelength, and angle response curve, and combine them to generate a fused feature vector.

[0104] For example, the surface roughness scattering mode and multi-angle incident spectrum response results of a glass fragment are fused after weighting according to the signal-to-noise ratio of each parameter, and 12 key parameters (such as roughness of 0.8 micrometers, main scattering wavelength of 550 nanometers, etc.) are extracted to generate a fused feature vector.

[0105] In step S62, the fusion feature consistency coefficient of the fusion feature vector is calculated. If the fusion feature consistency coefficient is greater than the preset fusion feature consistency threshold, the detection result is determined to be reliable, and a reliable detection result is output.

[0106] It should be noted that the correlation coefficient (i.e., the consistency coefficient of the fusion feature) between the surface roughness scattering mode related parameters and the multi-angle incident spectrum response results in the fusion feature vector is calculated. The preset consistency threshold of the fusion feature is 0.8. When the coefficient is greater than this threshold, the detection result is determined to be reliable, and a reliable detection result containing various feature parameters of foreign matter is output.

[0107] The specific operation process for calculating the consistency coefficient of the fused features is as follows: Two sets of parameters are separated from the fused feature vector: the first set (parameters related to the surface roughness scattering mode), including the mean scattering intensity, contrast, uniformity, main scattering wavelength, scattering angle range, and scattering intensity attenuation coefficient (usually six key parameters, all dimensionless or with defined units, such as the scattering angle range in degrees); the second set (parameters related to the multi-angle incident spectral response results), including the peak spectral intensity, peak wavelength, reflectance at different incident angles, angle response coefficient, spectral half-width, and response stability coefficient (again, six key parameters are selected, reflectance is dimensionless, wavelength is in nanometers, and angle is in degrees). The arithmetic mean of the six parameters in the first set is calculated and recorded as the mean of the surface roughness parameters (only the numerical value is used in the calculation, ignoring unit differences).

[0108] The arithmetic mean of the six parameters in the second group is calculated and denoted as the mean of the multi-angle response parameters (only numerical values ​​are used in the calculation). The covariance is calculated to measure the consistency of the overall trend of change between the two groups of parameters. Specifically, the mean of the surface roughness parameter is subtracted from the value of each parameter in the first group, resulting in six deviation values; the mean of the multi-angle response parameters is subtracted from the value of each parameter in the second group, resulting in six deviation values; the deviation values ​​at corresponding positions are multiplied to obtain six products; the arithmetic mean of these six products is the covariance of the two groups of parameters. The standard deviations of the two groups of parameters are calculated by taking the square root of the arithmetic mean of the squares of the six deviation values ​​in the first group; the square root of the arithmetic mean of the squares of the six deviation values ​​in the second group is also calculated. The correlation coefficient (fusion feature consistency coefficient) is calculated, where the correlation coefficient is equal to the covariance divided by the product of the standard deviations of the first and second groups. The coefficient ranges from [-1, 1]. The closer the value is to 1, the more consistent the trends of the two sets of parameters; the closer it is to -1, the more opposite the trends; and the closer it is to 0, the less obvious the correlation.

[0109] For example, in a set of fused feature vectors, the values ​​of the first group (surface roughness parameters) are: 0.8 (roughness), 8.5 (contrast), 0.23 (uniformity), 550 (primary scattering wavelength, nanometers), 30 (scattering angle range, degrees), and 0.4 (attenuation coefficient), with a mean of (0.8+8.5+0.23+550+30+0.4)÷6≈98.32; the values ​​of the second group (multi-angle response parameters) are: 0.25 (peak spectral intensity), 520 (peak wavelength, nanometers), 0.18 (reflectivity), 1.2 (angle response coefficient), 40 (half-width at half maximum), and 0.9 (stability coefficient), with a mean of (0.25+520+0.18+1.2+40+0.9)÷6≈93.76. The covariance is calculated to be 82.6, the standard deviation of the first group is 201.5, the standard deviation of the second group is 208.3, and the correlation coefficient is approximately 0.85 (82.6 ÷ (201.5 × 208.3)). Since this is greater than the preset threshold of 0.8, the detection result is deemed reliable, and a reliable detection result containing all characteristic parameters of the foreign object is output.

[0110] In step S63, based on the reliable detection results and combined with the preset foreign object material hazard level mapping relationship, the final identification result including foreign object material type, size estimation, and hazard level is output.

[0111] It should be noted that the preset mapping relationship of foreign object material hazard levels is stored in the database, which includes optical feature templates and corresponding hazard levels for materials such as glass, metal, plastic, and fiber (e.g., glass fragments are high risk). Reliable detection results are matched with templates to find the material type with the highest similarity. The size of the foreign object is estimated by combining feature parameters (e.g., the diameter is estimated based on scattering intensity), and the final identification result including material type, size, and hazard level is output.

[0112] For example, the reliable detection result has a similarity of 0.92 with the glass fragment template. Based on the scattering intensity, its diameter is estimated to be 5 micrometers. Combining the mapping relationship between glass fragments and high-risk in the database, the final identification result is output: the material type is glass fragment, the size is about 5 micrometers, and the danger level is high risk.

[0113] Reference Figure 2 The second embodiment of the present invention provides a machine vision-based foreign object detection device for liquid medicine, comprising: The template generation module is used to acquire multi-band spectral data of drug liquid samples and perform normalization processing to obtain the standard spectral feature template of drug liquid background. The foreign matter feature comparison and recording module is used to compare the standard spectral feature template of the drug solution background with the refractive index spectral feature of the drug solution sample containing foreign matter, and record the surface roughness scattering mode and polarized light reflection angle change data of the foreign matter area. The spectral peak calculation and edge extraction module is used to calculate the spectral peak displacement quantization value between the foreign object and the drug liquid background based on the polarized light reflection angle change data, and extract the spectral gradient information of the foreign object edge by combining the preset displacement threshold. The material parameter calculation module is used to calculate the transparency spectral attenuation coefficient based on the spectral gradient information of the foreign object edge, and to calculate the spectral contrast enhancement factor of different foreign objects by combining the spectral response characteristic data of different foreign objects obtained in advance. The parameter adjustment and multi-angle response module is used to dynamically adjust the spectral correction parameters and recalculate the spectral peak shift quantization value if the spectral contrast enhancement factor is lower than the preset recognition threshold, so as to obtain the multi-angle incident spectral response result. The result fusion and verification module is used to fuse the surface roughness scattering mode and the multi-angle incident spectrum response results, and perform consistency verification to obtain the final identification result.

[0114] It should be noted that the machine vision-based foreign object detection device for liquid medicine provided in this embodiment of the invention is used to execute all the process steps of the machine vision-based foreign object detection method for liquid medicine in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0115] This invention also provides an electronic device. The electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a machine vision-based drug foreign object detection program. When the processor executes the computer program, it implements the steps in the various machine vision-based drug foreign object detection method embodiments described above, for example... Figure 1 Step S1 is shown. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the above-described device embodiments, such as the function of the template generation module.

[0116] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0117] The electronic device may be a desktop computer, laptop, handheld computer, or smart tablet, etc. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. It may include more or fewer components than described above, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0118] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting various parts of the electronic device through various interfaces and lines.

[0119] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device (such as spectral data, temperature data, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, SmartMediaCard (SMC), Secure Digital (SD) card, FlashCard, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0120] If the modules / units integrated in the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0121] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0122] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A machine vision-based method for detecting foreign objects in liquid medicine, characterized in that, include: Multi-band spectral data of the drug solution sample were acquired and normalized to obtain the standard spectral feature template of the drug solution background. The refractive index spectral characteristics of the drug solution background standard spectral feature template are compared with those of the drug solution sample containing foreign matter, and the surface roughness scattering mode and polarized light reflection angle change data of the foreign matter region are recorded. Based on the polarized light reflection angle change data, the spectral peak displacement quantization value between the foreign object and the drug liquid background is calculated, and the spectral gradient information of the foreign object edge is extracted by combining the preset displacement threshold. The transparency spectral attenuation coefficient is calculated based on the spectral gradient information of the foreign object edge, and the spectral contrast enhancement factor of different foreign objects is calculated by combining the spectral response characteristic data of different foreign objects obtained in advance. If the spectral contrast enhancement factor is lower than the preset recognition threshold, the spectral correction parameters are dynamically adjusted, and the spectral peak shift quantization value is recalculated to obtain the multi-angle incident spectral response result. The surface roughness scattering mode and the multi-angle incident spectrum response results are fused and consistency verification is performed to obtain the final identification result; The method for obtaining the multi-angle incident spectral response results includes: Acquire surface roughness data of foreign objects at multiple different wavelengths, record the corresponding scattering mode parameters, and obtain a basic dataset containing the correspondence between wavelength features and scattering modes; The incident angle of the light source is adjusted based on the basic dataset, and a multi-angle spectral response test is performed in combination with the recalculated spectral peak displacement quantization value to obtain the multi-angle incident spectral response results.

2. The machine vision-based method for detecting foreign objects in liquid medicine according to claim 1, characterized in that, The process of acquiring multi-band spectral data of the drug solution sample and performing normalization processing to obtain a standard spectral feature template for the drug solution background includes: Obtain the raw spectral data of the drug sample in the preset ultraviolet to visible light band; The original spectral data was corrected by spectral baseline shift to obtain multi-band spectral data of the drug sample. Based on the solution concentration spectral response characteristics in the multi-band spectral data, the near-infrared reflectance intensity values ​​of different batches of drug solutions are normalized using a pre-established temperature-compensated spectral coefficient to obtain the background standard spectral feature template of the drug solution.

3. The machine vision-based method for detecting foreign objects in liquid medicine according to claim 2, characterized in that, The step of correcting the original spectral data through spectral baseline shift to obtain multi-band spectral data of the drug sample includes: If the ambient temperature deviates from the preset temperature threshold, a polynomial fitting process is performed on the original spectral data to obtain corrected spectral data. Based on the corrected spectral data, the absorption peak position of the ultraviolet spectrum is identified by the derivative method in the preset ultraviolet band, and the visible light transmittance curve is calculated in the preset visible light band. Based on the visible light transmittance curve and the ultraviolet spectral absorption peak position, a preset number of characteristic sub-bands are divided, the average light intensity is extracted, and the multi-band spectral data is generated through weighted fusion processing.

4. The machine vision-based method for detecting foreign objects in liquid medicine according to claim 1, characterized in that, The step of comparing the background standard spectral feature template of the drug solution with the refractive index spectral features of the drug solution sample containing foreign matter, and recording the surface roughness scattering mode and polarized light reflection angle change data of the foreign matter region, includes: Multi-angle spectral scanning was performed on the drug sample containing foreign matter to obtain reflectance spectral data; If the refractive index of the material in the reflection spectrum data deviates from the refractive index of the standard spectral feature template of the drug solution background by more than a preset refractive index deviation threshold, it is marked as a foreign object area. Based on the reflectance spectrum data of the foreign object region, surface roughness features are extracted and scattering modes are calculated to obtain a scattering intensity distribution map. Obtain polarized light reflection angle data, calculate the rate of change of polarized light reflection angle based on the scattering intensity distribution map, and record the surface roughness scattering mode and the polarized light reflection angle change data.

5. The machine vision-based method for detecting foreign objects in liquid medicine according to claim 4, characterized in that, The step of extracting surface roughness features and calculating scattering modes based on the reflectance spectral data of the foreign object region to obtain a scattering intensity distribution map includes: Based on the reflectance spectrum data of the foreign object region, surface roughness characteristic parameters are extracted; The gray-level co-occurrence matrix of the surface roughness feature parameters is calculated to obtain the spatial distribution relationship of pixel pairs with different gray levels; Based on the spatial distribution relationship, the scattering mode parameters are determined, and a scattering intensity distribution map is generated.

6. The machine vision-based method for detecting foreign objects in liquid medicine according to claim 1, characterized in that, The step of calculating the spectral peak displacement quantization value between the foreign object and the drug liquid background based on the polarized light reflection angle change data, and extracting the spectral gradient information of the foreign object edge by combining it with a preset displacement threshold, includes: The polarized light reflection angle change data is filtered to obtain denoised polarized light reflection data; The denoised polarized light reflection data is subjected to frequency domain transformation processing to calculate the quantized value of the spectral peak shift between the foreign object and the drug liquid background. If the spectral peak displacement quantization value exceeds the preset displacement threshold, the region where the foreign object exists is determined, and gradient calculation is performed on the edge of the region where the foreign object exists to extract the spectral gradient information of the foreign object edge.

7. The machine vision-based method for detecting foreign objects in liquid medicine according to claim 1, characterized in that, The step of calculating the transparency spectral attenuation coefficient based on the spectral gradient information of the foreign object edge, and combining it with pre-acquired spectral response characteristic data of different foreign objects, to calculate the spectral contrast enhancement factor of different foreign objects includes: Based on the spectral gradient information of the foreign object edge, the incident light intensity and transmitted light intensity are obtained, the absorbance is calculated, and the transparency spectral attenuation coefficient is obtained by combining the pre-acquired optical path length parameter with the analysis. By fusing the spectral gradient information of the foreign object's edge and the spectral attenuation coefficient of transparency, and combining it with the spectral response characteristic data of different foreign objects obtained in advance, the spectral contrast enhancement factor of different foreign objects is calculated.

8. The machine vision-based method for detecting foreign objects in liquid medicine according to claim 1, characterized in that, If the spectral contrast enhancement factor is lower than a preset recognition threshold, the spectral correction parameters are dynamically adjusted, and the spectral peak shift quantization value is recalculated, including: If the spectral contrast enhancement factor is lower than the preset recognition threshold, the spectral correction parameters are dynamically adjusted to obtain the adjusted spectral correction parameters. Based on the adjusted spectral correction parameters, the spectral peak shift quantization value is recalculated.

9. The machine vision-based method for detecting foreign objects in liquid medicine according to claim 1, characterized in that, The process of fusing the surface roughness scattering mode and the multi-angle incident spectral response results, and performing consistency verification to obtain the final identification result, includes: The data of the surface roughness scattering mode at different wavelengths and the multi-angle incident spectral response results are weighted and fused to generate a fused feature vector. Calculate the fusion feature consistency coefficient of the fused feature vector. If the fusion feature consistency coefficient is greater than the preset fusion feature consistency threshold, the detection result is determined to be reliable, and a reliable detection result is output. Based on the reliable detection results and combined with the preset mapping relationship of foreign object material hazard level, the final identification result including foreign object material type, size estimation, and hazard level is output.

10. A machine vision-based foreign object detection system for pharmaceutical liquids, characterized in that, include: The template generation module is used to acquire multi-band spectral data of drug liquid samples and perform normalization processing to obtain the standard spectral feature template of drug liquid background. The foreign matter feature comparison and recording module is used to compare the standard spectral feature template of the drug solution background with the refractive index spectral feature of the drug solution sample containing foreign matter, and record the surface roughness scattering mode and polarized light reflection angle change data of the foreign matter area. The spectral peak calculation and edge extraction module is used to calculate the spectral peak displacement quantization value between the foreign object and the drug liquid background based on the polarized light reflection angle change data, and extract the spectral gradient information of the foreign object edge by combining the preset displacement threshold. The material parameter calculation module is used to calculate the transparency spectral attenuation coefficient based on the spectral gradient information of the foreign object edge, and to calculate the spectral contrast enhancement factor of different foreign objects by combining the spectral response characteristic data of different foreign objects obtained in advance. The parameter adjustment and multi-angle response module is used to dynamically adjust the spectral correction parameters and recalculate the spectral peak shift quantization value if the spectral contrast enhancement factor is lower than the preset recognition threshold, so as to obtain the multi-angle incident spectral response result. The result fusion and verification module is used to fuse the surface roughness scattering mode and the multi-angle incident spectrum response results, and perform consistency verification to obtain the final identification result.

Images