Image or video recognition methods for extracting deterioration features of pre-cooked food
By using semantic segmentation and differential feature manifold space technology, the problem of environmental interference in industrial sites was solved, and the accurate identification of the deterioration characteristics of pre-cooked food and the real-time perception of its rheological state were realized.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUNAN XIANGDIAN FOOD CO LTD
- Filing Date
- 2026-03-26
- Publication Date
- 2026-05-26
Smart Images
Figure CN121937993B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image recognition technology, and more specifically, to an image or video recognition method for extracting deterioration features of pre-prepared vegetables. Background Technology
[0002] Currently, in modern industrialized food production systems, non-contact inspection technologies based on machine vision are used for quality inspection of pre-packaged foods. Existing general technical approaches typically follow the logic of absolute feature measurement. This involves acquiring product images from the production line using industrial cameras, extracting global features using convolutional neural networks, or calculating statistical values of regions of interest based on traditional image processing operators. The system pre-sets a set of absolute thresholds or templates based on the statistical distribution of normal samples. When the feature values of the sample to be tested deviate from this absolute threshold range, it is determined that degradation has occurred. Under ideal and constant laboratory lighting conditions, the above methods can achieve high recognition accuracy. However, when the above technologies are applied to high-throughput, long-cycle actual industrial production sites, they face severe engineering challenges. As industrial light sources accumulate operating time, their luminous flux and color temperature undergo nonlinear decay. At the same time, the surface of packaging films in cold chain environments is often accompanied by randomly distributed condensation and fogging, and the light transmittance of different batches of packaging materials has physical differences. These fluctuations in non-biological environmental factors can cause a global drift in the overall pixel statistical distribution of the image.
[0003] At this point, focusing solely on hardware standardization is insufficient to eliminate interference. Existing image recognition control logic has limitations in handling dynamic scenarios. For example, Chinese invention patent CN103914708B discloses a machine vision-based food variety detection method and system. Although it introduces a multi-dimensional feature comparison mechanism of RGB grayscale histogram and HSV hue value, and judges the food status by calculating the statistical distance between the tested image and the pre-stored standard image features in the database, the core criterion is still based on absolute feature value matching. The static comparison mode relying on pre-stored standard templates requires that the real-time lighting conditions at the detection site be consistent with the environmental parameters when the database was established. When the light source on the production line decays or the surface of the packaging film is diffusely reflected due to moisture, the color and texture features of the acquired image shift as a whole, resulting in a mismatch with the standard features in the database. Even with conventional white balance correction or filtering and noise reduction, it is difficult to decouple the complex optical distortion caused by changes in the transmittance of the medium in a single frame dimension, and it cannot meet the needs of accurately capturing weak deterioration features in industrial settings.
[0004] Therefore, the technical problem to be solved by this invention is how to construct an image recognition mechanism that can automatically decouple ambient light and media interference and extract relative degradation features with topological invariance based only on a single frame of image data without introducing additional hardware calibration devices, thereby solving the key technical problem of high false alarm rate caused by signal drift in industrial environment. Summary of the Invention
[0005] This invention provides an image or video recognition method for extracting deterioration features of prepared vegetables, comprising the following steps:
[0006] Step 100: Obtain optical image data of the target object. The target object includes a solid-liquid hybrid medium and a transparent high-barrier film covering the solid-liquid hybrid medium. Based on the preset semantic segmentation logic, perform pixel-level parsing on the optical image data to generate a semantic index mask containing a solid protein subspace, a liquid matrix subspace, and a specular reflection subspace. The specular reflection subspace is the connected pixel region on the surface of the transparent high-barrier film that produces specular reflection.
[0007] Step 200: In response to the generation of the semantic index mask, a dynamic lighting and geometric reference system is constructed using the image feature statistics of the specular reflection subspace. Under the constraints of the dynamic lighting and geometric reference system, relative chromaticity drift calculation is performed on the solid protein subspace to generate a relative chromaticity difference vector, and relative texture energy calculation is performed on the liquid matrix subspace to generate a relative texture signal-to-noise ratio index, thus establishing a relative feature set free from ambient lighting interference.
[0008] Step 300: Construct a differential feature manifold space, and map the relative color difference vector and the relative texture signal-to-noise ratio index as multidimensional coordinate components into the differential feature manifold space to form a feature difference vector characterizing the current physical state of the target object;
[0009] Step 400: Perform logical arbitration based on topology preservation, calculate the modulus change rate and directional change amount of the feature difference vector in the differential feature manifold space, compare the modulus change rate and directional change amount with the preset normal manifold topology threshold, and generate an identification instruction indicating that the target object has undergone physical property degradation when the modulus change rate or directional change amount exceeds the normal manifold topology threshold.
[0010] Preferably, the step of generating the semantic index mask in step 100 includes: using a pre-trained semantic segmentation network to identify different component regions in the optical image data; marking the identified diffuse reflection-dominated solid regions as solid protein subspaces; marking the identified transmission and refraction-dominated liquid regions as liquid matrix subspaces; marking the identified connected regions with brightness values greater than a preset specular threshold as specular reflection subspaces, and using the specular reflection subspaces as logical anchor points for eliminating global illumination fluctuations and monitoring surface micro-deformation; and outputting a pixel-level semantic index mask containing positional information of the solid protein subspace, liquid matrix subspace, and specular reflection subspace.
[0011] Preferably, step 200, which involves performing relative chromaticity drift calculation, includes: extracting the first chromaticity mean vector of the solid protein subspace in the Lab color space and the reference chromaticity mean vector of the specular reflection subspace in the Lab color space based on the spatial constraints of the semantic index mask; calculating the difference vector between the first chromaticity mean vector and the reference chromaticity mean vector to obtain the relative chromaticity difference vector; wherein the relative chromaticity difference vector is used to characterize the color deviation of the solid protein subspace relative to the real-time ambient light reflection reference.
[0012] Preferably, step 200, which involves performing relative texture energy calculation, includes: extracting the target gray-level co-occurrence matrix energy value of the liquid matrix subspace and the reference gray-level co-occurrence matrix energy value of the specular reflection subspace; and calculating the relative texture signal-to-noise ratio index based on the following formula: ,in, This is the relative texture signal-to-noise ratio index. The target gray-level co-occurrence matrix energy value, The reference gray-level co-occurrence matrix energy value is used, and δ is a preset minimum constant used to prevent the denominator from being zero. The relative texture signal-to-noise ratio index is used to characterize the texture complexity of the liquid matrix subspace relative to the background optical noise.
[0013] Preferably, step 400, which involves performing logical arbitration based on topology preservation, includes: defining a normal topological neighborhood within the differential feature manifold space, wherein the normal topological neighborhood is composed of a set of feature difference vectors of a preset qualified sample; calculating the Mahalanobis distance between the feature difference vectors and the geometric center of the normal topological neighborhood; comparing the Mahalanobis distance with a preset degradation judgment threshold; and determining that the feature difference vectors have destroyed the preset normal topological structure if the Mahalanobis distance is greater than the degradation judgment threshold.
[0014] Preferably, the method further includes the step of inverting the airtightness state of the packaging based on the morphological features of the specular reflection subspace: extracting the connected domain skeleton features and edge gradient continuity index of the specular reflection subspace; monitoring the topological morphological changes of the connected domain skeleton features over time; when the connected domain skeleton features evolve from a discrete and fragmented form to a continuous and closed form and the edge gradient continuity index shows an upward trend, determining that the surface curvature of the transparent high-barrier film has slightly protruded due to changes in internal pressure, and generating a micro-bulging bag defect warning instruction.
[0015] Preferably, the step of constructing the differential feature manifold space in step 300 includes: defining a multidimensional vector space, which contains at least one chromaticity dimension for mapping the relative chromaticity vector and a texture dimension for mapping the relative texture signal-to-noise ratio index; normalizing the magnitude of the relative chromaticity vector and the relative texture signal-to-noise ratio index to the same numerical order of magnitude; and projecting the normalized data into the multidimensional vector space to form a feature difference vector, thereby achieving orthogonal decoupling between solid component features and liquid component features in the vector space.
[0016] Preferably, the step of generating a degradation identification instruction in step 400 further includes classification logic for degradation types: if the projection magnitude of the feature difference vector on the chromaticity dimension of the differential feature manifold space is less than a preset chromaticity attenuation threshold, and the projection component on the texture dimension remains within a preset stable range, then a degradation identification instruction indicating a change in the properties of the solid medium is generated; if the projection component of the feature difference vector on the texture dimension of the differential feature manifold space is greater than a preset texture abrupt change threshold, then a degradation identification instruction indicating the appearance of abnormal textures inside the liquid medium is generated.
[0017] Preferably, the optical image data is derived from a continuously acquired video stream, and the method further includes a time-series stability verification step: performing steps 100 to 300 on multiple consecutive frames of images in the video stream to obtain multiple feature difference vectors in the time series; calculating the variance of the multiple feature difference vectors in the time series; when the variance is less than a preset jitter threshold, confirming that the current feature difference vector is a valid input and performing step 400.
[0018] Preferably, the method further includes a sorting control step based on the degradation identification command: converting the degradation identification command into a physical rejection signal; sending the physical rejection signal to a sorting execution mechanism associated with the image acquisition location; driving the sorting execution mechanism to perform mechanical actions to remove the physical entity corresponding to the target object from the transport path.
[0019] The embodiments of the present invention have at least the following beneficial effects:
[0020] 1. In image or video recognition, adaptive optical normalization based on intra-frame semantic anchoring divides the image space into a reference subspace and a test subspace with optical property stability through a semantic index mask. The difference feature vector of the test subspace relative to the reference subspace is calculated. By utilizing the physical invariance of packaging materials in the time domain, a dynamic optical reference system is constructed within a single frame image. Global common-mode interference introduced by the attenuation of industrial light source intensity, color temperature drift, or condensation and fogging on the packaging film surface is automatically canceled through vector difference operations. This intra-frame relative measurement method frees the feature extraction process from the rigid dependence on constant lighting environment and absolute threshold, ensuring the consistency of the system's measurement of meat color drift and sauce texture changes in complex and ever-changing industrial environments. This solves the problem of frequent false alarms caused by fluctuations in environmental parameters in traditional visual inspection.
[0021] 2. Non-contact rheological state perception based on mechanical vibration response: This method utilizes the inherent mechanical vibrations during conveyor belt operation as a physical excitation source. By analyzing the differential displacement field of the liquid matrix region relative to the rigid packaging edge within consecutive or single-frame exposure times, the physical viscosity characteristics of the fluid are mapped to the local motion fuzzy distribution of the image gradient. This transforms industrial vibration noise into an effective signal for detecting the physicochemical properties of pre-prepared food. Without introducing an active rheometer or contact probe, it achieves real-time perception of latent degradation characteristics such as shear thinning or gelation retrogradation of the liquid matrix. By capturing the differential optical response of rigid and flexible components to the same vibration excitation, it effectively identifies early spoilage and liquefaction phenomena where static texture features are not yet obvious, thus expanding the application boundaries of image recognition technology in the detection of food physicochemical properties.
[0022] 3. Micro-gas generation inversion based on specular reflection topology: The high-reflection area on the packaging film surface is defined as a topological probe characterizing the surface curvature change. By extracting the morphological skeleton features and edge gradient continuity index of the high-reflection connected domain, a mapping relationship between the two-dimensional spot morphology and the three-dimensional film tension state is established. Utilizing the focusing effect of the micro-protrusions on the specular reflection light path caused by the initial stage of microbial metabolic gas generation, the high-reflection artifacts, which are considered interference and filtered out in traditional algorithms, are transformed into key features for judging abnormal packaging airtightness. By monitoring the topological evolution of the high-reflection area from a discrete and fragmented form to a continuous and closed form, a high-sensitivity inversion of micro-bulging defects in packaging is achieved in the two-dimensional image space, making up for the technical blind spot of traditional contour detection methods being insensitive to small deformations. Attached Figure Description
[0023] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings, in which several embodiments of the invention are illustrated by way of example and not limitation, wherein:
[0024] Figure 1 This is a flowchart of the deterioration identification method for semantic segmentation and manifold mapping of the present invention;
[0025] Figure 2 This is a diagram illustrating the overall architecture and data processing logic of the image recognition system of the present invention. Detailed Implementation
[0026] The principles and spirit of the present invention will now be described with reference to several exemplary embodiments in conjunction with the accompanying drawings. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.
[0027] An image or video recognition method for extracting deterioration features of pre-prepared vegetables includes the following steps:
[0028] Step 100: Obtain optical image data of the target object. The target object includes a solid-liquid hybrid medium and a transparent high-barrier film covering the solid-liquid hybrid medium. Based on the preset semantic segmentation logic, perform pixel-level parsing on the optical image data to generate a semantic index mask containing a solid protein subspace, a liquid matrix subspace, and a specular reflection subspace. The specular reflection subspace is the connected pixel region on the surface of the transparent high-barrier film that produces specular reflection.
[0029] Step 200: In response to the generation of the semantic index mask, a dynamic lighting and geometric reference system is constructed using the image feature statistics of the specular reflection subspace. Under the constraints of the dynamic lighting and geometric reference system, relative chromaticity drift calculation is performed on the solid protein subspace to generate a relative chromaticity difference vector, and relative texture energy calculation is performed on the liquid matrix subspace to generate a relative texture signal-to-noise ratio index, thus establishing a relative feature set free from ambient lighting interference.
[0030] Step 300: Construct a differential feature manifold space, and map the relative color difference vector and the relative texture signal-to-noise ratio index as multidimensional coordinate components into the differential feature manifold space to form a feature difference vector characterizing the current physical state of the target object;
[0031] Step 400: Perform logical arbitration based on topology preservation, calculate the modulus change rate and directional change amount of the feature difference vector in the differential feature manifold space, compare the modulus change rate and directional change amount with the preset normal manifold topology threshold, and generate an identification instruction indicating that the target object has undergone physical property degradation when the modulus change rate or directional change amount exceeds the normal manifold topology threshold.
[0032] Preferably, the step of generating the semantic index mask in step 100 includes: using a pre-trained semantic segmentation network to identify different component regions in the optical image data; marking the identified diffuse reflection-dominated solid regions as solid protein subspaces; marking the identified transmission and refraction-dominated liquid regions as liquid matrix subspaces; marking the identified connected regions with brightness values greater than a preset specular threshold as specular reflection subspaces, and using the specular reflection subspaces as logical anchor points for eliminating global illumination fluctuations and monitoring surface micro-deformation; and outputting a pixel-level semantic index mask containing positional information of the solid protein subspace, liquid matrix subspace, and specular reflection subspace.
[0033] Preferably, step 200, which involves performing relative chromaticity drift calculation, includes: extracting the first chromaticity mean vector of the solid protein subspace in the Lab color space and the reference chromaticity mean vector of the specular reflection subspace in the Lab color space based on the spatial constraints of the semantic index mask; calculating the difference vector between the first chromaticity mean vector and the reference chromaticity mean vector to obtain the relative chromaticity difference vector; wherein the relative chromaticity difference vector is used to characterize the color deviation of the solid protein subspace relative to the real-time ambient light reflection reference.
[0034] Preferably, step 200, which involves performing relative texture energy calculation, includes: extracting the target gray-level co-occurrence matrix energy value of the liquid matrix subspace and the reference gray-level co-occurrence matrix energy value of the specular reflection subspace; and calculating the relative texture signal-to-noise ratio index based on the following formula: ,in, This is the relative texture signal-to-noise ratio index. The target gray-level co-occurrence matrix energy value, The reference gray-level co-occurrence matrix energy value is used, and δ is a preset minimum constant used to prevent the denominator from being zero. The relative texture signal-to-noise ratio index is used to characterize the texture complexity of the liquid matrix subspace relative to the background optical noise.
[0035] Preferably, step 400, which involves performing logical arbitration based on topology preservation, includes: defining a normal topological neighborhood within the differential feature manifold space, wherein the normal topological neighborhood is composed of a set of feature difference vectors of a preset qualified sample; calculating the Mahalanobis distance between the feature difference vectors and the geometric center of the normal topological neighborhood; comparing the Mahalanobis distance with a preset degradation judgment threshold; and determining that the feature difference vectors have destroyed the preset normal topological structure if the Mahalanobis distance is greater than the degradation judgment threshold.
[0036] Preferably, the method further includes the step of inverting the airtightness state of the packaging based on the morphological features of the specular reflection subspace: extracting the connected domain skeleton features and edge gradient continuity index of the specular reflection subspace; monitoring the topological morphological changes of the connected domain skeleton features over time; when the connected domain skeleton features evolve from a discrete and fragmented form to a continuous and closed form and the edge gradient continuity index shows an upward trend, determining that the surface curvature of the transparent high-barrier film has slightly protruded due to changes in internal pressure, and generating a micro-bulging bag defect warning instruction.
[0037] Preferably, the step of constructing the differential feature manifold space in step 300 includes: defining a multidimensional vector space, which contains at least one chromaticity dimension for mapping the relative chromaticity vector and a texture dimension for mapping the relative texture signal-to-noise ratio index; normalizing the magnitude of the relative chromaticity vector and the relative texture signal-to-noise ratio index to the same numerical order of magnitude; and projecting the normalized data into the multidimensional vector space to form a feature difference vector, thereby achieving orthogonal decoupling between solid component features and liquid component features in the vector space.
[0038] Preferably, the step of generating a degradation identification instruction in step 400 further includes classification logic for degradation types: if the projection magnitude of the feature difference vector on the chromaticity dimension of the differential feature manifold space is less than a preset chromaticity attenuation threshold, and the projection component on the texture dimension remains within a preset stable range, then a degradation identification instruction indicating a change in the properties of the solid medium is generated; if the projection component of the feature difference vector on the texture dimension of the differential feature manifold space is greater than a preset texture abrupt change threshold, then a degradation identification instruction indicating the appearance of abnormal textures inside the liquid medium is generated.
[0039] Preferably, the optical image data is derived from a continuously acquired video stream, and the method further includes a time-series stability verification step: performing steps 100 to 300 on multiple consecutive frames of images in the video stream to obtain multiple feature difference vectors in the time series; calculating the variance of the multiple feature difference vectors in the time series; when the variance is less than a preset jitter threshold, confirming that the current feature difference vector is a valid input and performing step 400.
[0040] Preferably, the method further includes a sorting control step based on the degradation identification command: converting the degradation identification command into a physical rejection signal; sending the physical rejection signal to a sorting execution mechanism associated with the image acquisition location; driving the sorting execution mechanism to perform mechanical actions to remove the physical entity corresponding to the target object from the transport path.
[0041] Example 1: In the quality inspection scenario of pre-prepared food on an industrial production line where both high reflectivity and non-uniform illumination coexist, facing the objective conditions of a conveyor belt speed of 1.2 m / s and non-linear decay of light source intensity over time, the image processing system acquires high-resolution RGB digital image data containing the target object to be inspected through an industrial camera. The target object is a solid-liquid hybrid pre-prepared food packaged under a transparent high-barrier film, whose surface exhibits randomly distributed high-noise spots due to reflection from the packaging film and internal water vapor condensation. In response to the acquisition of image data, the semantic parsing module within the system loads a pre-set lightweight semantic segmentation network model, performs pixel-level topological parsing operations on the digital image data, and outputs a semantic index mask with the same resolution as the original image. This mask logically divides the two-dimensional pixel coordinate system into non-overlapping solid protein subspace, liquid matrix subspace, and high-reflectivity specular reflection subspace. During this process, the system locks the high-reflectivity specular reflection subspace as a dynamic illumination and geometric reference system, using it as an intra-frame self-calibration logical anchor point for subsequent feature extraction. A reference chromaticity mean vector is established using a histogram statistical linear response interval screening procedure. Physical validity is achieved through real-time statistical analysis of the grayscale distribution of all pixel brightness channels within the specular reflection subspace. Non-linear saturated pixels with brightness values exceeding the sensor's saturation threshold (e.g., 255 for 8-bit images or 1023 for 10-bit images) are automatically truncated. Unsaturated specular pixels with brightness values between 85% and 95% of the saturation threshold are selected as sampling points. When the percentage of saturated pixels in the current specular region exceeds 30%, an electronic shutter adjustment command is triggered on the industrial camera. The exposure time is gradually reduced in 50-microsecond increments until the connected region area of the unsaturated specular pixels meets the minimum statistical sample size (e.g., no less than 500 pixels). This process acquires information including the untruncated light source color temperature and spectral shift. And substitute the relative color difference vector The difference calculation formula is based on the spatial constraints of the semantic index mask. The feature extraction unit does not directly calculate the absolute pixel statistics of each component, but performs heterogeneous feature difference projection operation based on semantic anchor points.
[0042] For pixel data falling within the solid protein subspace, the processor calculates its first chromaticity mean vector in the Lab color space. Simultaneously, the reference chromaticity mean vector of the specular reflection subspace in the isomorphic color space is calculated. A relative color difference vector is constructed through vector subtraction. ,Right now This mathematically cancels out the common-mode interference components introduced by the global illumination intensity change ΔL and color temperature drifts Δa and Δb; for pixel data falling within the liquid matrix subspace, the processor extracts the target gray-level co-occurrence matrix energy value. And extract the energy value of the reference gray-level co-occurrence matrix of the specular reflection subspace. Based on the formula Calculate the relative texture signal-to-noise ratio index δ is set as a non-zero minimum constant to prevent the denominator from being zero. This ratio calculation maps the texture complexity of the liquid matrix to a relative amount with respect to the background optical noise, thereby eliminating the numerical interference of global blurring caused by condensation and fogging on the packaging film surface on texture features.
[0043] The system constructs a multidimensional difference feature manifold space, and calculates the relative color difference vector. Relative texture signal-to-noise ratio index As orthogonal components projected into this space, they form a characteristic difference vector representing the current physical state. The difference feature manifold space is defined as a normalized multidimensional Euclidean metric space. Topology-preserving logical arbitration is performed based on Mahalanobis distance statistical distribution characteristics. A benchmark dataset is constructed by pre-collecting feature difference vectors of standard qualified samples, and the dataset covariance matrix Σ and its inverse matrix are calculated. In real-time detection, the feature difference vector of the current sample is calculated. Mahalanobis distance between the mean vector μ of the benchmark dataset and the target dataset The calculation formula is: The logic arbitration unit calculates the feature difference vector. Mahalanobis distance relative to the geometric center of a pre-defined normal topological neighborhood within the manifold space And monitor in real time the modulus change rate and directional change of this distance; when the calculated Mahalanobis distance Greater than the preset degradation threshold The threshold is set to 3.5. Alternatively, if the cosine of the angle between the feature vector's direction and the direction of the normal principal component is less than a preset angle threshold, the system determines that the feature difference vector has disrupted the preset normal manifold topology, indicating that the prepared food has undergone physical degradation such as solid browning or liquid stratification. Based on this, a degradation identification command is generated to drive the sorting mechanism to perform a rejection action. By converting the specular interference from ambient light into a dynamic reference benchmark for feature extraction, stable identification of weak degradation features of prepared food is achieved under uncontrolled optical conditions. The edge gradient continuity index... The contour normal vector dot product discretization method is used to determine the set of pixels on the edge contour of the specular connected region, and the normalized gradient vector along the contour normal direction is calculated. Iterate through all pixels on the contour and calculate the average of the dot product of the gradient vectors of adjacent pixels. The calculation formula is as follows: Where N is the total number of edge contour pixels. and These are the normalized gradient vectors of the i-th pixel and its neighboring pixels, respectively. Due to internal gas generation, the packaging film exhibits a continuous convex surface, and the gradient direction at the highlight edge changes gently. Approaching 1; the packaging film is in a relaxed or wrinkled state, with high-frequency abrupt changes in the edge gradient direction, and the exponent decreases, which quantitatively characterizes the changes in the micro-tension of the film surface.
[0044] Example 2: To verify the engineering stability and feature decoupling capability of the pre-prepared vegetable degradation feature extraction method of the present invention under uncontrolled optical environment, a closed verification platform was constructed, including a dynamic environment simulation unit and a high-precision optical acquisition unit. A 5-megapixel global shutter industrial camera was configured, with its lens vertically aligned with a simulated industrial conveyor belt. A set of programmable LED ring light sources was mounted above the conveyor belt to generate light intensities that continuously varied from 1000 lux to 5000 lux. An ultrasonic humidifier was also provided to generate condensation mists of different densities on the transparent high-barrier film on the surface of the sample to be tested. The test object was a standardized packaged solid-liquid mixed braised pork pre-prepared vegetable, including fresh qualified samples and degraded samples that had been artificially accelerated to exceed the acid value standard. The core objective of the experiment was to confirm the semantic anchor points based on the specular reflection subspace through quantitative data comparison. The differential mechanism was tested to determine whether it could maintain a linear response to solid browning and liquid matrix turbidity characteristics under the dual interference of light fluctuations and medium blurring, thus effectively avoiding the failure risk of traditional global feature extraction algorithms. A gradient test environment with three typical conditions was established: Condition A was set as a standard environment, where the light intensity was stable at 2500 lux and the packaging film surface was dry and fog-free; Condition B was set as a light disturbance environment, where the light intensity fluctuated sinusoidally between 1500 lux and 3500 lux at a frequency of 1 Hz; and Condition C was set as a medium interference environment, where the packaging film surface was covered with a uniform water mist with a 30% reduction in transmittance. For each condition, image data of the comparison group and the experimental group were collected. The comparison group used the traditional global RGB mean and global gray-level co-occurrence matrix energy value as feature indicators, while the experimental group used the relative color difference vector of this invention. Relative texture signal-to-noise ratio index As a characteristic indicator.
[0045] Data analysis showed that during the switch from operating condition A to operating condition B, the absolute Lab chromaticity values of the solid protein region extracted from the control group drifted, with the variance of its luminance component L reaching 15.2. This caused the chromaticity characteristics of qualified samples to fall into the judgment range of degraded samples, resulting in a false detection rate as high as 18.5%. In contrast, the experimental group calculated the solid protein subspace vector... With the specular reflection subspace vector The difference, its output relative color difference vector The modulus variance is only 0.8, indicating that the highlight region, as the illumination reference, effectively cancels the common-mode component introduced by global illumination fluctuations, thus maintaining high intra-class compactness of chromaticity features under dynamic illumination. In the test under condition C, due to the scattering effect caused by water mist, the absolute texture energy value of the liquid matrix region measured by the comparison sample group is lower. The value dropped sharply from 0.85 to 0.32, exhibiting numerical characteristics highly similar to matrix deterioration and turbidity, making it impossible for the system to distinguish between external blurring and internal deterioration; while the experimental group monitored the reference texture energy value of the specular reflection subspace. Synchronization decreased from 0.92 to 0.35, according to the formula. The calculated relative texture signal-to-noise ratio index The value remained stable within the range of 0.91 to 0.93 without any jumps. This data phenomenon confirms that the texture blurring in the highlight region can accurately characterize the changes in the optical transfer function of the external medium. Ratio calculations successfully filtered out the multiplicative interference of packaging film condensation on the texture features of the internal matrix. Further logical arbitration verification shows that when the above features are projected onto the differential feature manifold space, the feature difference vector of qualified samples under the three working conditions... They always cluster within the normal topological neighborhood centered at the origin, and their Mahalanobis distance relative to the geometric center is... The maximum value is 2.1, far below the preset degradation threshold of 3.5; while for the sample that has actually deteriorated, under the ideal environment of condition A, its The modulus increases due to browning caused by the Maillard reaction, resulting in a larger Mahalanobis distance. It jumped to 5.8; after the water mist interference of operating condition C was superimposed, its The value remained stable between 5.6 and 5.9, unaffected by environmental noise. This invention constructs a relative measurement system with the high-gloss specular reflection subspace as the physical anchor point, upgrading image recognition from an absolute feature extraction method susceptible to environmental influences to a relative feature difference method with environmental adaptability. This ensures the system's accuracy and stability in recognizing the deterioration of the physical properties of pre-prepared dishes under boundary conditions where lighting and packaging film conditions are uncontrollable in complex industrial settings.
[0046] Example 3: During the system initialization and parameter calibration phase, to ensure the relative texture signal-to-noise ratio index... Mathematical stability and degradation threshold To assess statistical validity, the system executes a standardized scalar calibration procedure. For the noise suppression constant δ used in the formula to prevent the denominator from being zero, the system controls an industrial camera to acquire dark current noise from a standard blackbody calibration plate under no-light conditions, calculating the sensor's inherent thermal noise texture energy base value. And set δ to this This is 3 to 5 times the value of the original value, ensuring that the denominator term remains constant even under extreme conditions where specular reflection energy is extremely low. It can still characterize an effective lower limit of optical background noise, thereby avoiding the risk of numerical singularity at the physical level.
[0047] Degradation judgment threshold Once the condition is determined, the system initiates a gold sample self-learning process, continuously collecting image data of N pre-prepared vegetable samples that have been manually confirmed as qualified, where N is an integer not less than 200; the processor calculates the feature difference vector for each qualified sample. The system maps these points to a difference feature manifold space to construct a reference point set, and then calculates the covariance matrix Σ and mean vector μ of this point set. Based on the statistical law that the squared Mahalanobis distance of normal samples follows a chi-square distribution, the system calculates the probability density function of the Mahalanobis distance for this batch of samples, and locks the value corresponding to the upper limit of the 99.7% confidence interval as the running state. This transforms empirical judgment boundaries into statistically significant probability boundaries. Finally, to address long-term transmittance drift caused by light source aging or batch changes in packaging films, the system activates a sliding time window update mechanism with a length of M cycles. Whenever a new sample is determined by the logic arbitration unit to be a high-confidence qualified sample, its feature vector is pushed into the tail of the reference queue, while old data at the head of the queue is popped out. Based on this, the system recursively updates the covariance matrix Σ and mean vector μ of the normal topological neighborhood, ensuring that the geometric benchmark used for anomaly detection is always dynamically aligned with the current physical conditions.
[0048] Example 4: Before the system is physically installed and put into operation on the production line for the first time, in order to eliminate model initialization deviations caused by differences in on-site lighting environment and different background texture characteristics of the conveyor belt, the control unit executes the cold start environment logic alignment procedure, drives the industrial camera to collect a continuous frame sequence in the no-load state of the conveyor belt to construct a static background optical noise distribution map, instructs the conveyor system to import a set of reference samples that have been manually calibrated to be qualified, and the feature extraction unit calculates the relative color difference vector of the solid protein subspace of the set of reference samples in real time. and the relative texture signal-to-noise ratio index of the liquid matrix subspace These initial feature data are then filled into the covariance matrix Σ and mean vector μ of the undetermined coefficients until the number of samples reaches the preset statistical significance lower limit N, thereby completing the normal topological neighborhood boundary locking for the current specific physical environment before the logical arbitration unit is activated.
[0049] After the system enters continuous monitoring mode, to prevent systemic misjudgments caused by nonlinear attenuation of the light source or batch changes in packaging film raw materials, the independently operating baseline validity verification process monitors the energy value of the reference gray-level co-occurrence matrix in the specular reflection subspace in real time. The trend of change of the first-order statistical moments; once detected If the drift amount within the preset time window exceeds the environmental stability threshold, or the calculated relative texture signal-to-noise ratio index... If an inexplicable abrupt change occurs in the global mean, the system will determine that the current environmental baseline has failed and automatically suspend the output of the degradation identification command. At the same time, it will trigger an alarm signal to request the execution of a recalibration process for the current new operating conditions. By updating the physical anchor point parameters used for normalization calculation, the system ensures that it always maintains a closed loop of logical judgment on the degradation characteristics of pre-prepared food in a dynamically changing industrial environment.
[0050] Example 5: To address the issue of fluctuating accuracy in specular reflection subspace recognition caused by differences in generalization ability of semantic segmentation network models during the initial deployment phase, the system incorporates a standardized on-site image data adaptive training and model fine-tuning procedure, which is automatically triggered after the system is first powered on and the cold start environment logic alignment is completed. The operator places a calibration plate containing standard specular features in the imaging area of the conveyor belt and controls the industrial camera to acquire multiple calibration images under a preset exposure parameter sequence, constructing the initial on-site training dataset. The system then activates the embedded transfer learning engine, loads the pre-built general semantic segmentation network model as the base network, and fine-tunes the parameters of the fully connected layer at the end of the model using the on-site training dataset. During the fine-tuning process, the cross-entropy loss function is used to calculate the pixel-level error between the predicted mask and the standard mask, and the weight matrix of the fully connected layer is iteratively updated through the backpropagation algorithm until the intersection-union ratio on the validation set reaches the preset engineering qualification line of 0.92, thereby completing the model adaptation for the current on-site optical environment.
[0051] During system operation, to ensure the stability of the model's recognition of atypical highlight features such as scratches or stains on the packaging film surface, the system periodically executes an online sample mining process based on confidence gating. For critical samples that are judged as suspicious by the logic arbitration unit but have not reached the degradation threshold, the system temporarily stores their corresponding image data in a difficult sample library and uses this sample library to incrementally train the model during idle periods. The incremental training uses a small learning rate of 0.001 to fine-tune the model's sensitivity to atypical features while avoiding catastrophic forgetting of learned normal features, ensuring that the system continuously improves its adaptability to complex working conditions during long-term operation.
[0052] The above description is only a few preferred embodiments of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, technical solutions formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention.
Claims
1. A method for image or video recognition in pre-prepared dish deterioration feature extraction, characterized in that, Includes the following steps: Step 100: Obtain optical image data of the target object. The target object includes a solid-liquid hybrid medium and a transparent high-barrier film covering the solid-liquid hybrid medium. Based on the preset semantic segmentation logic, perform pixel-level parsing on the optical image data to generate a semantic index mask containing a solid protein subspace, a liquid matrix subspace, and a specular reflection subspace. The specular reflection subspace is the connected pixel region on the surface of the transparent high-barrier film that produces specular reflection. Step 200: In response to the generation of the semantic index mask, a dynamic lighting and geometric reference system is constructed using the image feature statistics of the specular reflection subspace. Under the constraints of the dynamic lighting and geometric reference system, relative chromaticity drift calculation is performed on the solid protein subspace to generate a relative chromaticity difference vector, and relative texture energy calculation is performed on the liquid matrix subspace to generate a relative texture signal-to-noise ratio index, thus establishing a relative feature set free from ambient lighting interference. Step 300: Construct a differential feature manifold space, and map the relative color difference vector and the relative texture signal-to-noise ratio index as multidimensional coordinate components into the differential feature manifold space to form a feature difference vector characterizing the current physical state of the target object; Step 400: Perform logical arbitration based on topology preservation, calculate the modulus change rate and directional change amount of the feature difference vector in the differential feature manifold space, compare the modulus change rate and directional change amount with the preset normal manifold topology threshold, and generate an identification instruction indicating that the target object has undergone physical property degradation when the modulus change rate or directional change amount exceeds the normal manifold topology threshold.
2. The image or video recognition method for extracting deterioration features of pre-prepared vegetables according to claim 1, characterized in that, The step of generating the semantic index mask in step 100 includes: using a pre-trained semantic segmentation network to identify different component regions in the optical image data; labeling the identified diffuse reflection-dominated solid regions as solid protein subspaces; labeling the identified transmission and refraction-dominated liquid regions as liquid matrix subspaces; labeling the identified connected regions with brightness values greater than a preset specular threshold as specular reflection subspaces, and using the specular reflection subspaces as logical anchor points for eliminating global illumination fluctuations and monitoring surface micro-deformation; and outputting a pixel-level semantic index mask containing positional information of the solid protein subspace, liquid matrix subspace, and specular reflection subspace.
3. The image or video recognition method for extracting deterioration features of pre-prepared vegetables according to claim 1, characterized in that, The step of performing relative chromaticity drift calculation in step 200 includes: extracting the first chromaticity mean vector of the solid protein subspace in the Lab color space and the reference chromaticity mean vector of the specular reflection subspace in the Lab color space based on the spatial constraints of the semantic index mask; calculating the difference vector between the first chromaticity mean vector and the reference chromaticity mean vector to obtain the relative chromaticity difference vector; wherein, the relative chromaticity difference vector is used to characterize the color deviation of the solid protein subspace relative to the real-time ambient light reflection reference.
4. The image or video recognition method for extracting deterioration features of pre-prepared vegetables according to claim 1, characterized in that, Step 200, which involves calculating the relative texture energy, includes: extracting the target gray-level co-occurrence matrix energy value from the liquid matrix subspace and the reference gray-level co-occurrence matrix energy value from the specular reflection subspace; and calculating the relative texture signal-to-noise ratio index based on the following formula: ,in, This is the relative texture signal-to-noise ratio index. The target gray-level co-occurrence matrix energy value, The reference gray-level co-occurrence matrix energy value is used, and δ is a preset minimum constant used to prevent the denominator from being zero. The relative texture signal-to-noise ratio exponent is used to characterize the texture complexity of the liquid matrix subspace relative to the background optical noise.
5. The image or video recognition method for extracting deterioration features of pre-prepared vegetables according to claim 1, characterized in that, The step 400, which involves performing logical arbitration based on topology preservation, includes: defining a normal topological neighborhood within the differential feature manifold space, wherein the normal topological neighborhood is composed of a set of feature difference vectors of a preset qualified sample; calculating the Mahalanobis distance between the feature difference vectors and the geometric center of the normal topological neighborhood; comparing the Mahalanobis distance with a preset degradation judgment threshold; and determining that the feature difference vectors have destroyed the preset normal topological structure if the Mahalanobis distance is greater than the degradation judgment threshold.
6. The image or video recognition method for extracting deterioration features of pre-prepared vegetables according to claim 1, characterized in that, The method also includes the step of inverting the airtightness of the packaging based on the morphological features of the specular reflection subspace: extracting the connected domain skeleton features and edge gradient continuity index of the specular reflection subspace; monitoring the topological morphological changes of the connected domain skeleton features over time; when the connected domain skeleton features evolve from a discrete and fragmented form to a continuous and closed form and the edge gradient continuity index shows an upward trend, it is determined that the surface curvature of the transparent high-barrier film has a slight bulge due to changes in internal pressure, and a micro-bulging bag defect warning instruction is generated.
7. The image or video recognition method for extracting deterioration features of pre-prepared vegetables according to claim 1, characterized in that, The step of constructing the differential feature manifold space in step 300 includes: defining a multidimensional vector space, which contains at least one chromaticity dimension for mapping the relative chromatic difference vector and one texture dimension for mapping the relative texture signal-to-noise ratio index; normalizing the magnitude of the relative chromatic difference vector and the relative texture signal-to-noise ratio index to the same numerical order of magnitude; and projecting the normalized data into the multidimensional vector space to form a feature difference vector.
8. The image or video recognition method for extracting deterioration features of pre-prepared vegetables according to claim 5, characterized in that, The step of generating a degradation identification instruction in step 400 also includes classification logic for degradation types: if the projection modulus of the feature difference vector on the chromaticity dimension of the differential feature manifold space is less than the preset chromaticity attenuation threshold, and the projection component on the texture dimension remains within the preset stable range, then a degradation identification instruction indicating the change in the properties of the solid medium is generated. If the projection component of the feature difference vector on the texture dimension of the differential feature manifold space is greater than the preset texture mutation threshold, a degradation identification instruction indicating the presence of abnormal textures inside the liquid medium is generated.
9. The image or video recognition method for extracting deterioration features of pre-prepared vegetables according to claim 1, characterized in that, The optical image data is derived from a continuously acquired video stream. The method also includes a temporal stability verification step: steps 100 to 300 are performed on multiple consecutive frames of images in the video stream to obtain multiple feature difference vectors in the time series. Calculate the variance of multiple feature difference vectors on the time series; when the variance is less than the preset jitter threshold, confirm that the current feature difference vector is a valid input and execute step 400.
10. The image or video recognition method for extracting deterioration features of pre-prepared vegetables according to claim 1, characterized in that, The method also includes a sorting control step based on a degradation identification command: converting the degradation identification command into a physical rejection signal; sending the physical rejection signal to a sorting execution mechanism associated with the image acquisition location; and driving the sorting execution mechanism to perform mechanical actions to remove the physical entity corresponding to the target object from the transport path.