Artificial intelligence-based multi-dimensional intelligent analysis system for food ingredients

Abstract

The present application relates to the field of image enhancement and artificial intelligence identification, in particular to a multi-dimensional intelligent analysis system for food materials based on artificial intelligence. The system specifically includes six modules: a data acquisition module, an image processing module, a representation generation module, a feature extraction module, a feature fusion module and an analysis output module. The system uses image enhancement and artificial intelligence identification to comprehensively analyze the multi-dimensional attributes of food material samples, including food material types, freshness indicators and nutritional components, thereby achieving the purposes of saving analysis costs and increasing analysis efficiency, truly realizing mutual gain between analysis tasks, significantly improving the analysis accuracy and generalization ability of the entire system, and providing a good data basis and analysis suggestions for enterprise evaluation and quality control of food materials.

Description

Technical Field

[0001] This invention relates to the field of image enhancement and artificial intelligence recognition technology, specifically to an artificial intelligence-based multi-dimensional intelligent analysis system for food ingredients. Background Technology

[0002] Currently, using computer vision and artificial intelligence technologies to identify and analyze food ingredients in images has become a research hotspot in this field. Existing technical solutions typically follow a basic processing flow: first, images of the food ingredients or dishes to be analyzed are acquired using image acquisition equipment; then, a series of preprocessing operations are performed on the acquired raw images, such as image enhancement to improve image quality and highlight key features to improve the accuracy of subsequent recognition; finally, the preprocessed images are input into a pre-trained artificial intelligence model to identify information such as the type, quantity, or location of the food ingredients contained in the image. Some technologies can simultaneously identify and locate multiple different food ingredients in a single image, or attempt to combine the recognition results with external data to achieve analysis of a specific dimension of the food ingredients, such as assessing the cost of a dish by matching real-time market price data.

[0003] However, existing technologies still have certain shortcomings. Current solutions typically focus on single-dimensional analysis, such as assessing only cost or detecting only freshness, lacking a unified framework for comprehensive multi-dimensional evaluation of ingredients. This results in users not obtaining complete and multi-dimensional information about the ingredients. Furthermore, existing technologies fail to provide an effective technical architecture for integrating and processing information from different sensor modalities and heterogeneous data sources; the analytical dimensions are isolated and cannot form a synergistic effect. Therefore, existing technologies need to systematically integrate multi-dimensional data to build an intelligent system that can efficiently, collaboratively, and comprehensively analyze multiple dimensions such as ingredient type, cost, freshness, and safety to solve the current technical challenges.

[0004] To address this, an AI-based multi-dimensional intelligent analysis system for food ingredients is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide an artificial intelligence-based multi-dimensional intelligent analysis system for food ingredients to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An AI-based multi-dimensional intelligent analysis system for food ingredients includes:

[0008] Data acquisition module: acquires a visual image containing spatial and color information of the food sample to be analyzed and a hyperspectral data cube containing spectral information of the food sample in multiple spectral bands;

[0009] Image processing module: Performs preprocessing operations, including image enhancement and spatial alignment, on the visual image and data cube;

[0010] The characterization generation module constructs an observation matrix based on the preprocessed data, calculates the pairwise covariance between data channels using the observation matrix, generates a covariance matrix, and converts the covariance matrix into a two-dimensional characterization image.

[0011] Feature extraction module: Inputs the two-dimensional representation image into the multi-task intelligent model, and extracts shared feature representations through the shared feature extraction layer within the model;

[0012] Feature fusion module: Processes the shared feature representation through the attention bottleneck fusion mechanism within the model to generate collaborative feature representation. The fusion mechanism interacts with the shared feature representation across attention through a small set of learnable bottleneck tokens.

[0013] Analysis output module: Based on the collaborative feature representation, predict and output the multi-dimensional attributes of the food sample, including food type, freshness index and nutritional components.

[0014] Preferably, the process for acquiring visual images and hyperspectral data cubes includes:

[0015] The visual images are captured using a standard RGB sensor to record the texture, shape, and surface color information of the food sample. The hyperspectral data cube is sequentially captured using a line scan or snapshot hyperspectral camera across multiple consecutive narrow bands covering the visible to near-infrared spectrum to record the chemical composition information of the food sample.

[0016] Preferably, the specific implementation process for performing image preprocessing operations includes:

[0017] Image enhancement algorithms are applied to the visual image and the hyperspectral data cube respectively to identify common feature points in the two data sources, calculate affine or perspective transformation matrices, spatially align the visual image and the hyperspectral data cube, and perform data augmentation on the aligned data through geometric transformation.

[0018] Preferably, the construction of the observation matrix and the calculation of the covariance include:

[0019] For each spatially aligned pixel position, the values ​​of the R, G, and B color channels of the visual image are concatenated with the values ​​of all spectral band channels of the hyperspectral data cube to form a single feature vector. The feature vectors of all pixel positions are stacked to form an observation matrix. The mean of each column of the observation matrix is ​​calculated, and the mean of each column is subtracted to achieve data centralization. The product of the centralized observation matrix and its transpose is calculated and normalized to obtain the covariance matrix.

[0020] Preferably, the specific implementation process of converting the covariance matrix into a two-dimensional representation image includes:

[0021] Each element of the covariance matrix is ​​directly treated as the intensity value of a pixel; the element value of the position of the covariance matrix is ​​mapped to the pixel value of the two-dimensional representation image at the coordinates, generating a single-channel grayscale image with the same dimension as the covariance matrix, and making it suitable as the input of a convolutional neural network.

[0022] Preferably, the process of extracting shared feature representations in a multi-task intelligent model includes:

[0023] The two-dimensional representation image is input into a shared feature extraction network consisting of multiple convolutional layers, activation function layers, and pooling layers. The two-dimensional representation image is convolved through learnable filters in the convolutional layers to identify correlation patterns encoded in the image. A nonlinear activation function is applied after each convolutional layer to introduce nonlinearity, and the feature map is downsampled through the pooling layers to reduce spatial dimensionality and retain the most salient features. The final output of the shared feature extraction network is the shared feature representation.

[0024] Preferably, the specific implementation process for processing the shared feature representation includes:

[0025] A small set of fixed-size, learnable latent vectors is initialized as bottleneck tokens. In a cross-attention operation, the bottleneck tokens are used as queries, and the shared feature representation is linearly projected as keys and values. The attention weights are obtained by calculating the dot product of the query and the key and applying the Softmax function, and then weighted summation of the values ​​is performed to form the collaborative feature representation.

[0026] Preferably, the specific implementation process for predicting and outputting based on the collaborative feature representation includes:

[0027] The collaborative feature representation is simultaneously fed into multiple parallel, independent prediction heads. One prediction head is used to perform a classification task to output the type of food, another prediction head is used to perform a regression task to output the freshness index, and yet another prediction head is used to perform a regression task to output the nutritional components, thereby achieving parallel prediction and output of multiple tasks.

[0028] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0029] 1. This invention constructs a unified analytical framework capable of multi-dimensional comprehensive evaluation, effectively addressing the shortcomings of existing technologies that typically only perform single-dimensional analysis and cannot provide comprehensive information. By integrating a multi-task learning paradigm with a parallel prediction head structure, this system can output multi-dimensional attributes such as ingredient type, freshness, and nutritional components in parallel, based on a unified collaborative feature representation. This end-to-end processing approach improves analytical efficiency, ensures the inherent consistency and correlation of various output results, and provides users with a comprehensive and multi-dimensional ingredient quality assessment report, achieving a dual breakthrough in the breadth and depth of ingredient information acquisition.

[0030] 2. This invention proposes an innovative heterogeneous data representation method, solving the technical challenges faced by existing technologies in integrating modal data from different sensors. The system first concatenates the RGB channels of the visual image with all band channels of the hyperspectral data, then calculates their covariance matrix, and directly converts this matrix into a two-dimensional representation image. This representation image has a fixed size and explicitly encodes the second-order statistical relationships between all data channels, forming highly information-dense discriminative features. This enables efficient analysis using mature convolutional neural networks, significantly reducing the complexity and computational cost of model processing.

[0031] 3. This invention introduces an advanced attention bottleneck fusion mechanism to achieve deep collaboration between different analysis dimensions, overcoming the limitations of existing technologies where analysis dimensions are isolated and unable to form a synergistic effect. This mechanism introduces a small set of learnable bottleneck tokens to actively query and extract core information from the shared feature network through cross-attention. Compared to simple feature sharing, this sharing method forces the model to learn how to optimally combine general features to serve all downstream tasks, ultimately generating a unified collaborative feature representation with information efficiency far exceeding the simple addition of individual parts. This truly achieves mutual benefit between analysis tasks, significantly improving the overall system's analysis accuracy and generalization ability. Attached Figure Description

[0032] Figure 1 The flowchart is a multi-dimensional intelligent analysis system for food ingredients based on artificial intelligence as described in this invention.

[0033] Figure 2 This is a flowchart illustrating the image preprocessing process of the AI-based multi-dimensional intelligent food analysis system described in this invention.

[0034] Figure 3 This is a flowchart of the multi-task intelligent model in the AI-based multi-dimensional intelligent food analysis system described in this invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Please see Figures 1 to 3 The present invention relates to an artificial intelligence-based multi-dimensional intelligent analysis system for food ingredients, which specifically includes the following:

[0037] Data acquisition module: acquires a visual image containing spatial and color information of the food sample to be analyzed and a hyperspectral data cube containing spectral information of the food sample in multiple spectral bands;

[0038] Image processing module: Performs preprocessing operations, including image enhancement and spatial alignment, on the visual image and data cube;

[0039] The characterization generation module constructs an observation matrix based on the preprocessed data, calculates the pairwise covariance between data channels using the observation matrix, generates a covariance matrix, and converts the covariance matrix into a two-dimensional characterization image.

[0040] Feature extraction module: Inputs the two-dimensional representation image into the multi-task intelligent model, and extracts shared feature representations through the shared feature extraction layer within the model;

[0041] Feature fusion module: Processes the shared feature representation through the attention bottleneck fusion mechanism within the model to generate collaborative feature representation. The fusion mechanism interacts with the shared feature representation across attention through a small set of learnable bottleneck tokens.

[0042] Analysis output module: Based on the collaborative feature representation, predict and output the multi-dimensional attributes of the food sample, including food type, freshness index and nutritional components.

[0043] The technical solution of the present invention will be further described in detail below with reference to specific embodiments.

[0044] Example 1

[0045] This application discloses an artificial intelligence-based multi-dimensional intelligent analysis system for food ingredients, used for quality control, grading, and inventory management of food products, improving efficiency and accuracy. (See also...) Figure 1 The specific implementation of the system proposed in this invention includes: a data acquisition module: acquiring a visual image containing spatial and color information of the food sample to be analyzed and a hyperspectral data cube containing spectral information of the food sample in multiple spectral bands; an image processing module: performing preprocessing operations including image enhancement and spatial alignment on the visual image and the data cube; a representation generation module: constructing an observation matrix based on the preprocessed data, calculating the pairwise covariance between data channels through the observation matrix, generating a covariance matrix, and converting the covariance matrix into a two-dimensional representation image; a feature extraction module: inputting the two-dimensional representation image into a multi-task intelligent model, and extracting shared feature representations through a shared feature extraction layer within the model; a feature fusion module: processing the shared feature representations through an attention bottleneck fusion mechanism within the model to generate collaborative feature representations, wherein the fusion mechanism interacts with the shared feature representations across attention through a small set of learnable bottleneck tokens; and an analysis output module: predicting and outputting multi-dimensional attributes of the food sample, including food type, freshness index, and nutritional components, based on the collaborative feature representations.

[0046] In one specific embodiment of the present invention, the object of analysis is a fresh Atlantic salmon fillet.

[0047] Furthermore, acquiring a visual image containing spatial and color information of the food sample to be analyzed, and a hyperspectral data cube containing spectral information of the food sample in multiple spectral bands, corresponds to the system's data acquisition module, specifically including:

[0048] Top-view visual images of salmon fillets were captured under controlled lighting conditions using an industrial-grade standard RGB camera with a resolution of 1920x1080 pixels. These images clearly record the color of the fillet surface, the distribution pattern of fat stripes, the overall shape, and any visible imperfections. This information is crucial for subsequent species identification and macroscopic quality assessment.

[0049] A pushbroom or line-scan hyperspectral camera is used to acquire the hyperspectral data cube. This acquisition method is ideally suited for integration into conveyor belts on industrial production lines for continuous, online non-destructive testing. The camera acquires data over a broad spectral range covering visible to near-infrared light. This spectral range was chosen because it includes characteristic absorption bands associated with key chemical components in fish and hemoglobin related to oxidation states. Data is acquired across hundreds of consecutive narrow spectral bands, ultimately forming a three-dimensional data structure, the hyperspectral data cube.

[0050] By combining a standard RGB sensor and a hyperspectral camera, the system can capture complementary information about the ingredients. RGB images provide macroscopic physical features crucial for species identification, while hyperspectral data reveals microscopic chemical composition information necessary for assessing freshness and nutritional value. This multimodal data acquisition strategy provides a rich and comprehensive raw data foundation for subsequent precise and multidimensional analysis, which is unmatched by single-data-source solutions.

[0051] Furthermore, preprocessing operations, including image enhancement and spatial alignment, are performed on the visual image and data cube. Specifically, this corresponds to the image processing module of the system and includes:

[0052] Enhancement algorithms are applied to the acquired raw images. For RGB visual images, histogram equalization is used to enhance image contrast, making the texture and fat distribution of the fish meat clearer. For each band of the hyperspectral data cube, smoothing algorithms such as Savitzky-Golay filtering are applied to effectively remove sensor noise. The beneficial effect of this image enhancement algorithm is that by applying targeted processing to different types of image data, it significantly improves the quality of the raw data required for subsequent analysis. It can directly enhance image contrast, making key macroscopic physical features such as fish meat texture and fat distribution more clearly discernible, while preserving the true shape of spectral features crucial for chemical composition analysis to the greatest extent possible.

[0053] This task is accomplished using the Scale Invariant Feature Transform (SIFT) algorithm. SIFT is particularly effective for registration between multimodal images because it is invariant to scale, rotation, and illumination variations, which are ubiquitous across images acquired from different sensors. Specifically, SIFT is run on a grayscale version of the RGB image and a representative band of the hyperspectral data cube to detect a unique and stable set of keypoints.

[0054] Based on the successfully matched keypoint pairs in the two images, the system estimates a geometric transformation model. The system employs algorithms such as scale-invariant feature transformation to match the successfully matched keypoint pairs in the two images. To robustly handle potential false matches, a random sample consensus algorithm is used to compute an affine transformation matrix or a more flexible perspective transformation matrix. This matrix mathematically describes the geometric transformation required to map the coordinate system of one image to the coordinate system of another. Its accuracy is a fundamental prerequisite for effective multimodal data fusion, avoiding analytical errors caused by data misalignment and significantly improving the accuracy and reliability of subsequent feature fusion and analysis. This matrix mathematically describes the geometric transformation required to map the coordinate system of one image to the coordinate system of another.

[0055] The calculated transformation matrix is ​​applied to one of the data sources to geometrically warp it, resulting in a perfectly aligned RGB image pixel grid with the spatial grid of the hyperspectral data cube. Geometric data augmentation is then performed on the aligned data pairs, including random rotations, horizontal / vertical flips, and minor scaling. During this process, identical transformations must be applied simultaneously to both the RGB image and the hyperspectral data cube to ensure their spatial alignment is preserved.

[0056] By using image enhancement and spatial alignment as preprocessing techniques, it is possible to ensure that visual images and hyperspectral data acquired from different sensors correspond precisely in space. By identifying common feature points and calculating the transformation matrix, it can be guaranteed that at any pixel location, both color and spectral information originate from the same point on the food. This precise alignment is a prerequisite for effective multimodal data fusion, avoiding analytical errors caused by data misalignment and significantly improving the accuracy and reliability of subsequent feature fusion and analysis.

[0057] Furthermore, an observation matrix is ​​constructed based on the preprocessed data. The pairwise covariances between data channels are calculated using this observation matrix to generate a covariance matrix. This covariance matrix is ​​then converted into a two-dimensional representation image. The representation generation module of the corresponding system specifically includes:

[0058] The system concatenates the intensity values ​​of the R, G, and B channels of the RGB image at each spatially aligned pixel location with the intensity values ​​of all spectral bands in the hyperspectral data cube to form a single feature vector. Then, the feature vectors from all pixel locations in the image are stacked to form an observation matrix. In this matrix, each row represents the observation data of one pixel, each column represents a data channel, and the covariance matrix is ​​calculated from the observation matrix.

[0059] Calculate the mean of each column in the observation matrix, then subtract this mean from all elements in that column. This operation reduces the mean of the data for each channel to zero. Next, calculate the product of the centered matrix and its transpose, and then normalize it by dividing by the number of samples minus one to obtain the covariance matrix. Each element in this matrix represents the covariance between two data channels.

[0060] By transforming the raw multi-channel data stream into a structured representation that explicitly encodes the second-order statistical relationships between all data channels, each element in the covariance matrix quantifies the correlation between two channels. This approach transforms raw pixel values ​​into more discriminative relational features, providing extremely effective information for the model to learn the intrinsic properties of ingredients.

[0061] Next, the covariance matrix is ​​directly converted into a single-channel two-dimensional image. Specifically, the element value at each position in the covariance matrix is ​​normalized and directly used as the pixel intensity value of the two-dimensional image at the coordinates, thus generating a grayscale image with the same dimensions as the covariance matrix. The generated image has a regular structure and can be directly used as input to a standard two-dimensional convolutional neural network.

[0062] This step transforms a massive dataset that depends on the spatial dimensions of the original image into a fixed-size matrix independent of the original image size. It no longer preserves the original spatial information but explicitly encodes the second-order statistical relationships between all input channel pairs. For example, a value in the matrix might encode a strong negative correlation between "increased red channel intensity" and "decreased intensity in a near-infrared band associated with fat absorption." These correlation patterns between channels are themselves extremely powerful discriminative features for distinguishing different food properties.

[0063] By converting the covariance matrix into a two-dimensional representation image, this invention cleverly solves the technical challenge of inputting high-dimensional heterogeneous data with hundreds of channels into standard artificial intelligence models. By directly mapping the covariance matrix to a fixed-size two-dimensional grayscale image, this invention transforms a complex and computationally expensive high-dimensional data processing problem into a more manageable problem of finding patterns in two-dimensional images. This allows for efficient analysis using mature and powerful two-dimensional convolutional neural networks, significantly reducing model complexity and computational overhead.

[0064] Furthermore, the two-dimensional representation image is input into a multi-task intelligent model, and shared feature representations are extracted through a shared feature extraction layer within the model. The feature extraction module of the corresponding system specifically includes:

[0065] The core analysis engine of this invention is a deep intelligent model based on multi-task learning. This model receives a two-dimensional covariance representation image generated by the previous module and, through a shared and fused internal structure, ultimately outputs predictions for multiple dimensions of the ingredients in parallel. The two-dimensional covariance representation image is first fed into a shared convolutional neural feature extraction network, designed to learn and extract a hierarchical feature representation from the input covariance image.

[0066] A ResNet-based architecture is employed as the shared feature extractor. This network consists of multiple stacked residual blocks, each containing convolutional layers, batch normalization layers, ReLU activation functions, and pooling layers. The convolutional filters in the network learn to automatically identify local and global patterns in the covariance image. These patterns essentially correspond to complex interrelationships between different data channels; for example, a specific texture pattern in the covariance image might indicate a combination of high fat content and a specific species. The final output of this shared network is a high-dimensional feature map, which serves as the "shared feature representation" in this invention. Using CNNs to analyze hyperspectral correlated data has proven effective.

[0067] By allowing multiple analysis tasks to share a single feature extraction network, the model can learn low-level feature representations that are universal across all tasks. This hard parameter sharing mechanism not only significantly reduces the total number of model parameters and improves training efficiency, but also enables the model to have better generalization ability through joint learning across multiple tasks. That is, learning from one task can provide useful constraints and information for other related tasks, thereby improving overall performance.

[0068] Furthermore, the shared feature representation is processed through the attention bottleneck fusion mechanism within the model to generate a collaborative feature representation. This fusion mechanism interacts across attention levels with the shared feature representation using a small set of learnable bottleneck tokens. Specifically, the feature fusion module of the corresponding system includes:

[0069] This module receives high-dimensional shared feature representations from a shared feature extraction network and refines them into a more compact, information-dense collaborative feature representation. This process is achieved through an attention bottleneck mechanism that leverages attention to efficiently process and compress large-scale input information.

[0070] First, a small set of fixed-size, learnable latent vectors, called "bottleneck tokens," is initialized. For example, N = 32 bottleneck tokens can be set, each token being a vector of dimension D = 512. These tokens are the model's hyperparameters, and their number is much smaller than the dimension of the shared feature representation. A cross-attention operation is then performed. In this operation, the 32 bottleneck tokens act as queries, while the shared feature representation obtained in the previous step, after being flattened in the spatial dimension, generates keys and values ​​through two independent linear projection layers.

[0071] The model calculates the dot product similarity between the query and the key, converting it into attention weights using the Softmax function. These weights are then used to perform a weighted summation of the values. This process forces each bottleneck token to focus on or query the most critical and relevant information fragments in the entire shared feature map. Ultimately, the outputs of the 32 tokens are integrated to form a compact tensor, which is the collaborative feature representation. This tensor effectively distills and compresses the most essential information required for all subsequent tasks from the vast shared features, achieving deep collaboration between different analytical dimensions. This overcomes the limitations of existing technologies where analytical dimensions are isolated and unable to form synergistic effects. By introducing learnable "bottleneck tokens" to actively query and extract core information from the shared feature network through cross-attention, this mechanism forces the model to learn how to optimally combine general features to serve all downstream tasks. Ultimately, it generates a unified collaborative feature representation with information efficiency far exceeding the simple addition of the parts, thus truly realizing mutual gains between analytical tasks and significantly improving the analytical accuracy and generalization ability of the entire system.

[0072] This attention bottleneck mechanism is the core technology for achieving synergy among various analytical dimensions. Compared to simple hard parameter sharing, this mechanism provides a more advanced soft parameter sharing approach. Hard parameter sharing can lead to performance degradation when tasks vary significantly. The attention bottleneck mechanism, however, allows the shared CNN to learn common features, while the learnable bottleneck tokens learn how to selectively combine these common features in a way that benefits all downstream tasks. This approach decouples the depth of the backbone network from the size of the input data and allows for flexible querying of the latent space. Therefore, this is not merely parameter sharing, but an active, learnable fusion process that creates a unified representation optimized for multi-task objectives, with information efficiency far exceeding the simple summation of the parts.

[0073] The attention bottleneck fusion mechanism offers a more advanced and intelligent feature fusion approach than simple feature sharing. This mechanism actively queries and extracts key information from shared features using a small set of learnable bottleneck tokens, forming a compact and highly information-dense collaborative feature representation. This "attention bottleneck" forces the model to learn how to optimally combine general features to serve all downstream tasks, thus truly achieving synergy between different analytical dimensions and generating a unified representation with information efficiency far exceeding the simple addition of individual parts.

[0074] Furthermore, based on the collaborative feature representation, the system predicts and outputs multi-dimensional attributes of the food sample, including food type, freshness index, and nutritional components. The corresponding system analysis output module specifically includes:

[0075] The collaborative feature representations generated in the previous step are simultaneously fed into multiple parallel, structurally independent prediction heads. Using a multi-task learning framework, where a single model solves multiple related problems at once, improves learning efficiency and the model's generalization ability.

[0076] By using a parallel, multi-task prediction output approach, the system can output multiple different types of analysis results simultaneously and in parallel, based on the same collaborative feature representation. This architecture not only achieves efficient end-to-end analysis but also ensures that all output results are derived from a deeply fused, synergistic unified feature, guaranteeing the consistency and intrinsic correlation of results across various analysis dimensions. Ultimately, it provides users with a comprehensive, reliable, and timely food evaluation report.

[0077] Example 2

[0078] A large food processing company wants to use this system to conduct quality analysis on a batch of newly harvested tomatoes, aiming to achieve rapid, non-destructive, and multi-dimensional automated analysis of the variety, maturity, and key nutritional components of large quantities of tomatoes.

[0079] First, the system's data acquisition stage is deployed on an automated production line. Tomato samples to be analyzed move at a constant speed on a conveyor belt, passing sequentially through an integrated multimodal imaging station. This station is equipped with a standard RGB industrial camera and a hyperspectral camera. The RGB camera, with its high resolution and global shutter capabilities, captures a visual image of each tomato, clearly recording its shape, size, surface color, and physical texture information such as bruises or scars. A pushbroom hyperspectral camera covering the 400-1000 nm spectral range simultaneously scans the same tomato, generating a hyperspectral data cube. This data cube contains reflectance information of the sample across hundreds of consecutive narrow spectral bands, directly correlated with chemical components determining its intrinsic quality, such as lycopene, soluble solids, and chlorophyll. The entire acquisition process is conducted under uniform, shadowless lighting to ensure the stability and accuracy of the data quality.

[0080] The acquired raw data is then fed into the image processing module. To eliminate noise caused by the physical properties of the sample surface and variations in illumination, the system applies preprocessing algorithms such as standard normal variable transformation to correct the hyperspectral data. Subsequently, a spatial alignment step is performed, employing a scale-invariant feature transformation algorithm to automatically detect and match stable common feature points in representative band images of the RGB and hyperspectral data, such as unique spots or fruit stem contours on the sample surface. Based on these matching points, the system calculates an accurate geometric transformation matrix and applies this matrix to transform one of the data sources, ensuring that every pixel from the two heterogeneous data sources corresponds precisely in physical space. Simultaneously, the system applies data augmentation operations with the same parameters, such as random rotation and scaling, to the aligned pairs of data to enhance subsequent models.

[0081] Next, the system enters the innovative representation generation module. For each spatially aligned pixel in the image, the system concatenates the values ​​of its three RGB color channels with the values ​​of all hyperspectral bands to form a single high-dimensional feature vector. Stacking the feature vectors of all pixels in the image forms a large observation matrix. The system explicitly encodes the linear correlation information between different data channels by calculating the covariance matrix of this observation matrix. This covariance matrix is ​​then directly converted into a two-dimensional single-channel grayscale image, where the value of each element in the matrix corresponds to the intensity value of a pixel in the image. In this way, an abstract multivariate correlation analysis problem is cleverly transformed into an image pattern recognition problem in the field of computer vision. The generated two-dimensional representation image has extremely high information density, and its internal texture and patterns intuitively reflect the intrinsic relationship between the color, texture, and internal chemical composition of the food, making it very suitable as input for subsequent deep learning models.

[0082] The two-dimensional representation image is fed into a feature extraction module, the core of which is a shared feature extraction network based on a convolutional neural network. This network slides its learnable filters across the representation image through multiple layers of convolution and pooling operations to automatically identify and extract discriminative relevance patterns. These patterns are deep, abstract features valuable for all downstream analysis tasks. The network design follows a multi-task learning paradigm, allowing multiple related tasks to share a common underlying network, thereby achieving knowledge transfer and improving model efficiency. The network ultimately outputs a set of shared feature representations that highly summarize the common patterns learned from the original multimodal data.

[0083] The shared feature representation is then passed to a feature fusion module that employs an attention bottleneck fusion mechanism. This mechanism introduces a small set of fixed-size, learnable latent vectors, known as "bottleneck tokens." These tokens act as queries, engaging in cross-attention interactions with the high-dimensional shared feature representation. They proactively query and aggregate the information most relevant to the final task from the shared features. This approach not only efficiently compresses and refines high-dimensional features into a more focused and structured collaborative feature representation but also significantly reduces the enormous computational burden of traditional self-attention mechanisms.

[0084] Finally, in the analysis output module, this highly condensed collaborative feature representation is simultaneously fed into multiple parallel, independent prediction heads. Each prediction head is a simple network designed for a specific task. The classification head for variety identification uses a Softmax activation function at its terminal, outputting the probability that a sample belongs to different varieties such as "cherry tomato" or "Roman tomato." A regression head for maturity assessment uses linear activation at its terminal, outputting a continuous maturity index value. Another regression head also uses linear activation to predict the specific content of lycopene or soluble solids. This architecture allows all dimensions of analysis to be completed in parallel at once, greatly improving processing efficiency. Based on these high-precision, multi-dimensional analysis results, the system automatically generates a comprehensive quality report and triggers downstream sorting devices to automatically grade tomatoes into high-quality products for premium fresh consumption, sub-premium products for processing, or low-quality products for other uses, thereby achieving intelligent and automated quality control and grading processes.

[0085] Example 3

[0086] This invention discloses the specific application of the AI-based multi-dimensional intelligent analysis system for food ingredients in a large-scale automated pork processing enterprise. This application aims to perform automated, non-destructive, multi-dimensional comprehensive quality assessment of chilled pork tenderloin chunks moving along a high-speed processing line, replacing traditional grading methods that rely on manual labor, are highly subjective, and lack consistency. This achieves refined and data-driven management of pork quality and provides reliable technical support for downstream differentiated pricing, product diversion, and supply chain traceability.

[0087] First, the data acquisition module is executed. This module is integrated into a fixed imaging station above the automated conveyor belt in the processing plant. When a pork tenderloin sample enters the imaging area via the conveyor belt, the system simultaneously activates two different modal sensors for data acquisition. Specifically, an industrial-grade high-resolution RGB camera captures a visual image containing the spatial and color information of the pork tenderloin sample. This camera preferably uses a 5-megapixel sensor with a global shutter and is equipped with a standardized D65 light source to ensure no image blurring during high-speed movement and high accuracy and batch-to-batch consistency of color information. The acquired visual image clearly records the macroscopic physical characteristics of the tenderloin, including the distribution pattern and abundance of intramuscular fat, meat color, overall dimensions, and the presence of surface defects such as blood spots and bruises. Simultaneously, a pushbroom hyperspectral camera acquires a hyperspectral data cube containing spectral information of the pork tenderloin sample across multiple spectral bands. The hyperspectral camera is set to operate in a band covering 400 nanometers to 1000 nanometers. By acquiring hundreds of consecutive narrowband spectral data within this range, the system can obtain detailed information reflecting the internal chemical composition of the sample.

[0088] Next, the system enters the image processing module to preprocess the acquired raw data. A specular reflection correction algorithm is applied to the RGB visual image to eliminate localized highlight areas caused by the moisture on the pork tenderloin surface, ensuring the accuracy of color and texture analysis. Simultaneously, for the hyperspectral data cube, the system applies a standard normal variable transformation to correct spectral baseline drift caused by irregular scattering from the sample surface, and combines this with a Savitzky-Golay smoothing filter to remove sensor noise and enhance the signal-to-noise ratio of the characteristic spectral signals. Subsequently, the system employs a scale-invariant feature transformation (SIN) feature point detection algorithm to identify stable and unique keypoints in a representative band image of both the RGB image and the hyperspectral data cube. Then, using a random sample consensus algorithm, a perspective transformation matrix describing the spatial mapping relationship between the two data sources is robustly calculated based on successfully matched keypoint pairs. This matrix is ​​applied to one of the data sources for geometric correction, ensuring that in subsequent processing, any pixel in the RGB image and all its corresponding hyperspectral data originate precisely from the same point on the pork tenderloin sample. The accuracy of this step is the fundamental prerequisite for the effective fusion of multimodal information. Any spatial misalignment will lead to meaningless spurious correlations in subsequent covariance analysis, thus rendering the analysis results of the entire system invalid.

[0089] Subsequently, the system executes the representation generation module. For each precisely spatially aligned pixel location, the system concatenates the intensity values ​​of the R, G, and B channels of the RGB image at that location with the intensity values ​​of all hundreds of spectral bands in the hyperspectral data cube, forming a single high-dimensional feature vector. Stacking the feature vectors of all pixel locations in the sample image along one dimension creates a massive observation matrix. Based on this observation matrix, the system calculates the pairwise covariance between data channels and generates a covariance matrix. Each element of this matrix quantifies the linear correlation strength between any two data channels. Finally, the system directly converts this covariance matrix into a two-dimensional single-channel grayscale image, where the elements of the covariance matrix at the coordinates, after normalization, are directly mapped to the pixel intensity values ​​of the two-dimensional representation image at those coordinates. This step cleverly transforms a high-dimensional, heterogeneous data fusion problem into a classic problem of pattern recognition in a two-dimensional image. The generated two-dimensional representation image has a fixed size and is independent of the size and shape of the original pork tenderloin sample. Its internal texture and structural patterns are no longer a reproduction of physical space, but an abstraction and highly condensed visual encoding of the complex internal relationship between the color, texture and internal chemical composition of pork.

[0090] The two-dimensional representation image is then input into the feature extraction module. The core of this module is a shared feature extraction network based on a deep convolutional neural network, employing the ResNet architecture. This network processes the input covariance representation image through its stacked convolutional layers, activation function layers, and pooling layers. The learnable convolutional kernels in the network can automatically identify and extract discriminative local and global patterns in the image. These patterns essentially correspond to complex interrelationships between different data channels. For example, a specific texture pattern in the covariance map might uniquely identify the synergistic relationship between high intramuscular fat content and excellent water retention, while another pattern might represent the color and moisture abnormality characteristic of pale, soft, and exudative pork. By allowing all downstream analysis tasks, such as intramuscular fat content prediction, freshness assessment, and PSE defect classification, to share this feature extraction network, the model can discover universal low-level feature representations beneficial to all tasks during the learning process. This significantly reduces the number of model parameters, improves training efficiency, and, through joint optimization across multiple tasks, promotes knowledge transfer between different tasks, enhancing the model's generalization ability.

[0091] The high-dimensional shared feature representation output by the shared feature extraction network is passed to the feature fusion module, which processes the shared feature representation through an advanced attention bottleneck fusion mechanism. First, a small set of fixed-size, learnable latent vectors, called bottleneck tokens, is initialized. In this embodiment, 32 bottleneck tokens can be set. These tokens act as queries, interacting with keys and values ​​generated by the linear projection of the shared feature representation in a cross-attention operation. Essentially, each bottleneck token learns an optimal query strategy, actively querying the vast shared feature space and aggregating the most critical and relevant information fragments for all downstream tasks. For example, one token might learn to specifically extract feature combinations related to fat content and distribution patterns; another token might focus on querying features related to myoglobin oxidation state and water activity. This mechanism forces the model to actively and selectively extract and reorganize information from the shared feature pool, rather than passively sharing all features. The outputs of all bottleneck tokens are ultimately integrated to form a compact but highly information-dense collaborative feature representation. This collaborative feature representation is not a simple addition of the information from each part, but a unified representation that has been generated through deep fusion and negotiation and is optimized specifically to solve all current analysis tasks. Its information efficiency far exceeds that of the simple summation of each part, thus truly realizing the mutual benefit between each analysis task and significantly improving the analysis accuracy of the entire system.

[0092] Finally, in the analysis output module, the collaborative feature representation generated by the attention bottleneck mechanism is simultaneously fed into multiple parallel, structurally independent prediction heads. Each prediction head is a lightweight network designed for a specific task. Specifically, one prediction head performs a regression task, outputting a continuous value representing the intramuscular fat percentage of the pork tenderloin sample. A second prediction head also performs a regression task, outputting a comprehensive freshness index that correlates highly with traditional physicochemical indicators such as total bacterial count or volatile basic nitrogen. A third prediction head performs a multi-classification task, classifying the sample into different commercial grades, such as "premium," "grade 1," and "grade 2," based on comprehensive indicators such as intramuscular fat content and meat color. A fourth prediction head performs a binary classification task, specifically designed to determine whether the sample has PSE (Polymer Sedimentary Surface Area) quality defects.

[0093] Through this parallel, multi-task prediction output structure, the system can output evaluation results for multiple key dimensions of pork quality in a single, parallel manner, based on the same deeply collaboratively fused feature representation. This end-to-end processing approach ensures the inherent consistency and correlation of the various output results, providing enterprises with comprehensive, reliable, and timely pork quality assessment reports, and laying a solid technical foundation for achieving intelligent production and quality control.

[0094] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multi-dimensional intelligent analysis system for food ingredients based on artificial intelligence, characterized in that: include: A visual image containing spatial and color information of the food sample to be analyzed and a hyperspectral data cube containing spectral information of the food sample in multiple spectral bands are acquired, and preprocessing operations including image enhancement and spatial alignment are performed on them. The visual image is transformed into a high-dimensional feature vector, and the feature vectors of all pixels are stacked to construct an observation matrix. After centering the observation matrix, its product with its transpose is calculated, and then normalized to generate a covariance matrix. The construction of the observation matrix and the calculation of the covariance matrix include: for each spatially aligned pixel position, the values ​​of the R, G, and B color channels of the visual image are concatenated with the values ​​of all spectral band channels of the hyperspectral data cube to form a single feature vector, and the feature vectors of all pixel positions are stacked to form an observation matrix. The mean of each column of the observation matrix is ​​calculated, and the corresponding mean is subtracted from each column to achieve data centralization. The product of the centered observation matrix with its transpose is calculated and normalized to obtain the covariance matrix. Based on the covariance matrix, each element value is mapped to the pixel intensity value of the image to obtain a two-dimensional representation image. The two-dimensional representation image is input into a multi-task intelligent model, and convolution is performed through a learnable filter in the network. The most salient features are retained through a shared feature extraction layer in the model to generate a shared feature representation. The shared feature representation is processed by the attention bottleneck fusion mechanism within the model to generate a collaborative feature representation. The fusion mechanism interacts with the shared feature representation across attention through a learnable bottleneck token. Based on the collaborative feature representation, the multi-dimensional attributes of the food sample, including food type, freshness index, and nutritional components, are predicted and output. The specific implementation process of prediction and output based on the collaborative feature representation includes feeding the collaborative feature representation into multiple parallel and independent prediction heads simultaneously. One prediction head is used to perform a classification task to output the food type, another prediction head is used to perform a regression task to output the freshness index, and yet another prediction head is used to perform a regression task to output the nutritional components, thereby realizing parallel prediction and output of multiple tasks.

2. The artificial intelligence-based multi-dimensional intelligent analysis system for food ingredients according to claim 1, characterized in that, The process of acquiring visual images and hyperspectral data cubes includes: capturing the visual images using a standard RGB sensor to record the texture, shape, and surface color information of the food sample; and sequentially capturing the hyperspectral data cubes in multiple continuous narrow bands covering the visible to near-infrared spectrum using a line scan or snapshot hyperspectral camera to record the chemical composition information of the food sample.

3. The artificial intelligence-based multi-dimensional intelligent analysis system for food ingredients according to claim 1, characterized in that, The specific implementation process of image preprocessing includes: applying image enhancement algorithms to the visual image and the hyperspectral data cube respectively, identifying common feature points in the two data sources to calculate the transformation matrix, spatially aligning the visual image and the hyperspectral data cube, and performing data augmentation on the aligned data through geometric transformation.

4. The artificial intelligence-based multi-dimensional intelligent analysis system for food ingredients according to claim 1, characterized in that, The specific implementation process of converting the covariance matrix into a two-dimensional representation image includes: normalizing and mapping each element value of the covariance matrix as the intensity value of a pixel; mapping the element value of the position of the covariance matrix to the pixel value of the two-dimensional representation image at the coordinates, generating a single-channel grayscale image with the same size as the dimension of the covariance matrix, and making it suitable as the input of a convolutional neural network.

5. The artificial intelligence-based multi-dimensional intelligent analysis system for food ingredients according to claim 1, characterized in that, The process of extracting shared feature representations in a multi-task intelligent model includes: inputting the two-dimensional representation image into a shared feature extraction network consisting of multiple convolutional layers, activation function layers, and pooling layers; performing convolution operations on the two-dimensional representation image through learnable filters in the convolutional layers to identify correlation patterns encoded in the image; applying a nonlinear activation function after each convolutional layer to introduce nonlinearity, and downsampling the feature map through the pooling layers to reduce spatial dimensionality and retain the most salient features, wherein the final output of the shared feature extraction network is the shared feature representation.

6. The artificial intelligence-based multi-dimensional intelligent analysis system for food ingredients according to claim 1, characterized in that, The specific implementation process of processing the shared feature representation includes: initializing a small set of fixed-size, learnable latent vectors as bottleneck tokens; in a cross-attention operation, using the bottleneck tokens as queries and linearly projecting the shared feature representation as keys and values; and obtaining attention weights by calculating the dot product of the query and the key and applying the Softmax function to perform weighted summation of the values ​​to form the collaborative feature representation.

Images