Food adulteration detection method, model training method, system and related products

Abstract

The invention discloses a food adulteration detection method, a model training method, a system and a related product, and relates to the technical field of food detection. The training method of the food adulteration detection model comprises the steps of obtaining a multispectral image of sample food in a training data set, a corresponding real hyperspectral image and an adulteration category label, inputting the multispectral image into a spectrum reconstruction model to obtain a reconstructed hyperspectral image, determining reconstruction loss based on the reconstructed hyperspectral image and the real hyperspectral image, and determining the adulteration category label according to the reconstruction loss. And determining a contrast loss based on the reconstructed hyperspectral image and the hyperspectral image of the negative sample, the hyperspectral image of the negative sample being determined through an adulteration category label, optimizing parameters of a spectral reconstruction model based on the reconstruction loss and the contrast loss to complete training, and inputting the reconstructed hyperspectral image into a classification model to train the classification model. By executing the operation, the food adulteration detection cost is reduced, and the food adulteration detection accuracy based on low-cost equipment is improved.

Description

Technical Field

[0001] This application relates to the field of food testing technology, and in particular to a method for detecting food adulteration, a model training method, a system, and related products. Background Technology

[0002] Food adulteration is a growing concern with significant health and economic impacts. Adulteration includes substituting ingredients of similar origin, adding water, and using illegal additives, all of which can pose health risks such as foodborne illnesses, allergic reactions, cancer, and kidney damage. Furthermore, reports indicate that end-points in the supply chain, such as restaurants and retail outlets, are among the most vulnerable victims of food fraud.

[0003] However, food adulteration detection methods in related technologies are constrained by factors such as high costs, limited detection capabilities, or time-consuming sample preparation, making it difficult to strike a balance between high detection capabilities and reasonable system costs. This makes these methods only suitable for laboratory testing and unable to guarantee the safety of end users. Summary of the Invention

[0004] The main objective of this application is to provide a food adulteration detection method, model training method, system, and related products, aiming to improve the detection accuracy of low-cost food adulteration detection equipment.

[0005] To achieve the above objectives, one aspect of this application proposes a training method for a food adulteration detection model, wherein the food adulteration detection model includes a spectral reconstruction model and a classification model, and the training of the spectral reconstruction model includes the following steps:

[0006] Obtain multispectral images of sample food products in the training dataset, the corresponding real hyperspectral images, and adulteration category labels;

[0007] The multispectral image is input into the spectral reconstruction model to obtain the reconstructed hyperspectral image;

[0008] The reconstruction loss is determined based on the reconstructed hyperspectral image and the real hyperspectral image, and the contrast loss is determined based on the reconstructed hyperspectral image and the hyperspectral image of the negative sample, wherein the hyperspectral image of the negative sample is determined by the adulteration category label;

[0009] The parameters of the spectral reconstruction model are optimized based on the reconstruction loss and the contrast loss to complete the training;

[0010] The reconstructed hyperspectral image is input into the classification model to train the classification model.

[0011] In some embodiments, the training dataset is generated by one or any combination of the following:

[0012] Eliminate ambient light;

[0013] Multispectral and hyperspectral images of the food samples were collected at the same location and under the same conditions.

[0014] Median filtering is performed on the acquired multispectral and hyperspectral images of the sample food, and the region of interest is extracted.

[0015] Spatially align the multispectral image and the hyperspectral image;

[0016] The central regions of the multispectral image and the hyperspectral image are extracted to generate the training dataset.

[0017] In some embodiments, determining the contrast loss based on the reconstructed hyperspectral image and the hyperspectral image of the negative sample includes:

[0018] Negative samples of the sample food are selected by filtering out the adulteration category label. The negative samples are used to characterize samples that are different from the adulteration category of the sample food.

[0019] A contrast loss function is determined based on the reconstructed hyperspectral image and the hyperspectral image of the negative sample;

[0020] The contrast loss is determined based on the contrast loss function.

[0021] To achieve the above objectives, another aspect of this application proposes a training method for a food adulteration detection model. The food adulteration detection model includes a spectral reconstruction model and a classification model. The classification model includes a Student feature extraction model. The training of the classification model includes the following steps:

[0022] The reconstructed hyperspectral image is input into the student feature extraction model to obtain basic features, wherein the reconstructed hyperspectral image is generated by the spectral reconstruction model;

[0023] The real hyperspectral image corresponding to the reconstructed hyperspectral image is input into the teacher feature extraction model and the teacher feature extraction model is trained to obtain teacher features;

[0024] Distillation loss is determined using the aforementioned basic characteristics and the aforementioned teacher characteristics;

[0025] The parameters of the classification model are optimized based on the distillation loss to complete the training.

[0026] In some embodiments, the classification model further includes a visible light feature extraction model, and the training of the classification model further includes the following steps:

[0027] The visible light portion of the reconstructed hyperspectral image is input into the visible light feature extraction model to obtain visible light features;

[0028] The basic features and the visible light features are spliced ​​together to obtain the spliced ​​features;

[0029] The type of adulteration can be predicted by splicing features;

[0030] The classification loss is determined based on the predicted adulteration category and the adulteration category label;

[0031] The parameters of the classification model are optimized based on the distillation loss and the classification loss to complete the training.

[0032] In some embodiments, determining the classification loss based on the predicted adulteration category and the adulteration category label includes:

[0033] The average reconstruction error for determining adulteration categories;

[0034] The classification loss is determined based on the average reconstruction error, the predicted adulteration category, and the adulteration category label.

[0035] To achieve the above objectives, another aspect of this application provides a method for detecting food adulteration, the method comprising:

[0036] Acquire multispectral images of the food sample to be tested;

[0037] The multispectral image is input into the food adulteration detection model to obtain the adulteration category of the food sample to be tested;

[0038] Output the adulteration category of the food sample to be tested;

[0039] The food adulteration detection model includes a spectral reconstruction model and a classification model. The spectral reconstruction model is trained using the same training method as the spectral reconstruction model in the food adulteration detection model. The classification model is trained using the same training method as the classification model in the food adulteration detection model.

[0040] To achieve the above objectives, another aspect of this application proposes a food adulteration detection system, the system comprising:

[0041] The multispectral image acquisition module is used to acquire multispectral images of the food sample to be tested.

[0042] A spectral reconstruction module is used to reconstruct a hyperspectral image from the multispectral image;

[0043] A classification module is used to predict and output the adulteration category of the food sample to be tested based on the reconstructed hyperspectral image;

[0044] The spectral reconstruction module is trained using the training method of the spectral reconstruction model in the aforementioned food adulteration detection model, and the classification module is trained using the training method of the classification model in the aforementioned food adulteration detection model.

[0045] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0046] To achieve the above objectives, another aspect of this application provides a computer program product that, when run in an electronic device, causes the electronic device to perform the above-described method.

[0047] The embodiments of this application include at least the following beneficial effects:

[0048] This application provides a food adulteration detection method, model training method, system, and related products. The solution acquires multispectral images of sample food, corresponding real hyperspectral images, and adulteration category labels from the training dataset. The multispectral images are input into a spectral reconstruction model to obtain reconstructed hyperspectral images. The reconstruction loss is determined based on the reconstructed hyperspectral images and the real hyperspectral images. The contrast loss is determined based on the reconstructed hyperspectral images and the hyperspectral images of negative samples. The hyperspectral images of negative samples are determined by the adulteration category labels. The parameters of the spectral reconstruction model are optimized based on the reconstruction loss and the contrast loss to complete the training. The reconstructed hyperspectral images are then input into a classification model to train the classification model. Therefore, the technical solution provided in this application reconstructs hyperspectral images based on multispectral images of food using a spectral reconstruction model, which can reduce the high cost of hyperspectral acquisition equipment and thus reduce the cost of food adulteration detection. By optimizing the parameters of the spectral reconstruction model using the reconstruction loss determined by the reconstructed hyperspectral image and the real hyperspectral image, the contrast loss determined by the reconstructed hyperspectral image and the hyperspectral image of the negative sample, and the adulteration category label introduced during the reconstruction training stage, the differences between reconstructed samples of different adulteration categories can be increased, thereby improving the detection accuracy based on low-cost food adulteration equipment. Inputting the reconstructed hyperspectral image into the classification model for training will further improve the detection accuracy based on low-cost food adulteration equipment.

[0049] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0050] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0051] Figure 1 This is a flowchart of a training method for a food adulteration detection model provided in some embodiments of this application;

[0052] Figure 2 This is a flowchart of a training method for a spectral reconstruction model provided in some embodiments of this application;

[0053] Figure 3 This is a flowchart of another training method for the food adulteration detection model provided in some embodiments of this application;

[0054] Figure 4 These are schematic diagrams showing the comparison of the original hyperspectral and reconstructed hyperspectral at different spectral bands provided in some embodiments of this application;

[0055] Figure 5 This is a flowchart of a training method for a classification model provided in some embodiments of this application;

[0056] Figure 6 This is a flowchart of another training method for the food adulteration detection model provided in some embodiments of this application;

[0057] Figure 7 This is a flowchart of a food adulteration detection method provided in some embodiments of this application;

[0058] Figure 8 This is another flowchart of a food adulteration detection method provided in some embodiments of this application;

[0059] Figure 9 This is a comparison chart of the accuracy and F1 score of various food adulteration detection methods provided in some embodiments of this application;

[0060] Figure 10 These are schematic block diagrams of a food adulteration detection system provided in some embodiments of this application;

[0061] Figure 11 These are schematic diagrams of the hardware structure of electronic devices provided in some embodiments of this application. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the reference to "embodiment" herein means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the embodiments of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0063] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0064] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0066] To facilitate understanding of the inventive concept of this application, before providing a detailed description of the embodiments of this application, the English abbreviations (terms) / related concepts involved in the embodiments of this application will be explained first. The English abbreviations (terms) / related concepts involved in the embodiments of this application are subject to the following interpretation.

[0067] SOTA stands for state-of-the-art, meaning "state-of-the-art; cutting-edge; first-class." It is usually used to describe the latest and best technologies, equipment, methods, products, etc. in a certain field, representing the highest level and development trend in the field.

[0068] SR stands for spectral reconstruction.

[0069] State-of-the-art (SOTA) Spectral Reconstruction: In the field of spectral reconstruction, SOTA spectral reconstruction represents the state-of-the-art techniques, methods, or models currently employed in converting low-spectral-resolution images (such as common RGB images) to high-spectral-resolution images, or in recovering spectral information from other relevant data. With the development of technologies such as deep learning, many SOTA spectral reconstruction methods have emerged, including models based on deep convolutional neural networks, generative adversarial networks, and attention mechanisms. These methods have achieved significant results in terms of accuracy and efficiency in spectral reconstruction.

[0070] MSI stands for Multispectral Imaging, which refers to multispectral imaging.

[0071] HSI stands for Hyperspectral Imaging, which refers to hyperspectral imaging.

[0072] Food adulteration is a growing concern with significant health and economic implications. Adulteration includes substituting ingredients with similar components, adding water, and using illegal additives. These practices can pose health risks such as foodborne illnesses, allergic reactions, cancer, and kidney damage. Furthermore, reports indicate that end-points in the supply chain, such as restaurants and retail outlets, are among the most vulnerable victims of food fraud.

[0073] However, existing solutions fail to meet this need. Because adulterated and unadulterated food samples are extremely similar in composition, high detection capabilities are required. Most methods struggle to balance high detection capability with reasonable system cost. Standard detection methods are expensive and require specialized laboratory environments. While hyperspectral detection schemes can yield accurate results without sample preparation, they require complex and costly equipment. Conversely, some low-cost solutions, such as electronic noses or image classification, have limited ability to detect deeper chemical characteristics in food and may miss the most harmful adulteration issues, such as veterinary drug or antibiotic residues in meat adulteration. Currently, there is no consumer-grade system capable of detecting all types of food adulteration.

[0074] Among related technologies, there are methods for detecting food adulteration using spectral reconstruction algorithms. However, implementing a low-cost method for detecting food adulteration using spectral reconstruction algorithms is not easy. It faces the following challenges:

[0075] (1) High similarity of multispectral images. The similarity between genuine and adulterated samples, especially those with low concentrations of adulterants, produces similar spectral properties, making adulteration detection more complex. Low-cost multispectral cameras exacerbate this problem due to their coarse-grained spectral resolution. However, existing spectral reconstruction algorithms are unable to reconstruct distinguishable results between two samples with similar or even identical multispectral features.

[0076] (2) Full-band reconstruction error. Since the spectral absorption characteristics of different doping types are distributed across different wavelength ranges, it is crucial for the system to cover a sufficiently wide spectral range (typically 400-1000 nm). However, no low-cost multispectral device can cover such a wide wavelength range. When existing spectral reconstruction algorithms are applied for full-band (i.e., 400-1000 nm) reconstruction, errors and noise in the reconstructed data will affect its usability in doping detection.

[0077] (3) Lack of spectral reconstruction datasets. Spectral reconstruction algorithms require paired (multispectral image, hyperspectral image) data for training. However, obtaining perfectly paired data is challenging. Due to the different fields of view (FOV) and focal lengths of spectral cameras, the system needs to be used at specific fixed distances and angles to cover the same scene, which is impractical in real-world applications. Previous methods used publicly available hyperspectral image (HSI) datasets and downsampled the hyperspectral images as paired multispectral images (MSI) for training, but this approach may result in different distributions of training and testing data, thus reducing performance in practical applications.

[0078] In view of this, this application proposes a food adulteration detection method, model training method, system, and related products. In the embodiments of this application, multispectral images of sample food, corresponding real hyperspectral images, and adulteration category labels are obtained from the training dataset. The multispectral images are input into the spectral reconstruction model to obtain reconstructed hyperspectral images. The reconstruction loss is determined based on the reconstructed hyperspectral images and the real hyperspectral images. The contrast loss is determined based on the reconstructed hyperspectral images and the hyperspectral images of negative samples. The hyperspectral images of negative samples are determined by the adulteration category labels. The parameters of the spectral reconstruction model are optimized based on the reconstruction loss and the contrast loss to complete the training. The reconstructed hyperspectral images are input into the classification model to train the classification model. Therefore, the technical solution provided in this application reconstructs hyperspectral images based on multispectral images of food using a spectral reconstruction model, which can reduce the high cost of hyperspectral acquisition equipment and thus reduce the cost of food adulteration detection. By optimizing the parameters of the spectral reconstruction model using the reconstruction loss determined by the reconstructed hyperspectral image and the real hyperspectral image, the contrast loss determined by the reconstructed hyperspectral image and the hyperspectral image of the negative sample, and the adulteration category label introduced during the reconstruction training stage, the differences between reconstructed samples of different adulteration categories can be increased, thereby improving the detection accuracy of low-cost food adulteration detection equipment. Inputting the reconstructed hyperspectral image into the classification model for training will further improve the detection accuracy of low-cost food adulteration detection equipment.

[0079] The food adulteration detection method provided in this application relates to the field of food testing technology. It can be applied to the electronic device provided in this application. The electronic device can be a terminal or a server.

[0080] In some embodiments, the terminal may be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited thereto.

[0081] The server can be configured as a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network.

[0082] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0083] The implementation steps of the training method for the food adulteration detection model provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0084] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a training method for a food adulteration detection model provided in some embodiments of this application. The food adulteration detection model includes a spectral reconstruction model and a classification model. It should be noted that the steps shown in the flowchart can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0085] The training of the spectral reconstruction model in the embodiments of this application includes the following steps:

[0086] Step 101: Obtain the multispectral images of the sample food products in the training dataset, the corresponding real hyperspectral images, and the adulteration category labels;

[0087] Step 102: Input the multispectral image into the spectral reconstruction model to obtain the reconstructed hyperspectral image;

[0088] Step 103: Determine the reconstruction loss based on the reconstructed hyperspectral image and the real hyperspectral image, and determine the contrast loss based on the reconstructed hyperspectral image and the hyperspectral image of the negative sample. The hyperspectral image of the negative sample is determined by the adulteration category label.

[0089] Step 104: Optimize the parameters of the spectral reconstruction model based on reconstruction loss and contrast loss to complete training;

[0090] Step 105: Input the reconstructed hyperspectral image into the classification model to train the classification model.

[0091] Steps 101 to 105 as illustrated in this embodiment enable the reconstruction of hyperspectral images based on multispectral images of food using a spectral reconstruction model. This reduces the high cost associated with hyperspectral acquisition equipment, thereby lowering the cost of food adulteration detection. By optimizing the parameters of the spectral reconstruction model using the reconstruction loss determined by the reconstructed hyperspectral image and the real hyperspectral image, the contrast loss determined by the reconstructed hyperspectral image and the hyperspectral image of the negative sample, and the adulteration category labels introduced during the reconstruction training phase, the differences between reconstructed samples of different adulteration categories can be increased, improving the detection accuracy of low-cost food adulteration detection equipment. Inputting the reconstructed hyperspectral image into a classification model for training will further improve the detection accuracy of low-cost food adulteration detection equipment.

[0092] The specific implementation methods for each of the above steps are described below.

[0093] Before introducing the specific implementation of step 101, it is necessary to introduce the generation of the training dataset of the spectral reconstruction model provided in the embodiments of this application.

[0094] In some implementations, the training dataset is generated by one or any combination of the following: eliminating ambient light; acquiring multispectral images and hyperspectral images of the sample food at the same location and under the same environment; performing median filtering on the acquired multispectral and hyperspectral images of the sample food and extracting regions of interest; spatially aligning the multispectral and hyperspectral images; and extracting the central regions of the multispectral and hyperspectral images to generate the training dataset.

[0095] Spectral reconstruction algorithms in related technologies require training with paired (multispectral and hyperspectral) data sets from the same scene. This means they need to have identical spectral reflectance. However, acquiring perfectly paired data sets with the same spectral reflectance using multispectral and hyperspectral cameras is challenging. Because multispectral and hyperspectral cameras have different fields of view (FOV) and focal lengths, covering the same scene requires using them at specific fixed distances and angles, but this approach is difficult to implement in practice.

[0096] Existing methods partially utilize publicly available hyperspectral image (HSI) datasets, downsampling these HSI images to use as paired multispectral image (MSI) datasets for training. Others use known response curves of RGB cameras to simulate RGB data for training from HSI images. However, there is currently no publicly available library of multispectral image (MSI) device response curves, and it is also impossible to obtain known response curves for multispectral image devices. To address this, the applicant attempted to simulate several response curves using a Gaussian distribution. However, the simulations could not reproduce the acquisition process of multispectral image data, such as the response of the multispectral image detector and dark current noise, leading to differences in the distribution of training and testing data.

[0097] The different distributions of training and testing data result in poor generalization ability of the trained model in actual use, which affects the performance of the model and the user experience.

[0098] Therefore, this application provides a method for generating the dataset for training a food adulteration detection model, which can ensure the alignment of multispectral image (MSI) and hyperspectral image (HSI) data and solve the problems caused by ambient light influence, resolution differences and sample position changes during data collection.

[0099] In some embodiments, a method for generating a dataset suitable for training a food adulteration detection model may be:

[0100] Choose a multispectral camera or hyperspectral camera with low cost, suitable spectral range, resolution, and sensitivity. For example, a low-cost multispectral camera can cover the near-infrared region, but due to cost constraints, it may not simultaneously cover the visible light band. A hyperspectral camera, on the other hand, needs to cover a more continuous and finer spectral range, such as the 400-1000 nm wavelength range, and has a higher spectral resolution, such as below 10 nm.

[0101] Both cameras underwent rigorous calibration. This included radiometric calibration to ensure that the digital signal output by the camera accurately corresponds to the radiation intensity of the incident light; geometric calibration to correct image distortion caused by factors such as camera lens distortion; and spectral calibration to ensure the accuracy and stability of the spectral response.

[0102] Select a diverse range of food samples, including various types of foods such as fruits, vegetables, meats, and grains, to fully reflect the diversity and complexity of the food spectrum. For each food, select samples of different varieties, ripeness levels, and parts of the plant, such as different varieties of apples and bananas at different ripeness levels. Appropriately process the food samples, such as washing, cutting, and plating, to simulate the food's appearance in real-world application scenarios. However, care should be taken to avoid significantly affecting the food's spectral characteristics during processing; for example, prevent juice loss during cutting and avoid prolonged exposure to air to prevent oxidation.

[0103] Ambient light removal is crucial. In real-world scenarios, the uncontrollable intensity of ambient light can hinder the model from learning reliable reconstruction mappings. To overcome this, background subtraction can be performed by removing the spectral image illuminated solely by ambient light. This eliminates the influence of ambient light and improves the accuracy of subsequent analysis. Furthermore, ambient temperature and humidity can be controlled to prevent them from affecting the food's spectrum.

[0104] Subsequently, multispectral and hyperspectral images of the food samples were acquired at the same location and under the same conditions. Specifically, the food samples were placed in a suitable location and photographed using a hyperspectral camera to obtain hyperspectral image data. During the acquisition process, parameters such as exposure time and gain were adjusted according to the camera's performance and the characteristics of the food samples to ensure good image quality and avoid overexposure or underexposure. Simultaneously, the camera and environmental parameters at the time of acquisition were recorded. The multispectral camera was then used to photograph the food samples under the same location and lighting conditions to acquire multispectral image data. The spectral band settings of the multispectral camera could overlap to some extent with the spectral range of the hyperspectral camera to facilitate subsequent data fusion and matching.

[0105] Median filtering was applied to the acquired multispectral and hyperspectral images of the food samples to extract regions of interest (ROIs). Median filtering was used to reduce noise and improve the quality of the acquired multispectral and hyperspectral images. Furthermore, for food samples (e.g., meat) placed in circular petri dishes, Hough circle transform was used to extract ROIs. To achieve angular alignment of the extracted ROIs, a black circular marker was affixed to each petri dish, and the ROIs were rotated to align the center of the marker. This method compensates for variations in sample position and ensures accurate analysis.

[0106] After image cropping, resolution differences may exist between multispectral and hyperspectral images. To address this, the multispectral and hyperspectral images can be spatially aligned. Nearest neighbor interpolation can be used to downsample the larger image to ensure they have the same size. Then, misalignment is eliminated and an accurate correspondence between the multispectral and hyperspectral image pairs is ensured by aligning the centers of the labels and food samples along vertical lines.

[0107] To focus on the most information-rich regions, for example, a central region of size (320×320) pixels can be extracted from aligned multispectral and hyperspectral image data. This makes the analysis more efficient and reduces computational complexity while preserving key information for further processing.

[0108] The method for generating datasets for training food adulteration detection models provided in this application can ensure the alignment of multispectral image (MSI) and hyperspectral image (HSI) data, so that the distribution of training data and test data is as close as possible or the same, thereby improving the generalization performance of the trained model in actual use.

[0109] In step 101, the multispectral images of the sample food products in the training dataset, the corresponding real hyperspectral images, and the adulteration category labels are obtained.

[0110] In the dataset generated by the dataset generation method for training a food adulteration detection model provided in the embodiments of this application, multispectral images of various food samples, real hyperspectral images corresponding to the multispectral images, and adulteration category labels of the food samples can be obtained.

[0111] Optionally, a training dataset that meets the requirements of this application can be used to obtain multispectral images of the sample food, corresponding real hyperspectral images, and adulteration category labels. The multispectral images of the sample food and the corresponding real hyperspectral images in the training dataset have the same spectral reflectance.

[0112] By acquiring multispectral images of sample food, corresponding real hyperspectral images, and adulteration category labels, data support was provided for the subsequent training of food adulteration detection models.

[0113] In step 102, the multispectral image is input into the spectral reconstruction model to obtain the reconstructed hyperspectral image.

[0114] The multispectral images can be multispectral images from the training dataset, and the multispectral images can be acquired using a low-cost multispectral camera.

[0115] The spectral reconstruction model can use the current state-of-the-art (SOTA) SR algorithm as the backbone of the spectral reconstruction model to reconstruct hyperspectral images from input multispectral images.

[0116] The state-of-the-art (SOTA) SR model can be a model built using convolutional neural networks (CNNs) and their variants, a spectral reconstruction model based on conditional generative adversarial networks (GANs), or a model based on variational autoencoders (VAEs). This application does not impose any restrictions on this. It should be understood that when using these existing models to build the backbone of the spectral reconstruction model, adaptive modifications to the existing models are required based on the specific circumstances of this application.

[0117] For example, the state-of-the-art (SOTA) SR algorithm can be an "adaptive weighted attention network with camera spectral sensitivity prior for spectral reconstruction from RGB images." This algorithm is based on an adaptive weighted attention network (AWAN) for spectral reconstruction, with a backbone consisting of multiple stacked dual residual attention modules (DRAB) and long-short skip connections, thus forming dual residual learning. Specifically, the algorithm reallocates channel-level feature responses by integrating the correlations between channels. This algorithm is based on spectral reconstruction of RGB images, using the differences between RGB and hyperspectral images as finer constraints to achieve more accurate reconstruction. Therefore, when performing spectral reconstruction based on this algorithm in this application embodiment, it is necessary to modify the input and output of the corresponding network to adapt to the images in the training dataset provided in this application embodiment. Furthermore, the network can be further modified according to actual usage. This application does not impose any limitations on this.

[0118] The spectral reconstruction model can be used to obtain hyperspectral images reconstructed from multispectral images.

[0119] The reconstructed hyperspectral image obtained through the spectral reconstruction model can improve spectral resolution and expand spectral coverage. Therefore, the reconstructed hyperspectral image has a wider coverage and more bands than the multispectral image, thus providing support for the detection of food adulteration.

[0120] In step 103, the reconstruction loss is determined based on the reconstructed hyperspectral image and the real hyperspectral image, and the contrast loss is determined based on the reconstructed hyperspectral image and the hyperspectral image of the negative sample, wherein the hyperspectral image of the negative sample is determined by the adulteration category label.

[0121] This application embodiment determines the reconstruction loss by comparing the reconstructed hyperspectral image with the real hyperspectral image. Specifically, the reconstruction loss between the reconstructed hyperspectral image and the real hyperspectral image for each sample can be calculated. The reconstruction loss can be expressed as:

[0122]

[0123] Where M is the total number of pixels, and R i Let the reconstructed hyperspectrum and the true hyperspectrum of the i-th pixel in the sample be represented respectively. By minimizing the loss... The spectral reconstruction model can reduce the difference between the reconstructed hyperspectral image (HSI) and the real hyperspectral image.

[0124] In determining the contrast loss, negative samples can be screened by adulteration category labels, where all samples that do not belong to the same adulteration category can be defined as negative samples.

[0125] After identifying the negative samples, for each batch of data, the contrast loss between the reconstructed hyperspectral image output by the spectral reconstruction model and the negative sample image is calculated.

[0126] In some embodiments, the negative sample image can be a reconstructed hyperspectral image corresponding to the negative sample, obtained by reconstructing the multispectral images of samples of different categories using a spectral reconstruction model. Then, the contrast loss between the reconstructed hyperspectral image of the negative sample and the reconstructed hyperspectral image corresponding to the negative sample is calculated.

[0127] Specifically, the contrast loss can be expressed as:

[0128]

[0129] Where B is the number of samples in each batch, and M is the total number of pixels. and These are sample b and negative sample b, respectively. - The i-th reconstructed pixel. b and y b - are sample b and negative sample b - The function represents the category label. It suggests that it's sufficient to maximize the distance between the target sample b and other samples of different categories within the batch. Therefore, contrastive learning enables the spectral reconstruction model to have better discriminative power.

[0130] By introducing prior information about food adulteration categories (adulteration category labels) and employing contrastive learning during the training phase of the spectral reconstruction model, the model can distinguish between different categories of food samples with similar or even identical multispectral features. Verification shows that the training method provided in this application can improve the Euclidean distance between samples of different adulteration categories in the reconstructed hyperspectral image from 0.1764 to 0.2311.

[0131] In step 104, the parameters of the spectral reconstruction model are optimized based on the reconstruction loss and contrast loss to complete the training.

[0132] The reconstruction loss and contrast loss can be combined to optimize the adjustable parameters of the spectral reconstruction model for training. Training is completed when the preset training termination condition is met.

[0133] Reconstruction loss measures the difference between the reconstructed hyperspectral image and the true hyperspectral image, while contrastive loss aims to teach the model more discriminative feature representations. Based on the idea of ​​contrastive learning, the model can learn to bring the feature representations of positive samples closer together while pushing the feature representations of negative samples further apart. Combining the reconstruction loss and contrastive loss yields the total loss function L. The reconstruction loss and contrastive loss can be weighted by α to adjust the balance between different learning objectives. Alternatively, different weighting methods can be used separately, such as L = αL. h +βL c The weighted summation method uses α and β as weighting parameters to balance the two types of losses, which need to be adjusted according to the specific situation.

[0134] Optimization algorithms can be used to minimize the total loss function, thereby updating the parameters of the spectral reconstruction model. These algorithms include stochastic gradient descent (SGD), Adagrad, Adadelta, RMSProp, and Adam. Taking the Adam algorithm as an example, it updates the parameters based on the gradient of the loss function with respect to the model parameters. In each training step, the gradient of the total loss function with respect to the model parameters is first calculated, and then the parameters are updated according to the Adam algorithm's update rules. Specifically, the Adam algorithm maintains two exponential moving average vectors to estimate the first and second moments of the gradient, and then adjusts the learning rate based on these estimates to update the model parameters. After multiple training steps, the model parameters are continuously adjusted, causing the total loss function to gradually decrease until the model converges or reaches a preset training termination condition, such as reaching the maximum number of training epochs or the loss function falling below a certain threshold. At this point, the model training is complete.

[0135] Please refer to Figure 2 , Figure 2 This is a flowchart of a training method for a spectral reconstruction model provided in some embodiments of this application, where M(x, y, w0) represents the multispectral image of the input sample, and x, y, and w0 represent the width, height, and wavelength channels, respectively. This represents the reconstructed hyperspectral image. R represents the hyperspectral image of the reconstructed negative sample. c (x, y, w) represents the true hyperspectral image of the sample.

[0136] Figure 2In this process, the multispectral image of the input sample is used to obtain the reconstructed hyperspectral image through State-of-the-art (SOTA) hyperspectral image. Then, a negative sample set that does not belong to the adulteration category of the sample is filtered out using adulteration category labels (category prior). The contrastive learning loss between the sample within each batch and its negative sample set is calculated; the reconstruction loss between the reconstructed hyperspectral image and the real hyperspectral image of each sample is calculated; the two loss functions are weighted and summed; and the parameters in SOTA SR are tuned to complete the model training.

[0137] This application embodiment utilizes a low-cost multispectral camera for food adulteration detection. Therefore, the multispectral images of adulterated and unadulterated food samples acquired by the camera are similar. However, because the low-cost multispectral camera can only capture a limited number of spectral bands, the resulting coarse-grained spectra lack detailed information. Within these limited spectral bands, the multispectral images of normal and adulterated food samples may appear more similar, or even identical.

[0138] However, for two multispectral images that are similar or even identical but have different hyperspectral images, existing spectral reconstruction methods struggle to reproduce the subtle differences present in the original hyperspectral images. This makes it difficult to distinguish between adulterated and genuine food products from the reconstructed spectra. Therefore, the spectral reconstruction model proposed in this application incorporates prior information about adulteration categories during the training phase and employs a contrastive learning method to amplify the differences between reconstructed samples with similar multispectral image inputs. Through this method, the reconstructed spectra can better distinguish between different categories of food samples, thereby improving the accuracy and reliability of food adulteration detection.

[0139] In step 105, the reconstructed hyperspectral image is input into the classification model to train the classification model.

[0140] The reconstructed hyperspectral image can be the output of a trained spectral reconstruction model. Because the spectral reconstruction model employs contrastive learning, it can extract new information from the reconstructed hyperspectral image. The training method for the classification model is described below.

[0141] Please refer to Figure 3 , Figure 3 This is a flowchart illustrating another training method for a food adulteration detection model provided in some embodiments of this application. The food adulteration detection model includes a spectral reconstruction model and a classification model. It should be noted that the steps shown in the flowchart can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0142] The training of the classification model in the embodiments of this application includes the following steps:

[0143] Step 301: Input the reconstructed hyperspectral image into the student feature extraction model to obtain basic features. The reconstructed hyperspectral image is generated by the spectral reconstruction model.

[0144] Step 302: Input the real hyperspectral image corresponding to the reconstructed hyperspectral image into the teacher feature extraction model and train the teacher feature extraction model to obtain teacher features;

[0145] Step 303: Determine distillation loss using basic and teacher characteristics;

[0146] Step 304: Optimize the parameters of the classification model based on distillation loss to complete the training.

[0147] Steps 301 to 304, as illustrated in the embodiments of this application, can use a feature extractor (teacher feature extraction model) of a real hyperspectral image to guide a feature extractor (student feature extraction model) of a reconstructed hyperspectral image, thereby transferring useful knowledge across the entire spectrum, especially the visible light spectrum, to improve the accuracy of adulteration detection.

[0148] Before training a classification model, the data input to the classification model needs to be optimized.

[0149] For example, each reconstructed hyperspectral image can be divided into 40×40 image blocks, and average pooling can be performed on each block to reduce spatial redundancy and speed up the training process. Next, a mean filter with a window size of 3 can be used to smooth the spectrum of each pixel to achieve spectral denoising. Then, a common spectral preprocessing method, Standardized Normalized Variable (SNV), can be applied to the spectrum of each pixel to reduce scattering effects. After these operations, each processed hyperspectral image data block with 40×40 pixels and 138 channels will be input into the classification model.

[0150] SVN is a variable that has been standardized to conform to a standard normal distribution. It can unify the spectrum of each pixel to the scale of a standard normal distribution, which can reduce scattering effects and help improve the accuracy and stability of the model.

[0151] The training of the classification model will be introduced below.

[0152] In step 301, the reconstructed hyperspectral image is input into the student feature extraction model to obtain basic features. The reconstructed hyperspectral image is generated by the spectral reconstruction model.

[0153] Optionally, the reconstructed hyperspectral image is the output of the spectral reconstruction model provided in the embodiments of this application. The spectral reconstruction model is trained using steps 101 to 104 provided in the embodiments of this application.

[0154] Please see Figure 5 The student feature extraction model, also known as the base model, takes optimized reconstructed hyperspectral image data (e.g., 40×40×138) as input and employs three 2D convolutional layers, three batch normalization (BN) layers, three pooling layers, and one fully connected (FC) layer.

[0155] Considering the significant noise in the reconstructed hyperspectral image, the first convolutional layer performs downsampling to reduce the impact of noise and redundant information. Simultaneously, it doesn't reduce the spectral dimension and assigns a weight to each channel for better feature extraction. Furthermore, to further avoid overfitting due to reconstruction noise, a batch normalization layer and a max-pooling layer are added after the first convolutional layer. These three layers together constitute a dedicated denoising module for the reconstructed hyperspectral image. The output is then fed into the second convolutional layer. In this layer, a larger kernel size is used to extract more global information, as the reconstructed spatial texture may contain erroneous noise and lead to overfitting. Therefore, greater attention is paid to global information and spectral dimension. The third convolutional layer has the same settings as the second, further extracting finer-grained features.

[0156] It should be understood that although the third and second convolutional layers use the same kernel, they cannot be considered completely identical convolutional layers. First, each convolutional layer has its own independent weights and bias parameters. Even if the kernel size is the same, their outputs will differ. Furthermore, due to their different locations, they are used to capture features at different scales. Afterwards, average pooling is performed to remove the spatial dimension, resulting in an embedding vector of length 32, denoted as the base feature F. s .

[0157] Figure 5 In ConvBN, a combination of convolutional layers and batch normalization layers, it is commonly used in computer vision tasks. ConvBN can reduce model complexity and improve training efficiency. The numbers in ConvBN represent: ConvBN(number of input channels, number of output channels, kernel size, stride, padding).

[0158] By inputting the reconstructed hyperspectral image generated by the reconstruction model into the student feature extraction model for feature extraction, basic features can be obtained.

[0159] In step 302, the real hyperspectral image corresponding to the reconstructed hyperspectral image is input into the teacher feature extraction model and the teacher feature extraction model is trained to obtain teacher features.

[0160] Since this application aims to reduce the cost of food adulteration detection and improve the detection accuracy of low-cost food adulteration detection equipment, the multispectral cameras used are mostly inexpensive. However, due to cost constraints, low-cost multispectral cameras may not cover the visible light band. Accurately reconstructing the visible light band using relevant technologies is quite difficult. Furthermore, because the spectral absorption characteristics of different adulteration types are distributed across different bands, it is necessary to utilize the full-band information of a true hyperspectral image (HSI) for effective adulteration detection.

[0161] In view of this, this application embodiment adds a teacher feature extraction model that uses real hyperspectral images as input to the student feature extraction model. The teacher feature extraction model guides the training of the student feature extraction model.

[0162] Since real hyperspectral images have almost no noise, standard normalization (SNV) can reduce the spatial dimension of the processed image to 20×20, thus accelerating the training speed of the teacher feature extraction model. The teacher feature extraction model only requires two convolutional layers, with kernels smaller than those in the student feature extraction model. The processed image is input into the first ConvBN layer, then into a max pooling layer, followed by a second ConvBN layer, and finally into an average pooling layer.

[0163] For example, after training the teacher feature extraction model, the parameters of the teacher feature extraction model are frozen, and the embedding vector of length 32 before the fully connected (FC) layer is extracted as the teacher feature, denoted as F. t To perform feature distillation, the underlying feature F extracted by the student feature extraction model can be minimized. s Teacher characteristics F of the frozen teacher model t The gap between them.

[0164] In step 303, distillation loss is determined using basic features and teacher features. Basic features refer to the features extracted using the student feature extraction model; teacher features refer to the features extracted using the teacher feature extraction model.

[0165] The similarity between the output distribution of the student model and the output distribution of the teacher model can be measured using a distillation loss function. This distillation function can be based on Kullback-Leibler divergence, which measures the difference between two probability distributions. Alternatively, the distillation function can be based on mean squared error (MSE) loss. This application does not impose any restrictions on this.

[0166] In implementation, the outputs of the teacher and student models can be converted into probability distributions using a softmax function, and then the KL divergence or mean squared error between them can be calculated. Furthermore, to better balance distillation loss and classification loss, cross-entropy loss can be introduced to measure the difference between the student feature extraction model's output and the true labels.

[0167] In step 304, the parameters of the classification model are optimized based on distillation loss to complete the training.

[0168] In optimizing the parameters of a classification model, a suitable optimizer can be selected. This optimizer can be stochastic gradient descent (SGD), Adagrad, Adadelta, RMSProp, Adam, etc., and this application does not impose any restrictions on this. All of these optimizers can be used to optimize the parameters of a classification model. The purpose of these optimizers is to minimize the distillation loss by adjusting the model's parameters. The optimizer updates the parameter values ​​based on the gradient of the loss function with respect to the parameters. Different optimizedrs will result in different methods for updating parameters and different learning rate adjustment strategies.

[0169] In each training iteration of the optimized classification model, the gradient of the distillation loss with respect to the parameters of the student model (also known as the student feature extraction model) is first calculated. This can be achieved using automatic differentiation techniques (such as backpropagation in deep learning frameworks). Then, the optimizer updates the parameters of the student model based on the calculated gradient and a preset learning rate. This process is repeated until preset training stopping conditions are met, such as reaching a certain number of training epochs, the loss value converging below a certain threshold, or performance on the validation set no longer improving. By continuously updating the parameters, the student model gradually learns how to mimic the behavior of the teacher model and can effectively classify adulteration categories in the original food, thus completing the model training.

[0170] By training the classification model by optimizing the parameters through distillation loss, the extracted features can be made consistent with the features of the real hyperspectral image (HSI) in the latent feature space, thereby reducing the error in the full-band reconstructed spectrum and capturing features related to adulteration.

[0171] In some implementations, the classification model further includes a visible light feature extraction model, and the training of the classification model may also include: inputting the visible light portion of the reconstructed hyperspectral image into the visible light feature extraction model to obtain visible light features; concatenating the basic features and the visible light features to obtain concatenated features; determining the predicted adulteration category through the concatenated features; determining the classification loss based on the predicted adulteration category and the adulteration category label; and optimizing the parameters of the classification model based on the distillation loss and the classification loss to complete the training.

[0172] Although the visible light band is difficult to reconstruct, the use of negative samples and contrastive learning in the spectral reconstruction model allows for the extraction of new information from the reconstructed visible light band. For example... Figure 4 As shown, Figure 4 This is a schematic diagram comparing the original hyperspectral and reconstructed hyperspectral in different bands according to some embodiments of this application. Compared with the visible light band of the original hyperspectral (real hyperspectral image), the visible light band of the reconstructed hyperspectral has more information that can be used to distinguish different categories. Figure 4 In the original hyperspectral image, the Euclidean distance in the visible light band is 0.240, while the Euclidean distance in the visible light band of the reconstructed hyperspectral image is 0.282. This additional information can help improve classification performance.

[0173] Therefore, embodiments of this application provide a method for adding a visible light feature extraction model to a classification model. The visible light portion of the reconstructed hyperspectral image is input into the visible light feature extraction model to obtain visible light features. The visible light feature extraction model is used to extract additional features within the visible light band (e.g., the first 66 channels) of the reconstructed hyperspectral image (HSI).

[0174] For example, an embedding vector F of length 8 can be extracted. v As a characteristic of visible light.

[0175] Next, the extracted visible light features and the basic features extracted by distillation are spliced ​​together. For example, the length of the spliced ​​feature is 40. The spliced ​​feature is then input into a fully connected layer for classification.

[0176] The structure of the visible light feature extraction model can be similar to that of the teacher feature extraction model.

[0177] During model training, the applicant discovered that different types of adulteration exhibit varying degrees of reconstruction error. Larger errors may make it more difficult to distinguish between these adulteration types. Therefore, the average reconstruction error for each type of adulteration can be added as a parameter to the loss function.

[0178] In some implementations, determining the classification loss based on the predicted adulteration category and the adulteration category label can be the average reconstruction error used to determine the adulteration category; or the classification loss can be determined based on the average reconstruction error, the predicted adulteration category, and the adulteration category label.

[0179] For example, the reconstruction error of each adulterated category can be calculated and used as the weight information of that category in the classification loss function to obtain better classification performance.

[0180] Ultimately, the overall loss function of the classification model can be composed of distillation loss and weighted cross-entropy (WCE) loss. It can be expressed as:

[0181]

[0182] Where β is a weighting parameter used to weigh the distillation intensity of features for different objectives. This is knowledge distillation loss. The classification task uses a novel weighted cross-entropy loss function, which can be expressed as:

[0183]

[0184] Where, N c Let ΔR represent the number of samples of the c-th type of adulteration. i P represents the average reconstruction error of the i-th sample. c This represents the classification probability of belonging to class c. In this way, the classification model will pay more attention to classes with poor reconstruction performance. Furthermore, for feature distillation, the mean squared error (MSE) loss can be used as the knowledge distillation loss to measure the difference between the extracted student model features and the teacher model features, i.e.: Where N f This represents the number of samples (feature length). During training, distillation loss and cross-entropy loss are used together to ensure that distillation proceeds in the direction of improving classification accuracy.

[0185] Please refer to Figure 6 , Figure 6 This is a flowchart of another training method for a food adulteration detection model provided in some embodiments of this application. Figure 6 In this process, a training dataset for a food adulteration detection model can be established through hardware imaging. Then, the data from this dataset is input into the adulteration detection model to train it. Specifically, the multispectral images of sample food products, their corresponding real hyperspectral images, and adulteration category labels from the training dataset can be input into the spectral reconstruction model to train it. Next, the reconstructed hyperspectral images output by the trained spectral reconstruction model are input into the classification model to train it. Finally, the adulteration type information is input.

[0186] In some embodiments of this application, the food adulteration detection model includes a spectral reconstruction model and a classification model.

[0187] The training of the spectral reconstruction model can be achieved by acquiring multispectral images of sample food products, corresponding real hyperspectral images, and adulteration category labels from the training dataset; inputting the multispectral images into the spectral reconstruction model to obtain reconstructed hyperspectral images; determining the reconstruction loss based on the reconstructed hyperspectral images and real hyperspectral images, and determining the contrast loss based on the reconstructed hyperspectral images and hyperspectral images of negative samples, where the hyperspectral images of negative samples are determined by adulteration category labels; optimizing the parameters of the spectral reconstruction model based on the reconstruction loss and contrast loss to complete the training; and inputting the reconstructed hyperspectral images into the classification model to train the classification model.

[0188] The training of the classification model can be achieved by inputting the reconstructed hyperspectral image output by the spectral reconstruction model into the student feature extraction model to obtain basic features; inputting the real hyperspectral image corresponding to the reconstructed hyperspectral image into the teacher feature extraction model and training the teacher feature extraction model to obtain teacher features; determining the distillation loss using the basic features and teacher features; inputting the visible light portion of the reconstructed hyperspectral image into the visible light feature extraction model to obtain visible light features; concatenating the basic features and visible light features to obtain concatenated features; determining the predicted adulteration category using the concatenated features; determining the classification loss based on the predicted adulteration category and adulteration category label; and optimizing the parameters of the classification model based on the distillation loss and classification loss to complete the training.

[0189] For example, the data acquisition for the training dataset can be achieved by placing the food sample within the field of view of a low-cost multispectral camera to capture a multispectral image (M(x,y,w0)), obtaining the adulteration category label (Y) of the sample during the sample preparation process, and acquiring a hyperspectral image of the sample (H(x,y,w0)) using a laboratory-grade hyperspectral camera to obtain the data for the training dataset.

[0190] Next, the multispectral images (M(x,y,w0)) collected in the training dataset are input into the spectral reconstruction model (φ) for training. Simultaneously, negative samples are filtered using an adulteration category label (Y) (all adulterated samples not belonging to the same adulteration type as the original sample are defined as negative samples). For each batch of data, the contrast loss between negative samples of the hyperspectral (φ(M)) images output by the spectral reconstruction model, and the reconstruction error between each sample and the true hyperspectral (H) image are calculated.

[0191] The teacher network (β) is trained using real hyperspectral data (H), and then reconstructed hyperspectral data is used. Train the student network (α). Calculate the features extracted by the teacher network and the student network. The loss. Using the visible light portion of the reconstructed hyperspectral data. Train the feature extractor (σ) in the visible light band, and then concatenate the features extracted by the student network with the features extracted by the visible light feature extractor. The spliced ​​features are input into the fully connected layer to predict the adulteration category. Calculate the loss function with the true adulteration category (Y) and complete the training of the model.

[0192] For example, in the actual training process, the size of the processed paired hyperspectral images (HSI) and multispectral images (MSI) in the dataset is set to 320×320 pixels. To train the spectral reconstruction model, the images can be segmented into 64×64 image patches, the batch size is set to 32, and the initial learning rate is 10. -4 And a cosine preselection annealing strategy is used to reduce the learning rate to 10. -6 After 20,000 iterations, the model converged when the weight α of the loss function was 0.95.

[0193] To train the classification model, the batch size can be set to 64, the distillation weight β to 0.8, and the initial learning rate to 5 × 10⁻⁶. -4 And using a cosine preselection annealing strategy, the final learning rate is 10. -6 After 1000 training cycles, to prevent getting stuck in local optima during cosine annealing, the learning rate can be reset to a fixed value of 5 × 10. -5 The model was then trained for an additional 1000 epochs. After 2000 epochs of training, the classification model converged to the final result.

[0194] The above is an introduction to the training method of food adulteration detection model.

[0195] Please refer to Figure 7 , Figure 7 This is a flowchart of a food adulteration detection method provided by some embodiments of this application. It should be noted that the steps shown in the flowchart can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be performed in a different order than that shown here.

[0196] The food adulteration detection method of this application includes the following steps:

[0197] Step 701: Acquire multispectral images of the food sample to be tested;

[0198] Step 702: Input the multispectral image into the food adulteration detection model to obtain the adulteration category of the food sample to be tested;

[0199] Step 703: Output the adulteration category of the food sample to be tested;

[0200] The food adulteration detection model includes a spectral reconstruction model and a classification model. The spectral reconstruction model is trained using the same training method as the spectral reconstruction model in the food adulteration detection model, and the classification model is trained using the same training method as the classification model in the food adulteration detection model.

[0201] See Figure 8 Multispectral images of the food samples to be tested can be acquired using a low-cost multispectral camera. The acquired multispectral images are input into a spectral reconstruction model to obtain reconstructed hyperspectral images. The reconstructed hyperspectral images are then input into a classification model, and the final adulteration type is obtained through a Student feature extraction model and a visible light band feature extraction model.

[0202] For example, a food sample (e.g., a meat sample) can be placed within the field of view of a low-cost multispectral camera to capture a multispectral image (M(x,y,w0)). The acquired multispectral image (M(x,y,w0)) is then input into a trained spectral reconstruction model (φ) for spectral reconstruction. The reconstructed hyperspectral data (φ(M)) is then input into a student feature extractor (α). The visible light portion of the reconstructed hyperspectral data (φ(M)) is then extracted. RGB The input is fed into the visible light band feature extractor (σ), which concatenates the features extracted by the student feature extractor with the features extracted by the visible light feature extractor [α(φ(M)), σ(φ(M)]. RGB The concatenated features are input into the fully connected layer to predict the adulteration category.

[0203] A low-cost multispectral camera could be the Monarch camera, which operates in the near-infrared band from 690 nm to 950 nm, providing more spectral features relevant to food, such as detecting adulteration in meat. Secondly, it is more affordable. The Monarch camera employs a novel microelectromechanical system (MEMS) spectral imaging chip based on a Fabry-Perot broadband filter, significantly reducing hardware costs.

[0204] When acquiring multispectral images, multiple tungsten filament lamps can be evenly distributed around the acquisition device to ensure uniform illumination.

[0205] The above is an introduction to the food adulteration detection method of the present application embodiments.

[0206] Please see Figure 9To verify the performance of the food adulteration detection method provided in this application, the detection method proposed in this application was compared with several benchmarks (for meat adulteration detection), including: (i) RGB: Classification is performed using the corresponding RGB images generated from the acquired hyperspectral images (HSI). Specifically, a residual network is constructed to classify the generated RGB images. (ii) Multispectral images (MSI): Classification is performed directly using multispectral images. A random forest algorithm is used to classify the spectrum after averaging in the spatial dimension. The equipment for acquiring this type of multispectral image costs approximately US$60. (iii) Multispectral images (state-of-the-art spectral reconstruction algorithm, i.e., SOTA SR): The hyperspectral image (HSI) is reconstructed using the existing state-of-the-art spectral reconstruction algorithm, and then the spectrum after averaging in the spatial dimension is classified using a random forest algorithm. (iv) Hyperspectral images (HSI): Classification is performed using hyperspectral data acquired by an expensive hyperspectral imaging camera (costing more than US$10,000), and the classification model proposed in this application is used.

[0207] As can be seen from the comparison of the proposed food adulteration detection method with various benchmarks in terms of accuracy and F1 score, three important conclusions can be drawn. First, the performance of adulteration detection using unprocessed multispectral images (MSI) or RGB images is poor, with an accuracy of less than 70%, because they contain limited and coarse spectral information. Second, applying the existing state-of-the-art spectral reconstruction (SOTA SR) algorithm to multispectral images only slightly improves the system performance. This is because the algorithm cannot recover the fine-grained differences in the reconstructed hyperspectral image (HSI) and cannot effectively handle reconstruction errors, resulting in low classification accuracy. Finally, the proposed food adulteration detection method achieves an accuracy of 91.06% in the meat adulteration type detection task (the corresponding multispectral image acquisition equipment is much cheaper than a hyperspectral image camera), which is 21.58% higher than the method of directly using multispectral images and 14.95% higher than the method of processing multispectral images using existing reconstruction algorithms, demonstrating the effectiveness of our system design. It should be understood that these are only the test results for the corresponding environment and meat samples presented in the embodiments of this application. The test results may vary depending on the environment and samples, but the overall trend should be consistent.

[0208] The implementation of the food adulteration detection system provided in this application will now be described in detail with reference to the accompanying drawings.

[0209] In addition to the food adulteration detection method provided in the above embodiments, this application also provides a food adulteration detection system for implementing the above method, such as... Figure 10 As shown, Figure 10This is a schematic block diagram of a food adulteration detection system 1000 according to an embodiment of this application. The food adulteration detection system includes:

[0210] The multispectral image acquisition module 1010 is used to acquire multispectral images of the food sample to be tested.

[0211] The spectral reconstruction module 1020 is used to reconstruct hyperspectral images from multispectral images;

[0212] The classification module 1030 is used to predict the adulteration category of the food sample to be tested by reconstructing the hyperspectral image;

[0213] Among them, the spectral reconstruction module 1020 is trained using the training method of the spectral reconstruction model in the above-mentioned food adulteration detection model, and the classification module 1030 is trained using the training method of the classification model in the above-mentioned food adulteration detection model.

[0214] It is understood that the content of the above-described food adulteration detection method embodiments is applicable to this food adulteration detection system embodiment. The specific functions implemented by this food adulteration detection system embodiment are the same as those of the above-described food adulteration detection method embodiments, and the beneficial effects achieved are also the same as those achieved by the above-described food adulteration detection method embodiments.

[0215] Reference Figure 11 This application also provides an electronic device 1100, which includes a memory 1110 and one or more processors 1120. Figure 11 (Only one is shown in the image) and a computer program stored in memory 1110 and executable on processor 1120. Memory 1110 stores software programs and units. Processor 1120 executes various functional applications and data processing by running the software programs and units stored in memory 1110 to obtain resources corresponding to the aforementioned preset events. Optionally, processor 1120 implements the training method for the food adulteration detection model or the detection method for food adulteration by running the computer program stored in memory 1110.

[0216] Memory 1110 serves as a non-transitory computer-readable medium for storing non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory 1110 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 1110 may optionally include memory remotely located relative to the processor, which can be connected to the processor 1120 via a network.

[0217] It is understood that the content of the above method embodiments is applicable to the embodiments of this electronic device. The specific functions implemented by the embodiments of this electronic device are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0218] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the training method for the food adulteration detection model or the detection method for food adulteration.

[0219] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0220] This application also provides a computer program product, which includes a computer program. When the computer program is executed by one or more processors, it can implement the training method of the food adulteration detection model or the detection method of food adulteration.

[0221] It is understood that the content of the above method embodiments is applicable to this computer program product. The specific functions implemented by the embodiments of this computer program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0222] The food adulteration detection method, model training method, system, electronic equipment, media, and computer program products provided in this application maintain low-cost hardware configuration while ensuring the fine-grained spectral capabilities required for food adulteration detection. To achieve this goal, this application utilizes existing spectral reconstruction techniques and incorporates two novel modules: a spectral reconstruction module and a classification module. The spectral reconstruction module aims to recover more discriminative reconstructed images from cost-effective multispectral input data. It incorporates prior information related to adulteration and contrastive learning methods to reconstruct more easily distinguishable full-band hyperspectral images. The classification module, based on the error distribution characteristics of the reconstructed hyperspectral image (HSI), designs an adulteration detection model based on a convolutional neural network (CNN) and enhances it through a knowledge distillation structure. Furthermore, it designs unique feature extractors for different bands. In addition, this application also compiles a large-scale dataset containing various adulterants. Experimental evaluation shows that the method provided in this application achieves an accuracy of 91.06% in detecting various types of adulteration in meat, which is a 21.58% improvement in performance compared to a benchmark system that does not use the method proposed in this application.

[0223] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0224] Although specific embodiments are described herein, those skilled in the art will recognize that many other modifications or alternative embodiments are also within the scope of this disclosure. For example, any of the functions and / or processing capabilities described in connection with a particular device or component can be performed by any other device or component. Furthermore, while various exemplary embodiments and architectures have been described according to embodiments of this disclosure, those skilled in the art will recognize that many other modifications to the exemplary embodiments and architectures described herein are also within the scope of this disclosure.

[0225] The foregoing description, with reference to block diagrams and flowcharts of systems, methods, systems, and / or computer program products according to exemplary embodiments, has described certain aspects of this disclosure. It should be understood that one or more blocks in the block diagrams and flowcharts, as well as combinations of blocks in the block diagrams and flowcharts, can be implemented by executing computer-executable program instructions, respectively. Similarly, according to some embodiments, some blocks in the block diagrams and flowcharts may not need to be executed in the order shown, or may not all need to be executed. Furthermore, additional components and / or operations beyond those shown in the blocks in the block diagrams and flowcharts may exist in some embodiments.

[0226] Therefore, blocks in block diagrams and flowcharts support combinations of means for performing a specified function, combinations of elements or steps for performing a specified function, and program instruction means for performing a specified function. It should also be understood that each block in a block diagram and flowchart, and combinations of blocks in block diagrams and flowcharts, can be implemented by a dedicated hardware computer system or a combination of dedicated hardware and computer instructions that performs a specific function, element, or step.

[0227] The program modules, applications, etc., described herein may include one or more software components, including, for example, software objects, methods, data structures, etc. Each such software component may include computer-executable instructions that, in response to execution, cause at least a portion of the functionality described herein (e.g., one or more operations of the exemplary methods described herein) to be performed.

[0228] Software components can be coded using any of a variety of programming languages. An exemplary programming language could be a low-level programming language, such as assembly language associated with a specific hardware architecture and / or operating system platform. Software components including assembly language instructions may need to be converted into executable machine code by an assembler before being executed by the hardware architecture and / or platform. Another exemplary programming language could be a higher-level programming language that is portable across multiple architectures. Software components including higher-level programming languages ​​may need to be converted into an intermediate representation by an interpreter or compiler before execution. Other examples of programming languages ​​include, but are not limited to, macro languages, shell or command languages, job control languages, scripting languages, database query or search languages, or report writing languages. In one or more exemplary embodiments, a software component containing instructions from one of the above-described programming language examples can be executed directly by the operating system or other software components without first being converted into another form.

[0229] Software components can be stored as files or other data storage structures. Software components of similar type or related function can be stored together in a specific directory, folder, or library. Software components can be static (e.g., pre-defined or fixed) or dynamic (e.g., created or modified at runtime).

[0230] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. A training method for a food adulteration detection model, characterized in that, The food adulteration detection model includes a spectral reconstruction model and a classification model. The training of the spectral reconstruction model includes the following steps: Obtain multispectral images of sample food products in the training dataset, the corresponding real hyperspectral images, and adulteration category labels; The multispectral image is input into the spectral reconstruction model to obtain the reconstructed hyperspectral image; The reconstruction loss is determined based on the reconstructed hyperspectral image and the real hyperspectral image, and the contrast loss is determined based on the reconstructed hyperspectral image and the hyperspectral image of the negative sample, wherein the hyperspectral image of the negative sample is determined by the adulteration category label; The parameters of the spectral reconstruction model are optimized based on the reconstruction loss and the contrast loss to complete the training; The reconstructed hyperspectral image is input into the classification model to train the classification model.

2. The training method for the food adulteration detection model according to claim 1, characterized in that, The training dataset is generated by one or any combination of the following: Eliminate ambient light; Multispectral and hyperspectral images of the food samples were collected at the same location and under the same conditions. Median filtering is performed on the acquired multispectral and hyperspectral images of the sample food, and the region of interest is extracted. Spatially align the multispectral image and the hyperspectral image; The central regions of the multispectral image and the hyperspectral image are extracted to generate the training dataset.

3. The training method for the food adulteration detection model according to claim 1, characterized in that, The determination of contrast loss based on the reconstructed hyperspectral image and the hyperspectral image of the negative sample includes: Negative samples of the sample food are selected by filtering out the adulteration category label. The negative samples are used to characterize samples that are different from the adulteration category of the sample food. A contrast loss function is determined based on the reconstructed hyperspectral image and the hyperspectral image of the negative sample; The contrast loss is determined based on the contrast loss function.

4. A training method for a food adulteration detection model, characterized in that, The food adulteration detection model includes a spectral reconstruction model and a classification model. The classification model includes a Student feature extraction model. The training of the classification model includes the following steps: The reconstructed hyperspectral image is input into the student feature extraction model to obtain basic features, wherein the reconstructed hyperspectral image is generated by the spectral reconstruction model; The real hyperspectral image corresponding to the reconstructed hyperspectral image is input into the teacher feature extraction model and the teacher feature extraction model is trained to obtain teacher features; Distillation loss is determined using the aforementioned basic characteristics and the aforementioned teacher characteristics; The parameters of the classification model are optimized based on the distillation loss to complete the training.

5. The training method for the food adulteration detection model according to claim 4, characterized in that, The classification model also includes a visible light feature extraction model, and the training of the classification model further includes the following steps: The visible light portion of the reconstructed hyperspectral image is input into the visible light feature extraction model to obtain visible light features; The basic features and the visible light features are spliced ​​together to obtain the spliced ​​features; The type of adulteration can be predicted by splicing features; The classification loss is determined based on the predicted adulteration category and the adulteration category label; The parameters of the classification model are optimized based on the distillation loss and the classification loss to complete the training.

6. The training method for the food adulteration detection model according to claim 5, characterized in that, The determination of classification loss based on the predicted adulteration category and adulteration category label includes: The average reconstruction error for determining adulteration categories; The classification loss is determined based on the average reconstruction error, the predicted adulteration category, and the adulteration category label.

7. A method for detecting adulteration in food, characterized in that, The method for detecting food adulteration includes: Acquire multispectral images of the food sample to be tested; The multispectral image is input into the food adulteration detection model to obtain the adulteration category of the food sample to be tested; Output the adulteration category of the food sample to be tested; The food adulteration detection model includes a spectral reconstruction model and a classification model. The spectral reconstruction model is trained using the training method of the food adulteration detection model according to any one of claims 1 to 3, and the classification model is trained using the training method of the food adulteration detection model according to any one of claims 4 to 6.

8. A food adulteration detection system, characterized in that, The system includes: The multispectral image acquisition module is used to acquire multispectral images of the food sample to be tested. A spectral reconstruction module is used to reconstruct a hyperspectral image from the multispectral image; A classification module is used to predict and output the adulteration category of the food sample to be tested based on the reconstructed hyperspectral image; The spectral reconstruction module is trained using the training method of the food adulteration detection model according to any one of claims 1 to 3, and the classification module is trained using the training method of the food adulteration detection model according to any one of claims 4 to 6.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the training method of the food adulteration detection model as described in any one of claims 1 to 3, the training method of the food adulteration detection model as described in any one of claims 4 to 6, or the food adulteration detection method as described in claim 7.

10. A computer program product, characterized in that, When the computer program product is run in an electronic device, the electronic device performs the training method of the food adulteration detection model as described in any one of claims 1 to 3, the training method of the food adulteration detection model as described in any one of claims 4 to 6, or the food adulteration detection method as described in claim 7.

Images