Intelligent commodity quality inspection method and system based on Internet of Things

Abstract

The invention discloses an intelligent commodity quality inspection method and system based on the Internet of Things, and relates to the technical field of artificial intelligence, and the method comprises the steps: firstly obtaining a first training set composed of simulation multi-modal sensor data, and the first training set comprises a fresh food state target value and a type label; calling a commodity quality inspection model to extract multi-modal features, and respectively carrying out target value regression and state type classification speculation; calculating the state speculation loss of the fresh food based on the labeled value and the speculation value, and calculating the alignment loss through the multi-modal characteristic deviation; and fusing the two types of loss optimization models to obtain a target commodity quality inspection model. The model can process fresh food monitoring data collected by an internet-of-things multi-mode sensor in real time, speculate state target values and types of the fresh food monitoring data and output quality inspection results, the problems that a traditional method depends on manpower, single-mode data is limited, and multi-mode feature alignment is insufficient are solved, and the method is suitable for intelligent quality control of scenes such as fresh food cold chains and retail.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an intelligent commodity quality inspection method and system based on the Internet of Things. Background Art

[0002] Quality inspection of fresh produce, such as fruits and vegetables and meat, is a critical component of cold chain logistics and retail, directly impacting both consumer experience and food safety. Traditional quality inspection methods rely on manual sensory judgment, such as observing color and touching firmness. These methods are subject to subjectivity, low efficiency, and the inability to monitor real-time conditions. Some scenarios utilize single-mode sensors, such as temperature and humidity sensors, to assist in detection. However, a single data dimension cannot fully reflect the state of fresh produce, as external decay and internal deterioration may not sync. Summary of the Invention

[0003] The purpose of the present invention is to provide a commodity intelligent quality inspection method and system based on the Internet of Things.

[0004] In a first aspect, an embodiment of the present invention provides an IoT-based intelligent commodity quality inspection method, comprising:

[0005] Obtaining a first training set, the first training set including a plurality of multimodal fresh food monitoring training data, and a first preset fresh food state target value and a preset fresh food state type that are matched to each of the plurality of multimodal fresh food monitoring training data, wherein the multimodal fresh food monitoring training data is monitoring data collected by simulating a multimodal sensor;

[0006] Extracting fresh food monitoring features that match each of the plurality of multimodal fresh food monitoring training data using a product quality inspection model;

[0007] The commodity quality inspection model is used to perform target value inference on each of the multiple multimodal fresh food monitoring training data based on the fresh food monitoring features that each of the multiple multimodal fresh food monitoring training data matches, thereby obtaining a first inferred fresh food target value that each of the multiple multimodal fresh food monitoring training data matches. The commodity quality inspection model is used to perform state type inference on the multiple multimodal fresh food monitoring training data, thereby obtaining an inferred fresh food state type that each of the multiple multimodal fresh food monitoring training data matches;

[0008] Determining the freshness estimation loss of the commodity quality inspection model according to the first preset freshness state target value, the preset freshness state type, the first estimated freshness target value, and the estimated freshness state type that are matched to each of the plurality of multimodal freshness monitoring training data;

[0009] Determining a multimodal alignment loss of the product quality inspection model based on the fresh food monitoring features matched to the plurality of multimodal fresh food monitoring training data;

[0010] According to the freshness estimation loss of the commodity quality inspection model and the multimodal alignment loss of the commodity quality inspection model, the commodity quality inspection model is subjected to a first optimization process to obtain a target commodity quality inspection model; the target commodity quality inspection model is used to infer the freshness target value and / or freshness type of the target multimodal freshness monitoring data, and determine the quality inspection result corresponding to the target multimodal freshness monitoring data based on the freshness target value and / or freshness type, and the target multimodal freshness monitoring data is the monitoring data collected by the multimodal sensor.

[0011] In a second aspect, an embodiment of the present invention provides a server system, including a server, wherein the server is configured to execute the method described in the first aspect.

[0012] Compared with the existing technology, the beneficial effects provided by the present invention include: using the IoT-based intelligent commodity quality inspection method and system disclosed by the present invention, which relates to the field of artificial intelligence technology, by obtaining a first training set consisting of simulated multimodal sensor data, including fresh food state target values and type annotations; calling the commodity quality inspection model to extract multimodal features, and performing target value regression and state type classification inference respectively; calculating the fresh food state inference loss based on the labeled value and the inferred value, and calculating the alignment loss through the multimodal feature deviation; fusing the two types of loss optimization models to obtain the target commodity quality inspection model. This model can process the fresh food monitoring data collected by the IoT multimodal sensors in real time, infer its state target value and type, and output the quality inspection results. It solves the problems of traditional methods that rely on manual labor, the limitations of single-modal data, and the lack of multimodal feature alignment, and is suitable for intelligent quality control in scenarios such as fresh food cold chain and retail. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly describes the drawings required for use in the embodiments. It should be understood that the following drawings illustrate only certain embodiments of the present invention and should not be construed as limiting the scope of the present invention. Those skilled in the art can, without inventive effort, derive other relevant drawings from these drawings.

[0014] Figure 1 A schematic diagram of the steps of the IoT-based intelligent quality inspection method for commodities provided in an embodiment of the present invention;

[0015] Figure 2 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the embodiments of the present invention. It should be understood that the described embodiments are only a portion of the embodiments of the present invention, not all of them. Generally, the components of the embodiments of the present invention described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

[0017] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0018] In order to solve the technical problems in the above background technology, Figure 1 This is a flow chart of the IoT-based intelligent commodity quality inspection method provided in an embodiment of the present disclosure. The IoT-based intelligent commodity quality inspection method is introduced in detail below.

[0019] Step S201: Obtain a first training set, where the first training set includes a plurality of multimodal fresh food monitoring training data, and a first preset fresh food state target value and a preset fresh food state type that match each of the plurality of multimodal fresh food monitoring training data. The multimodal fresh food monitoring training data is monitoring data collected by a simulated multimodal sensor.

[0020] Step S202: extracting fresh produce monitoring features that match each of the plurality of multimodal fresh produce monitoring training data using a product quality inspection model;

[0021] Step S203: Using the commodity quality inspection model, target values are estimated for each of the multiple multimodal fresh food monitoring training data based on the fresh food monitoring features that each of the multiple multimodal fresh food monitoring training data matches, thereby obtaining first estimated fresh food target values that each of the multiple multimodal fresh food monitoring training data matches. Furthermore, using the commodity quality inspection model, state types are estimated for each of the multiple multimodal fresh food monitoring training data, thereby obtaining estimated fresh food state types that each of the multiple multimodal fresh food monitoring training data matches.

[0022] Step S204: determining the freshness estimation loss of the commodity quality inspection model based on the first preset freshness target value, the preset freshness type, the first estimated freshness target value, and the estimated freshness type that match each of the plurality of multimodal freshness monitoring training data;

[0023] Step S205: determining the multimodal alignment loss of the commodity quality inspection model based on the fresh food monitoring features matched to the plurality of multimodal fresh food monitoring training data;

[0024] Step S206: Perform a first optimization process on the commodity quality inspection model based on the freshness estimation loss of the commodity quality inspection model and the multimodal alignment loss of the commodity quality inspection model to obtain a target commodity quality inspection model; the target commodity quality inspection model is used to infer the freshness target value and / or freshness type of the target multimodal freshness monitoring data, and determine the quality inspection result corresponding to the target multimodal freshness monitoring data based on the freshness target value and / or freshness type, and the target multimodal freshness monitoring data is the monitoring data collected by the multimodal sensor.

[0025] In the embodiments of the present invention, this embodiment takes the intelligent quality inspection scenario of fresh goods (such as apples) as an example to explain in detail the specific implementation process of the commodity intelligent quality inspection method and system based on the Internet of Things. The execution subject of this method is a server deployed with a commodity quality inspection model. The server is connected to multimodal sensors (such as temperature and humidity sensors, image acquisition cameras, gas sensors, etc.) through the Internet of Things to receive sensor data and perform model training and quality inspection reasoning.

[0026] The server first needs to obtain the first training set, which is used to initially train the product quality inspection model. Taking into account that the real-time data of multimodal sensors in actual fresh food monitoring scenarios (such as sensors for collecting ambient temperature and humidity, cameras for photographing the appearance of fresh food, and sensors for detecting the concentration of gases such as ethylene) may be limited, the first training set is constructed using simulated data generation. For example, for the quality inspection scenario of apples, the server generates multiple sets of multimodal fresh food monitoring training data by simulating different storage environments (such as 0-10°C refrigeration, 15-25°C normal temperature, and high temperature above 25°C), different storage times (1 day, 3 days, 7 days), and different initial freshness (such as 0 hours, 6 hours, and 12 hours after picking). Each set of data includes: (1) time series data simulated by temperature and humidity sensors (e.g., temperature 2°C, humidity 85%; temperature 5°C, humidity 80%, etc. recorded every hour); (2) RGB image data simulated by image sensors (e.g., images of bright red apples, images with brown spots on some parts, and images of apples that are black and rotten); (3) ethylene concentration data simulated by gas sensors (e.g., 0.1ppm, 0.5ppm, 2.0ppm). Each set of data corresponds to the first preset freshness status target value (e.g., freshness score based on industry standards, ranging from 0 to 100 points, with fresh apples scored 90-100, semi-fresh apples scored 70-89, and rotten apples scored 0-69) and the preset freshness status type (e.g., "fresh," "semi-fresh," and "rotten"). For example, a set of simulated data might correspond to refrigerated apples stored for three days. Its multimodal monitoring data includes: temperature and humidity time series data [(2°C, 85%), (3°C, 83%)], an image showing no spots on the fruit surface (RGB values corresponding to a 90% red component), and an ethylene concentration of 0.3 ppm. The corresponding first preset freshness target score is 85, and the preset freshness type is "sub-fresh." The server integrates all of this simulated data and its annotations to form a first training set containing thousands of samples.

[0027] The server invokes the product quality inspection model to be trained and performs feature extraction on each multimodal fresh produce monitoring training data point in the first training set. The structural design of the product quality inspection model must adapt to the characteristics of multimodal data. For temperature and humidity time series data, the model uses an LSTM (Long Short-Term Memory) network to extract time series features, such as capturing temperature fluctuation trends (e.g., a slow increase from 2°C to 3°C) and humidity stability (e.g., a slight decrease from 85% to 83%). For image data, the model uses a ResNet-50 convolutional neural network to extract visual features, such as identifying the color distribution (percentage of red area) and the number of spots (0) on the apple surface. For gas sensor data, the model uses a fully connected layer to extract statistical features of ethylene concentration (e.g., mean 0.3 ppm, maximum 0.3 ppm). The extracted features of each modality (referred to as first sensor monitoring features and second sensor monitoring features, e.g., image features as first sensor monitoring features and temperature and humidity features as second sensor monitoring features) are fused through a cross-modal fusion layer (e.g., a gated fusion unit) to produce a unified fresh produce monitoring feature vector (with a dimension of 256) corresponding to each training data point. For example, in the refrigerated apple data stored for three days, after model processing, the image feature vector is [0.8, 0.2, 0.1, ...] (indicating a high proportion of red), the temperature and humidity feature vector is [0.1, 0.9, 0.0, ...] (indicating stable temperature and slightly decreased humidity), and the gas feature vector is [0.3, 0.0, ...] (indicating low ethylene concentration). The fused fresh food monitoring feature vector comprehensively reflects the semantic information of "sub-fresh but in good condition."

[0028] Based on the extracted freshness monitoring features, the server performs two inference tasks through the commodity quality inspection model: (1) target value inference (regression task): predicting the freshness status target value (such as freshness score); (2) status type inference (classification task): predicting the freshness status type (such as "fresh", "secondary fresh", "rotten"). Specifically, the regression branch of the model contains two fully connected layers, which map the 256-dimensional freshness monitoring features to a one-dimensional output (i.e., the first inferred freshness target value); the classification branch contains one fully connected layer plus a softmax activation function, which maps the 256-dimensional features to a three-dimensional probability distribution (i.e., the probability of inferring the freshness status type). For example, for the above-mentioned refrigerated apple data stored for 3 days, after its freshness monitoring features are input into the model, the regression branch outputs the first inferred freshness target value of 83 points (close to the preset 85 points), and the classification branch outputs a probability distribution of [0.1, 0.8, 0.1] (i.e., the probability of inferring "secondary fresh" is 80%, which is consistent with the preset "secondary fresh" type).

[0029] The server calculates the freshness prediction loss based on the labeled values of the training data (the first preset freshness target value and the preset freshness type) and the model's inferred values (the first inferred freshness target value and the inferred freshness type). This loss consists of two parts: Freshness indicator regression loss: For regression tasks, the mean squared error (MSE) is used to calculate the loss for a single training data point. For example, if the first preset freshness target value for a training data point is 90 points and the model-inferred first inferred freshness target value is 88 points, the regression loss is (90-88)² = 4. Freshness classification loss: For classification tasks, the cross-entropy loss is used to calculate the loss for a single training data point. For example, if the preset freshness type for a training data point is "fresh" (corresponding to the one-hot encoding [1,0,0]) and the model-inferred probability distribution is [0.7,0.2,0.1], the classification loss is -(1×log0.7+0×log0.2+0×log0.1)≈0.3567.

[0030] The server traverses all the data in the first training set, calculates the regression loss and classification loss for each data point, and finally adds the average of all regression losses to the average of all classification losses to obtain the freshness estimation loss of the product quality inspection model. For example, if the first training set contains 1000 data points, the sum of the regression loss is 2000, and the sum of the classification loss is 300, then the freshness estimation loss is (2000 / 1000) + (300 / 1000) = 2 + 0.3 = 2.3.

[0031] To ensure that the multimodal features extracted by the model (such as image features and temperature and humidity features) are semantically aligned (i.e., different modal features of the same product should describe the same freshness), the server needs to calculate the multimodal alignment loss. The specific steps are as follows:

[0032] For any two training data in the first training set (denoted as data A and data B), the server first extracts the first sensor monitoring feature (such as image feature) and the second sensor monitoring feature (such as temperature and humidity feature) of data A, and calculates the cosine similarity between the two as the first feature deviation (the higher the similarity, the smaller the deviation); then, it extracts the cosine similarity between the first sensor monitoring feature of data A and the second sensor monitoring feature of data B as the second feature deviation, and extracts the cosine similarity between the second sensor monitoring feature of data A and the first sensor monitoring feature of data B as the third feature deviation. The design goal of the multimodal alignment loss is: different modal features of the same data should be highly similar (the first feature deviation approaches 1), and cross-modal features of different data should be less similar (the second and third feature deviations approach 0). Therefore, for data A, its multimodal alignment loss can be defined as: loss A = (1-{first feature deviation}) + ({second feature deviation} + {third feature deviation}). For example, if data A is a "fresh apple" (image features show bright colors, and temperature and humidity features show refrigeration at 0°C), and data B is a "rotten apple" (image features show blackness, and temperature and humidity features show a high temperature of 30°C), then the first feature deviation of data A (the similarity between the image and temperature and humidity features) is 0.9 (high similarity), the second feature deviation (the similarity between the image features of data A and the temperature and humidity features of data B) is 0.2 (low similarity), and the third feature deviation (the similarity between the temperature and humidity features of data A and the image features of data B) is 0.1 (low similarity). The multimodal alignment loss of data A is (1-0.9)+(0.2+0.1)=0.1+0.3=0.4. The server traverses all training data pairs, calculates the multimodal alignment loss for each pair of data, and takes the average as the multimodal alignment loss of the product quality inspection model.

[0033] The server takes the weighted sum of the freshness state inference loss and the multimodal alignment loss (e.g., weights of 0.7 and 0.3 respectively) to obtain the target loss of the model (e.g., 2.3×0.7+0.4×0.3=1.61+0.12=1.73). If the model has not reached convergence (e.g., the decrease in target loss is less than 0.01 after 10 consecutive iterations), the target loss is backpropagated through the Adam optimizer to update the model parameters (such as LSTM weights, ResNet convolution kernel parameters, fully connected layer biases, etc.). Repeat this process until the model converges to obtain the optimized product quality inspection model. In order to further improve the generalization ability of the model on real data, the server needs to use a second training set (containing freshness monitoring data collected by real multimodal sensors and its second preset freshness state target value) to calibrate the optimized model. First, the server groups the second training set according to the preset freshness status target values (such as 80-100 points, 60-79 points, and 0-59 points), and randomly extracts 70% of the data from each group as the target calibration training data set (for fine-tuning the model), and the remaining 30% as the target quality inspection verification data set (for evaluating model performance). For example, the second training set contains 1,000 sets of real data, of which 500 are in the "80-100 points" group, 350 of which are extracted as calibration data and 150 as verification data; 300 are in the "60-79 points" group, 210 of which are extracted as calibration data and 90 as verification data; and 200 are in the "0-59 points" group, 140 of which are extracted as calibration data and 60 as verification data. The server uses the target calibration training data set to fine-tune the optimized model (only adjusting the parameters of the regression branch), and then evaluates the classification performance of the model using the target quality inspection verification data set. Specifically, for an apple sample in the validation data (with a preset target value of 85 points), the model extracts its freshness monitoring features and calculates the cosine similarity between the features and each preset target value (such as the mean feature of 80-100 points and the mean feature of 60-79 points), and obtains the confidence that the sample belongs to each target value interval (such as the confidence of 80-100 points is 0.9, and the confidence of 60-79 points is 0.1). If the confidence predicted by the model is low in matching the true target value (such as the true value is 85 points but the model predicts a confidence of 60-79 points of 0.6), the cross entropy loss is calculated and the model parameters are continuously optimized until the classification loss on the validation set is lower than the threshold (such as 0.1), and the target product quality inspection model is finally obtained.

[0034] After the target commodity quality inspection model is deployed, it can receive target multimodal fresh food monitoring data collected by IoT multimodal sensors in real time (such as temperature and humidity time series data, real-time images, and ethylene concentration data of apples in a supermarket refrigerator). The server calls the model to extract fresh food monitoring features, infers the fresh food status target value (such as 92 points) and status type (such as "fresh"), and outputs the quality inspection results based on the preset quality inspection rules (such as freshness ≥85 points is "qualified", and <85 points is "unqualified"). For example, after the multimodal monitoring data of a batch of apples is processed by the model, the inferred target value is 92 points and the status type is "fresh". The server determines that the batch has passed the quality inspection and is allowed to enter the sales link; if the inferred target value is 75 points and the status type is "sub-fresh", it is judged to be unqualified and needs to be sorted or discounted.

[0035] In summary, this method constructs an intelligent commodity quality inspection model suitable for IoT scenarios through steps such as simulated data pre-training, multimodal feature alignment, and real data calibration. It can efficiently and accurately evaluate the freshness and status of fresh products, providing an intelligent solution for quality control in the fresh food supply chain.

[0036] In an embodiment of the present invention, the fresh food status estimation loss of the commodity quality inspection model is determined based on the first preset fresh food status target value, preset fresh food status type, first inferred fresh food target value and inferred fresh food status type that are matched with each of the multiple multimodal fresh food monitoring training data. This can be implemented through the following examples.

[0037] Determining a freshness indicator regression loss corresponding to the first multimodal freshness monitoring data based on a first preset freshness state target value and the first estimated freshness target value corresponding to the first multimodal freshness monitoring data; the first multimodal freshness monitoring data belonging to the plurality of multimodal freshness monitoring training data;

[0038] Determining a fresh food status classification loss corresponding to the first multimodal fresh food monitoring data according to a preset fresh food status type corresponding to the first multimodal fresh food monitoring data and the inferred fresh food status type;

[0039] When the fresh food status classification loss and fresh food index regression loss respectively matched by the multiple multimodal fresh food monitoring training data are determined, the fresh food status classification loss and fresh food index regression loss respectively matched by the multiple multimodal fresh food monitoring training data are used as the fresh food status inference loss of the commodity quality inspection model.

[0040] In an embodiment of the present invention, the server first selects a random set of multimodal fresh produce monitoring training data from the first training set (referred to as the "first multimodal fresh produce monitoring data"). For example, consider a set of refrigerated apple data stored for three days: its first preset freshness target value (manually labeled according to industry standards) is 85 (indicating "sub-fresh but close to fresh"). However, after the product quality inspection model estimates this target value, it outputs a first estimated freshness target value of 83 (the model's predicted freshness score). To measure the error in the model's estimated target value, the server calculates the difference between the two. Specifically, the regression loss calculation logic is to take the square of the difference between the preset target value and the estimated target value as the regression loss for that data. For example, the difference between 85 and 83 is 2 points, which is squared to 4, representing the freshness indicator regression loss for this set of data. The smaller this value, the closer the model's estimated target value is to the actual value. For the same set of apple data, the preset freshness status type (also manually labeled) is "sub-fresh" (status types are categorized as "fresh," "sub-fresh," and "rotten"). The product quality inspection model outputs a probability distribution of the inferred freshness status through the classification branch, for example, [10% probability of "fresh," 80% probability of "less fresh," 10% probability of "rotten"]. The server needs to evaluate the accuracy of the model's inferred status type. The classification loss is calculated based on the degree of match between the model's inferred status type probability and the true status type. For example, if the true status type is "less fresh" and the model infers an 80% probability of "less fresh," the classification loss is calculated based on this probability—the higher the probability, the smaller the loss. Specifically, the server takes the logarithm of the probability of "less fresh" (logarithmic operations amplify errors with low probabilities) and then takes its inverse to calculate the freshness classification loss for that data set. For example, the logarithm corresponding to an 80% probability is approximately -0.223, and the inverse gives 0.223, which is the classification loss for this data set. The server iterates through all the multimodal freshness monitoring training data in the first training set (for example, 1000 sets of apple monitoring data), repeating the above steps for each set, and calculating the corresponding regression loss and classification loss. For example, another set of data in the first training set consists of fresh apples stored for one day: the preset target value is 95 ("fresh"), the model infers a target value of 94, and the regression loss is (95 - 94) squared, or 1. The preset state type is "fresh," and the model infers a 90% probability of "fresh." The classification loss is the inverse of the logarithm of 90%, or approximately 0.105. Another example is a set of rotten apples stored for seven days: the preset target value is 50, the model infers a 52, and the regression loss is (50 - 52) squared, or 4. The preset state type is "rotten," and the model infers an 80% probability of "rotten." The classification loss is also approximately 0.223.The server adds up the regression losses of all 1,000 sets of data to obtain the total regression loss (e.g., 2,000), and then divides it by the amount of data (1,000) to obtain the average regression loss (2). Similarly, all classification losses are added up (e.g., 300) and divided by the amount of data to obtain the average classification loss (0.3). Finally, the server adds the average regression loss and the average classification loss (2+0.3=2.3) as the freshness status inference loss of the product quality inspection model. This loss value comprehensively reflects the overall error of the model in the target value regression and status type classification tasks. The server will subsequently adjust the model parameters (such as the weights and biases of the neural network) based on this loss value, and improve the accuracy of the model's inference of the freshness status by reducing the loss value, ensuring that it can more accurately judge the freshness and status type of fresh products based on multimodal monitoring data. In summary, the server calculates the regression loss and classification loss of the training data one by one and summarizes the model's freshness status inference loss, providing a clear quantitative target for model optimization training. This ensures that the model can effectively learn the relationship between multimodal monitoring data and freshness status, and ultimately achieve intelligent quality inspection of fresh products.

[0041] In an embodiment of the present invention, the fresh food monitoring feature includes a first sensor monitoring feature and a second sensor monitoring feature;

[0042] The determination of the multimodal alignment loss of the commodity quality inspection model based on the fresh food monitoring features matched to the multiple multimodal fresh food monitoring training data can be implemented through the following example.

[0043] Obtaining a first feature deviation between a first sensor monitoring feature of the first multimodal fresh food monitoring training data and a second sensor monitoring feature of the first multimodal fresh food monitoring training data;

[0044] Obtaining a second feature deviation between a first sensor monitoring feature of the first multimodal fresh food monitoring training data and a second sensor monitoring feature of the second multimodal fresh food monitoring training data, and determining a third feature deviation between the second sensor monitoring feature of the first multimodal fresh food monitoring training data and the first sensor monitoring feature of the second multimodal fresh food monitoring training data; the second multimodal fresh food monitoring training data is any multimodal fresh food monitoring training data among the multiple multimodal fresh food monitoring training data except the first multimodal fresh food monitoring training data;

[0045] Determining a multimodal alignment loss of the commodity quality inspection model with respect to the first multimodal fresh produce monitoring training data based on the first feature deviation, the second feature deviation, and the third feature deviation;

[0046] When the multimodal alignment losses of the product quality inspection model for the multiple multimodal fresh food monitoring training data are determined, the multimodal alignment losses of the product quality inspection model for the multiple multimodal fresh food monitoring training data are used as the multimodal alignment losses of the product quality inspection model.

[0047] In an exemplary embodiment of the present invention, the server first specifies that freshness monitoring features are extracted from two types of sensor data: first sensor monitoring features (e.g., visual features such as the apple's color and spots) extracted by a first sensor (e.g., an image acquisition camera), and second sensor monitoring features (e.g., temporal features such as the storage environment's temperature and humidity) extracted by a second sensor (e.g., a temperature and humidity sensor). For example, two sets of data from the first training set are as follows: Data A: Refrigerated apples stored for three days. The image captured by the image sensor shows the fruit's surface as rosy and spotless (the first sensor monitoring feature can be understood as a "bright color visual feature"). The temperature and humidity sensor records the environment as 0°C refrigerated and 85% humidity (the second sensor monitoring feature can be understood as a "low-temperature, stable temperature and humidity feature"). Its true freshness state is "sub-fresh." Data B: Apples stored at room temperature for seven days. The image captured by the image sensor shows the fruit's surface as black and spotted (the first sensor monitoring feature can be understood as a "rotten visual feature"). The temperature and humidity sensor records the environment as 25°C normal temperature and 70% humidity (the second sensor monitoring feature can be understood as a "high-temperature, fluctuating temperature and humidity feature"). Its true freshness state is "rotten." The server needs to use data A as the "first multimodal fresh food monitoring training data" and data B as the "second multimodal fresh food monitoring training data" (that is, any other data except data A) to calculate the feature deviation between the two. The server first calculates the "first feature deviation" between the first sensor monitoring feature of data A and the second sensor monitoring feature. This deviation reflects whether the image features and temperature and humidity features of the same apple describe the same fresh state. For example, the image features (bright colors) and temperature and humidity features (low temperature and stability) of data A both point to a "sub-fresh but well-preserved" state, so the features of the two should be highly similar. The server quantifies this similarity by calculating the cosine similarity of the feature vectors (a method to measure the consistency of the direction of the vector): the higher the similarity, the smaller the deviation. For example, the image feature vector of data A is [0.8, 0.2, 0.1] (indicating a high proportion of red components), and the temperature and humidity feature vector is [0.9, 0.1, 0.0] (indicating stable temperature and suitable humidity). The cosine similarity between the two is 0.9 (close to 1). The first feature deviation can be understood as "1 similarity", that is, 0.1 (small deviation, indicating good modal alignment). To ensure that the model can distinguish the states of different apples, the server needs to calculate the cross-modal feature deviation of data A and data B: Second feature deviation: The deviation between the first sensor monitoring feature of data A (bright color visual feature) and the second sensor monitoring feature of data B (high temperature and humidity feature). Since data A is "semi-fresh" and data B is "rotten", the two states are different, so these two features should be quite different.For example, if the cosine similarity between the image feature vector [0.8, 0.2, 0.1] of data A and the temperature and humidity feature vector [0.2, 0.7, 0.1] of data B (indicating high temperature and low humidity) is 0.3 (low), then the second feature deviation is "1 similarity," or 0.7 (large deviation, indicating high modal discrimination). The third feature deviation is the deviation between the second sensor monitoring feature of data A (low temperature and stable temperature and humidity features) and the first sensor monitoring feature of data B (visual features of decay). Similarly, since data A and data B have different states, these two features should also differ significantly. For example, if the cosine similarity between the temperature and humidity feature vector [0.9, 0.1, 0.0] of data A and the image feature vector [0.1, 0.8, 0.1] of data B (indicating dark color) is 0.2 (lower), then the third feature deviation is "1 similarity," or 0.8 (large deviation, indicating strong modal discrimination). Based on these three deviations, the server calculates the multimodal alignment loss for data A. The loss is designed based on the following logic: cross-modal features of the same data should be as similar as possible (small deviation in the first feature), while cross-modal features of different data should be as dissimilar as possible (large deviation in the second and third features). Therefore, the multimodal alignment loss for data A is "first feature deviation + second feature deviation + third feature deviation." Taking data A as an example, the first feature deviation is 0.1, the second feature deviation is 0.7, and the third feature deviation is 0.8, resulting in a total loss of 0.1 + 0.7 + 0.8 = 1.6. A larger value indicates a poorer alignment of the multimodal features; conversely, a smaller value indicates a better ability of the model to associate different sensor features from the same apple while distinguishing between different apples. The server iterates through all the multimodal fresh produce monitoring training data in the first training set (e.g., 1,000 sets of apple data) and repeats the above steps for each pair of data (data A and data B, data A and data C, etc.). The multimodal alignment loss is calculated for each pair and the average of all losses is taken as the multimodal alignment loss for the product quality inspection model. For example, if the average loss for each pair of data in 1,000 sets of data is 1.2, then the model's multimodal alignment loss is 1.2. This loss value will be combined with the previously calculated "freshness state inference loss" to jointly guide model optimization. The server adjusts model parameters (such as the convolution kernel of the image feature extraction layer and the LSTM weights of the temperature and humidity feature extraction layer) to reduce the multimodal alignment loss, so that the multimodal features extracted by the model are more in line with the semantic alignment requirement of "different sensor data for the same product describes the same state, and sensor data for different products describes different states." In summary, by calculating the cross-modal feature consistency of the same data, the cross-modal feature difference of different data, and summarizing the loss of all data, the server ensures that the product quality inspection model can effectively integrate multimodal sensor data, improve the accuracy of freshness state inference, and provide a reliable feature foundation for subsequent intelligent quality inspection.

[0048] In an embodiment of the present invention, the product quality inspection model is first optimized based on the freshness estimation loss of the product quality inspection model and the multimodal alignment loss of the product quality inspection model to obtain a target product quality inspection model, which can be implemented through the following examples.

[0049] Determining a target loss of the product quality inspection model based on the freshness estimation loss of the product quality inspection model and the multimodal alignment loss of the product quality inspection model;

[0050] If the product quality inspection model has not reached a convergence state, performing a first optimization process on the product quality inspection model using the target loss until the product quality inspection model reaches the convergence state, thereby obtaining an optimized product quality inspection model;

[0051] The target product quality inspection model is determined based on the optimized product quality inspection model.

[0052] In an embodiment of the present invention, illustratively, the server first needs to combine the fresh state inference loss with the multimodal alignment loss to form the "target loss" of the model as the core indicator of optimization. For example, the server calculates through the first training set: the fresh state inference loss is 2.3 (obtained by the weighted sum of the regression loss 2 and the classification loss 0.3), and the multimodal alignment loss is 1.2 (obtained by the average of the multimodal feature deviations of all data). In order to balance the importance of the two types of tasks, the server assigns weights to the two (such as the fresh state inference loss weight 0.7, the multimodal alignment loss weight 0.3), and the target loss is calculated as: target loss = 2.3 × 0.7 + 1.2 × 0.3 = 1.61 + 0.36 = 1.97. This value comprehensively reflects the overall error of the model in "inference accuracy" and "multimodal feature alignment", and is the core goal of subsequent optimization. If the model has not reached convergence (such as the loss is still significantly decreasing), the server will optimize the model through the target loss. The optimization process is based on the principle of gradient descent: the server first calculates the gradient of the target loss with respect to the parameters of each model layer (such as the convolution kernel weights in the image feature extraction layer, the LSTM unit biases in the temperature and humidity feature extraction layer, and the weights of the fully connected layers in the regression / classification branches). It then adjusts the parameters in the opposite direction of the gradient to reduce the target loss. Taking Apple monitoring data as an example, in the model's initial state, the weight of a convolution kernel in the image feature extraction layer is 0.5, and the bias of the LSTM unit in the temperature and humidity feature extraction layer is 0.3. After calculating the gradient of the target loss, the server found that increasing the convolution kernel weight (for example, to 0.52) reduces the deviation between image features and temperature and humidity features (reducing multimodal alignment loss). Simultaneously, adjusting the LSTM bias (for example, to 0.28) improves the accuracy of freshness score estimation (reducing freshness estimation loss). Through these parameter adjustments, the model's target loss dropped from 1.97 to 1.85 (after the first round of optimization). The server continuously monitors changes in the target loss until the model reaches convergence. The convergence criteria are usually: in multiple consecutive iterations (such as 10 rounds), the decrease in the target loss is less than a preset threshold (such as 0.01), or the maximum number of iterations (such as 100 rounds) is reached. For example, the server iteratively optimizes the model: after the first round of optimization, the target loss drops to 1.85; after the fifth round of optimization, the loss drops to 1.52; after the 15th round of optimization, the loss drops to 1.21; from the 20th to the 30th round, the loss stabilizes between 1.18-1.19 (the change in each round is less than 0.01). At this point, the server determines that the model has converged, stops optimization, and obtains the "optimized product quality inspection model." The optimized model has achieved good performance on the first training set (simulated data), but to ensure that it works accurately in real IoT scenarios (such as processing apple monitoring data collected by real sensors), the server needs to further verify and adjust the model.For example, the server obtains the second training set (containing apple data collected by real multimodal sensors, such as real-time temperature and humidity, images, ethylene concentration data of apples in supermarket refrigerators, and manually labeled freshness scores), extracts part of the data (such as 70%) as the calibration set, and the remaining (30%) as the validation set. The server uses the calibration set to fine-tune the optimized model (only adjusting the parameters of the regression branch) and evaluates the model performance through the validation set: if the model's inferred value for an apple sample in the validation set (real freshness 85 points) is 84 points (error 1 point), and the state type is inferred to be "sub-fresh" (consistent with the real type), it means that the model performs well on the real data; if the inferred value error exceeds 2 points or the state type is wrong, the parameters are further adjusted. Finally, the server will determine the optimized model that has passed the verification as the "target product quality inspection model." The optimized target product quality inspection model has shown significant improvement in actual applications. For example, for a set of real apple data (refrigerated apples stored for 2 days, with a real freshness of 90 points and a status type of "fresh"): the model before optimization may infer that the freshness is 87 points (an error of 3 points) and the status type is "sub-fresh" (wrong); the optimized target model infers that the freshness is 89 points (an error of 1 point) and the status type is "fresh" (correct), and the similarity between the image features and the temperature and humidity features is improved from 0.7 to 0.9 (better multimodal alignment). This shows that by integrating the optimization of the fresh state inference loss and the multimodal alignment loss, the model not only improves the accuracy of the inference of freshness and status type, but also enhances the fusion capability of multimodal sensor data, and can more reliably serve the intelligent quality inspection of fresh products. In summary, the server finally obtains a target product quality inspection model suitable for real IoT scenarios through the calculation of target loss, iterative optimization of parameters and convergence judgment, providing an intelligent core tool for quality control of the fresh supply chain.

[0053] In an embodiment of the present invention, the freshness state estimation loss of the commodity quality inspection model includes the freshness index regression loss and the freshness state classification loss respectively matched with the multiple multimodal freshness monitoring training data; the multimodal alignment loss of the commodity quality inspection model includes the multimodal alignment loss of the commodity quality inspection model for the multiple multimodal freshness monitoring training data respectively;

[0054] The target loss of the product quality inspection model is determined based on the freshness estimation loss of the product quality inspection model and the multimodal alignment loss of the product quality inspection model. This can be implemented through the following example.

[0055] Performing a multimodal fusion calculation on the fresh food indicator regression loss, the fresh food state classification loss, and the multimodal alignment loss corresponding to the first multimodal fresh food monitoring training data to obtain the fresh food quality inspection loss of the first multimodal fresh food monitoring training data; the first multimodal fresh food monitoring data belongs to the multiple multimodal fresh food monitoring training data;

[0056] When the fresh food quality inspection losses matched by each of the multiple multimodal fresh food monitoring training data are determined, the fresh food quality inspection losses matched by each of the multiple multimodal fresh food monitoring training data are accumulated and integrated to obtain the target loss of the commodity quality inspection model.

[0057] In an embodiment of the present invention, illustratively, the server first processes any one of the multimodal fresh food monitoring training data in the first training set (referred to as the "first multimodal fresh food monitoring data"). For example, a set of refrigerated apple data stored for 3 days (data A) is selected: Freshness index regression loss: The first preset freshness state target value of the data is 85 points (secondary freshness), and the first inferred freshness target value inferred by the model is 83 points. The regression loss is the square of (85-83), that is, 4 (reflecting the target value inference error). Freshness state classification loss: The preset freshness state type of the data is "secondary freshness", and the model infers that the probability of "secondary freshness" is 80%. The classification loss is the inverse of the logarithm of 80%, which is about 0.223 (reflecting the state type inference error). Multimodal alignment loss: The deviation between the first sensor feature (image features, such as vibrant color) and the second sensor feature (temperature and humidity features, such as low-temperature stability) for this data is 0.1. The cross-modal feature deviations from another set of rotten apple data (data B) are 0.7 and 0.8, respectively. The total multimodal alignment loss is 0.1 + 0.7 + 0.8 = 1.6 (reflecting the semantic consistency error of the multimodal features). These three types of losses reflect model deficiencies from different perspectives: the regression loss focuses on the accuracy of numerical inference, the classification loss focuses on the correctness of type judgment, and the multimodal alignment loss focuses on the effectiveness of sensor data fusion. The server needs to fuse these three types of losses for data A into a comprehensive "fresh produce quality inspection loss" to comprehensively evaluate the model's performance on this data. This fusion method should balance the importance of the three tasks. For example, a weight of 0.5 is assigned to the regression loss (prioritizing numerical accuracy), a weight of 0.3 to the classification loss (secondarily prioritizing type judgment), and a weight of 0.2 to the multimodal alignment loss (lastly prioritizing feature fusion). During the specific calculation, the server multiplies the regression loss (4) of data A by 0.5 to get 2; the classification loss (0.223) by 0.3 to get 0.067; and the multimodal alignment loss (1.6) by 0.2 to get 0.32. After adding the three together, the fresh food quality inspection loss of data A is 2+0.067+0.32=2.387. The smaller this value is, the better the overall performance of the model on this data. The server needs to repeat the above steps for all multimodal fresh food monitoring training data in the first training set (for example, 1000 sets of apple data), calculate the fresh food quality inspection loss of each data one by one, and finally accumulate the losses of all data to obtain the target loss of the commodity quality inspection model.For example, another set of fresh apple data stored for one day (data C): regression loss: the preset target value is 95 points, the estimated value is 94 points, and the loss is (95-94)2=1; classification loss: the preset type is "fresh", the estimated probability is 90%, and the loss is about 0.105; multimodal alignment loss: the deviation between the image feature (bright) and the temperature and humidity feature (0℃ refrigerated) is 0.05, and the cross-modal deviations from the other set of less fresh data are 0.6 and 0.7 respectively, and the total alignment loss is 0.05+0.6+0.7=1.35; the fresh quality inspection loss after fusion is: 1×0.5+0.105×0.3+1.35×0.2=0.5+0.0315+0.27=0.8015. For another example, consider a set of rotten apple data (data D) stored for seven days: regression loss: given a preset target value of 50 and an estimated value of 52, the loss is (50-52)² = 4; classification loss: given a preset class of "rotten" and an estimated probability of 80%, the loss is approximately 0.223; multimodal alignment loss: the deviation between the image feature (blackening) and the temperature and humidity feature (high temperature of 25°C) is 0.1, and the cross-modal deviations from the fresh data are 0.8 and 0.9, respectively, resulting in a total alignment loss of 0.1 + 0.8 + 0.9 = 1.8; the fused fresh food quality inspection loss is 4 × 0.5 + 0.223 × 0.3 + 1.8 × 0.2 = 2 + 0.067 + 0.36 = 2.427. After the server traverses 1000 sets of data, for example, if the total fresh food quality inspection loss for all data is 2000, the model's target loss is 2000 (total loss) divided by 1000 (data volume), or 2.0. This numerical value comprehensively reflects the overall error of the model on all training data and is the core goal of model optimization. The design of the target loss ensures that the model pays attention to both "inference accuracy" and "multimodal feature alignment" during the optimization process. For example, if after a round of optimization, the regression loss of data A drops to 3 (the inferred value is closer to 85 points), the classification loss drops to 0.15 (the probability of inferring "sub-freshness" increases to 85%), and the multimodal alignment loss drops to 1.2 (the image is more consistent with the temperature and humidity features), then its fresh food quality inspection loss becomes 3×0.5+0.15×0.3+1.2×0.2=1.5+0.045+0.24=1.785, which is lower than the 2.387 before optimization, indicating that the overall performance of the model on this data has improved. Through continuous optimization, the target loss of the model will gradually decrease and eventually reach a convergence state (such as the loss change is less than 0.01 for 10 consecutive rounds). At this point, the model can not only accurately infer the freshness score and condition type of fresh produce, but also effectively integrate multimodal sensor data (for example, images are highly correlated with temperature and humidity features), thereby reliably performing intelligent quality inspection tasks for fresh produce in real-world IoT scenarios. In summary, the server fuses the three types of losses for each training data point and aggregates the results across all data points to form the core target loss for model optimization.This process ensures balanced optimization of the model across multiple tasks (regression, classification, and feature alignment), providing key guarantees for the accuracy and robustness of intelligent quality inspection.

[0058] In the embodiment of the present invention, determining the target commodity quality inspection model based on the optimized commodity quality inspection model can be implemented through the following examples.

[0059] Obtaining a second training set, the second training set including a plurality of multimodal fresh food monitoring training data and a second preset fresh food status target value matched to each of the plurality of multimodal fresh food monitoring training data;

[0060] Acquire multiple multimodal fresh food monitoring training data from the multiple multimodal fresh food monitoring training data and determine them as a target calibration training data set; acquire multiple multimodal fresh food monitoring training data from the multiple multimodal fresh food monitoring training data and determine them as a target quality inspection verification data set; the target calibration training data set and the target quality inspection verification data set are mutually disjoint;

[0061] Executing a training process on the optimized product quality inspection model according to the target calibration training data set to obtain a pending product quality inspection model;

[0062] Determining the freshness status target value classification loss of the pending commodity quality inspection model based on the target quality inspection verification data set;

[0063] The to-be-determined commodity quality inspection model is subjected to a second optimization process through the freshness state target value classification loss of the to-be-determined commodity quality inspection model to obtain a target commodity quality inspection model.

[0064] In an embodiment of the present invention, for example, the server first obtains a second training set, which is composed of apple monitoring data collected by real multimodal sensors (such as temperature and humidity sensors, cameras, and ethylene sensors in supermarket refrigerators), and contains thousands of sets of samples. Each set of data contains: (1) temperature and humidity time series data (such as temperature fluctuations within ±0.5°C within 24 hours under 0°C refrigeration); (2) image data (such as apple images taken from different angles, showing that the fruit surface has no spots or a small number of spots); (3) ethylene concentration data (such as a low concentration of 0.2ppm). Each set of data is also marked with a second preset fresh state target value (such as a freshness score manually evaluated based on appearance, smell, etc., ranging from 0 to 100 points). For example, a set of real data corresponds to apples refrigerated in a supermarket for 3 days, and its multimodal monitoring data is: temperature and humidity [(0.2°C, 86%), (0.5°C, 85%)], the image shows that the fruit surface is rosy (the red component accounts for 85%), the ethylene concentration is 0.25ppm, and the second preset target value is 88 points (sub-fresh). The server needs to divide the second training set into a target calibration training data set (for fine-tuning the model) and a target quality inspection verification data set (for evaluating performance), and the two do not overlap. During the specific operation, the server groups the data according to the second preset target value (such as three groups of "80-100 points", "60-79 points" and "0-59 points"), and randomly extracts 70% of each group as a calibration set and 30% as a verification set. For example, the second training set contains 1,000 groups of data, of which 500 are in the "80-100 points" group, 350 of which are extracted as calibration sets and 150 as verification sets; 300 are in the "60-79 points" group, 210 of which are extracted as calibration sets and 90 as verification sets; and 200 are in the "0-59 points" group, 140 of which are extracted as calibration sets and 60 as verification sets. The server uses the target calibration training data set to fine-tune the optimized product quality inspection model to obtain the "pending product quality inspection model". During fine-tuning, the server only adjusts the regression branch of the model (the fully connected layer responsible for inferring the freshness score), because the distribution of simulated data and real data may be different (such as smaller fluctuations in real temperature and humidity, and more uniform image lighting), and the regression parameters need to be corrected through real data. For example, the second preset target value of an apple data in the calibration set is 92 points (fresh), and the initial inferred value of the model is 90 points (an error of 2 points). The server adjusts the weight of the regression layer through backpropagation to make the inferred value closer to 92 points (such as the adjusted inferred value is 91.5 points, with an error of 0.5 points). The server uses the target quality inspection verification data set to evaluate the performance of the pending model, focusing on calculating the "fresh status target value classification loss" (that is, the model's classification accuracy for the freshness score interval). For example, the second preset target value of an apple in the verification set is 85 points (belonging to the "80-100 points" interval), and the model needs to infer the probability that it belongs to this interval.If the model infers that the apple is "80-100 points" with a probability of 90% (correct), the loss is low; if the inferred probability is 60% (with an error tendency towards "60-79 points"), the loss is high. The server traverses all the data in the validation set and calculates the average classification loss (such as 0.15). If the loss is higher than the threshold (such as 0.1), it continues to adjust the model parameters (such as the bias of the regression layer) until the loss drops to 0.08 (meets the target). After calibration and verification optimization, the server determines the final pending product quality inspection model as the "target product quality inspection model." The model shows higher accuracy on real data: for example, for a newly-stocked apple in a supermarket (with an actual freshness score of 90 points), the model infers a value of 89.5 points (with an error of 0.5 points), and the probability of judging that it belongs to the "80-100 points" range is 95% (correct), which is a significant improvement compared to the pre-optimization (error of 2 points, probability of 70%). In summary, the server ensures that the product quality inspection model is adapted from the simulation scenario to the actual IoT environment through calibration and verification with real data, and ultimately obtains the target model that can reliably perform intelligent quality inspection of fresh food.

[0065] In an embodiment of the present invention, the steps of obtaining multiple multimodal fresh food monitoring training data from the multiple multimodal fresh food monitoring training data and determining them as a target calibration training data set, and obtaining multiple multimodal fresh food monitoring training data from the multiple multimodal fresh food monitoring training data and determining them as a target quality inspection verification data set can be implemented through the following examples.

[0066] Extracting a plurality of second preset fresh food state target values from the second preset fresh food state target values respectively matched by the plurality of multimodal fresh food monitoring training data;

[0067] Extracting a plurality of multimodal fresh food monitoring training data corresponding to a reference preset fresh food state target value from the plurality of multimodal fresh food monitoring training data; the reference preset fresh food state target value belongs to a plurality of second preset fresh food state target values;

[0068] If a plurality of multimodal fresh food monitoring training data that each matches the plurality of second preset fresh food state target values are extracted from the plurality of multimodal fresh food monitoring training data, the plurality of multimodal fresh food monitoring training data that each matches the plurality of second preset fresh food state target values are grouped to obtain a calibration training data set and a quality inspection verification data set that each matches the plurality of second preset fresh food state target values;

[0069] The calibration training data sets that each match the plurality of second preset fresh food state target values are used as target calibration training data sets; the total number of multimodal fresh food monitoring training data in the calibration training data sets that each match the plurality of second preset fresh food state target values is the number of data in the target calibration training data sets;

[0070] The quality inspection and verification data sets that each match the multiple second preset fresh food status target values are used as the target quality inspection and verification data sets; the total number of multimodal fresh food monitoring training data in the quality inspection and verification data sets that each match the multiple second preset fresh food status target values is the number of data in the target quality inspection and verification data sets.

[0071] In an embodiment of the present invention, illustratively, the server first reads the "second preset fresh state target value" (i.e., manually labeled freshness score, such as 85 points, 72 points, 50 points, etc.) of all multimodal fresh food monitoring training data in the second training set. For example, the second training set contains 1,000 groups of data, and its second preset target value distribution is: 80-100 points (500 groups), 60-79 points (300 groups), and 0-59 points (200 groups). The server determines the target values of these three intervals as "reference preset fresh state target values" (i.e., the target value range that needs to be covered). The server extracts corresponding data from the second training set for each reference preset fresh state target value. For example: for the reference target value of "80-100 points", the server filters out 500 groups of data with the second preset target value between 80-100 points (such as a group of data marked as 88 points, which belongs to this interval); for the reference target value of "60-79 points", 300 groups of data are filtered out (such as a group of data marked as 72 points, which belongs to this interval); for the reference target value of "0-59 points", 200 groups of data are filtered out (such as a group of data marked as 50 points, which belongs to this interval). This step ensures that the real data corresponding to each reference target value are covered to avoid data loss during subsequent calibration or verification. The server divides the data set corresponding to each reference target value into a calibration training subset and a quality inspection verification subset according to a fixed ratio (such as 7:3). For example, of the 500 data sets for "80-100 points," 350 sets (70%) were randomly selected as the calibration training subset for that reference target value, and the remaining 150 sets (30%) were used as the quality inspection and verification subset. Of the 300 data sets for "60-79 points," 210 sets (70%) were randomly selected as the calibration training subset for that reference target value, and the remaining 90 sets (30%) were used as the quality inspection and verification subset. Of the 200 data sets for "0-59 points," 140 sets (70%) were randomly selected as the calibration training subset for that reference target value, and the remaining 60 sets (30%) were used as the quality inspection and verification subset. During grouping, the server uses a random number generator to ensure uniform data distribution (for example, avoiding continuous sampling of apple data from the same stored batch) to ensure objectivity in calibration and verification. The server then merges all calibration training subsets for all reference target values to create the "target calibration training dataset." For example, 350 data sets for "80-100 points" + 210 data sets for "60-79 points" + 140 data sets for "0-59 points" = 700 data sets, constituting the target calibration training dataset. Similarly, all the QC validation subsets for the reference target values are combined to form the "target QC validation dataset." For example, 150 data sets for "80-100 points" + 90 data sets for "60-79 points" + 60 data sets for "0-59 points" = 300 data sets, constituting the target QC validation dataset. These two datasets are mutually exclusive (no duplicate data), ensuring that the calibration process is independent of the validation data and that the validation results truly reflect the model's performance on new data.By grouping by reference target values, the server ensures that the calibration set and validation set cover apple data in all states: fresh, semi-fresh, and rotten. For example, the calibration set contains 350 groups of fresh / semi-fresh apple data, 210 groups of semi-fresh / nearly rotten data, and 140 groups of rotten data. During training, the model can learn the characteristic differences of multimodal data in different states; the validation set contains 150 groups of fresh / semi-fresh data, 90 groups of semi-fresh / nearly rotten data, and 60 groups of rotten data, which can comprehensively evaluate the accuracy of the model's inferences about various states. In summary, by extracting and grouping data according to reference target values, the server ensures the representativeness and balance of the calibration set and validation set, providing a reliable data foundation for the subsequent real-world adaptation of the model.

[0072] In an embodiment of the present invention, the target quality inspection and verification data set includes a plurality of quality inspection and verification data sets that match each of the second preset fresh status target values. The determination of the fresh status target value classification loss of the pending commodity quality inspection model based on the target quality inspection and verification data set can be implemented through the following example.

[0073] The target value classification is performed on the target multimodal fresh food monitoring training data in the quality inspection verification data set corresponding to the reference preset fresh food state target value by the pending commodity quality inspection model, and target value speculation information about the target multimodal fresh food monitoring training data is obtained; the reference preset fresh food state target value belongs to a plurality of second preset fresh food state target values; the target value speculation information includes that the second speculated fresh food target value of the target multimodal fresh food monitoring training data is a first confidence level of the reference preset fresh food state target value and the second speculated fresh food target value of the target multimodal fresh food monitoring training data is a second confidence level of the remaining preset fresh food state target values; the remaining preset fresh food state target value is any preset fresh food state target value among the plurality of second preset fresh food state target values except the reference preset fresh food state target value;

[0074] Determining, based on the first confidence level and the second confidence level, a training inference loss of the reference preset fresh food state target value on the target multimodal fresh food monitoring training data;

[0075] Determining a target training loss of the reference preset fresh food state target value on the corresponding quality inspection verification data set based on a training inference loss of the reference preset fresh food state target value on the target multimodal fresh food monitoring training data;

[0076] If the target training losses of the multiple second preset fresh status target values on the corresponding quality inspection verification data sets are obtained, the fresh status target value classification loss of the pending commodity quality inspection model is determined based on the multiple target training losses.

[0077] In an embodiment of the present invention, illustratively, the server first determines the reference range of the second preset fresh state target value (such as "80-100 points", "60-79 points", and "0-59 points"), and obtains the quality inspection verification subset corresponding to each reference target value. For example, the quality inspection verification subset of "80-100 points" contains 150 groups of real apple data (such as the second preset target value of a group of data is 88 points, which belongs to this interval); the verification subset of "60-79 points" contains 90 groups of data (such as the second preset target value of a group of data is 72 points); the verification subset of "0-59 points" contains 60 groups of data (such as the second preset target value of a group of data is 50 points). The server calls the pending commodity quality inspection model, performs target value classification on the "target multimodal fresh monitoring training data" in the verification subset of each reference target value, and obtains "target value inference information". For example, for a set of apple data in the "80-100 score" validation subset (true target score: 88), the model extracts its multimodal features (images showing rosy redness without spots, stable temperature and humidity at 0°C, and ethylene concentration of 0.2 ppm). It then outputs a first confidence level of 0.9 for the data belonging to the "80-100 score" range (i.e., the model has a 90% probability of assigning a freshness score of 80-100), a second confidence level of 0.08 for the data belonging to the "60-79 score" range, and a second confidence level of 0.02 for the data belonging to the "0-59 score" range. Based on the confidence levels output by the model, the server calculates the "training inference loss" for the data on the reference target value. The loss calculation logic is: the higher the model's confidence in the true target value, the lower the loss. For example, if the true target value for the apple data above is "80-100 score," the model outputs a first confidence level of 0.9. The server uses cross-entropy loss for this calculation: loss = -log(0.9) ≈ 0.105 (the smaller the logarithm, the lower the loss). If the first confidence level of the model for the other set of "80-100 points" data is only 0.6, the loss is -log(0.6)≈0.511 (a larger loss indicates inaccurate model inference). The server traverses all the data in the validation subset corresponding to the reference target value, calculates the training inference loss for each set of data, and then takes the average as the "target training loss" for the reference target value. For example, the total loss of the 150 sets of data in the "80-100 points" validation subset is 15.75 (e.g., the average loss per set is 0.105), so its target training loss is 15.75 / 150=0.105; the total loss of the 90 sets of data in the "60-79 points" validation subset is 18 (e.g., the average loss per set is 0.2), and the target training loss is 18 / 90=0.2; the total loss of the 60 sets of data in the "0-59 points" validation subset is 18 (e.g., the average loss per set is 0.3), and the target training loss is 18 / 60=0.3.The server takes a weighted average of the target training losses of all reference target values (the weights can be allocated according to the amount of data, such as 50% for "80-100 points", 30% for "60-79 points", and 20% for "0-59 points") to obtain the "fresh status target value classification loss" of the pending product quality inspection model. For example, total loss = 0.105×0.5+0.2×0.3+0.3×0.2=0.0525+0.06+0.06=0.1725. If the loss is higher than the preset threshold (such as 0.15), the server will adjust the model parameters (such as the weight of the regression layer) until the loss drops to 0.12 (meeting the standard), and finally determine the target product quality inspection model. In summary, the server ensures that the model can accurately classify the fresh status target value in real scenarios through confidence assessment, loss calculation and summary of each group of data, providing a key guarantee for the reliability of intelligent quality inspection.

[0078] In an embodiment of the present invention, the target value classification is performed on the target multimodal fresh food monitoring training data in the quality inspection verification data set corresponding to the preset fresh food status target value by the pending commodity quality inspection model to obtain target value inference information about the target multimodal fresh food monitoring training data, which can be implemented through the following examples.

[0079] Determining a mean characteristic corresponding to the reference preset fresh state target value;

[0080] Calling the pending commodity quality inspection model to extract fresh food monitoring features corresponding to the target multimodal fresh food monitoring training data;

[0081] Calculating, based on the mean feature corresponding to the reference preset fresh food state target value and the fresh food monitoring feature corresponding to the target multimodal fresh food monitoring training data, a second inferred fresh food target value corresponding to the target multimodal fresh food monitoring training data as a first confidence level of the reference preset fresh food state target value;

[0082] Determining the mean characteristics corresponding to the remaining preset fresh state target values;

[0083] The pending commodity quality inspection model is called to calculate the second inferred fresh target value corresponding to the target multimodal fresh monitoring training data as the second confidence level of the remaining preset fresh status target value based on the mean feature corresponding to the remaining preset fresh status target value and the fresh monitoring feature corresponding to the target multimodal fresh monitoring training data.

[0084] In an embodiment of the present invention, illustratively, the server first extracts the freshness monitoring features of all apple data belonging to the interval from the target calibration training data set with reference to a preset freshness status target value (such as "80-100 points"), and calculates its mean feature. For example, the calibration training subset of "80-100 points" contains 350 groups of data, and the freshness monitoring features of each group of data are 256-dimensional vectors (obtained by fusing multimodal features such as image, temperature and humidity, and ethylene concentration). The server averages these 350 groups of feature vectors dimension by dimension to obtain the mean feature vector of "80-100 points" (for example, the first three dimensions are [0.85, 0.12, 0.03], indicating that the image color is bright, the temperature and humidity are stable, and the ethylene concentration is low). This mean feature represents the typical feature pattern of "80-100 points" fresh apples. The server selects a set of target multimodal freshness monitoring training data (for example, apple data refrigerated in a supermarket for 3 days) from the quality inspection verification subset of "80-100 points". The multimodal monitoring information of this data includes: the image shows that the fruit surface is rosy and spotless (the RGB value corresponds to the red component accounting for 88%), the temperature and humidity sensors record the environment as refrigerated at 0.5°C (temperature fluctuation within 24 hours is ±0.3°C, humidity is 85% ±2%), and the ethylene sensor detects a concentration of 0.2 ppm. The server calls the pending product quality inspection model to perform multimodal feature extraction on this data, obtaining its freshness monitoring feature vector (for example, the first three dimensions are [0.82, 0.15, 0.02], which is highly similar to the mean feature of "80-100 points"). The server calculates the similarity between the freshness monitoring feature of the target data and the mean feature of "80-100 points" (for example, cosine similarity, a method to measure the consistency of vector direction). For example, if the target data feature vector is [0.82, 0.15, 0.02] and the mean feature vector of "80-100 points" is [0.85, 0.12, 0.03], the cosine similarity between the two is 0.98 (close to 1), indicating that the characteristics of the target data are highly consistent with the typical characteristics of "80-100 points." The server uses this similarity value as the first confidence level (for example, 0.98) for "the second inferred fresh food target value is 80-100 points." The server repeats the above steps, sequentially calculating the similarity between the target data features and the mean features of the remaining preset fresh food status target values (such as "60-79 points" and "0-59 points") to obtain the corresponding second confidence levels. For example, the mean feature vector of "60-79 points" is [0.65, 0.25, 0.1] (the image color is darker, the temperature and humidity fluctuate slightly, and the ethylene concentration is slightly higher). The cosine similarity between the target data feature and the mean feature is 0.75, corresponding to the second confidence level of 0.01 (because the total confidence level needs to be normalized, the actual value may be 0.01); the mean feature vector of "0-59 points" is [0.4, 0.4, 0.2] (the image is dark and spotted, the temperature and humidity fluctuate greatly, and the ethylene concentration is high). The cosine similarity between the target data feature and the mean feature is 0.5, corresponding to the second confidence level of 0.01.Ultimately, the target value inference information for the target data is: 98% confidence for "80-100 points", 1% confidence for "60-79 points", and 1% confidence for "0-59 points". Through the above steps, the server generates confidence levels for each target value interval for each set of verification data. If the true target value of a certain apple data is 88 points (belonging to "80-100 points"), and the model outputs a confidence level of 98% for "80-100 points", it means that the model has accurately identified its freshness interval; if the confidence level is only 60%, the model parameters (such as the weights of the feature extraction layer) need to be adjusted to improve classification accuracy. In summary, the server generates confidence levels for each target value interval for the target multimodal fresh food monitoring training data through mean feature comparison and similarity calculation, providing a quantitative basis for subsequent evaluation of model classification loss and optimization of model performance, ensuring that the model can reliably judge the freshness level of fresh products in real scenarios.

[0085] In the embodiment of the present invention, it is worth noting that the following implementation methods are also provided.

[0086] Obtaining a plurality of second preset fresh food status target values, each of which matches a plurality of reference multimodal fresh food monitoring data; the plurality of second preset fresh food status target values including the plurality of second preset fresh food status target values, each of which matches a plurality of multimodal fresh food monitoring training data; the reference multimodal fresh food monitoring data being independent of the second training set;

[0087] According to the fresh food monitoring characteristics of the benchmark multimodal fresh food monitoring data corresponding to the candidate preset fresh food state target value, multimodal fresh food monitoring data that matches the candidate preset fresh food state target value is searched from the multimodal fresh food monitoring data set, and determined as the candidate multimodal fresh food monitoring data corresponding to the candidate preset fresh food state target value; the candidate preset fresh food state target value belongs to a plurality of second preset fresh food state target values;

[0088] When the candidate multimodal fresh food monitoring data that each matches the multiple second preset fresh food status target values are determined, the second training set is constructed based on the candidate multimodal fresh food monitoring data that each matches the multiple second preset fresh food status target values.

[0089] In this embodiment of the present invention, the server first obtains "baseline multimodal fresh produce monitoring data" that matches multiple second preset fresh produce status target values (e.g., "80-100 points," "60-79 points," and "0-59 points"). This baseline data is independent of the subsequently constructed second training set and typically comes from authoritative laboratories or long-accumulated standard samples. For example, the "80-100 score" benchmark data consists of apple data collected in a laboratory under controlled conditions (0°C refrigeration, 85% humidity, and storage for 1-3 days). The data includes multimodal information such as images (rosy and spotless), temperature and humidity (0.2°C ± 0.5°C), and ethylene concentration (0.1-0.3 ppm). The "60-79 score" benchmark data consists of apple data stored in a laboratory for 5-7 days. The images show a small amount of brown spots, the temperature and humidity are (5°C ± 1°C), and the ethylene concentration is (0.5-1.0 ppm). The "0-59 score" benchmark data consists of rotten apple data stored in a laboratory for more than 10 days. The images are black and have numerous spots, the temperature and humidity are (25°C ± 2°C), and the ethylene concentration is (above 2.0 ppm). These benchmark data were not used for the initial training of the model to ensure the independence of the candidate data subsequently screened. For each candidate preset freshness status target value (such as "80-100 points"), the server extracts the freshness monitoring features of its benchmark data (a 256-dimensional vector fused by multimodal features such as image, temperature and humidity, and ethylene concentration). For example, the feature vector of the "80-100 points" benchmark data is [0.85, 0.12, 0.03, ...] (indicating bright color, stable temperature and humidity, and low ethylene concentration). Subsequently, the server queries the candidate data that matches the benchmark feature from the "multimodal freshness monitoring data set" (such as unlabeled apple data collected by a fresh food warehouse in the past three months, containing tens of thousands of sets of multimodal monitoring records). The matching method is to calculate the similarity between the data features and the benchmark features (such as cosine similarity), and filter out data with a similarity higher than a threshold (such as 0.85) as candidates. For example, a set of apple data was found from warehouse data: the image showed a rosy red surface (red component accounting for 88%), temperature and humidity (0.5℃±0.3℃, humidity 84%), and ethylene concentration of 0.25ppm. The similarity between its feature vector and the "80-100 points" benchmark feature was 0.92 (higher than the threshold), and it was selected as the candidate data for "80-100 points"; another set of data: the image had a small amount of spots (red component accounting for 65%), temperature and humidity (4.8℃±1℃, humidity 80%), and ethylene concentration of 0.7ppm. The similarity with the "60-79 points" benchmark feature was 0.88, and it was selected as the candidate data for "60-79 points"; the third set of data: the image was black (red component accounting for 30%), temperature and humidity (24℃±2℃, humidity 75%), and ethylene concentration of 2.5ppm. The similarity with the "0-59 points" benchmark feature was 0.90, and it was selected as the candidate data for "0-59 points".The server traverses all candidate preset freshness status target values ("80-100 points", "60-79 points", "0-59 points"), summarizes the candidate data corresponding to each target value, and constructs the final second training set. For example, "80-100 points" screens out 500 groups of candidate data; "60-79 points" screens out 300 groups of candidate data; "0-59 points" screens out 200 groups of candidate data. These candidate data are all matched with benchmark features to ensure a strong correlation with the corresponding target value, and are independent of the simulated data of the initial training, and can truly reflect the multimodal feature distribution of apples in real scenes. The second training set constructed by screening candidate data with benchmark data covers apple data in all states of "fresh-sub-fresh-rotten", and each type of data is highly consistent with the typical features of the corresponding freshness level. For example, candidate data of "80-100 points" all have the characteristics of "bright color, stable at low temperature, and low ethylene concentration", and candidate data of "0-59 points" all have the characteristics of "blackened color, high temperature fluctuation, and high ethylene concentration". After calibrating the model with this training set, it can more accurately identify the freshness level of apples in real-world scenarios, avoiding inference bias caused by differences between simulated and real-world data. In summary, by matching features of the baseline data and screening candidate data, the server constructed a highly representative second training set, providing reliable data support for the model's real-world adaptation, ultimately improving the accuracy and practicality of intelligent quality inspection.

[0090] In an embodiment of the present invention, the fresh food monitoring characteristics of the benchmark multimodal fresh food monitoring data corresponding to the candidate preset fresh food status target value are queried from the multimodal fresh food monitoring data set to obtain multimodal fresh food monitoring data that matches the candidate preset fresh food status target value. This can be implemented through the following example.

[0091] Acquire fresh produce monitoring features of each multimodal fresh produce monitoring data in the multimodal fresh produce monitoring data set;

[0092] Determine the multimodal feature deviations between the fresh food monitoring features of the benchmark multimodal fresh food monitoring data corresponding to the candidate preset fresh food status target value and the fresh food monitoring features of each multimodal fresh food monitoring data in the multimodal fresh food monitoring data set; based on the determined multimodal feature deviations, query the multimodal fresh food monitoring data set for the multimodal fresh food monitoring data that matches the candidate preset fresh food status target value, and determine it as the multimodal fresh food monitoring data that matches the candidate preset fresh food status target value.

[0093] In an embodiment of the present invention, the server first accesses a "multimodal fresh produce monitoring data set" (e.g., unlabeled apple monitoring data from a fresh produce warehouse over the past three months, comprising 100,000 records) and invokes a commodity quality inspection model to extract fresh produce monitoring features for each set of data. The features of each set of data are generated by fusing multimodal sensor data such as image, temperature and humidity, and ethylene concentration to form a 256-dimensional feature vector. For example, one set of data images shows a rosy red fruit surface (88% red component), a refrigerated temperature and humidity of 0.5°C (temperature fluctuation of ±0.3°C within 24 hours, humidity 85%), and an ethylene concentration of 0.2ppm. The first three dimensions of its feature vector are [0.82, 0.15, 0.02]. Another set of data images has a small number of spots (65% red component), a temperature and humidity of 5°C (fluctuation of ±1°C, humidity 80%), and an ethylene concentration of 0.7ppm. The first three dimensions of its feature vector are [0.63, 0.28, 0.08]. The server stores the feature vectors of all 100,000 sets of data as a feature library. The server obtains benchmark multimodal fresh produce monitoring data (e.g., laboratory standard samples) corresponding to the candidate preset fresh produce status target value of "80-100 points" and extracts its fresh produce monitoring features (e.g., the first three dimensions are [0.85, 0.12, 0.03]). The server then iterates over each data set in the feature library and calculates the "multimodal feature deviation" between that data feature and the benchmark feature. This deviation is calculated using cosine similarity (a method for measuring the directional consistency of vectors): higher similarity indicates smaller deviation; lower similarity indicates larger deviation. For example: the eigenvector of the first set of data is [0.82, 0.15, 0.02], and its cosine similarity with the benchmark feature [0.85, 0.12, 0.03] is 0.98 (close to 1), and the deviation is 1-0.98=0.02 (the deviation is extremely small, indicating that the features are highly consistent); the eigenvector of the second set of data is [0.63, 0.28, 0.08], and its cosine similarity with the benchmark feature is 0.75, and the deviation is 1-0.75=0.25 (the deviation is large, indicating that the features are significantly different); the eigenvector of the third set of data is [0.4, 0.4, 0.2] (the image is dark, the temperature and humidity fluctuate greatly, and the ethylene concentration is high), and its cosine similarity with the benchmark feature is 0.5, and the deviation is 1-0.5=0.5 (the deviation is extremely large, and the features are significantly different). The server sets a deviation threshold (e.g., 0.15) and retains only data with deviations below this threshold as candidate data for the pre-set freshness status target value of "80-100 points" for multimodal fresh produce monitoring. For example, the first set of data above, with a deviation of 0.02 less than 0.15, is selected; the second set of data, with a deviation of 0.25 greater than 0.15, is excluded; and the third set of data, with a deviation of 0.5, is also excluded. By traversing 100,000 sets of data, the server screened out 500 sets of candidate data with deviations less than 0.15.The multimodal features of these data are highly consistent with the "80-100 points" benchmark features (such as rosy images, stable temperature and humidity, and low ethylene concentration), ensuring a strong correlation between the candidate data and the target value. The candidate data screened through multimodal feature deviations truly reflects the typical characteristic patterns of "80-100 points" fresh apples. For example, among the 500 selected data sets, 90% of the images had a red component accounting for more than 80%, 85% had temperature and humidity fluctuations within ±0.5°C, and 80% had ethylene concentrations below 0.3ppm, which are highly consistent with the characteristic distribution of laboratory benchmark data. This enables the subsequent construction of a second training set to effectively calibrate the model and improve its recognition accuracy for "80-100 points" apples in real scenarios. In summary, through feature extraction, deviation calculation, and threshold screening, the server accurately locates candidate data that matches the candidate preset target value from a large-scale data set, providing key support for constructing a high-quality second training set, ultimately ensuring that the model can reliably perform intelligent quality inspection tasks in real scenarios.

[0094] The embodiment of the present invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned intelligent quality inspection method for goods based on the Internet of Things. Figure 2 As shown, Figure 2 This is a block diagram of the structure of a computer device 100 provided in an embodiment of the present invention. Computer device 100 includes a memory 111, a processor 112, and a communication unit 113. To enable data transmission or exchange, memory 111, processor 112, and communication unit 113 are electrically connected to each other, directly or indirectly. For example, these components can be electrically connected via one or more communication buses or signal lines.

[0095] For illustrative purposes, the foregoing description has been made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed. Numerous modifications and variations are possible in light of the above teachings. These embodiments have been selected and described in order to best illustrate the principles of the present disclosure and its practical application, thereby enabling those skilled in the art to best utilize the present disclosure and to utilize various embodiments with various modifications as appropriate for the specific application contemplated.

Claims

1. A commodity intelligent quality inspection method based on the Internet of Things, characterized in that: include: Obtaining a first training set, the first training set including a plurality of multimodal fresh food monitoring training data, and a first preset fresh food state target value and a preset fresh food state type that are matched to each of the plurality of multimodal fresh food monitoring training data, wherein the multimodal fresh food monitoring training data is monitoring data collected by simulating a multimodal sensor; Extracting fresh food monitoring features that match each of the plurality of multimodal fresh food monitoring training data using a product quality inspection model; The commodity quality inspection model is used to perform target value inference on each of the multiple multimodal fresh food monitoring training data based on the fresh food monitoring features that each of the multiple multimodal fresh food monitoring training data matches, thereby obtaining a first inferred fresh food target value that each of the multiple multimodal fresh food monitoring training data matches. The commodity quality inspection model is used to perform state type inference on the multiple multimodal fresh food monitoring training data, thereby obtaining an inferred fresh food state type that each of the multiple multimodal fresh food monitoring training data matches; Determining the freshness estimation loss of the commodity quality inspection model according to the first preset freshness state target value, the preset freshness state type, the first estimated freshness target value, and the estimated freshness state type that are matched to each of the plurality of multimodal freshness monitoring training data; Determining a multimodal alignment loss of the product quality inspection model based on the fresh food monitoring features matched to the plurality of multimodal fresh food monitoring training data; According to the freshness estimation loss of the commodity quality inspection model and the multimodal alignment loss of the commodity quality inspection model, the commodity quality inspection model is subjected to a first optimization process to obtain a target commodity quality inspection model; the target commodity quality inspection model is used to infer the freshness target value and / or freshness type of the target multimodal freshness monitoring data, and determine the quality inspection result corresponding to the target multimodal freshness monitoring data based on the freshness target value and / or freshness type, and the target multimodal freshness monitoring data is the monitoring data collected by the multimodal sensor.

2. The method according to claim 1, characterized in that The determining of the freshness estimation loss of the commodity quality inspection model according to the first preset freshness target value, the preset freshness type, the first estimated freshness target value, and the estimated freshness type that are matched to each of the plurality of multimodal freshness monitoring training data includes: Determining a freshness indicator regression loss corresponding to the first multimodal freshness monitoring data based on a first preset freshness state target value and the first estimated freshness target value corresponding to the first multimodal freshness monitoring data; the first multimodal freshness monitoring data belonging to the plurality of multimodal freshness monitoring training data; Determining a fresh food status classification loss corresponding to the first multimodal fresh food monitoring data according to a preset fresh food status type corresponding to the first multimodal fresh food monitoring data and the inferred fresh food status type; When the fresh food status classification loss and fresh food index regression loss respectively matched by the multiple multimodal fresh food monitoring training data are determined, the fresh food status classification loss and fresh food index regression loss respectively matched by the multiple multimodal fresh food monitoring training data are used as the fresh food status inference loss of the commodity quality inspection model.

3. The method according to claim 2, characterized in that The fresh food monitoring feature includes a first sensor monitoring feature and a second sensor monitoring feature; Determining the multimodal alignment loss of the commodity quality inspection model based on the fresh food monitoring features matched to the plurality of multimodal fresh food monitoring training data includes: Obtaining a first feature deviation between a first sensor monitoring feature of the first multimodal fresh food monitoring training data and a second sensor monitoring feature of the first multimodal fresh food monitoring training data; Obtaining a second feature deviation between a first sensor monitoring feature of the first multimodal fresh food monitoring training data and a second sensor monitoring feature of the second multimodal fresh food monitoring training data, and determining a third feature deviation between the second sensor monitoring feature of the first multimodal fresh food monitoring training data and the first sensor monitoring feature of the second multimodal fresh food monitoring training data; the second multimodal fresh food monitoring training data is any multimodal fresh food monitoring training data among the multiple multimodal fresh food monitoring training data except the first multimodal fresh food monitoring training data; Determining a multimodal alignment loss of the commodity quality inspection model with respect to the first multimodal fresh produce monitoring training data based on the first feature deviation, the second feature deviation, and the third feature deviation; When the multimodal alignment losses of the product quality inspection model for the multiple multimodal fresh food monitoring training data are determined, the multimodal alignment losses of the product quality inspection model for the multiple multimodal fresh food monitoring training data are used as the multimodal alignment losses of the product quality inspection model.

4. The method according to claim 1, wherein The first optimization process is performed on the product quality inspection model based on the freshness estimation loss of the product quality inspection model and the multimodal alignment loss of the product quality inspection model to obtain a target product quality inspection model, including: Determining a target loss of the product quality inspection model based on the freshness estimation loss of the product quality inspection model and the multimodal alignment loss of the product quality inspection model; If the product quality inspection model has not reached a convergence state, performing a first optimization process on the product quality inspection model using the target loss until the product quality inspection model reaches the convergence state, thereby obtaining an optimized product quality inspection model; The target product quality inspection model is determined based on the optimized product quality inspection model.

5. The method according to claim 4, characterized in that The freshness estimation loss of the product quality inspection model includes the freshness index regression loss and freshness status classification loss of each of the multiple multimodal freshness monitoring training data; the multimodal alignment loss of the product quality inspection model includes the multimodal alignment loss of the product quality inspection model for each of the multiple multimodal freshness monitoring training data; Determining the target loss of the product quality inspection model based on the freshness estimation loss of the product quality inspection model and the multimodal alignment loss of the product quality inspection model includes: Performing a multimodal fusion calculation on the fresh food indicator regression loss, the fresh food state classification loss, and the multimodal alignment loss corresponding to the first multimodal fresh food monitoring training data to obtain the fresh food quality inspection loss of the first multimodal fresh food monitoring training data; the first multimodal fresh food monitoring data belongs to the multiple multimodal fresh food monitoring training data; When the fresh food quality inspection losses matched by each of the multiple multimodal fresh food monitoring training data are determined, the fresh food quality inspection losses matched by each of the multiple multimodal fresh food monitoring training data are accumulated and integrated to obtain the target loss of the commodity quality inspection model.

6. The method according to claim 4, characterized in that The step of determining a target product quality inspection model based on the optimized product quality inspection model includes: Obtaining a second training set, the second training set including a plurality of multimodal fresh food monitoring training data and a second preset fresh food status target value matched to each of the plurality of multimodal fresh food monitoring training data; Acquire multiple multimodal fresh food monitoring training data from the multiple multimodal fresh food monitoring training data and determine them as a target calibration training data set; acquire multiple multimodal fresh food monitoring training data from the multiple multimodal fresh food monitoring training data and determine them as a target quality inspection verification data set; the target calibration training data set and the target quality inspection verification data set are mutually disjoint; Executing a training process on the optimized product quality inspection model according to the target calibration training data set to obtain a pending product quality inspection model; Determining the freshness status target value classification loss of the pending commodity quality inspection model based on the target quality inspection verification data set; The to-be-determined commodity quality inspection model is subjected to a second optimization process through the freshness state target value classification loss of the to-be-determined commodity quality inspection model to obtain a target commodity quality inspection model.

7. The method according to claim 6, characterized in that The step of obtaining a plurality of multimodal fresh food monitoring training data from the plurality of multimodal fresh food monitoring training data and determining the data as a target calibration training data set, and obtaining a plurality of multimodal fresh food monitoring training data from the plurality of multimodal fresh food monitoring training data and determining the data as a target quality inspection verification data set includes: Extracting a plurality of second preset fresh food state target values from the second preset fresh food state target values respectively matched by the plurality of multimodal fresh food monitoring training data; Extracting a plurality of multimodal fresh food monitoring training data corresponding to a reference preset fresh food state target value from the plurality of multimodal fresh food monitoring training data; the reference preset fresh food state target value belongs to a plurality of second preset fresh food state target values; If a plurality of multimodal fresh food monitoring training data that each matches the plurality of second preset fresh food state target values are extracted from the plurality of multimodal fresh food monitoring training data, the plurality of multimodal fresh food monitoring training data that each matches the plurality of second preset fresh food state target values are grouped to obtain a calibration training data set and a quality inspection verification data set that each matches the plurality of second preset fresh food state target values; The calibration training data sets that each match the plurality of second preset fresh food state target values are used as target calibration training data sets; the total number of multimodal fresh food monitoring training data in the calibration training data sets that each match the plurality of second preset fresh food state target values is the number of data in the target calibration training data sets; The quality inspection and verification data sets that each match the multiple second preset fresh food status target values are used as the target quality inspection and verification data sets; the total number of multimodal fresh food monitoring training data in the quality inspection and verification data sets that each match the multiple second preset fresh food status target values is the number of data in the target quality inspection and verification data sets.

8. The method according to claim 6, characterized in that The target quality inspection and verification data set includes a plurality of quality inspection and verification data sets that are matched with respective second preset freshness status target values, and determining the freshness status target value classification loss of the undetermined commodity quality inspection model based on the target quality inspection and verification data set includes: The target value classification is performed on the target multimodal fresh food monitoring training data in the quality inspection verification data set corresponding to the reference preset fresh food state target value by the pending commodity quality inspection model, and target value speculation information about the target multimodal fresh food monitoring training data is obtained; the reference preset fresh food state target value belongs to a plurality of second preset fresh food state target values; the target value speculation information includes that the second speculated fresh food target value of the target multimodal fresh food monitoring training data is a first confidence level of the reference preset fresh food state target value and the second speculated fresh food target value of the target multimodal fresh food monitoring training data is a second confidence level of the remaining preset fresh food state target values; the remaining preset fresh food state target value is any preset fresh food state target value among the plurality of second preset fresh food state target values except the reference preset fresh food state target value; Determining, based on the first confidence level and the second confidence level, a training inference loss of the reference preset fresh food state target value on the target multimodal fresh food monitoring training data; Determining a target training loss of the reference preset fresh food state target value on the corresponding quality inspection verification data set based on a training inference loss of the reference preset fresh food state target value on the target multimodal fresh food monitoring training data; If the target training losses of the multiple second preset fresh status target values on the corresponding quality inspection verification data sets are obtained, the fresh status target value classification loss of the pending commodity quality inspection model is determined based on the multiple target training losses.

9. The method according to claim 8, characterized in that The target value classification is performed on the target multimodal fresh food monitoring training data in the quality inspection verification data set corresponding to the preset fresh food status target value by the undetermined commodity quality inspection model to obtain target value estimation information about the target multimodal fresh food monitoring training data, including: Determining a mean characteristic corresponding to the reference preset fresh state target value; Calling the pending commodity quality inspection model to extract fresh food monitoring features corresponding to the target multimodal fresh food monitoring training data; Calculating, based on the mean feature corresponding to the reference preset fresh food state target value and the fresh food monitoring feature corresponding to the target multimodal fresh food monitoring training data, a second inferred fresh food target value corresponding to the target multimodal fresh food monitoring training data as a first confidence level of the reference preset fresh food state target value; Determining the mean characteristics corresponding to the remaining preset fresh state target values; The pending commodity quality inspection model is called to calculate the second inferred fresh target value corresponding to the target multimodal fresh monitoring training data as the second confidence level of the remaining preset fresh status target value based on the mean feature corresponding to the remaining preset fresh status target value and the fresh monitoring feature corresponding to the target multimodal fresh monitoring training data.

10. A server system, characterized in that: The method comprises a server, wherein the server is configured to execute the method according to any one of claims 1 to 9.

Images