Food material abnormity identification method and system based on image and weight dual modes

Abstract

The invention relates to the technical field of food material abnormity identification, in particular to a food material abnormity identification method and system based on image and weight dual modes. Comprising the following steps: respectively collecting image data and weight data of a to-be-detected food material, dividing abnormal grades of the food material based on the freshness of the food material, and constructing a food material abnormal grade identification control group; a dual-channel identification model is constructed, abnormal degree identification of the food materials is carried out, the abnormal degree identification comprises food material abnormal degree identification based on an image data feature identification channel and food material abnormal degree identification based on a weight data feature identification channel, comprehensive judgment is carried out based on an identification result of the dual-channel identification model, and dual-mode identification of food material abnormity is realized; according to the method, multi-dimensional, multi-level and quantitative judgment of food material anomaly recognition is realized, the accuracy and interpretability of image recognition are improved, the distinguishable capability of weight anomaly analysis is enhanced, a reusable control group mechanism is provided, and rapid expansion of new types of food materials is supported.

Description

Technical Field

[0001] The present invention relates to the technical field of food anomaly recognition, and in particular to a method and system for food anomaly recognition based on image and weight dual modes. Background Art

[0002] With the continuous development of science and technology, people's quality of life has been improved, and the safety requirements for food ingredients have also increased. However, in traditional food freshness or abnormality identification technologies, single image visual analysis or weight detection methods are usually used for judgment. Single-channel analysis has the following limitations:

[0003] Image methods are easily affected by lighting, shooting angle, occlusion, etc., resulting in inaccurate recognition of surface anomalies;

[0004] Weight detection cannot identify appearance changes such as surface spoilage and is easily affected by moisture fluctuations or weighing errors;

[0005] The recognition level is rough, it is difficult to form a systematic quantitative abnormality level, and there is a lack of a control sample reference mechanism, making it impossible to objectively compare the recognition results. Summary of the Invention

[0006] In view of the above-mentioned problems, the present invention is proposed.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for identifying abnormal food materials based on image and weight dual modes, comprising the following steps:

[0008] Collect image data and weight data of the food to be tested respectively, and classify the abnormality level of the food based on the freshness of the food. At the same time, build a control group for identifying the abnormality level of the food;

[0009] Construct a dual-channel recognition model and identify the abnormality level of ingredients, including the abnormality level of ingredients based on the image data feature recognition channel, specifically:

[0010] The collected food image data is used as the image data mapping base, and the basic layer judgment of the abnormality of the food is performed based on the overlap of the image data;

[0011] Extracting food feature signals from the image data that has completed the basic layer judgment, and calculating the feature similarity between the food feature signals and the feature signals of each abnormality level, and performing a secondary judgment on the abnormality level of the food;

[0012] And, the weight data feature recognition channel identifies the abnormality of ingredients, specifically:

[0013] Based on the weight data of the current food to be tested, the deviation from each abnormality level is calculated, and a unified static deviation factor is set to identify the abnormality level of the food;

[0014] Based on the judgment results of the image data on the degree of abnormality of the food, and the judgment results of the image data on the degree of abnormality of the food, the two are comprehensively judged to achieve dual-mode recognition of food abnormalities.

[0015] As a preferred solution of the method for identifying abnormal food materials based on image and weight dual-mode according to the present invention, the construction of the control group for identifying abnormal food materials level is as follows:

[0016] The freshness of the set ingredients is divided into five levels according to the same time period, including level one, level two, level three, level four and level five. Level one is for completely damaged ingredients, and level five is for the freshest ingredients.

[0017] According to the divided food abnormality levels, the image data and weight data corresponding to each abnormal level of food are collected respectively, and the collected image data and weight data are used as the control group for identifying the abnormality level of food.

[0018] As a preferred solution of the method for identifying abnormal food materials based on image and weight dual modes of the present invention, the basic layer judgment of the abnormality degree of the food materials is specifically as follows:

[0019] The collected food image data is used as the image data to map the base. At the same time, the image data in the constructed control group is stacked with the base image data in order of abnormality level from large to small. The basic layer judgment of the abnormality level of the food is made according to the overlap of the image data. Then,

[0020] If the base image data has a portion that completely overlaps with the control group image data, the food corresponding to the overlapping portion is extracted from the base image data, and the abnormality level of the overlapping portion is extracted from the control group image data, and the abnormality level of the food corresponding to the overlapping portion extracted from the base image data is the abnormality level of the overlapping portion extracted from the control group image data;

[0021] According to the level of abnormality, the degree of overlap with the base image data is judged in turn, and the corresponding ingredients in the image data are divided into abnormality levels according to the degree of overlap, until the control group image data has no complete overlap with the base image data, which means that the basic layer judgment of the ingredients is completed.

[0022] As a preferred solution of the method for identifying abnormal food materials based on image and weight dual-mode according to the present invention, the secondary judgment of the abnormality degree of the food materials is as follows:

[0023] The cosine similarities between the food feature signal and the feature signal of each abnormality level are set to be cos(I C (t),I C，1 )、cos(I C (t),I C，2 )、cos(I C (t),I C，3 )、cos(I C (t),I C，4 ) and cos(I C (t),I C，5 );

[0024] Arrange all the results in ascending order, and judge the abnormality of the ingredients according to the final arrangement order, specifically:

[0025] If there is only one minimum cosine similarity among the cosine similarities sorted in ascending order, it means that the abnormality level of the current food feature signal is the food abnormality level corresponding to the minimum cosine similarity;

[0026] If there are multiple smallest cosine similarities among the cosine similarities sorted in ascending order, the abnormality level of the food is judged by setting a cosine similarity threshold between the food feature signal and the feature signal of each abnormality level.

[0027] As a preferred solution of the method for identifying abnormal food materials based on image and weight dual modes of the present invention, wherein: a secondary judgment of the abnormality degree of food materials is performed based on the calculated difference, then,

[0028] For the cosine similarities sorted in ascending order, there are multiple minimum cosine similarities. For the same minimum cosine similarity, the difference is calculated to perform a secondary judgment on the abnormality of the food. The specific judgment process is as follows:

[0029] Set the same minimum cosine similarity as cos(I C (t),I C，1 )、cos(I C (t),I C，2 ), then extract the difference D1 between the cosine similarity between the food feature signal and the feature signal of the first abnormality level and the corresponding cosine similarity threshold, and the difference D2 between the cosine similarity between the food feature signal and the feature signal of the second abnormality level and the corresponding cosine similarity threshold;

[0030] Based on the extracted difference, a secondary judgment of the abnormality of the food is made,

[0031] If the extracted difference satisfies the formula D1≤D2, it means that the abnormality level of the food is level one;

[0032] If the extracted difference satisfies the formula D1>D2, it means that the abnormality level of the food is the second abnormality level.

[0033] As a preferred solution of the method for identifying abnormal food materials based on image and weight dual-mode according to the present invention, the deviation of the calculation from each abnormality level is specifically as follows:

[0034] Collect the weight data W(t) of the food to be tested, and calculate the mean and standard deviation of each abnormality level in the control group based on the constructed control group;

[0035] According to the weight data of the current food to be tested, the deviation from each abnormality level is calculated, and then,

[0036]

[0037] i∈[1,5]

[0038] Among them, W(t) represents the collected food weight data, μ i represents the mean of the weight data corresponding to the i-th abnormal level in the control group, σ i represents the variance of the weight data corresponding to the i-th abnormal level in the control group, D s,i It represents the deviation between the weight data of the current food to be tested and the i-th abnormality level, and is used to identify the abnormality level of the food based on the weight data.

[0039] As a preferred solution of the method for identifying abnormal food materials based on image and weight dual modes of the present invention, the method for setting a unified static deviation factor to identify the abnormality degree of food materials is as follows:

[0040] Set a uniform static deviation factor D s ', and combined with the calculated deviation to identify the abnormality of the food, we have,

[0041] If there is only one deviation in the calculated deviation that satisfies the formula D when compared with the static deviation factor s,i ≥D s When , it means that the deviation between the current ingredient and the abnormal level exceeds the set static deviation factor, and the abnormality level of the current ingredient is the abnormality level corresponding to the deviation;

[0042] If there are multiple deviations in the calculated deviations that exceed the static deviation factor, a secondary judgment of the abnormality of the food is performed through a mutual mapping algorithm, specifically:

[0043] Calculate the Euclidean distance between the two deviations and the static deviation factor respectively. According to the calculated Euclidean distance, judge the abnormality level of the food and determine which deviation corresponds to the abnormality level. Specifically:

[0044] If D s,1 The Euclidean distance between the static deviation factor is lower than,D s,2 If the Euclidean distance between the current ingredient and the static deviation factor is greater than the normal distance between the two ingredients, it means that the abnormality level of the current ingredient is the first level abnormality level. Otherwise, it means that the abnormality level of the current ingredient is the second level abnormality level.

[0045] As a preferred solution of the method for identifying abnormal food materials based on image and weight dual modes of the present invention, the dual mode identification of abnormal food materials is specifically as follows:

[0046] Assuming the comprehensive recognition level of food anomalies is L, which is composed of the output results of the dual-channel model, we have:

[0047]

[0048] Among them, L represents the comprehensive recognition level of food abnormality, and L is the comprehensive recognition result of food abnormality. I Indicates the abnormal recognition level of food ingredients by the image data feature recognition channel, L W Indicates the abnormal recognition level of the weight data feature recognition channel for food ingredients, Indicates the correction coefficient, which is dynamically corrected according to the difference in recognition levels between the two channels. The specific corrections are as follows:

[0049] If the difference in recognition levels between the two channels satisfies the formula |L I -L W |≥2, it means that there is an abnormality in the two channels' recognition of the abnormal level of food ingredients. By adjusting the correction coefficient Correction is performed, and the comprehensive recognition result of food abnormality is calculated by the formula L=max(L I ,L W )+1 to confirm;

[0050] If the difference in recognition levels between the two channels satisfies the formula |L I -L W |<2, it means that there is an abnormality in the single-channel judgment result between the two channels for the abnormal level recognition of the food. By adjusting the correction coefficient Correction is performed, and the comprehensive recognition result of food abnormality is calculated by the formula L=max(L I ,LW ) to confirm.

[0051] As a preferred solution of the food anomaly identification system based on image and weight dual modes described in the present invention, it includes: a data collection and abnormality degree control group construction module, a dual-channel model recognition module, and a comprehensive recognition module; the data collection and abnormality degree control group construction module is used to construct a control group for food anomaly degree identification while collecting image data and weight data of food; the dual-channel model recognition module is used to use the image data features and weight data features of the food as input data of the dual-channel recognition model according to the constructed dual-channel recognition model, and identify the abnormality degree of the food according to the output results of the dual channels; the comprehensive recognition module is used to make a comprehensive judgment on the output results of the image data feature recognition channel and the output results of the weight data feature recognition channel according to realize dual-mode identification of food anomalies.

[0052] Beneficial effects of the present invention:

[0053] The present invention improves the accuracy and robustness of food anomaly identification by establishing an image + weight dual-channel recognition model and combining visual and physical attribute recognition;

[0054] By adopting a multi-layer structure of "base layer coincidence judgment + feature signal similarity cosine analysis + threshold difference judgment", a recognition path of image information is realized from coarse to fine and step-by-step enhancement, improving the recognition accuracy of subtle anomalies;

[0055] By matching the mean and variance with the control group and introducing the "static deviation factor + Euclidean distance secondary judgment" mechanism, detailed distinction can be made even when multiple abnormal levels are close, improving the resolution of weight recognition.

[0056] By introducing a correction coefficient, the comprehensive judgment level is dynamically adjusted according to the level difference between the image and weight channels, reducing the impact of inter-channel judgment deviation on the results and improving the consistency of the system's comprehensive judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0058] Figure 1 This is a schematic diagram of the overall method steps of the food anomaly identification method based on image and weight dual modes of the present invention. DETAILED DESCRIPTION

[0059] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0060] Example 1

[0061] Reference Figure 1 , which is the first embodiment of the present invention, provides a method for identifying abnormal food materials based on image and weight dual modes, comprising the following steps:

[0062] S1: Collect and construct an abnormality control group.

[0063] Specifically, the acquisition and construction of the abnormality control group is to construct a control group for identifying the abnormality of the food while acquiring the image data and weight data of the food, which is specifically implemented as follows:

[0064] For the first ingredient among all ingredients, use the image sensor and weight meter to collect the current image data I1(t) and weight data W1(t), and build the image data set and weight data set corresponding to the ingredient set based on all types of ingredients. Then,

[0065] Image data set, I(t)={I1(t),I2(t),...,I n (t)}

[0066] Weight data set, W(t)={W1(t),W2(t),...,W n (t)}

[0067] Among them, I1(t) and W1(t) represent the image data and weight data of the first food at the current moment, respectively. n (t), W n (t) represents the image data and weight data of the nth ingredient at the current moment, n represents the total number of ingredients, I(t) and W(t) represent the image data set and weight data set corresponding to the constructed ingredient set at the current moment.

[0068] Furthermore, a control group for identifying the abnormality of ingredients is constructed based on fresh ingredients obtained historically. The specific construction is as follows:

[0069] Based on the fresh ingredients obtained historically, the freshness of the ingredients is set in the order of the time in which the ingredients were obtained, from small to large.

[0070] F={F(t-t0),F(t-t1),...,F(t)}

[0071] Among them, F(t-t0) represents the freshness of the ingredients obtained at time t0 in the past, which is the freshness of the current ingredients when the time length of acquisition is t0. F(t-t1) represents the freshness of the ingredients obtained at time t0 in the past, which is the freshness of the current ingredients when the time length of acquisition is t1. F(t) represents the freshness of the ingredients obtained at the current moment, which is the freshest ingredient. F represents the freshness set of ingredients, which is the freshness of the ingredients obtained within the time period [t, t0]. In actual application, it is set by the implementers according to the actual application scenario.

[0072] According to the freshness of the set ingredients, a control group for identifying the abnormality of ingredients is constructed, specifically:

[0073] The freshness of the set ingredients is divided into five levels according to the same time period, including level one, level two, level three, level four and level five. Level one is for completely damaged ingredients, level five is for the freshest ingredients, and levels two, three and four indicate that the damage degree of ingredients decreases as the level of abnormality increases.

[0074] F1=F(T1,T2), F2=F(T2,T3), F3=F(T3,T4), F4=F(T4,T5), F5=F(T5,T6)

[0075] F={F1∪F2∪F3∪F4∪F5}

[0076] Among them, F1 represents the first level of abnormality, which is the freshness of the ingredients corresponding to the time period [T1, T2]; F2 represents the second level of abnormality, which is the freshness of the ingredients corresponding to the time period (T2, T3); F3 represents the third level of abnormality, which is the freshness of the ingredients corresponding to the time period (T3, T4); F4 represents the fourth level of abnormality, which is the freshness of the ingredients corresponding to the time period (T4, T5); F5 represents the fifth level of abnormality, which is the freshness of the ingredients corresponding to the time period (T5, T6).

[0077] It should be noted that according to the abnormality level of the ingredients, the image data and weight data corresponding to each abnormal level of ingredients are collected respectively, and the collected image data and weight data are used as the control group for identifying the abnormality level of ingredients. Then,

[0078] I′={I1,I2,I3,I4,I5}

[0079] W′={W1,W2,W3,W4,W5}

[0080] Among them, I′ represents the constructed food image data abnormal control group, W′ represents the constructed food weight data abnormal control group, I1, I2, I3, I4, and I5 represent the image data corresponding to the first, second, third, fourth, and fifth levels of abnormality, respectively, and W1, W2, W3, W4, and W5 represent the weight data corresponding to the first, second, third, fourth, and fifth levels of abnormality, respectively.

[0081] S2: Identify the abnormality of ingredients based on the dual-channel recognition model.

[0082] Specifically, the method for identifying the abnormality degree of food ingredients based on the dual-channel recognition model is to construct a dual-channel recognition model, and according to the constructed dual-channel recognition model, use the image data features and weight data features of the food ingredients as input data of the dual-channel recognition model, and identify the abnormality degree of the food ingredients according to the output results of the dual channels, which is specifically implemented as follows:

[0083] The dual-channel recognition model consists of two recognition channels: an image data feature recognition channel and a weight data feature recognition channel. The image data feature recognition channel identifies the degree of abnormality of ingredients based on their image data, while the weight data feature recognition channel identifies the degree of abnormality of ingredients based on their weight data. Specifically:

[0084] The image data feature recognition channel uses the collected food image data as input features and performs multi-angle analysis on the input data features through a layer-by-layer analysis mechanism. Finally, the abnormality level of the food is identified based on the constructed control group. Specifically:

[0085] Based on the collected food image data I(t), and the constructed food freshness set F, the basic layer judgment of food abnormality is performed, specifically:

[0086] The collected food image data is used as the image data to map the base. At the same time, the image data in the constructed control group is stacked with the base image data in order of abnormality level from large to small. The basic layer judgment of the abnormality level of the food is made according to the overlap of the image data. Then,

[0087] If the base image data has a part that completely overlaps with the control group image data, the food corresponding to the overlapping part is extracted from the base image data, and the abnormality level of the overlapping part is extracted from the control group image data according to the completely overlapping part, and the abnormality level of the food corresponding to the overlapping part extracted from the base image data is the abnormality level of the overlapping part extracted from the control group image data.

[0088] According to the level of abnormality, the degree of overlap with the base image data is judged in turn, and the corresponding ingredients in the image data are divided into abnormality levels according to the degree of overlap, until the control group image data has no complete overlap with the base image data, which means that the basic layer judgment of the ingredients is completed.

[0089] For the image data that has completed the basic layer judgment, the food feature signal is extracted from the image data. At the same time, the corresponding feature signal is extracted from the image data corresponding to each abnormality level of the control group, and the feature similarity between the food feature signal and the feature signal of each abnormality level is calculated. Based on the calculated similarity, a secondary judgment of the abnormality level of the food is performed, specifically as follows:

[0090] Set the food feature signal extracted from the image data to I C (t), the characteristic signals of the abnormality levels are I C,1 , I C,2 , I C,3 , I C,4 and I C,5 , respectively represent the characteristic signal of the first abnormality level, the characteristic signal of the second abnormality level, the characteristic signal of the third abnormality level, the characteristic signal of the fourth abnormality level, and the characteristic signal of the fifth abnormality level;

[0091] Select the food feature signal and calculate the cosine similarity between it and the feature signal of each abnormality level in turn, then we have:

[0092] The cosine similarities between the food feature signal and the feature signal of each abnormality level are set to be cos(I C (t),I C，1 )、cos(I C (t),I C，2 )、cos(I C (t),I C，3 )、cos(I C (t),I C，4 ) and cos(I C (t),I C，5 );

[0093] Arrange all the results in ascending order, and judge the abnormality of the ingredients according to the final arrangement order, specifically:

[0094] If there is only one minimum cosine similarity among the cosine similarities sorted in ascending order, it means that the abnormality level of the current food feature signal is the food abnormality level corresponding to the minimum cosine similarity;

[0095] If there are multiple minimum cosine similarities among the cosine similarities sorted in ascending order, the abnormality level of the food is determined by setting the cosine similarity threshold between the food feature signal and the feature signal of each abnormality level, specifically:

[0096] Set the cosine similarity threshold between the food feature signal and the feature signal of each abnormality level, respectively, cos′(I C (t),I C，1 ), cos′(I C (t),I C，2 ), cos′(I C (t),I C，3 ), cos′(I C (t),I C，4 ) and cos′(I C (t),I C，5 );

[0097] A secondary judgment of the abnormality of ingredients is made based on the calculated cosine similarity and the set cosine similarity threshold, specifically:

[0098] Calculate the difference between each cosine similarity and the cosine similarity threshold in turn, then we have,

[0099] D1=|cos(I C (t),I C,1 )-cos′(I C (t),I C,1 )|

[0100] D2=|cos(I C (t),I C,2 )-cos′(I C (t),I C,2 )|

[0101] D3=|cos(I C (t),I C,3 )-cos′(I C (t),I C,3 )|

[0102] D4=|cos(IC (t),I C,4 )-cos′(I C (t),I C,4 )|

[0103] D5=|cos(I C (t),I C,5 )-cos′(I C (t),I C,5 )|

[0104] Wherein, D1 represents the cosine similarity between the food feature signal and the feature signal of the first abnormality level, and the difference between the cosine similarity threshold value; D2 represents the cosine similarity between the food feature signal and the feature signal of the second abnormality level, and the difference between the cosine similarity threshold value; D3 represents the cosine similarity between the food feature signal and the feature signal of the third abnormality level, and the difference between the cosine similarity threshold value; D4 represents the cosine similarity between the food feature signal and the feature signal of the fourth abnormality level, and the difference between the cosine similarity threshold value; D5 represents the cosine similarity between the food feature signal and the feature signal of the fifth abnormality level, and the difference between the cosine similarity threshold value; cos(I C (t),I C，1 )、cos(I C (t),I C，2 )、cos(I C (t),I C，3 )、cos(I C (t),I C，4 ) and cos(I C (t),I C，5 ) represent the cosine similarity between the food feature signal and the feature signal of the first abnormality level, the feature signal of the second abnormality level, the feature signal of the third abnormality level, the feature signal of the fourth abnormality level, and the feature signal of the fifth abnormality level, cos′(I C (t),I C，1 ), cos′(I C (t),I C，2 ), cos′(I C (t),I C，3 ), cos′(I C (t),I C，4 ) and cos′(I C (t),I C，5 ) represent the cosine similarity threshold corresponding to each abnormality level.

[0105] Based on the calculated difference, a secondary judgment of the abnormality of the food is made,

[0106] For the cosine similarities sorted in ascending order, there are multiple minimum cosine similarities. For the same minimum cosine similarity, the difference is calculated to perform a secondary judgment on the abnormality of the food. The specific judgment process is as follows:

[0107] In this embodiment, the same minimum cosine similarity is taken as cos(I C (t),I C，1 )、cos(I C (t),I C，2 ) is used as an example only as an example of the implementation of the technical solution, and does not limit the implementation of the technical solution. The specific implementation is determined by the implementer according to the actual application scenario;

[0108] Set the same minimum cosine similarity as cos(I C (t),I C，1 )、cos(I C (t),I C，2 ), then extract the difference D1 between the cosine similarity between the food feature signal and the feature signal of the first abnormality level and the corresponding cosine similarity threshold, and the difference D2 between the cosine similarity between the food feature signal and the feature signal of the second abnormality level and the corresponding cosine similarity threshold;

[0109] Based on the extracted difference, a secondary judgment of the abnormality of the food is made,

[0110] If the extracted difference satisfies the formula D1≤D2, it means that among the two identical cosine similarities, the cosine similarity corresponding to the first abnormality level is closer to the set cosine similarity threshold than the cosine similarity corresponding to the second abnormality level, indicating that the abnormality level of the food is the first abnormality level;

[0111] If the extracted difference satisfies the formula D1>D2, it means that the abnormality level of the food is the second abnormality level.

[0112] It should be noted that when three identical minimum cosine similarities exist, the difference between the cosine similarity and the cosine similarity threshold is also determined to determine which cosine similarity is closest to the corresponding cosine similarity threshold. Based on the determined cosine similarity, a secondary judgment on the abnormality degree of the current food is achieved.

[0113] Furthermore, the weight data feature recognition channel combines the collected food weight data with the constructed control group to identify the abnormality of the food, specifically:

[0114] Based on the collected food weight data and combined with the weight data of different levels of the control group, a basic weight deviation index for the current food is constructed. The constructed deviation index is used as the basis for the initial judgment of the abnormality of the food. Specifically:

[0115] Collect the weight data W(t) of the food to be tested, and calculate the mean and standard deviation of each abnormality level in the control group based on the constructed control group. At the same time, set a unified static deviation factor for each abnormality level;

[0116] According to the weight data of the current food to be tested, the deviation from each abnormality level is calculated, and then,

[0117]

[0118] i∈[1,5]

[0119] Among them, W(t) represents the collected food weight data, μ i represents the mean of the weight data corresponding to the i-th abnormal level in the control group, σ i represents the variance of the weight data corresponding to the i-th abnormal level in the control group, D s,i Indicates the deviation between the weight data of the current food to be tested and the i-th abnormality level, which is used to identify the abnormality level of the food based on the weight data. Specifically:

[0120] Set a uniform static deviation factor D s ', and combined with the calculated deviation to identify the abnormality of the food, we have,

[0121] If there is only one deviation in the calculated deviation that satisfies the formula D when compared with the static deviation factor s,i ≥D s When , it means that the deviation between the current ingredient and the abnormal level exceeds the set static deviation factor, and the abnormality level of the current ingredient is the abnormality level corresponding to the deviation.

[0122] If there are multiple deviations in the calculated deviations that exceed the static deviation factor, a secondary judgment of the abnormality of the food is performed through a mutual mapping algorithm, specifically:

[0123] For the food that exceeds the static deviation factor, the deviation of all static deviation factors is extracted. In this embodiment, D s,1 and D s,2 To explain, there is,

[0124] Calculate the Euclidean distance between the two deviations and the static deviation factor respectively. According to the calculated Euclidean distance, judge the abnormality level of the food and determine which deviation corresponds to the abnormality level. Specifically:

[0125] If D s,1 The Euclidean distance between the static deviation factor is lower than,D s,2 If the Euclidean distance between the current ingredient and the static deviation factor is greater than the normal distance between the two ingredients, it means that the abnormality level of the current ingredient is the first level abnormality level. Otherwise, it means that the abnormality level of the current ingredient is the second level abnormality level.

[0126] It should be noted that the abnormality level of ingredients identified by the image data feature recognition channel in the dual-channel recognition model, and the abnormality level of ingredients identified by the weight data feature recognition channel, are both used as abnormality indicators of their respective channels to facilitate the subsequent comprehensive abnormality recognition of ingredients.

[0127] S3: Comprehensively identify food abnormalities based on the judgment results of image data and weight data.

[0128] Specifically, the comprehensive identification of the degree of abnormality of food ingredients based on the judgment results of image data and weight data is to comprehensively determine the judgment results of the degree of abnormality of food ingredients based on image data and the judgment results of the degree of abnormality of food ingredients based on image data, so as to realize dual-mode identification of abnormal food ingredients, specifically:

[0129] Based on the output results of the dual-channel model, including the output results of the image data feature recognition channel and the output results of the weight data feature recognition channel, a comprehensive identification of food anomalies is performed.

[0130] If the output results of the image data feature recognition channel and the weight data feature recognition channel are consistent in their identification results of the abnormality degree of the food, it means that the current comprehensive identification result of the abnormality of the food is the output result of the dual-channel model;

[0131] If the output results of the image data feature recognition channel and the output results of the weight data feature recognition channel have different results for the degree of abnormality of the food, a comprehensive identification of the abnormality of the food is performed through a multi-channel fusion mechanism, specifically:

[0132] Assuming the comprehensive recognition level of food anomalies is L, which is composed of the output results of the dual-channel model, we have:

[0133]

[0134] Among them, L represents the comprehensive recognition level of food abnormality, and L is the comprehensive recognition result of food abnormality. I Indicates the abnormal recognition level of food ingredients by the image data feature recognition channel, L W Indicates the abnormal recognition level of the weight data feature recognition channel for food ingredients, Indicates the correction coefficient, which is dynamically corrected according to the difference in recognition levels between the two channels. The specific corrections are as follows:

[0135] If the difference in recognition levels between the two channels satisfies the formula |L I -L W |≥2, it means that there is an abnormality in the two channels' recognition of the abnormal level of food ingredients. By adjusting the correction coefficient Correction is performed, and the comprehensive recognition result of food abnormality is calculated by the formula L=max(L I ,L W )+1 to confirm;

[0136] If the difference in recognition levels between the two channels satisfies the formula |L I -L W |<2, it means that there is an abnormality in the single-channel judgment result between the two channels for the abnormal level recognition of the food. By adjusting the correction coefficient Correction is performed, and the comprehensive recognition result of food abnormality is calculated by the formula L=max(L I ,L W ) to confirm.

[0137] It should be noted that the adjustment of the correction coefficient in this embodiment is based on five abnormal levels. In specific applications, the correction coefficient can be set by the implementer according to the actual application scenario.

[0138] Example 2

[0139] The second embodiment of the present invention provides an abnormal food material identification system based on image and weight dual modes, including a data collection and abnormality control group construction module, a dual-channel model identification module, and a comprehensive identification module;

[0140] Specifically, the data collection and abnormality control group construction module is used to construct a control group for identifying the abnormality level of food while collecting the image data and weight data of the food; the dual-channel model recognition module is used to use the image data features and weight data features of the food as input data of the dual-channel recognition model based on the constructed dual-channel recognition model, and identify the abnormality level of the food based on the output results of the dual channels; the comprehensive recognition module is used to make a comprehensive judgment on the output results of the image data feature recognition channel and the output results of the weight data feature recognition channel based on the two, so as to realize dual-mode recognition of food abnormalities.

[0141] Furthermore, if the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0142] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0143] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0144] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for identifying abnormal food ingredients based on image and weight dual-mode, characterized by: The following steps are included: Collect image data and weight data of the food to be tested respectively, and classify the abnormality level of the food based on the freshness of the food. At the same time, build a control group for identifying the abnormality level of the food; Construct a dual-channel recognition model and identify the abnormality level of ingredients, including the abnormality level of ingredients based on the image data feature recognition channel, specifically: The collected food image data is used as the image data mapping base, and the basic layer judgment of the abnormality of the food is performed based on the overlap of the image data; Extracting food feature signals from the image data that has completed the basic layer judgment, and calculating the feature similarity between the food feature signals and the feature signals of each abnormality level, and performing a secondary judgment on the abnormality level of the food; And, the weight data feature recognition channel identifies the abnormality of ingredients, specifically: Based on the weight data of the current food to be tested, the deviation from each abnormality level is calculated, and a unified static deviation factor is set to identify the abnormality level of the food; Based on the judgment results of the image data on the degree of abnormality of the food, and the judgment results of the image data on the degree of abnormality of the food, the two are comprehensively judged to achieve dual-mode recognition of food abnormalities.

2. The method for identifying abnormal food materials based on image and weight dual modes according to claim 1, characterized in that: The specific construction of the food abnormality level identification control group is as follows: The freshness of the set ingredients is divided into five levels according to the same time period, including level one, level two, level three, level four and level five. Level one is for completely damaged ingredients, and level five is for the freshest ingredients. According to the divided food abnormality levels, the image data and weight data corresponding to each abnormal level of food are collected respectively, and the collected image data and weight data are used as the control group for identifying the abnormality level of food.

3. The method for identifying abnormal food materials based on image and weight dual modes according to claim 2, characterized in that: The basic level judgment of the abnormality of the food ingredients is as follows: The collected food image data is used as the image data to map the base. At the same time, the image data in the constructed control group is stacked with the base image data in order of abnormality level from large to small. The basic layer judgment of the abnormality level of the food is made according to the overlap of the image data. Then, If the base image data has a portion that completely overlaps with the control group image data, the food corresponding to the overlapping portion is extracted from the base image data, and the abnormality level of the overlapping portion is extracted from the control group image data, and the abnormality level of the food corresponding to the overlapping portion extracted from the base image data is the abnormality level of the overlapping portion extracted from the control group image data; According to the level of abnormality, the degree of overlap with the base image data is judged in turn, and the corresponding ingredients in the image data are divided into abnormality levels according to the degree of overlap, until the control group image data has no complete overlap with the base image data, which means that the basic layer judgment of the ingredients is completed.

4. The method for identifying abnormal food materials based on image and weight dual modes according to claim 3, characterized in that: The secondary judgment of the abnormality degree of the food ingredients is as follows: The cosine similarities between the food feature signal and the feature signal of each abnormality level are set to be cos(I C (t),I C，1 )、cos(I C (t),I C，2 )、cos(I C (t),I C，3 )、cos(I C (t),I C，4 ) and cos(I C (t),I C，5 ); Arrange all the results in ascending order, and judge the abnormality of the ingredients according to the final arrangement order, specifically: If there is only one minimum cosine similarity among the cosine similarities sorted in ascending order, it means that the abnormality level of the current food feature signal is the food abnormality level corresponding to the minimum cosine similarity; If there are multiple smallest cosine similarities among the cosine similarities sorted in ascending order, the abnormality level of the food is judged by setting a cosine similarity threshold between the food feature signal and the feature signal of each abnormality level.

5. The method for identifying abnormal food materials based on image and weight dual modes according to claim 4 is characterized in that: Based on the calculated difference, a secondary judgment of the abnormality of the food is made, For the cosine similarities sorted in ascending order, there are multiple minimum cosine similarities. For the same minimum cosine similarity, the difference is calculated to perform a secondary judgment on the abnormality of the food. The specific judgment process is as follows: Set the same minimum cosine similarity as cos(I C (t),I C，1 )、cos(I C (t),I C，2 ), then extract the difference D1 between the cosine similarity between the food feature signal and the feature signal of the first abnormality level and the corresponding cosine similarity threshold, and the difference D2 between the cosine similarity between the food feature signal and the feature signal of the second abnormality level and the corresponding cosine similarity threshold; Based on the extracted difference, a secondary judgment of the abnormality of the food is made, If the extracted difference satisfies the formula D1≤D2, it means that the abnormality level of the food is level one; If the extracted difference satisfies the formula D1>D2, it means that the abnormality level of the food is the second abnormality level.

6. The method for identifying abnormal food materials based on image and weight dual modes according to claim 5, characterized in that: The calculations for each level of anomaly deviation are as follows: Collect the weight data W(t) of the food to be tested, and calculate the mean and standard deviation of each abnormality level in the control group based on the constructed control group; According to the weight data of the current food to be tested, the deviation from each abnormality level is calculated, and then, i∈[1,5] Among them, W(t) represents the collected food weight data, μ i represents the mean of the weight data corresponding to the i-th abnormal level in the control group, σ i represents the variance of the weight data corresponding to the i-th abnormal level in the control group, D s,i It represents the deviation between the weight data of the current food to be tested and the i-th abnormality level, and is used to identify the abnormality level of the food based on the weight data.

7. The method for identifying abnormal food materials based on image and weight dual modes according to claim 6, characterized in that: The method of setting a unified static deviation factor to identify the abnormality of food ingredients is as follows: Set a uniform static deviation factor D s ', and combined with the calculated deviation to identify the abnormality of the food, we have, If there is only one deviation in the calculated deviation that satisfies the formula D when compared with the static deviation factor s,i ≥D s When , it means that the deviation between the current ingredient and the abnormal level exceeds the set static deviation factor, and the abnormality level of the current ingredient is the abnormality level corresponding to the deviation; If there are multiple deviations in the calculated deviations that exceed the static deviation factor, a secondary judgment of the abnormality of the food is performed through a mutual mapping algorithm, specifically: Calculate the Euclidean distance between the two deviations and the static deviation factor respectively. According to the calculated Euclidean distance, judge the abnormality level of the food and determine which deviation corresponds to the abnormality level. Specifically: If D s,1 The Euclidean distance between the static deviation factor is lower than,D s,2 If the Euclidean distance between the current ingredient and the static deviation factor is greater than the normal distance between the two ingredients, it means that the abnormality level of the current ingredient is the first level abnormality level. Otherwise, it means that the abnormality level of the current ingredient is the second level abnormality level.

8. The method for identifying abnormal food materials based on image and weight dual modes according to claim 7, characterized in that: The dual-mode recognition of food abnormalities is specifically as follows: Assuming the comprehensive recognition level of food anomalies is L, which is composed of the output results of the dual-channel model, we have: Among them, L represents the comprehensive recognition level of food abnormality, and L is the comprehensive recognition result of food abnormality. I Indicates the abnormal recognition level of food ingredients by the image data feature recognition channel, L W Indicates the abnormal recognition level of the weight data feature recognition channel for food ingredients, Indicates the correction coefficient, which is dynamically corrected according to the difference in recognition levels between the two channels. The specific corrections are as follows: If the difference in recognition levels between the two channels satisfies the formula |L I -L W |≥2, it means that there is an abnormality in the two channels' recognition of the abnormal level of food ingredients. By adjusting the correction coefficient Correction is performed, and the comprehensive recognition result of food abnormality is calculated by the formula L=max(L I ,L W )+1 to confirm; If the difference in recognition levels between the two channels satisfies the formula |L I -L W |<2, it means that there is an abnormality in the single-channel judgment result between the two channels for the abnormal level recognition of the food. By adjusting the correction coefficient Correction is performed, and the comprehensive recognition result of food abnormality is calculated by the formula L=max(L I ,L W ) to confirm.

9. The food anomaly recognition system based on image and weight dual-mode is applied to the food anomaly recognition method based on image and weight dual-mode according to claims 1 to 8, characterized in that: It includes a data collection and abnormality control group construction module, a dual-channel model recognition module, and a comprehensive recognition module; The data collection and abnormality control group construction module is used to construct a control group for identifying the abnormality level of ingredients while collecting image data and weight data of the ingredients; The dual-channel model recognition module is used to use the image data features and weight data features of the food as input data of the dual-channel recognition model according to the constructed dual-channel recognition model, and to identify the abnormality degree of the food according to the output results of the dual channels; The comprehensive recognition module is used to make a comprehensive judgment based on the output results of the image data feature recognition channel and the output results of the weight data feature recognition channel, so as to realize dual-mode recognition of food anomalies.

Images