Method, device, medium and product for identifying raw and cooked state of meal meat

Abstract

The present application relates to a meal meat raw state recognition method, device, medium and product. The method comprises: obtaining a to-be-recognized meal image; performing meat image detection on the to-be-recognized meal image; in the case where the to-be-recognized meal image is determined as a meat image, inputting the to-be-recognized meal image into a raw state recognition model to obtain a meat raw state recognition result in the to-be-recognized meal image; wherein the raw state recognition model is obtained by training through the following steps: obtaining meal meat training images labeled with raw state; performing image transformation for simulating a target shooting interference type on the meal meat training images to obtain enhanced training images; the image transformation has a preset simulation corresponding relationship with the target shooting interference type; and using the enhanced training images and their corresponding raw state labels, the raw state recognition model is trained.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to methods, equipment, media, and products for identifying the raw or cooked state of meat in meals. Background Technology

[0002] With the increasing prevalence of online complaint channels and social media platforms, the number of complaint tickets involving food photos in the food service system is constantly rising. Among these, complaint tickets concerning the raw or cooked status of meat dishes require customer service personnel to determine the raw or cooked status of the meat based on the photos uploaded by customers and to determine the appropriate course of action by referring to similar historical tickets.

[0003] In the current processing flow, customers can upload photos of their meals through apps, social media platforms, or other complaint channels. Customer service personnel manually review these photos and, based on experience, determine the doneness of the meat. Subsequently, they search for similar cases in the historical work order system using keyword searches or manual review, and provide a handling opinion based on their manual judgment and historical cases. This approach relies heavily on the experience of customer service personnel, resulting in low efficiency, inconsistent judgment standards, strong subjectivity, and difficulty in handling high-concurrency work orders.

[0004] In existing technologies, general image classification models can be used to classify images. For example, pre-trained models based on network structures such as ResNet and VGG can be used to recognize images. However, recognizing the raw / cooked state of meat in meals is a fine-grained visual recognition scenario, where the recognition result depends on features such as the color, texture, and gloss of the meat surface. General image classification models are usually not specifically trained for recognizing the raw / cooked state of meat in meals, and when directly transferred to this scenario, they lack the ability to distinguish the fine-grained differences between raw and cooked meat.

[0005] Furthermore, the food images uploaded by customers come from a wide range of sources, potentially involving different brands of mobile phones, different imaging sensors, different image signal processing pipelines, and different lens distortion characteristics. During actual shooting, there may also be shooting interference factors such as focus differences, imaging noise, angled or overhead shots, low indoor lighting or strong outdoor lighting, partial obstruction, and reflections from plates. These shooting interferences can affect the image presentation of discriminative features such as the color, texture, and gloss of the meat surface, making the model prone to decreased recognition accuracy across different devices, lighting conditions, and focus states.

[0006] Existing technologies may employ perceptual hashing or differential hashing for image deduplication or similar image retrieval. However, these methods are typically suitable for detecting completely identical or highly similar images. Their retrieval capabilities are limited for food images that differ in shooting angle, lighting, and composition but possess semantic similarity. Therefore, hash-based image retrieval methods are insufficient to effectively assist customer service personnel in finding semantically similar food examples from historical work orders.

[0007] Therefore, how to identify the raw or cooked state of meat in food photos under complex shooting conditions, and how to combine similar food photos from historical work orders to provide a reference for customer service processing, is a problem that needs to be solved in related technologies. Summary of the Invention

[0008] The purpose of this invention is to provide a method, device, medium, and product for identifying the raw or cooked state of meat in food products, and to solve the technical problem of how to identify the raw or cooked state of meat in food product images under complex shooting conditions, and to provide a reference for customer service processing by combining similar food product images from historical work orders.

[0009] The first aspect of this invention discloses a method for identifying the raw / cooked state of meat in a meal, comprising:

[0010] Acquire the image of the food item to be identified;

[0011] Meat image detection is performed on the image of the food item to be identified;

[0012] If the image of the food to be identified is determined to be a meat image, the image of the food to be identified is input into multiple sub-recognition models of the raw / cooked state recognition model to obtain multiple raw / cooked state probability distributions output by each sub-recognition model; the multiple raw / cooked state probability distributions are fused and calculated, and the raw / cooked state recognition result of the meat in the image of the food to be identified is determined based on the fused probability result.

[0013] The raw / cooked state recognition model is trained through the following steps:

[0014] Obtain training images of meat products labeled with their raw / cooked status;

[0015] Based on the principle of consistent category proportion, the training images of meat dishes labeled with raw and cooked status are divided into multiple training subsets;

[0016] An image transformation simulating target shooting interference type is performed on the training image of the meat dish to obtain an enhanced training image; the image transformation and the target shooting interference type have a preset simulation correspondence;

[0017] Multiple sub-recognition models are obtained by independently training the enhanced training images and their corresponding raw / cooked state labels of the multiple training subsets respectively;

[0018] The multiple sub-recognition models are combined to determine the raw / cooked state recognition model.

[0019] According to the method of the first aspect of the present invention, the target shooting interference type includes at least two of imaging color difference, focus difference, imaging noise, shooting angle difference, partial occlusion and brightness difference;

[0020] The simulated correspondence between the image transformation and the target shooting interference type includes at least two of the following:

[0021] At least one of hue transformation, saturation transformation, and luminance channel transformation is used to simulate imaging color differences;

[0022] Fuzzy transformation is used to simulate focusing differences;

[0023] Noise superposition transformation is used to simulate imaging noise;

[0024] At least one of translation transformation, scaling transformation, and rotation transformation is used to simulate differences in shooting perspective;

[0025] Local occlusion is simulated using local region dropout transformation;

[0026] At least one of luminance transformation and contrast transformation is used to simulate luminance difference.

[0027] According to a first aspect of the invention, performing an image transformation on the training image of the meat dish to simulate a type of target shooting interference includes:

[0028] According to a preset probability, at least two image transformations are performed serially or in parallel on the same meat training image to obtain an enhanced training image corresponding to the same raw / cooked status label.

[0029] According to a method of a first aspect of the present invention, the raw / cooked state recognition model is trained, comprising:

[0030] The enhanced training image is input into a convolutional neural network including a channel attention structure and a self-gated nonlinear activation function to extract the meat color, texture and gloss features in the enhanced training image;

[0031] Based on the raw / cooked state prediction results and corresponding raw / cooked state labels output by the convolutional neural network, the network parameters of the convolutional neural network are adjusted.

[0032] According to a first aspect of the present invention, acquiring training images of cooked meat labeled with its raw / cooked state includes:

[0033] The sampling weights for different raw / ripe states are determined based on the number of image samples corresponding to each raw / ripe state. Among them, the sampling weights for raw / ripe states with fewer samples are higher.

[0034] According to the sampling weights, the training images of meat products are extracted from the original meat product image library.

[0035] According to a first aspect of the present invention, a method for detecting meat in the image of the food to be identified includes:

[0036] Extract global image features from the image of the food item to be identified;

[0037] The global image features are input into a single-class anomaly detection model trained on a single positive class sample to obtain the detection result of whether the food image to be identified is a meat image.

[0038] According to the method of the first aspect of the present invention, the single-class anomaly detection model is trained through the following steps:

[0039] Obtain images containing meat as positive class samples;

[0040] The global image features of the positive class samples are extracted using a pre-trained feature extraction network;

[0041] The single-class anomaly detection model is trained with the optimization objective of identifying and enclosing the feature distribution boundary of the positive class samples.

[0042] According to a first aspect of the method of the present invention, after obtaining the recognition result of the raw / cooked state of the meat in the image of the food to be identified, the method further includes:

[0043] Based on the image features of the food image to be identified, a semantic similarity search is performed in the image index library to obtain historical food images whose similarity scores meet the set conditions;

[0044] Obtain historical work order information associated with the historical food images;

[0045] Output the results of the meat raw / cooked status identification and the historical work order information.

[0046] A second aspect of the present invention discloses an electronic device comprising a memory storing computer-executable instructions and a processor, wherein when the instructions are executed by the processor, the electronic device implements a method for identifying the raw / cooked state of meat according to a first aspect of the present invention.

[0047] A third aspect of the present invention discloses a computer storage medium storing instructions that, when executed on a computer, cause the computer to perform a method for identifying the raw / cooked state of meat according to a first aspect of the present invention.

[0048] A fourth aspect of the present invention discloses a computer program product comprising computer-executable instructions, which are executed by a processor to implement a method for identifying the raw / cooked state of meat in a meal according to a first aspect of the present invention.

[0049] The main differences and effects of this invention compared to existing technologies are as follows:

[0050] In this invention, the method for identifying the raw / cooked state of meat in a meal includes: acquiring an image of the meal to be identified; performing meat image detection on the image of the meal to be identified; if the image of the meal to be identified is determined to be a meat image, inputting the image of the meal to be identified into a raw / cooked state identification model to obtain the raw / cooked state identification result of the meat in the image of the meal to be identified; wherein, the raw / cooked state identification model is trained through the following steps: acquiring training images of the meal meat labeled with raw / cooked state; performing image transformation on the training images of the meal meat to simulate target shooting interference types to obtain enhanced training images; the image transformation and the target shooting interference types have a preset simulated correspondence; using the enhanced training images and their corresponding raw / cooked state labels, the raw / cooked state identification model is trained. Therefore, on the one hand, before identifying the raw / cooked state, detecting meat images in the image of the food to be identified can reduce the possibility of misjudging raw / cooked status when non-meat images are input into the raw / cooked status identification model. On the other hand, when training the raw / cooked status identification model, generating enhanced training images through image transformations that have a preset simulated correspondence with the target shooting interference type can enable the raw / cooked status identification model to learn image change features related to shooting interferences such as imaging color differences, focus differences, imaging noise, shooting angle differences, partial occlusion, and brightness differences during the training process. This reduces the impact of different shooting equipment, different lighting conditions, different shooting angles, or different focus states on the results of meat raw / cooked status identification, and improves the stability and applicability of meat raw / cooked status identification for meals. Attached Figure Description

[0051] Figure 1 A flowchart illustrating a method for identifying the raw / cooked state of meat in a meal according to an embodiment of this application is shown.

[0052] Figure 2 A flowchart illustrating the training process of a raw / cooked state recognition model according to an embodiment of this application is provided.

[0053] Figure 3 This is a hardware structure block diagram of an electronic device implementing the embodiments of this application. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0055] To address the problems in related technologies, such as the strong subjectivity of manual judgment, the insufficient fine-grained ability of general image classification models to distinguish the raw / cooked state of meat in meals, the impact of complex shooting interference on recognition stability, and the low efficiency of retrieving similar cases from historical work orders, embodiments of this application disclose a method for recognizing the raw / cooked state of meat in meals. This method first performs meat image detection on the image of the meal to be identified. Once it is determined that the image of the meal to be identified is a meat image, it then uses a raw / cooked state recognition model trained with shooting interference simulation enhancement to determine the meat's raw / cooked state recognition result. Furthermore, it can retrieve historical meal images semantically similar to the image of the meal to be identified and their associated historical work order information. For example... Figure 1 As shown in the flowchart, the method includes:

[0056] S101, acquire the image of the food item to be identified.

[0057] In some implementations, the food image to be identified can be a food image uploaded by a customer in the complaint channel, a food image selected or uploaded by customer service personnel on the work order processing page, a food image in a social media post, or a food image read during the batch review of historical work orders.

[0058] In one specific engineering implementation, customer service personnel can upload images of food items to be identified via a browser plugin. The browser plugin, implemented using Chrome Extension, Vue3, and TypeScript, provides image upload, recognition result display, and historical work order association functionality. The browser plugin sends the image binary file and user session information to the business service layer. The business service layer, implemented using Spring Boot, MyBatis-Plus, and Java, handles user authentication, whitelist management, image upload, request forwarding, and result return. The AI ​​model service layer, implemented using FastAPI, Python, ONNX Runtime, and PyTorch, is used for meat image detection, raw / cooked status determination, image search, and historical work order retrieval.

[0059] In some implementations, after receiving the image of the food item to be identified, the service layer can perform file verification on the image. File verification may include file size verification, image size verification, and file format verification. For example, the file size may be limited to no more than 50MB; the image size may be limited to between 100px and 4096px; and the file format may include jpg, png, webp, and bmp. For images exceeding a preset size, such as images exceeding 4MB, image compression may be performed. For WebP format images, they may be converted to JPEG format. The verified and processed image can be uploaded to the object storage service, and the corresponding image URL will be returned.

[0060] In some implementations, the data passed from the browser plugin layer to the business service layer may include image binary data in MultipartFile format and user identifiers from the Session; the business service layer passes compressed image binary data to the object storage service and receives the image URL; the business service layer sends a recognition request to the AI ​​model service layer, which may include the image URL, user identifier, and a field indicating whether meat detection is performed. For example, the recognition request may include the following fields: imageUrl, userId, and check_meat, where check_meat indicates whether meat image detection should be performed. The recognition results returned by the AI ​​model service layer may include whether it is raw meat, raw meat confidence, cooked meat confidence, and a list of associated historical work orders, such as isRawMeat, rawConfidence, cookedConfidence, and relatedOrders fields.

[0061] S102, perform meat image detection on the food image to be identified.

[0062] In some implementations, meat image detection in the image of the food to be identified refers to determining whether the image contains meat. By performing meat image detection before raw / cooked status identification, non-meat images can be filtered out, reducing the likelihood of high-confidence false positives when performing raw / cooked status determination on irrelevant inputs.

[0063] In some implementations, global image features of the food image to be identified can be extracted. These global image features are then input into a single-class anomaly detection model trained on a single positive class sample to obtain a detection result indicating whether the food image to be identified is a meat image. This single positive class sample can be an image sample containing meat. The single-class anomaly detection model can learn the distribution boundary of meat images in the feature space and identify images falling outside this distribution boundary as anomaly samples or non-meat images.

[0064] In one specific implementation, a ResNet50 pre-trained on ImageNet can be used as the feature extraction network. The last fully connected layer of the ResNet50 is removed, retaining its feature extraction portion to output a 2048-dimensional global feature vector. This 2048-dimensional global feature vector is then input into a One-Class SVM classifier for meat / non-meat detection. The One-Class SVM model file can be oc_svm_model.pkl. This model is trained with meat images as positive class samples and performs anomaly detection on out-of-distribution samples. In a further implementation, the 2048-dimensional global feature vector output by ResNet50 can be reduced to 50 dimensions using PCA; the parameter nu of the One-Class SVM can be set to 0.07, and a threshold strategy of percentile=7 can be used to determine the anomaly discrimination boundary. Feature extraction supports batch processing with a batch size of 32 and CPU inference can be performed in torch.no_grad() and eval modes.

[0065] In some implementations, when the output of the single-class anomaly detection model indicates that the image of the food to be identified is a non-meat image, an error message or rejection result can be returned directly, and the subsequent raw / cooked status identification process can be terminated. When the output of the single-class anomaly detection model indicates that the image of the food to be identified is a meat image, step S103 continues.

[0066] S103, if the image of the food to be identified is determined to be a meat image, the image of the food to be identified is input into the raw / cooked state recognition model to obtain the raw / cooked state recognition result of the meat in the image of the food to be identified.

[0067] In some implementations, the raw / cooked state recognition model can be a binary classification model, whose output includes a confidence score corresponding to the raw meat category and a confidence score corresponding to the cooked meat category. In other implementations, the raw / cooked state recognition model can also be a multi-class classification model, whose output includes probabilities or confidence scores corresponding to multiple preset raw / cooked state categories respectively. The image of the food to be recognized can be preprocessed uniformly before being input into the raw / cooked state recognition model. The uniform preprocessing can include adjusting the image size and image normalization. For example, the input image can be adjusted to 224×224 and normalized using ImageNet mean and standard deviation, where mean=[0.485,0.456,0.406] and std=[0.229,0.224,0.225].

[0068] In one specific implementation, the raw / cooked status recognition model can include multiple ONNX sub-models. The AI ​​model service layer can load the `raw_cooked_model_fold_N.onnx` sub-model from the `onnx` directory; each sub-model independently infers for the same food image to be recognized, outputting logits; the logits are processed using Softmax to obtain a probability vector; the probability vectors output by multiple sub-models are averaged to obtain the ensemble probability result; and finally, the final predicted category is determined by argmax. Here, `index=0` corresponds to the raw meat category, and `index=1` corresponds to the cooked meat category. The output results can include `is_raw_meat`, `raw_confidence`, and `cooked_confidence`.

[0069] The raw / cooked status recognition model is trained through the following steps: First, training images of meat dishes labeled with their raw / cooked status are acquired. Then, image transformations are applied to the meat training images to simulate target shooting interference types, resulting in enhanced training images. The image transformations and target shooting interference types have a pre-defined simulated correspondence. Finally, the raw / cooked status recognition model is trained using the enhanced training images and their corresponding raw / cooked status labels.

[0070] In some implementations, the training images for meat dishes can be derived from historical work orders containing images associated with the raw / cooked state of the meat, or from pre-collected images of meat dishes. Each training image for meat dishes is associated with a corresponding raw / cooked state label. The raw / cooked state label indicates that the meat in the training image belongs to one of the preset raw / cooked state categories. For example, the training dataset may include 1098 raw meat samples, 1059 cooked meat samples, and an additional 2481 non-meat anomaly samples in the base database, used to train or evaluate the meat / non-meat detection model. These samples can cover imaging from multiple brands of mobile phones, multiple shooting states, and multiple lighting conditions.

[0071] In some implementations, the raw / cooked status recognition model can employ an EfficientNet backbone network, such as EfficientNet-B3. The model can be built based on pre-trained weights from the timm module, for example, using `timm.create_model('efficientnet_b3', pretrained=True, num_classes=2)` to construct a binary classification model. This model can retain the Swish activation function and the Squeeze-and-Excitation channel attention mechanism to extract features from discriminative characteristics such as meat color, texture, and gloss. The Swish activation function can be represented as the input multiplied by its Sigmoid function output, i.e., x×σ(x). The Squeeze-and-Excitation channel attention structure can generate channel weights in the MBConv module using operators such as ReduceMean, conv_reduce, Sigmoid, and Mul, and adjust the feature response based on these channel weights.

[0072] In one specific implementation, the network structure of the raw / ripe state recognition model may include an initial convolutional layer (conv_stem), multiple MBConv modules, a GlobalAveragePool, and fully connected classification heads (Flatten and Gemm). The MBConv modules may include depthwise convolution and SE attention structures. The model input can be batch×3×224×224 RGB images, and the output can be batch×2 binary logits. ONNX model structure analysis shows the model may include 386 nodes, including 130 Conv nodes, 104 Sigmoid nodes, 104 Mul nodes, 26 ReduceMean nodes, and 19 Add nodes. The model training framework can be PyTorch 2.4.1, and data augmentation can use the alumnimentations library. The model deployment format can be ONNX, and inference can be executed on the CPU using the ONNX Runtime, eliminating the need for a GPU.

[0073] According to the method in this application embodiment, non-meat images are first pre-rejected through meat image detection, and then the raw / cooked state recognition model is used to determine the raw / cooked state of the meat images. This decouples "whether it is meat" and "the raw / cooked state of the meat" into two relatively independent recognition tasks. This reduces the likelihood of misjudgments when non-meat images are incorrectly input into the raw / cooked state recognition model. Furthermore, through image enhancement training accommodating real-world shooting interference, the model can learn more stable raw / cooked meat discrimination features under different devices, lighting conditions, focal lengths, and shooting angles.

[0074] According to the method of the embodiments of this application, the target shooting interference type includes at least two of imaging color difference, focus difference, imaging noise, shooting angle difference, partial occlusion, and brightness difference; the simulated correspondence between image transformation and target shooting interference type includes at least two of the following: using at least one of hue transformation, saturation transformation, and brightness channel transformation to simulate imaging color difference; using blur transformation to simulate focus difference; using noise superposition transformation to simulate imaging noise; using at least one of translation transformation, scaling transformation, and rotation transformation to simulate shooting angle difference; using local region discard transformation to simulate partial occlusion; and using at least one of brightness transformation and contrast transformation to simulate brightness difference.

[0075] In some implementations, differences in image color can stem from variations in white balance strategies, ISP color processing styles, warm / cool tone differences, and saturation differences between different mobile phone brands. These differences can be simulated using HueSaturationValue transformation. For example, the hue perturbation range can be ±10, the saturation perturbation range can be ±30, the luminance channel perturbation range can be ±20, and the execution probability can be 0.5.

[0076] In some implementations, focus differences can include image blurring caused by hand shake, inaccurate focusing, or out-of-focus conditions. This type of difference can be simulated using the Gaussian Blur transform. For example, the blur kernel limit parameter `blur_limit` can be 3, and the execution probability can be 0.2.

[0077] In some implementations, imaging noise may include sensor readout noise or compressed noise under low-light conditions. This type of difference can be modeled using the GaussNoise transform. For example, the noise variance parameter var_limit can be 0.1, and the execution probability can be 0.2.

[0078] In some implementations, differences in shooting perspective can include different holding angles, lens distortion, and angled or overhead shots. These differences can be simulated using the ShiftScaleRotate transform. For example, the shift parameter can be 0.1, the scale parameter can be 0.1, the rotation angle can be 30°, and the execution probability can be 0.5. Different framing compositions can also be simulated using the RandomResizedCrop transform. For example, the crop size can be 256, and the scale range can be from 0.8 to 1.0. Differences in plate orientation can also be simulated using HorizontalFlip and VerticalFlip. For example, the execution probability of HorizontalFlip can be 0.5, and the execution probability of VerticalFlip can be 0.2.

[0079] In some implementations, partial occlusion may include the loss of local information caused by plates, side dishes, cutlery, packaging, or reflective highlights. This type of difference can be simulated using the CoarseDropout transform. For example, the maximum number of drop-out regions, max_holes, can be 8, and the execution probability can be 0.2.

[0080] In some implementations, brightness differences can include low indoor light, strong outdoor light, overexposure, or underexposure. Such differences can be simulated using the RandomBrightnessContrast transform. For example, the brightness and contrast perturbation range can be set to ±0.2, and the execution probability can be 0.5.

[0081] In a specific data augmentation pipeline, the training phase can sequentially perform the following processing on each meat training image: RandomResizedCrop(size=256, scale=0.8~1.0); HorizontalFlip(p=0.5) and VerticalFlip(p=0.2); ShiftScaleRotate(shift=0.1, scale=0.1, rotate=30°, p=0.5); RandomBrightnessContrast(±0.2, p=0.5); HueSaturationValue(hue±10, sat±30, val±20, p=0.5); GaussNoise(var_limit=0.1, p=0.2); GaussianBlur(blur_limit=3, p=0.2); CoarseDropout(max_holes=8, p=0.2); Normalize and ToTensor. The above image transformations correspond to differences in framing and composition, plate orientation, shooting angle, lighting, mobile phone imaging color, low light noise, out-of-focus, and partial occlusion in real shooting scenarios.

[0082] According to the method of the embodiments of this application, performing image transformation on a training image of meat for simulating target shooting interference includes: performing at least two image transformations serially or in parallel on the same training image of meat according to a preset probability to obtain an enhanced training image corresponding to the same raw / cooked status label.

[0083] In some implementations, performing at least two image transformations sequentially can refer to performing multiple image transformations on the same meat training image in turn, so that the same enhanced training image simultaneously contains multiple simulated shooting interference results. For example, random cropping can be performed first, followed by translation, scaling, and rotation, then brightness and contrast transformations, and finally noise superposition and blurring transformations. Performing at least two image transformations in parallel can refer to generating multiple enhancement branches for the same meat training image, with each enhancement branch performing one or more image transformations, thereby obtaining multiple enhanced training images corresponding to the same raw / cooked state label. The enhanced training images inherit the raw / cooked state label of the original meat training image.

[0084] By performing multiple image transformations according to preset probabilities, all training samples can be protected from being processed with a fixed enhancement method, allowing the model to encounter different combinations of shooting interference during training. This reduces the model's dependence on single shooting conditions, single device imaging styles, or single lighting environments, improving the model's adaptability to real-world complaint images.

[0085] In some exemplary embodiments, such as Figure 2 As shown, the raw / cooked state recognition model can be obtained through the following training process.

[0086] S201, Obtain training images of meat products labeled with their raw / cooked status.

[0087] The training images for meat dishes can include images of dishes associated with the raw / cooked status of meat from historical work orders, or pre-collected images of meat dishes. Each training image of meat dishes is associated with a corresponding raw / cooked status label, which indicates that the meat in the training image belongs to one of the preset raw / cooked status categories.

[0088] In practical implementation, training images of meat dishes can be stored in an original meat dish image library. The raw / cooked status of images in the original meat dish image library can be determined based on manual annotation results, historical work order processing results, or review results. For the anomaly sample base library used for meat / non-meat detection, images that do not contain meat or are unrelated to the raw / cooked status of meat can be stored for training or validating the rejection capability of single-class anomaly detection models.

[0089] S202, based on the labeling of raw and cooked status, organize the training images of meat dishes into samples.

[0090] In some implementations, the sampling weights for different raw / cooked states can be determined based on the number of image samples corresponding to each state. Training image samples are then extracted from the meat training images according to these sampling weights. A higher sampling weight corresponds to a raw / cooked state with fewer samples. For example, WeightedRandomSampler can be used to perform weighted sampling based on the inverse of the class frequency to alleviate the class imbalance problem between raw and cooked meat samples.

[0091] In other implementations, the training images of meat dishes can be divided into multiple training subsets based on the principle of consistent category proportions. This ensures that the proportion of samples in each training subset representing different states of raw / cooked meat corresponds to the category proportions in the overall training images of meat dishes. For example, StratifiedKFold can be used for hierarchical partitioning, with K value set to 5. Hierarchical partitioning maintains a similar proportion of raw / cooked categories in different training subsets, reducing training instability caused by data partitioning bias.

[0092] S203, determine the type of target shooting interference used for training enhancement.

[0093] The target shooting interference type can be determined from a preset set of shooting interference types. This preset set can include at least two of the following: imaging color difference, focus difference, imaging noise, shooting angle difference, partial occlusion, and brightness difference. The target shooting interference type can be determined based on the source of interference in the actual complaint images. For example, in scenarios with many images uploaded from multiple brands of mobile phones, imaging color difference can be selected as the target shooting interference type; in scenarios with many low-light complaint images, brightness difference and imaging noise can be selected as the target shooting interference types; in scenarios where plate obstruction or reflective highlights are common, partial occlusion can be selected as the target shooting interference type.

[0094] S204, determine the simulated correspondence between the target shooting interference type and the image transformation.

[0095] The simulated correspondence is used to represent the target shooting interference type simulated by corresponding image transformations. For example, at least one of hue transformation, saturation transformation, and luminance channel transformation is used to simulate imaging color differences; blur transformation is used to simulate focus differences; noise superposition transformation is used to simulate imaging noise; at least one of translation transformation, scaling transformation, and rotation transformation is used to simulate shooting angle differences; local region discard transformation is used to simulate local occlusion; and at least one of luminance transformation and contrast transformation is used to simulate luminance differences.

[0096] S205. According to the simulated correspondence, perform image transformation on the training image of meat in the meal to obtain an enhanced training image.

[0097] In some implementations, at least one image transformation can be performed on the same meat training image according to a preset probability. In other implementations, at least two image transformations can be performed sequentially or in parallel on the same meat training image according to a preset probability to obtain one or more augmented training images corresponding to the same raw / cooked state label. The augmented training images inherit the raw / cooked state label of the corresponding meat training image.

[0098] In the specific implementation, the input size during the training phase can be set to 256×256, and when deployed for ONNX inference, the input can be preprocessed to 224×224. The normalization parameters can be the ImageNet mean and standard deviation, i.e., mean=[0.485,0.456,0.406] and std=[0.229,0.224,0.225].

[0099] S206, The enhanced training images and their corresponding raw / cooked state labels are used as training samples and input into the raw / cooked state recognition model to be trained.

[0100] The raw / cooked state recognition model to be trained may include a convolutional neural network for extracting image features. The convolutional neural network can extract color features, texture features, and gloss features from the enhanced training image and output raw / cooked state prediction results corresponding to the preset raw / cooked state categories.

[0101] In one specific implementation, the raw / cooked state recognition model to be trained can be an EfficientNet-B3 binary classification model with up to 12 million parameters. This model can perform transfer learning based on ImageNet pre-trained weights and set the classification head to binary output. The model can retain the Swish activation function and SE channel attention structure, allowing the model to assign different weights to the color, texture, and gloss channels related to the raw / cooked state of meat during training.

[0102] S207. Based on the prediction results of the raw / cooked state and the corresponding raw / cooked state labels, adjust the model parameters of the raw / cooked state recognition model to be trained.

[0103] Specifically, the model training loss can be determined based on the difference between the predicted raw / cooked state and the corresponding raw / cooked state label, and the model parameters of the raw / cooked state recognition model to be trained can be adjusted based on the model training loss.

[0104] In one specific implementation, the loss function can be CrossEntropyLoss; the optimizer can be AdamW, the learning rate (lr) can be set to 1e-3, and the weight decay (weight_decay) can be set to 1e-4; the learning rate scheduling strategy can be CosineAnnealingLR, and T_max can be set to 25; the batch size (batch_size) can be set to 32, and the number of training epochs can be set to 25. Mixed-precision training can also be used during training, such as using torch.amp.GradScaler and autocast, to improve training efficiency. The random seed can be set to seed=42 to improve the reproducibility of the training process.

[0105] S208, when training with multiple training subsets, train multiple sub-recognition models using the enhanced training images corresponding to the multiple training subsets respectively.

[0106] Each training subset can undergo simulated image interference enhancement and model training according to steps S203 to S207 to obtain a corresponding sub-recognition model. In one specific implementation, the training data can be divided into 5 training subsets based on StratifiedKFold, and 5 EfficientNet-B3 sub-models can be trained respectively. Each sub-model can be exported in ONNX format and named raw_cooked_model_fold_N.onnx.

[0107] S209, determine one or more recognition models obtained from training as raw / cooked state recognition models.

[0108] If a single recognition model is trained, it is designated as the "raw / ripe state" recognition model. If multiple sub-recognition models are trained, they are combined to obtain the "raw / ripe state" recognition model. This combination can include calling multiple sub-recognition models separately during the inference phase and fusing their output probabilities. The probability fusion method can be probability averaging, confidence weighting, weighted confidence fusion, or majority voting. In a specific implementation, ensemble methods such as average, confidence, weighted confidence, and majority voting can be tested, and the method with the highest test accuracy can be selected as the deployment scheme. For example, K=5 fold Softmax probability averaging can be used as the ensemble method.

[0109] According to the method of the embodiments of this application, a raw / cooked state recognition model is trained, including: inputting an enhanced training image into a convolutional neural network including a channel attention structure and a self-gated nonlinear activation function to extract the color, texture and gloss features of meat in the enhanced training image; and adjusting the network parameters of the convolutional neural network according to the raw / cooked state prediction results output by the convolutional neural network and the corresponding raw / cooked state labels.

[0110] In some implementations, the channel attention structure can be a Squeeze-and-Excitation structure. This structure can compress global information from the convolutional feature map, generate weights to characterize the importance of each channel, and apply these weights to the feature responses of the corresponding channels. For the task of recognizing the raw / cooked state of meat, features such as color, texture, and gloss may be distributed across different feature channels. The channel attention structure can enhance the discriminative channels related to the raw / cooked state.

[0111] In some implementations, the self-gated nonlinear activation function can be the Swish function. The Swish function forms a smooth nonlinear mapping by multiplying the input with the sigmoid-gated result, which, compared to hard-truncated activation functions, can retain more continuous variation information in fine-grained visual recognition tasks. For fine-grained features such as color gradients, gloss reflections, and texture patterns on meat surfaces, this activation function helps the model learn more stable feature representations.

[0112] According to the method of the embodiments of this application, training a raw / cooked state recognition model includes: dividing training images of meat dishes labeled with raw / cooked state into multiple training subsets based on the principle of consistent category proportions; training independently using the multiple training subsets to obtain multiple sub-recognition models; and combining the multiple sub-recognition models to determine a raw / cooked state recognition model.

[0113] In some implementations, the principle of consistent category proportions means that the proportion of samples in each training subset representing different states of raw and cooked food corresponds to the proportion of each category in all training images of meat dishes. A hierarchical K-fold cross-validation method can be used to divide the training subsets. Each fold of training yields a sub-recognition model. Multiple sub-recognition models can learn the feature distribution under different training partitions, thereby reducing the dependence of a single model on specific training sample partitions.

[0114] During the inference phase, all sub-recognition models can be loaded, and after performing uniform preprocessing on the same image of the food to be recognized, they can be inferred separately to obtain multiple probability distributions. The mean of multiple probability distributions can be used as the ensemble probability distribution. Compared with a single model, multi-fold ensemble can reduce the risk of overfitting in scenarios with limited datasets and improve the stability of prediction results.

[0115] According to the method of the embodiments of this application, the raw and cooked status recognition model of the food to be identified is input into the food image to be identified to obtain the raw and cooked status recognition result of the meat in the food image to be identified, including: inputting the food image to be identified into multiple sub-recognition models respectively to obtain multiple raw and cooked status probability distributions output by each sub-recognition model; performing fusion calculation on the multiple raw and cooked status probability distributions, and determining the raw and cooked status recognition result of the meat based on the fused probability result.

[0116] In some implementations, the multiple sub-recognition models can be multiple ONNX sub-models trained with K-Fold. For each sub-recognition model, the image of the food to be recognized can be adjusted to 224×224 and normalized to obtain the model input tensor. After the sub-recognition model outputs logits, a Softmax operation is performed on the logits to obtain the probability distributions corresponding to the raw meat category and the cooked meat category. The probability distributions output by the multiple sub-recognition models are averaged element-wise to obtain the fused probability distribution. If the fused probability of raw meat is greater than the probability of cooked meat, the meat's raw / cooked state is determined to be raw meat; if the fused probability of cooked meat is greater than the probability of raw meat, the meat's raw / cooked state is determined to be cooked meat. The output results may also include the confidence scores for raw meat and cooked meat for customer service personnel to refer to.

[0117] According to the method of the embodiments of this application, obtaining training images of meat products labeled with raw or cooked status includes: determining the sampling weight of different raw or cooked statuses based on the number of image samples corresponding to different raw or cooked statuses; wherein, the sampling weight of a raw or cooked status with fewer samples is higher; and extracting training images of meat products from the original meat product image library according to the sampling weights.

[0118] In some implementations, the number of samples for each raw / cooked state category in the original food meat image database can be counted, and the sampling weight can be determined based on the reciprocal of the sample count. For categories with fewer samples, the probability of their sampling is increased; for categories with more samples, the probability of their sampling is decreased. This approach allows the model to learn the features of minority class samples more fully during training, reducing bias caused by class imbalance.

[0119] According to the method of the embodiments of this application, meat image detection is performed on the image of the food to be identified, including: extracting global image features of the image of the food to be identified; inputting the global image features into a single-class anomaly detection model trained based on a single positive class sample, and obtaining a detection result of whether the image of the food to be identified is a meat image.

[0120] In some implementations, positive class samples are images containing meat. The single-class anomaly detection model does not require supervised training with a large number of non-meat images as negative class samples; instead, it learns the distribution region of positive class samples in the feature space. For images deviating from this distribution region, the single-class anomaly detection model outputs an anomaly result. This approach is suitable for scenarios where the types of non-meat images are complex and the negative class space is difficult to exhaustively explore.

[0121] According to the method of the embodiments of this application, the single-class anomaly detection model is trained by the following steps: acquiring images containing meat as positive class samples; extracting global image features of positive class samples using a pre-trained feature extraction network; and training the single-class anomaly detection model with the goal of identifying and surrounding the feature distribution boundary of positive class samples.

[0122] In one specific implementation, a ResNet50 pre-trained on ImageNet can be used to extract 2048-dimensional global image features from positive class samples. These features are then subjected to PCA dimensionality reduction, for example, to 50 dimensions. The dimensionality-reduced features are then input into a One-Class SVM for training, resulting in a single-class anomaly detection model. The nu parameter of the One-Class SVM can be set to 0.07, and a threshold strategy with percentile=7 can be used to determine the detection boundary. The trained model can be saved as oc_svm_model.pkl and loaded for use in the AI ​​model service layer.

[0123] According to the method of the embodiments of this application, after obtaining the recognition result of the raw / cooked state of meat in the image of the food to be identified, the method further includes: performing semantic similarity retrieval in the image index library based on the image features of the image of the food to be identified, and obtaining historical food images whose similarity scores meet the set conditions; obtaining historical work order information associated with the historical food images; and outputting the recognition result of the raw / cooked state of meat and the historical work order information.

[0124] In some implementations, the image index can be an image search index built based on historical food images. These historical food images can be stored in association with historical work order information. The image index can employ image semantic vector retrieval, which, compared to keyword-based work order retrieval, can discover historical cases with different text descriptions but similar image content; and compared to perceptual hashing or difference hashing, it can retrieve food images with different shooting angles, compositions, and brightness but similar semantics.

[0125] In one specific implementation, the AI ​​model service layer can call the SearchImageByPic interface of the image search service to perform similar image retrieval on the image of the food to be identified. Threshold filtering can be applied based on similarity scores, for example, retaining historical food images with a score greater than 0.8. Subsequently, a batch query of the MySQL database can be performed to retrieve historical work order information associated with the historical food images. Historical work order information may include work order number, complaint type, processing result, historical image address, raw / cooked judgment result, or other business fields. The AI ​​model service layer can record the query history, for example, by writing it to the query_history table, and return the recognition results and a list of associated historical work orders to the business service layer.

[0126] In some implementations, a unified image archiving mechanism can also be established. The image of the food item to be identified, its identification result, image URL, uploading user, upload time, and associated work order information can be stored in the `image_info` table and simultaneously added to the image search index. Through unified archiving and index maintenance, subsequent searches for similar images from historical work orders can cover more processed cases.

[0127] In a complete engineering embodiment, the system may include a browser plugin layer, a business service layer, and an AI model service layer. The browser plugin layer is responsible for receiving user operations, uploading images, displaying raw / cooked food identification results, and associating historical work orders. The business service layer is responsible for providing external interfaces for image recognition, image archiving, user authentication, whitelist verification, and request forwarding. The AI ​​model service layer is responsible for providing core recognition interfaces, separate raw / cooked food identification interfaces, separate meat detection interfaces, and image archiving interfaces.

[0128] For example, the business service layer can provide a POST / image / recognize interface to receive image files and initiate the recognition process; the business service layer can also provide a POST / image / archive interface to receive image archiving requests externally. The AI ​​model service layer can provide a POST / image-recognition interface to perform meat detection, raw / cooked judgment, and similarity retrieval; a POST / image-cooked-or-raw interface to perform raw / cooked judgment alone; a POST / image-meat-or-none interface to perform meat detection alone; and a POST / upload-image interface for image archiving.

[0129] In some implementations, the system can be configured with security mechanisms. Users can log in via SMS verification code sent to their mobile phone number. The verification code can be 6 digits long and can be set to be used only once within a validity period. The system can control customer service account login permissions through the `login_white_list` table, for example, by maintaining a whitelist of customer service accounts. The system can also control the range of interfaces that different accounts can access through the `api_model` field. User behavior logs can record the user's login IP address, the reason for successful or failed login, and store them in the `user_behavior` table.

[0130] In some implementations, the system's technology selection may include: browser plugins using ChromeExtension, Vue3, and TypeScript; business services using Spring Boot 3, MyBatis-Plus, and Java 17; AI model services using FastAPI, Python 3.10, ONNX Runtime, and PyTorch; image search using Imagesearch; object storage using Tencent Cloud COS; database using MySQL; caching using Redis; and deployment using Docker containerization. Model files may include the raw_cooked_model_fold_N.onnx file for raw / cooked status recognition and the oc_svm_model.pkl file for meat / non-meat detection.

[0131] In some test environments, when using ImageNet pre-trained EfficientNet-B3 for direct transfer without scene enhancement, the test set accuracy is 88% to 90%, but the accuracy drops to 75% to 80% on out-of-focus, strong light, and color-shift subsets. After adding the data augmentation pipeline for shooting interference in the embodiments of this application, the test set accuracy improves to 93% to 95%, and the accuracy on out-of-focus, strong light, and color-shift subsets remains above 90%. Based on this, using K=5 fold ensemble probability mean, the accuracy is further improved to 95% to 97%, and the F1 scores for raw and processed images are more balanced. Compared with a single model, the multi-fold ensemble model can improve the accuracy on the test set by 3% to 5% and reduce the false positive rate.

[0132] In some test environments, the single-recognition response time for ONNX Runtime CPU inference is less than 2000ms, which includes image download, dual-model inference, and image search. The system's peak TPS can be set to 20, and the availability metric can be set to greater than 99.9%. Data storage can be configured with MySQL 10G and COS 75G / year. Because the raw / familiar state recognition model is deployed in the ONNX format and supports CPU inference, the system can run without relying on GPUs, resulting in lower deployment costs. Since the system can be integrated into existing customer service systems as a browser plugin, no deep modification to existing systems is required, making it suitable for cross-platform, low-intrusion deployment.

[0133] Through the above implementation method, this application can, after customer service personnel upload food images, first determine whether the image is a meat image. If it is determined to be a meat image, then it performs raw / cooked status identification and further retrieves semantically similar historical food images and corresponding historical work order information. This processing flow can reduce the subjective differences of manual judgment, improve the efficiency of complaint work order processing, and provide customer service personnel with historical processing cases that are semantically similar to the current image.

[0134] Figure 3 This is a hardware structure block diagram of an electronic device implementing the embodiments of this application.

[0135] like Figure 3 As shown, the electronic device 300 may include one or more processors 302, a system motherboard 308 connected to at least one of the processors 302, system memory 304 connected to the system motherboard 308, non-volatile memory (NVM) 306 connected to the system motherboard 308, and a network interface 310 connected to the system motherboard 308.

[0136] Processor 302 may include one or more single-core or multi-core processors. Processor 302 may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In embodiments of the invention, processor 302 may be configured to perform one or more embodiments according to various embodiments of this application.

[0137] In some embodiments, the system motherboard 308 may include any suitable interface controller to provide any suitable interface to at least one of the processors 302 and / or any suitable device or component communicating with the system motherboard 308.

[0138] In some embodiments, system motherboard 308 may include one or more memory controllers to provide an interface to system memory 304. System memory 304 may be used to load and store data and / or instructions. In some embodiments, system memory 304 of electronic device 300 may include any suitable volatile memory, such as suitable dynamic random access memory (DRAM).

[0139] The NVM 306 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the NVM 306 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of an HDD (Hard Disk Drive), a CD (Compact Disc) drive, or a DVD (Digital Versatile Disc) drive.

[0140] NVM 306 may include a portion of the storage resources on a device installed on electronic device 300, or it may be accessible by the device, but is not necessarily part of the device. For example, NVM 306 may be accessed over a network via network interface 310.

[0141] Specifically, system memory 304 and NVM 306 may each include a temporary copy and a permanent copy of instruction 320. Instruction 320 may include instructions that, when executed by at least one of processors 302, cause electronic device 300 to perform methods as described in any embodiment of this application. In some embodiments, instruction 320, hardware, firmware, and / or its software components may additionally / alternatively be located in system motherboard 308, network interface 310, and / or processor 302.

[0142] Network interface 310 may include a transceiver for providing a radio interface to electronic device 300, thereby enabling communication with any other suitable device (e.g., front-end module, antenna, etc.) via one or more networks. In some embodiments, network interface 310 may be integrated into other components of electronic device 300. For example, network interface 310 may be integrated into at least one of processor 302, system memory 304, NVM 306, and firmware device (not shown) with instructions, wherein when at least one of processor 302 executes the instructions, electronic device 300 implements one or more embodiments of various embodiments of this application.

[0143] The network interface 310 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 310 may be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.

[0144] In one embodiment, at least one of the processors 302 may be packaged together with one or more controllers for the system motherboard 308 to form a system-in-package (SiP). In another embodiment, at least one of the processors 302 may be integrated on the same die with one or more controllers for the system motherboard 308 to form a system-on-a-chip (SoC).

[0145] The electronic device 300 may further include: an input / output (I / O) device 312 connected to the system motherboard 308. The I / O device 312 may include a user interface enabling a user to interact with the electronic device 300; the peripheral component interface is designed to allow peripheral components to also interact with the electronic device 300. In some embodiments, the electronic device 300 may also include sensors for determining at least one of environmental conditions and location information related to the electronic device 300.

[0146] In some embodiments, the input / output (I / O) device 312 may include, but is not limited to, a display (e.g., a liquid crystal display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., a light-emitting diode flash) and a keyboard.

[0147] In some embodiments, the peripheral component interface may include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.

[0148] In some embodiments, the sensor may include, but is not limited to, a gyroscope sensor, an accelerometer, a proximity sensor, an ambient light sensor, and a positioning unit. The positioning unit may also be part of or interact with the network interface 310 to communicate with components of the positioning network, such as Global Positioning System (GPS) satellites.

[0149] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 300. In other embodiments of this application, the electronic device 300 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0150] Program code can be applied to input instructions to perform the functions described in this invention and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a system for processing instructions including processor 302 includes any system having a processor such as a digital signal processor (DSP), microcontroller, application-specific integrated circuit (ASIC), or microprocessor.

[0151] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this invention are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0152] One or more aspects of at least one embodiment can be implemented by instructions stored on a computer-readable storage medium, which, when read and executed by a processor, enable an electronic device to implement the methods of the embodiments described in this invention.

[0153] According to some embodiments of this application, a computer storage medium is disclosed, on which instructions are stored, which, when executed on a computer, cause the computer to perform a method for identifying the raw or cooked state of meat according to embodiments of this application.

[0154] The method embodiments of this application correspond to this embodiment, and this embodiment can be implemented in conjunction with the method embodiments of this application. The relevant technical details mentioned in the method embodiments of this application are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the method embodiments of this application.

[0155] According to some embodiments of this application, a computer program product is disclosed, including computer-executable instructions that are executed by a processor to implement a method for identifying the raw / cooked state of meat in a meal according to embodiments of this application.

[0156] The method embodiments of this application correspond to this embodiment, and this embodiment can be implemented in conjunction with the method embodiments of this application. The relevant technical details mentioned in the method embodiments of this application are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the method embodiments of this application.

[0157] It is understood that the specific embodiments described herein are merely for illustrative purposes and not for limiting the scope of this application. Furthermore, for ease of description, the accompanying drawings show only the parts relevant to this application, and not all of the structures or processes. It should be noted that similar reference numerals and letters in the drawings denote similar items throughout this application.

[0158] It should be understood that although the terms "first," "second," etc., may be used herein to describe various features, these features should not be limited by these terms. The use of these terms is merely for distinction and should not be construed as indicating or implying relative importance. For example, without departing from the scope of the exemplary embodiments, a first feature may be referred to as a second feature, and similarly, a second feature may be referred to as a first feature.

[0159] In the description of this application, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set up," "connected," and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this embodiment based on the specific circumstances.

[0160] The illustrative embodiments of this application include, but are not limited to, methods, devices, media, and products for identifying the raw or cooked state of meat in meals.

[0161] Various aspects of the illustrative embodiments will be described using terminology commonly employed by those skilled in the art to convey the essence of their work to others skilled in the art. However, it will be apparent to those skilled in the art that some alternative embodiments will be practiced using the features partially described. Specific figures and configurations are set forth for purposes of explanation in order to provide a more thorough understanding of the illustrative embodiments. However, it will be apparent to those skilled in the art that alternative embodiments may be practiced without specific details. In some other instances, well-known features have been omitted or simplified herein to avoid obscuring the illustrative embodiments of this application.

[0162] Furthermore, the various operations will be described as multiple separate operations in a manner most conducive to understanding the illustrative embodiments; however, the order of description should not be construed as implying that these operations must depend on the order of description, and many of these operations may be performed in parallel, concurrently, or simultaneously. Moreover, the order of the operations may also be rearranged. The process may be terminated when the described operations are completed, but may also include additional steps not included in the figures. The process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0163] References to "an embodiment," "embodiment," "illustrative embodiment," etc., in this application indicate that the described embodiment may include specific features, structures, or properties; however, each embodiment may or may not necessarily include specific features, structures, or properties. Furthermore, these phrases are not necessarily directed at the same embodiment. Moreover, when specific features are described in conjunction with specific embodiments, the knowledge of those skilled in the art can influence the combination of these features with other embodiments, whether or not those embodiments are explicitly described.

[0164] Unless the context otherwise specifies, the terms “comprising,” “having,” and “including” are synonyms. The phrase “A and / or B” means “(A), (B), or (A and B).”

[0165] As used herein, the term "module" may refer to, as part of, or include: a memory (shared, dedicated, or grouped), an application-specific integrated circuit (ASIC), electronic circuitry and / or a processor (shared, dedicated, or grouped), combinational logic circuitry, and / or other suitable components that provide the said functionality for running one or more software or firmware programs.

[0166] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order is not necessary. Rather, in some embodiments, these features may be illustrated in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular drawing does not mean that all embodiments need to include such features; in some embodiments, these features may be omitted or may be combined with other features.

[0167] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions or programs carried or stored on one or more transient or non-transient machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors, etc. When the instructions or program are run by a machine, the machine may perform the various methods described above. For example, the instructions may be distributed via a network or other computer-readable media. Therefore, machine-readable media may include, but are not limited to, any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, such as floppy disks, optical disks, optical disc read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electronically erasable programmable read-only memories (EEPROMs), magnetic cards or optical cards, or flash memory or tangible machine-readable storage for transmitting network information via electrical, optical, acoustic, or other forms of signals (e.g., carrier waves, infrared signals, digital signals, etc.). Therefore, machine-readable media includes any form of machine-readable medium suitable for storing or transmitting electronic instructions or machine-readable (e.g., computer-readable) information.

[0168] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, the use of the technical solutions of this application is not limited to the various applications mentioned in the embodiments of this application. Various structures and modifications can be easily implemented with reference to the technical solutions of this application to achieve the various beneficial effects mentioned herein. Within the scope of knowledge possessed by those skilled in the art, all changes made without departing from the spirit of this application should be considered within the scope of this patent application.

Claims

1. A method for identifying the raw / cooked state of meat in a meal, characterized in that, include: Acquire the image of the food item to be identified; Meat image detection is performed on the image of the food item to be identified; If the image of the food to be identified is determined to be a meat image, the image of the food to be identified is input into multiple sub-recognition models of the raw / cooked state recognition model to obtain multiple raw / cooked state probability distributions output by each sub-recognition model; the multiple raw / cooked state probability distributions are fused and calculated, and the raw / cooked state recognition result of the meat in the image of the food to be identified is determined based on the fused probability result. The raw / cooked state recognition model is trained through the following steps: Obtain training images of meat products labeled with their raw / cooked status; Based on the principle of consistent category proportion, the training images of meat dishes labeled with raw and cooked status are divided into multiple training subsets; An image transformation simulating target shooting interference type is performed on the training image of the meat dish to obtain an enhanced training image; the image transformation and the target shooting interference type have a preset simulation correspondence; Multiple sub-recognition models are obtained by independently training the enhanced training images and their corresponding raw / cooked state labels of the multiple training subsets respectively; The multiple sub-recognition models are combined to determine the raw / cooked state recognition model.

2. The method according to claim 1, characterized in that, The target shooting interference types include at least two of the following: imaging color difference, focus difference, imaging noise, shooting angle difference, partial occlusion, and brightness difference; The simulated correspondence between the image transformation and the target shooting interference type includes at least two of the following: At least one of hue transformation, saturation transformation, and luminance channel transformation is used to simulate imaging color differences; Fuzzy transformation is used to simulate focusing differences; Noise superposition transformation is used to simulate imaging noise; At least one of translation transformation, scaling transformation, and rotation transformation is used to simulate differences in shooting perspective; Local occlusion is simulated using local region dropout transformation; At least one of luminance transformation and contrast transformation is used to simulate luminance difference.

3. The method according to claim 1, characterized in that, Performing image transformation on the training images of the meat dishes to simulate target shooting interference types includes: According to a preset probability, at least two image transformations are performed serially or in parallel on the same meat training image to obtain an enhanced training image corresponding to the same raw / cooked status label.

4. The method according to claim 1, characterized in that, The raw / cooked state recognition model is trained and includes: The enhanced training image is input into a convolutional neural network including a channel attention structure and a self-gated nonlinear activation function to extract the meat color, texture and gloss features in the enhanced training image; Based on the raw / cooked state prediction results and corresponding raw / cooked state labels output by the convolutional neural network, the network parameters of the convolutional neural network are adjusted.

5. The method according to claim 1, characterized in that, Obtain training images of meat dishes labeled with their raw / cooked status, including: The sampling weights for different raw / ripe states are determined based on the number of image samples corresponding to each raw / ripe state. Among them, the sampling weights for raw / ripe states with fewer samples are higher. According to the sampling weights, the training images of meat products are extracted from the original meat product image library.

6. The method according to claim 1, characterized in that, Meat image detection is performed on the image of the food to be identified, including: Extract global image features from the image of the food item to be identified; The global image features are input into a single-class anomaly detection model trained on a single positive class sample to obtain the detection result of whether the food image to be identified is a meat image.

7. The method according to claim 6, characterized in that, The single-class anomaly detection model is trained through the following steps: Obtain images containing meat as positive class samples; The global image features of the positive class samples are extracted using a pre-trained feature extraction network; The single-class anomaly detection model is trained with the optimization objective of identifying and enclosing the feature distribution boundary of the positive class samples.

8. The method according to any one of claims 1-7, characterized in that, After obtaining the result of identifying the raw / cooked state of the meat in the image of the food to be identified, the method further includes: Based on the image features of the food image to be identified, a semantic similarity search is performed in the image index library to obtain historical food images whose similarity scores meet the set conditions; Obtain historical work order information associated with the historical food images; Output the results of the meat raw / cooked status identification and the historical work order information.

9. An electronic device, characterized in that, The electronic device includes a memory storing computer-executable instructions and a processor, which, when executed by the processor, causes the electronic device to implement the method for identifying the raw / cooked state of meat in a meal according to any one of claims 1-8.

10. A computer storage medium, characterized in that, The computer storage medium stores instructions that, when executed on the computer, cause the computer to perform the method for identifying the raw / cooked state of meat in a meal according to any one of claims 1-8.

11. A computer program product, characterized in that, It includes computer-executable instructions, which are executed by a processor to implement the method for identifying the raw / cooked state of meat in a meal according to any one of claims 1-8.

Images