Food quality evaluation method, device, electronic device and readable storage medium
By segmenting out images containing only the objects to be evaluated in the food quality evaluation, avoiding background information interference, the problem of inaccurate food quality evaluation in the prior art is solved, and higher evaluation accuracy is achieved.
Patent Information
- Application Number
- CN202111452334.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-01
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-12-01
AI Technical Summary
In the prior art In the evaluation of food quality, the target image segmented by the target segmentation model contains background information, resulting in the results of the evaluation of food quality inadequately accurate.
The pre-trained evaluation model obtains the image containing the meal to be evaluated, determines the candidate area and divides out the second image containing only the object to be evaluated, and uses the image classification module to evaluate the distribution uniformity to avoid interference from background information and improves the accuracy of evaluation sex.
It improves the accuracy of the assessment of the uniformity of the meal distribution, thereby improving the accuracy of the assessment of the results of the meal quality.
Smart Images

Figure CN114091969B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a method, device, electronic device and readable storage medium for evaluating food quality. Background Art
[0002] In the catering industry, the quality of food determines sales to a certain extent. In the process of making food, due to the different technical proficiency of chefs, the quality of the food may also be different, affecting the user's impression of the food. The uniformity of food distribution is the most intuitive visual expression and the first impression of people. If a dish is scattered in a messy way and the ingredients are scattered irregularly, it will affect the taste and sales of the food. Therefore, in order to improve the quality of food and enhance the user's dining experience, it is usually necessary to monitor the uniformity of the distribution of the prepared food.
[0003] In the prior art, when monitoring the distribution uniformity of food, a target segmentation model is usually used to segment the acquired image containing the food to be evaluated, and the target image containing the food to be evaluated is segmented out by the target segmentation model, and then the food to be evaluated in the target image is evaluated to determine the quality of the food to be evaluated. Since the target image segmented out by the target segmentation model contains not only the food to be evaluated but also part of the background area, when evaluating the target image containing the food to be evaluated and part of the background area, it is disturbed by the part of the background area, making the quality result of the food to be evaluated determined by the method in the prior art not accurate enough. Summary of the invention
[0004] In view of this, the purpose of the present application is to provide a food quality assessment method, device, electronic device and readable storage medium to improve the accuracy of the distribution uniformity assessment of the objects to be assessed on the food to be assessed, thereby improving the accuracy of the quality assessment of the food to be assessed.
[0005] In a first aspect, an embodiment of the present application provides a method for evaluating food quality, the method being applied to a server, the method comprising:
[0006] Acquire a first image containing a food to be evaluated;
[0007] Input the first image into a pre-trained evaluation model, determine a candidate region including the food to be evaluated and part of the background information in the first image through the evaluation model, and segment a second image including the object to be evaluated from the candidate region; wherein at least one of the objects to be evaluated is distributed on the surface of the food to be evaluated;
[0008] Calculating the second image by using the evaluation model to obtain a distribution uniformity result of the object to be evaluated;
[0009] The quality of the food to be evaluated is determined according to the distribution uniformity result of the object to be evaluated in the food to be evaluated.
[0010] In combination with the first aspect, an embodiment of the present application provides a first possible implementation of the first aspect, wherein the evaluation model is obtained by training multiple GPUs connected in a ring; the same initial evaluation model is deployed on each GPU, and the training samples on different GPUs are different; wherein the method for training the evaluation model includes:
[0011] For each GPU, a first model parameter sent by an upstream GPU of the GPU is received, where the first model parameter is obtained by the upstream GPU based on training of the current evaluation model deployed thereon; the first model parameter is fused and calculated with the model parameters corresponding to its own current evaluation model to obtain fused model parameters, and the current evaluation model including the fused model parameters is further trained using the training samples corresponding to the GPU to obtain second model parameters obtained by training the current evaluation model, the current evaluation model of the GPU is updated based on the second model parameters, and the second model parameters are sent to the downstream GPUs of the GPU until the current evaluation model including the second model parameters on each GPU meets the first training cutoff condition, and the current evaluation model on any GPU is used as the evaluation model.
[0012] In combination with the first possible implementation of the first aspect, an embodiment of the present application provides a second possible implementation of the first aspect, wherein the current evaluation model including the second model parameters on each GPU satisfies the first training cutoff condition including: the parameter index of the current evaluation model including the second model parameters on each GPU satisfies a preset threshold; the parameter index includes the first prediction accuracy and / or error rate of the current evaluation model; when the current evaluation model including the second model parameters on each GPU satisfies the first training cutoff condition, the second model parameters of the current evaluation model on each GPU are the same.
[0013] In combination with the first possible implementation of the first aspect, the embodiment of the present application provides a third possible implementation of the first aspect, wherein the training sample is constructed in the following manner:
[0014] Constructing a plurality of initial training samples for training the evaluation model;
[0015] Randomly selecting at least two of the initial training samples as a group of samples to be fused;
[0016] For each group of samples to be fused, the initial training samples in the group of samples to be fused are subjected to image fusion processing to obtain enhanced training samples corresponding to the group of samples to be fused;
[0017] All the initial training samples and all the enhanced training samples are determined as the training samples.
[0018] In combination with the first aspect, the embodiment of the present application provides a fourth possible implementation of the first aspect, wherein the evaluation model calculates the second image through an included image classification module to obtain a distribution uniformity result of the object to be evaluated; the image classification module is trained in the following manner:
[0019] Constructing a first training sample for training the image classification module; the first training sample includes: a target image containing only a target object and a preset label of the target image; at least one of the target objects is distributed on the surface of the target food;
[0020] Using the first training sample to train an initial image classification module including a third model parameter to obtain an intermediate image classification module; the prediction accuracy of the image classification module is greater than the prediction accuracy of the intermediate image classification module;
[0021] The target image is input into the intermediate image classification module, and the target image is calculated by the intermediate image classification module to determine the predicted distribution uniformity result of the target object in the target image and the weight of the target image; the predicted distribution uniformity result includes a first probability that the distribution of the target object in the target image predicted by the intermediate image classification module is a uniform distribution and a second probability that the distribution of the target object is a non-uniform distribution;
[0022] According to the weight of each of the target images, the target images in the first training sample are divided into a first category of images and a second category of images by the intermediate image classification module; wherein the weight of the first category of images is greater than the weight of the second category of images;
[0023] For each second-category image, when the first probability, the second probability, and the preset label corresponding to the second-category image satisfy the following relationship, it is determined that the preset label of the second-category image is incorrectly labeled; wherein P1 is the first probability corresponding to the second-category image, P2 is the second probability corresponding to the second-category image, P is the value of the probability corresponding to the preset label of the second-category image, and k is a preset value;
[0024] Max(P1,P2)>P+k
[0025] When the preset label of the second category image is incorrectly marked, the preset label of the second category image is corrected to obtain a corrected label of the second category image, and the second category image and the corrected label are used as second training samples for training the intermediate image classification module;
[0026] The intermediate image classification module is trained using the second training sample until the intermediate image classification module meets a second training cutoff condition, and the trained intermediate image classification module is used as the image classification module.
[0027] In combination with the fourth possible implementation of the first aspect, the embodiment of the present application provides a fifth possible implementation of the first aspect, wherein the evaluation model obtains the second image by including a region segmentation module; and the evaluation model is deployed to the server in the following manner:
[0028] Converting the trained formats of the region segmentation module and the image classification module into an open neural network exchange format;
[0029] Converting the region segmentation module and the image classification module in the open neural network exchange format into an accelerated format;
[0030] The prediction speeds of the region segmentation module and the image classification module under the acceleration format are tested respectively. When the prediction speed of the region segmentation module is greater than a first preset speed, and the prediction speed of the image classification module is greater than a second preset speed, the region segmentation module and the image classification module are encapsulated to obtain the evaluation model, and the evaluation model is deployed to the server.
[0031] In combination with the first aspect, the embodiment of the present application provides a sixth possible implementation of the first aspect, wherein the acquiring of the first image containing the food to be evaluated includes:
[0032] Receive the first images of the multiple dishes to be evaluated sent by the photographing device; each dish to be evaluated corresponds to a first image less than the target number, and the first image is a picture that meets the preset filtering conditions selected by the photographing device from the multiple candidate images of the dishes to be evaluated; the preset filtering conditions include: containing at least one of the dishes to be evaluated, the clarity of at least one of the dishes to be evaluated is greater than or equal to a preset clarity threshold, and the completeness of at least one of the dishes to be evaluated is greater than or equal to a completeness threshold.
[0033] In a second aspect, the present application also provides a food quality assessment device, which resides in a server and includes:
[0034] An acquisition module, used for acquiring a first image containing a meal to be evaluated;
[0035] An input module, used to input the first image into a pre-trained evaluation model, determine a candidate region including the food to be evaluated and part of the background information in the first image through the evaluation model, and segment a second image including the object to be evaluated from the candidate region; wherein at least one of the objects to be evaluated is distributed on the surface of the food to be evaluated;
[0036] A calculation module, used to calculate the second image by using the evaluation model to obtain a distribution uniformity result of the object to be evaluated;
[0037] The first determination module is used to determine the quality of the food to be evaluated according to the distribution uniformity result of the object to be evaluated in the food to be evaluated.
[0038] In combination with the second aspect, an embodiment of the present application provides a first possible implementation of the second aspect, wherein the evaluation model is obtained by training multiple GPUs connected in a ring; the same initial evaluation model is deployed on each GPU, and the training samples on different GPUs are different;
[0039] The first training module, when used to train the evaluation model, is specifically used to receive, for each GPU, a first model parameter sent by an upstream GPU of the GPU, where the first model parameter is obtained by the upstream GPU based on the training of the current evaluation model deployed thereon; the first model parameter is fused and calculated with the model parameters corresponding to its own current evaluation model to obtain fused model parameters, and the current evaluation model including the fused model parameters is further trained using the training samples corresponding to the GPU to obtain second model parameters obtained by the current evaluation model training; the current evaluation model of the GPU is updated based on the second model parameters, and the second model parameters are sent to the downstream GPU of the GPU until the current evaluation model including the second model parameters on each GPU meets the first training cutoff condition, and the current evaluation model on any GPU is used as the evaluation model.
[0040] In combination with the first possible implementation of the second aspect, the embodiment of the present application provides a second possible implementation of the second aspect, which further includes:
[0041] A construction module, used to construct a plurality of initial training samples for training the evaluation model;
[0042] A selection module, used for arbitrarily selecting at least two of the initial training samples as a group of samples to be fused;
[0043] A fusion module is used to perform image fusion processing on the initial training samples in each group of samples to be fused, so as to obtain enhanced training samples corresponding to the group of samples to be fused;
[0044] The second determining module is used to determine all the initial training samples and all the enhanced training samples as the training samples.
[0045] In combination with the second aspect, the embodiment of the present application provides a third possible implementation of the second aspect, wherein the evaluation model calculates the second image through the included image classification module to obtain the distribution uniformity result of the object to be evaluated; and further includes:
[0046] The second training module, when used to train the image classification module, is specifically used to:
[0047] Constructing a first training sample for training the image classification module; the first training sample includes: the target image containing only the target object and a preset label of the target image; at least one of the target objects is distributed on the surface of the target food;
[0048] Using the first training sample to train an initial image classification module including a third model parameter to obtain an intermediate image classification module; the prediction accuracy of the image classification module is greater than the prediction accuracy of the intermediate image classification module;
[0049] The target image is input into the intermediate image classification module, and the target image is calculated by the intermediate image classification module to determine the predicted distribution uniformity result of the target object in the target image and the weight of the target image; the predicted distribution uniformity result includes a first probability that the distribution of the target object in the target image predicted by the intermediate image classification module is a uniform distribution and a second probability that the distribution of the target object is a non-uniform distribution;
[0050] According to the weight of each of the target images, the target images in the first training sample are divided into a first category of images and a second category of images by the intermediate image classification module; wherein the weight of the first category of images is greater than the weight of the second category of images;
[0051] For each second-category image, when the first probability, the second probability, and the preset label corresponding to the second-category image satisfy the following relationship, it is determined that the preset label of the second-category image is incorrectly labeled; wherein P1 is the first probability corresponding to the second-category image, P2 is the second probability corresponding to the second-category image, P is the value of the probability corresponding to the preset label of the second-category image, and k is a preset value;
[0052] Max(P1,P2)>P+k
[0053] When the preset label of the second category image is incorrectly marked, the preset label of the second category image is corrected to obtain a corrected label of the second category image, and the second category image and the corrected label are used as second training samples for training the intermediate image classification module;
[0054] The intermediate image classification module is trained using the second training sample until the intermediate image classification module meets a second training cutoff condition, and the trained intermediate image classification module is used as the image classification module.
[0055] In combination with the third possible implementation manner of the second aspect, the embodiment of the present application provides a fourth possible implementation manner of the second aspect, wherein the evaluation model obtains the second image by including a region segmentation module; and further includes:
[0056] A first conversion module, used for converting the formats of the trained region segmentation module and the image classification module into an open neural network exchange format;
[0057] A second conversion module, used to convert the region segmentation module and the image classification module in the open neural network exchange format into an accelerated format;
[0058] A test module is used to test the prediction speeds of the region segmentation module and the image classification module under the acceleration format respectively. When the prediction speed of the region segmentation module is greater than a first preset speed, and the prediction speed of the image classification module is greater than a second preset speed, the region segmentation module and the image classification module are encapsulated to obtain the evaluation model, and the evaluation model is deployed to the server.
[0059] In combination with the second aspect, the embodiment of the present application provides a fifth possible implementation of the second aspect, wherein the acquisition module, when used to acquire a first image containing a meal to be evaluated, is specifically used to:
[0060] Receive the first images of the multiple dishes to be evaluated sent by the photographing device; each dish to be evaluated corresponds to a first image less than the target number, and the first image is a picture that meets the preset filtering conditions selected by the photographing device from the multiple candidate images of the dishes to be evaluated; the preset filtering conditions include: containing at least one of the dishes to be evaluated, the clarity of at least one of the dishes to be evaluated is greater than or equal to a preset clarity threshold, and the completeness of at least one of the dishes to be evaluated is greater than or equal to a completeness threshold.
[0061] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps in any possible implementation of the first aspect above are performed.
[0062] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in any possible implementation of the first aspect described above are executed.
[0063] The embodiments of the present application provide a method, device, electronic device and readable storage medium for evaluating the quality of food. The evaluation model is used to segment the object to be evaluated on the food to be evaluated from the first image. Specifically, a candidate area including the food to be evaluated and part of the background information in the first image is determined, and a second image including the object to be evaluated is segmented from the candidate area, wherein at least one object to be evaluated is distributed on the surface of the food to be evaluated; so that the segmented second image only includes the object to be evaluated on the food to be evaluated, but does not include the background information, and the distribution uniformity of the object to be evaluated on the second image is evaluated to determine the distribution uniformity result of the object to be evaluated; and then the quality of the food to be evaluated is determined according to the distribution uniformity result of the object to be evaluated in the food to be evaluated. Compared with the method of evaluating the target image containing the food to be evaluated and part of the background area in the prior art, the present application evaluates the distribution uniformity of the second image only including the object to be evaluated, determines the distribution uniformity result of the object to be evaluated, avoids the interference of background information, improves the accuracy of the distribution uniformity evaluation of the object to be evaluated, and thus improves the accuracy of the quality result evaluation of the food to be evaluated.
[0064] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0066] Figure 1 A flow chart of a method for evaluating food quality provided in an embodiment of the present application is shown;
[0067] Figure 2A schematic diagram showing a candidate area provided in an embodiment of the present application;
[0068] Figure 3 A schematic diagram of a distributed structure provided in an embodiment of the present application is shown;
[0069] Figure 4 A schematic diagram of the structure of the ring multi-GPU distributed training provided in an embodiment of the present application is shown;
[0070] Figure 5 A schematic diagram of the structure of a food quality assessment device provided in an embodiment of the present application is shown;
[0071] Figure 6 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0072] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.
[0073] The uniformity of food distribution is the most intuitive visual expression and the first impression of people. If a dish is scattered in a disorderly manner, the taste and sales of the food will be affected. In the process of making food, due to the different levels of chefs' technical mastery, the appearance and quality of the food may also be different, affecting the user's impression of the food and even the taste, resulting in different evaluations of the food by diners, affecting the sales of the store. Therefore, in order to improve the quality of food and enhance the user's dining experience, it is usually necessary to monitor the uniformity of the distribution of the prepared food.
[0074] In the prior art, when monitoring the distribution uniformity of food, a target segmentation model is usually used to segment the acquired image containing the food to be evaluated, and the target image containing the food to be evaluated is segmented out by the target segmentation model, and then the food to be evaluated in the target image is evaluated to determine the quality of the food to be evaluated. Since the target image segmented out by the target segmentation model contains not only the food to be evaluated but also part of the background area, when evaluating the target image containing the food to be evaluated and part of the background area, it is disturbed by the part of the background area, making the quality result of the food to be evaluated determined by the method in the prior art not accurate enough.
[0075] In view of the above problems, based on this, the embodiments of the present application provide a food quality assessment method, device, electronic device and readable storage medium to improve the accuracy of the distribution uniformity assessment of the object to be evaluated, thereby improving the accuracy of the quality assessment of the food to be evaluated, which is described below through embodiments.
[0076] Embodiment 1:
[0077] To facilitate understanding of this embodiment, a method for evaluating food quality disclosed in an embodiment of the present application is first introduced in detail. Figure 1 A flow chart of a method for evaluating food quality provided in an embodiment of the present application is shown. Figure 1 As shown, the following steps are included:
[0078] S101: Acquire a first image containing a meal to be evaluated.
[0079] The food refers to the food consumed by the user during dining, and the food to be evaluated refers to the food to be evaluated. The food to be evaluated can be any kind of food, such as pizza, spicy hot pot, etc. In addition to the object to be evaluated, the first image also includes other background information.
[0080] S102: Input the first image into a pre-trained evaluation model, determine a candidate region in the first image including the food to be evaluated and part of the background information through the evaluation model, and segment a second image including the object to be evaluated from the candidate region; wherein at least one object to be evaluated is distributed on the surface of the food to be evaluated.
[0081] The evaluation model includes a region segmentation module and an image classification module, wherein the region segmentation module may be specifically an instance segmentation model, and the image classification module may be specifically a fine-grained classification model. The image features of the first image are calculated by the region segmentation module in the quality evaluation model, a candidate region including the food to be evaluated and part of the background information in the first image is determined, and a second image including the object to be evaluated in the food to be evaluated is segmented from the candidate region to obtain a second image containing only the object to be evaluated on the food to be evaluated, and the second image is input into the image classification module of the evaluation model.
[0082] Exemplarily, when the food to be evaluated is a pineapple pizza, the object to be evaluated is the pineapple on the pineapple pizza, that is, at this time, the second image only contains the pineapple. Figure 2 A schematic diagram of the candidate area provided in the embodiment of the present application is shown. Figure 2 As shown, the region segmentation module first frames a candidate region including the food to be evaluated and part of the background information in the first image, and then segments the object to be evaluated on the food to be evaluated from the candidate region to obtain a second image containing only the object to be evaluated.
[0083] S103: Calculate the second image using the evaluation model to obtain a distribution uniformity result of the object to be evaluated.
[0084] The distribution uniformity feature of the second image is calculated by the image classification module in the evaluation model to obtain the distribution uniformity result of the object to be evaluated in the meal to be evaluated.
[0085] The distribution uniformity result includes uniform distribution and non-uniform distribution. For example, following the example in step S102, when the first image includes a pineapple pizza and background information, a second image containing only the pineapple on the pineapple pizza is obtained by the region segmentation module, and at this time, the image classification module is used to determine whether the distribution of the pineapple in the second image is uniform distribution or non-uniform distribution.
[0086] In a possible embodiment, the image classification module is used to determine a third probability that the distribution of the object to be evaluated is uniformly distributed and a fourth probability that the distribution of the object to be evaluated is non-uniformly distributed. When the third probability is greater than the fourth probability, it means that the distribution uniformity result of the object to be evaluated is uniformly distributed; when the third probability is not greater than the fourth probability, it means that the distribution uniformity result of the object to be evaluated is non-uniformly distributed.
[0087] S104: Determine the quality of the meal to be evaluated according to the distribution uniformity result of the objects to be evaluated in the meal to be evaluated.
[0088] When the distribution uniformity result of the object to be evaluated is uniform distribution, it means that the quality of the food to be evaluated is good; when the distribution uniformity result of the object to be evaluated is non-uniform distribution, it means that the quality of the food to be evaluated is poor.
[0089] The object to be evaluated on the food to be evaluated is segmented from the first image through the evaluation model, specifically, a candidate area including the food to be evaluated and part of the background information in the first image is determined, and a second image including the object to be evaluated is segmented from the candidate area, wherein at least one object to be evaluated is distributed on the surface of the food to be evaluated; so that the segmented second image only includes the object to be evaluated and does not include background information, and the second image is evaluated for distribution uniformity to determine the distribution uniformity result of the object to be evaluated; and then the quality of the food to be evaluated is determined based on the distribution uniformity result of the object to be evaluated in the food to be evaluated. Compared with the method of evaluating a target image including the food to be evaluated and part of the background area in the prior art, the present application evaluates the distribution uniformity of the second image including only the object to be evaluated, determines the distribution uniformity result of the object to be evaluated, avoids the interference of background information, improves the accuracy of the distribution uniformity evaluation of the object to be evaluated, and further improves the accuracy of the quality evaluation of the food to be evaluated.
[0090] When training the evaluation model, it is necessary to use multiple training samples on the GPU for training. If all training samples are deployed on one GPU (graphics processing unit) and all training samples are trained by only one GPU, then this training method will cause a large amount of data processing on the GPU, resulting in a slow training speed. Therefore, in order to avoid using the same GPU to train all training samples, the prior art usually uses a distributed method to train all training samples, that is, multiple GPUs are used to train all training samples, wherein a portion of the training samples is allocated to each GPU. For example, when there are 1,000 training samples in total and there are 5 GPUs, 200 training samples are allocated to each GPU, thereby reducing the data processing amount of a single GPU.
[0091] Figure 3 A schematic diagram of a distributed structure provided in an embodiment of the present application is shown. Figure 3As shown, in the prior art, the same initial model is preset on each GPU. When the initial model is trained in each training round using training samples on each GPU, the GPU (including A2, A3, A4, A5) needs to send the model parameters obtained in each training round to the same GPU (i.e., A1). The GPU (A1) calculates the model parameters sent by other GPUs (including A2, A3, A4, A5) and its own model parameters to obtain new model parameters, and then sends the new model parameters to other GPUs (including A2, A3, A4, A5). All GPUs (including A1, A2, A3, A4, A5) use the new model parameters to update their own model parameters, and repeat the above training process until the training is completed.
[0092] When using the distributed method in the prior art for training, GPU (A1) needs to calculate the model parameters sent by all GPUs (including A2, A3, A4, A5) and its own model parameters. When the number of GPUs is large, the data processing volume of GPU (A1) is large, which in turn affects the model training speed. Based on this problem, the embodiment of the present application provides a ring multi-GPU distributed training method to improve the model training speed, which is described in detail below.
[0093] In a possible implementation, the evaluation model is obtained by training multiple GPUs connected in a ring; the same initial evaluation model is deployed on each GPU, and each GPU corresponds to multiple training samples for training the initial evaluation model included in the GPU; the number of training samples on different GPUs is the same, but the training samples are different; each GPU is used to receive a first model parameter obtained by each training round sent by an upstream GPU connected to it upstream; and send a second model parameter obtained by each training round to a downstream GPU connected to it downstream; wherein the method for training the evaluation model includes:
[0094] For each GPU, a first model parameter sent by an upstream GPU of the GPU is received, where the first model parameter is obtained by the upstream GPU based on training of the current evaluation model deployed thereon; the first model parameter is fused and calculated with the model parameter corresponding to the current evaluation model of the GPU to obtain fused model parameters, and the current evaluation model including the fused model parameters is further trained using the training samples corresponding to the GPU to obtain second model parameters obtained by training the current evaluation model, the current evaluation model of the GPU is updated based on the second model parameters, and the second model parameters are sent to the downstream GPUs of the GPU until the current evaluation model including the second model parameters on each GPU meets the first training cutoff condition, and the current evaluation model on any GPU is used as the evaluation model.
[0095] Among them, if the current training round is the first round of training, the first model parameters are the initial model parameters of the initial evaluation model in the upstream GPU; if the current training round is not the first round of training, the first model parameters are trained by the upstream GPU based on the current evaluation model deployed thereon; the region segmentation module and image classification module in the evaluation model are trained separately or simultaneously.
[0096] In an embodiment of the present application, the number of GPUs connected in a ring is at least three. Figure 4 The schematic diagram of the structure of the ring multi-GPU distributed training provided by the embodiment of the present application is shown as follows Figure 4 As shown, when the number of GPUs is 5, each GPU (including B1, B2, B3, B4, B5) is connected in a ring in sequence.
[0097] Each GPU corresponds to an upstream GPU and a downstream GPU of the GPU, for example, Figure 4 As shown, the upstream GPU of B1 is B5, and the downstream GPU of B1 is B2.
[0098] In a specific embodiment, when training the initial evaluation model on B1, upon receiving the first model parameters obtained through training sent by B5, B1 performs a fusion calculation on the first model parameters and the model parameters corresponding to its own current evaluation model to obtain fusion model parameters, and uses the training samples corresponding to B1 to perform current evaluation model training on the current evaluation model on B1 including the fusion model parameters. The fusion calculation may be a mean calculation of the first model parameters and the model parameters corresponding to its own current evaluation model.
[0099] In one possible implementation, the current evaluation model including the second model parameters on each GPU satisfies the first training cutoff condition including: the parameter index of the current evaluation model including the second model parameters on each GPU satisfies a preset threshold; the parameter index includes the first prediction accuracy and / or error rate of the current evaluation model; when the current evaluation model including the second model parameters on each GPU satisfies the first training cutoff condition, the second model parameters of the current evaluation model on each GPU are the same.
[0100] The parameter indicator satisfies the preset threshold value specifically when: the first prediction accuracy is greater than the first preset accuracy and / or the error rate is less than the preset error rate.
[0101] Among them, when the fusion model parameters corresponding to the current evaluation model on at least one GPU are different from the fusion model parameters corresponding to the current evaluation models on other GPUs, the current evaluation models on all GPUs continue to be trained until the fusion model parameters corresponding to the current evaluation models on each GPU are the same, and the training stops.
[0102] In this application, by adopting a circular multi-GPU distributed training method, the amount of calculation on each GPU can be reduced. No matter how many GPUs are set, each GPU only needs to calculate the model parameters on two GPUs; in the distributed training method of the prior art, it is necessary to calculate the model parameters of all GPUs through an intermediate GPU, resulting in a large amount of calculation on the intermediate GPU, which affects the model training speed. Therefore, through the method in this application, the amount of model calculation on each GPU can be reduced, thereby increasing the model training speed.
[0103] In a possible implementation, the training samples are constructed in the following manner:
[0104] S111: construct multiple initial training samples for training and evaluating models;
[0105] S112: arbitrarily selecting at least two initial training samples from the initial training samples as a group of samples to be fused.
[0106] S113: For each group of samples to be fused, perform image fusion processing on the initial training samples in the group of samples to be fused to obtain enhanced training samples corresponding to the group of samples to be fused.
[0107] S114: Determine all initial training samples and all enhanced training samples as training samples.
[0108] In step S113, for each group of samples to be fused, the initial training samples in the group of samples to be fused can be subjected to image fusion processing by using the Mixup data enhancement technology to obtain an enhanced training sample corresponding to the group of samples to be fused. For example, when the group of samples to be fused includes a picture of an apple and a picture of an orange, the picture of an apple and the picture of an orange are fused into a new enhanced training sample.
[0109] In step S114, all initial training samples and all obtained enhanced training samples are used as training samples, thereby increasing the number of training samples.
[0110] In a possible implementation, the evaluation model calculates the second image through an included image classification module to obtain a distribution uniformity result of the object to be evaluated; the image classification module is trained in the following manner:
[0111] S121: Constructing a first training sample for training an image classification module; the first training sample includes: a target image containing only a target object and a preset label of the target image; and at least one target object is distributed on the surface of the target food.
[0112] The preset labels include uniform distribution labels and non-uniform distribution labels.
[0113] S122: Using the first training sample to train the initial image classification module including the third model parameters to obtain an intermediate image classification module; the prediction accuracy of the image classification module is greater than the prediction accuracy of the intermediate image classification module.
[0114] In the present application, when the initial image classification module is trained, the initial image classification module including the third model parameters can be trained using the first training sample until the training time reaches a preset time, and the initial image classification module obtained at this time is used as the intermediate image classification module. The prediction accuracy of the image classification module is greater than the prediction accuracy of the intermediate image classification module, and the prediction accuracy of the initial image classification module is greater than the prediction accuracy of the initial image classification module.
[0115] S123: Input the target image into the intermediate image classification module, calculate the target image through the intermediate image classification module, and determine the predicted distribution uniformity result of the target object in the target image and the weight of the target image; the predicted distribution uniformity result includes the first probability that the distribution of the target object in the target image predicted by the intermediate image classification module is a uniform distribution and the second probability that the distribution of the target object is a non-uniform distribution.
[0116] When determining the weight of the target image through the intermediate image classification module, it can be specifically performed in the following way:
[0117] The key features of each target image are extracted through the CNN Backbone (convolutional layer backbone network) in the intermediate image classification module, where each target image corresponds to a key feature. For each target image, the key features of the target image are input into the fully connected layer and Sigmoid (activation function) of the intermediate image classification module to obtain the weight of the target image.
[0118] Exemplarily, the target image M is input into the intermediate image classification module to obtain the predicted distribution uniformity result of the target object m (including the first probability P1=0.7 and the second probability P2=0.3) and the weight of the target image M.
[0119] S124: Classifying the target images in the first training sample into first-category images and second-category images through an intermediate image classification module according to the weight of each target image; wherein the weight of the first-category images is greater than the weight of the second-category images.
[0120] According to the order of the weight of each target image from large to small, all target images are sorted, and according to the total number of target images, the weight of the target image and the preset ratio, all target images in the first training sample are divided into the first category of images and the second category of images. Exemplarily, when the total number of target images in the first training sample is 20 and the preset ratio is 75%, by 20×75%=15, the first 15 target images are sorted as the first category of images, and the remaining target images are sorted as the second category of images. At this time, the second category of images includes 5 target images.
[0121] Since the preset labels are marked by people, for the same object, some people think that the distribution of the object is uniform, while others think that the distribution of the object is non-uniform. Therefore, the preset labels may be incorrectly marked. For example, the distribution of the target object in a target image is actually uniform, but the preset labels of the target object are non-uniformly distributed, which makes the preset labels incorrectly marked. If some of the preset labels in the first training sample are incorrectly marked, the prediction accuracy of the image classification module obtained by training the first training sample will not be high enough. Therefore, the present application corrects the incorrectly marked preset labels through a self-healing mechanism.
[0122] For each second-category image, the intermediate image classification module is used to determine whether the preset label of the second-category image is incorrectly labeled; wherein the method for determining whether the preset label of the second-category image is incorrectly labeled includes:
[0123] S125: For each second-category image, when the first probability, the second probability and the preset label corresponding to the second-category image satisfy the following relationship, it is determined that the preset label of the second-category image is incorrectly labeled; wherein P1 is the first probability corresponding to the second-category image, P2 is the second probability corresponding to the second-category image, P is the probability value corresponding to the preset label of the second-category image, and k is a preset value.
[0124] Max(P1,P2)>P+k
[0125] Continuing with the example in step S123, when the second type of image includes the target image M, the first probability of the second type of image M is P1=0.7, the second probability is P2=0.3, and the preset label is non-uniformly distributed. At this time, Max(P1, P2)=0.7, P=P2=0.3. If the constant term K is preset to 0.2, the first probability, second probability and preset label corresponding to the second type of image M satisfy Max(P1, P2)=0.7>0.3+0.2, and it is judged that the preset label of the second type of image M is wrong.
[0126] Continuing with the example in step S123, when the second type of image includes the target image M, the first probability of the second type of image M is P1=0.7, the second probability is P2=0.3, and the preset label is uniformly distributed. At this time, Max(P1, P2)=0.7, P=P1=0.7. If the constant term K is preset to 0.2, the first probability, the second probability, and the preset label corresponding to the second type of image M do not satisfy Max(P1, P2)=0.7>0.7+0.2, and it is judged that the preset label of the second type of image M is correctly marked.
[0127] S126: When the preset label of the second category image is incorrectly marked, the preset label of the second category image is corrected to obtain the corrected label of the second category image, and the second category image and the corrected label are used as second training samples for training the intermediate image classification module.
[0128] Continuing with the example in step S125, when the preset label of the second type of image M is incorrectly marked, the preset label of the second type of image M is corrected, that is, the original preset label is changed from non-uniform distribution to uniform distribution. At this time, the corrected label of the second type of image M is uniformly distributed.
[0129] S127: Use the second training sample to train the intermediate image classification module until the intermediate image classification module meets the second training cutoff condition, and use the trained intermediate image classification module as the image classification module.
[0130] The second training cutoff condition includes one or more of a second prediction accuracy of the intermediate image classification module and a second training number.
[0131] In the present application, the intermediate image classification module is trained by modifying the second training sample with a label, thereby improving the prediction accuracy of the trained image classification module.
[0132] In a possible implementation, the region segmentation module is trained in the following manner:
[0133] S131: Construct a third training sample for training the region segmentation module; the third training sample includes: a sample image containing a target meal and a target image segmented from the sample image and including a target object contained in the target meal; at least one target object is distributed on the surface of the target meal.
[0134] In the embodiments of the present application, the sample image includes not only the target food but also other background information; the target image only includes the target object in the target food but does not include other background information.
[0135] When training the image classification module and the region segmentation module, the original images of the training samples used are the same. Specifically, the sample image is an original image of a picture, the target image is an image containing only the target object in the original image, and the preset label is a preset uniformity label of the target object on the target image.
[0136] When constructing the third training sample, there are multiple target dishes, such as pizza, spicy hot pot, etc. In each sample image, the sample image contains at least one target dish, the clarity of at least one target dish is greater than or equal to a preset clarity threshold, and the completeness of at least one target dish is greater than or equal to a completeness threshold.
[0137] S132: Input the sample image into the region segmentation module to be trained. The region segmentation module to be trained calculates the image features of the sample image, determines the predicted candidate region including the target food and part of the background information in the sample image, and segments the third image including the target object from the predicted candidate region.
[0138] The predicted candidate region is a candidate region predicted by the to-be-trained region segmentation module, and the third image is an image predicted by the to-be-trained region segmentation module to contain only the target object.
[0139] S133: Determine a first prediction result of the region segmentation module to be trained according to the third image corresponding to the sample image and the target image.
[0140] Since the third image is an image that only contains the target object predicted by the region segmentation module to be trained, the third image may contain not only the image of the target object but also other image information. The first prediction result of the region segmentation module to be trained is determined by comparing the third image with the target image.
[0141] S134: using the first prediction result to train the region segmentation module to be trained including the fourth model parameter until the region segmentation module to be trained meets the third training cutoff condition, and using the trained region segmentation module to be trained as the region segmentation module.
[0142] The third training cutoff condition includes one or more of a third prediction accuracy and a third training number of the region segmentation module to be trained.
[0143] In a possible implementation, the evaluation model obtains the second image by including a region segmentation module; and the evaluation model is deployed to the server in the following manner:
[0144] S141: Convert the formats of the trained region segmentation module and the image classification module into an open neural network exchange format.
[0145] After the training of the region segmentation module and the image classification module is completed, the formats of the trained region segmentation module and the image classification module are converted into ONNX format (open neural network exchange format).
[0146] S142: Convert the region segmentation module and the image classification module in the open neural network exchange format into an accelerated format.
[0147] The acceleration format is specifically the TensorRT format. The preset speed of the model in the acceleration format is greater than the predicted speed of the model in other formats.
[0148] S143: respectively testing the prediction speeds of the region segmentation module and the image classification module in the accelerated format; when the prediction speed of the region segmentation module is greater than a first preset speed and the prediction speed of the image classification module is greater than a second preset speed, encapsulating the region segmentation module and the image classification module to obtain an evaluation model, and deploying the evaluation model to the server.
[0149] The first test sample is used to test the region segmentation module in the accelerated format, and the second test sample is used to test the prediction speed of the image classification module.
[0150] In a possible implementation manner, obtaining a first image containing a meal to be evaluated includes:
[0151] Receive multiple first images of meals to be evaluated sent by a photographing device; each meal to be evaluated corresponds to a first image less than a target number, and the first image is a picture that meets preset filtering conditions and is selected by the photographing device from multiple candidate images of the meal to be evaluated; the preset filtering conditions include: containing at least one meal to be evaluated, the clarity of at least one meal to be evaluated is greater than or equal to a preset clarity threshold, and the completeness of at least one meal to be evaluated is greater than or equal to a completeness threshold.
[0152] When the photographing device photographs the food to be evaluated and obtains the first image, multiple first images may be obtained for the same food to be evaluated. In order to avoid the server from repeatedly evaluating the same food to be evaluated and reduce the data processing workload of the server, in the present application, after the photographing device obtains the first image of the food to be evaluated, the photographing device filters the first image of each food to be evaluated according to preset filtering conditions, and for each food to be evaluated, the filtered first images that are less than the target number are sent to the server for food quality evaluation.
[0153] In a possible implementation manner, the first image corresponding to each meal to be evaluated is selected by the photographing device in the following manner:
[0154] S151: Selecting a first image to be evaluated that includes at least one meal to be evaluated from a plurality of candidate images according to the items included in each candidate image.
[0155] When the shooting device is shooting, it can be shot at a preset time interval. Therefore, the candidate image captured may contain the food to be evaluated, or may not contain the food to be evaluated. In the present application, based on the items contained in each candidate image, a first image to be evaluated that contains at least one food to be evaluated is selected from multiple candidate images.
[0156] S152: According to the clarity of the food to be evaluated contained in each first image to be evaluated, selecting from the first images to be evaluated at least one second image to be evaluated whose clarity of the food to be evaluated is greater than or equal to a preset clarity threshold.
[0157] The first image to be evaluated may include one or more meals to be evaluated. In the present application, based on the clarity of the meals to be evaluated contained in each first image to be evaluated, a second image to be evaluated is selected from the first images to be evaluated, the clarity of at least one meal to be evaluated being greater than or equal to a preset clarity threshold.
[0158] S153: According to the completeness of the food to be evaluated contained in each second image to be evaluated, selecting at least one first image whose completeness of the food to be evaluated is greater than or equal to a completeness threshold from the second images to be evaluated.
[0159] The completeness of the meal to be evaluated refers to the completeness of the meal to be evaluated contained in the second image to be evaluated. For example, some second images to be evaluated contain half of the meal to be evaluated, and some second images to be evaluated contain the entire image of the meal to be evaluated.
[0160] Embodiment 2:
[0161] Based on the same technical concept, the embodiment of the present application also provides a food quality evaluation device, which resides in the server. Figure 5 FIG. 1 shows a schematic diagram of the structure of a food quality assessment device provided in an embodiment of the present application. Figure 5 As shown, the device comprises:
[0162] An acquisition module 501 is used to acquire a first image containing a food to be evaluated;
[0163] An input module 502 is used to input the first image into a pre-trained evaluation model, determine a candidate region including the food to be evaluated and part of the background information in the first image through the evaluation model, and segment a second image including the object to be evaluated from the candidate region; wherein at least one of the objects to be evaluated is distributed on the surface of the food to be evaluated;
[0164] A calculation module 503 is used to calculate the second image by using the evaluation model to obtain a distribution uniformity result of the object to be evaluated;
[0165] The first determination module 504 is used to determine the quality of the food to be evaluated according to the distribution uniformity result of the object to be evaluated in the food to be evaluated.
[0166] Optionally, the evaluation model is obtained by training multiple GPUs connected in a ring; the same initial evaluation model is deployed on each GPU, and the training samples on different GPUs are different; and the method further includes:
[0167] The first training module, when used to train the evaluation model, is specifically used to receive, for each GPU, a first model parameter sent by an upstream GPU of the GPU, where the first model parameter is obtained by the upstream GPU based on the training of the current evaluation model deployed thereon; the first model parameter is fused and calculated with the model parameters corresponding to its own current evaluation model to obtain fused model parameters, and the current evaluation model including the fused model parameters is further trained using the training samples corresponding to the GPU to obtain second model parameters obtained by the current evaluation model training; the current evaluation model of the GPU is updated based on the second model parameters, and the second model parameters are sent to the downstream GPU of the GPU until the current evaluation model including the second model parameters on each GPU meets the first training cutoff condition, and the current evaluation model on any GPU is used as the evaluation model.
[0168] Optionally, also include:
[0169] A construction module, used to construct a plurality of initial training samples for training the evaluation model;
[0170] A selection module, used for arbitrarily selecting at least two of the initial training samples as a group of samples to be fused;
[0171] A fusion module is used to perform image fusion processing on the initial training samples in each group of samples to be fused, so as to obtain enhanced training samples corresponding to the group of samples to be fused;
[0172] The second determining module is used to determine all the initial training samples and all the enhanced training samples as the training samples.
[0173] Optionally, the evaluation model calculates the second image through an included image classification module to obtain a distribution uniformity result of the object to be evaluated; and further includes:
[0174] The second training module, when used to train the image classification module, is specifically used to:
[0175] Constructing a first training sample for training the image classification module; the first training sample includes: a target image containing only a target object and a preset label of the target image; at least one of the target objects is distributed on the surface of the target food;
[0176] Using the first training sample to train an initial image classification module including a third model parameter to obtain an intermediate image classification module; the prediction accuracy of the image classification module is greater than the prediction accuracy of the intermediate image classification module;
[0177] The target image is input into the intermediate image classification module, and the target image is calculated by the intermediate image classification module to determine the predicted distribution uniformity result of the target object in the target image and the weight of the target image; the predicted distribution uniformity result includes a first probability that the distribution of the target object in the target image predicted by the intermediate image classification module is a uniform distribution and a second probability that the distribution of the target object is a non-uniform distribution;
[0178] According to the weight of each of the target images, the target images in the first training sample are divided into a first category of images and a second category of images by the intermediate image classification module; wherein the weight of the first category of images is greater than the weight of the second category of images;
[0179] For each second-category image, when the first probability, the second probability, and the preset label corresponding to the second-category image satisfy the following relationship, it is determined that the preset label of the second-category image is incorrectly labeled; wherein P1 is the first probability corresponding to the second-category image, P2 is the second probability corresponding to the second-category image, P is the value of the probability corresponding to the preset label of the second-category image, and k is a preset value;
[0180] Max(P1,P2)>P+k
[0181] When the preset label of the second category image is incorrectly marked, the preset label of the second category image is corrected to obtain a corrected label of the second category image, and the second category image and the corrected label are used as second training samples for training the intermediate image classification module;
[0182] The intermediate image classification module is trained using the second training sample until the intermediate image classification module meets a second training cutoff condition, and the trained intermediate image classification module is used as the image classification module.
[0183] Optionally, the evaluation model obtains the second image by means of a region segmentation module; and further comprises:
[0184] A first conversion module, used for converting the formats of the trained region segmentation module and the image classification module into an open neural network exchange format;
[0185] A second conversion module, used to convert the region segmentation module and the image classification module in the open neural network exchange format into an accelerated format;
[0186] A test module is used to test the prediction speeds of the region segmentation module and the image classification module under the acceleration format respectively. When the prediction speed of the region segmentation module is greater than a first preset speed, and the prediction speed of the image classification module is greater than a second preset speed, the region segmentation module and the image classification module are encapsulated to obtain the evaluation model, and the evaluation model is deployed to the server.
[0187] Optionally, when the acquisition module 501 is used to acquire the first image containing the food to be evaluated, it is specifically used to:
[0188] Receive the first images of the multiple dishes to be evaluated sent by the photographing device; each dish to be evaluated corresponds to a first image less than the target number, and the first image is a picture that meets the preset filtering conditions selected by the photographing device from the multiple candidate images of the dishes to be evaluated; the preset filtering conditions include: containing at least one of the dishes to be evaluated, the clarity of at least one of the dishes to be evaluated is greater than or equal to a preset clarity threshold, and the completeness of at least one of the dishes to be evaluated is greater than or equal to a completeness threshold.
[0189] For the specific implementation method steps and principles, please refer to the description of Example 1, which will not be repeated here in detail.
[0190] Embodiment three:
[0191] Based on the same technical concept, the embodiment of the present application also provides an electronic device, Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. Figure 6 As shown, the electronic device 600 includes: a processor 601, a memory 602 and a bus 603. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor 601 communicates with the memory 602 through the bus 603, and the processor 601 executes the machine-readable instructions to perform the method steps described in Example 1.
[0192] For the specific implementation method steps and principles, please refer to the description of Example 1, which will not be repeated here in detail.
[0193] Embodiment 4:
[0194] Based on the same technical concept, embodiment 4 of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method steps described in embodiment 1 are executed.
[0195] For the specific implementation method steps and principles, please refer to the description of Example 1, which will not be repeated here in detail.
[0196] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0197] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0198] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0199] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0200] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0201] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A method for evaluating food quality, characterized in that: The method is applied to a server, and the method comprises: Acquire a first image containing a food to be evaluated; Input the first image into a pre-trained evaluation model, determine a candidate region including the food to be evaluated and part of the background information in the first image through the evaluation model, and segment a second image including the object to be evaluated from the candidate region; wherein at least one of the objects to be evaluated is distributed on the surface of the food to be evaluated; Calculating the second image by using the evaluation model to obtain a distribution uniformity result of the object to be evaluated; Determining the quality of the food to be evaluated according to the distribution uniformity result of the object to be evaluated in the food to be evaluated; The evaluation model calculates the second image through the included image classification module to obtain the distribution uniformity result of the object to be evaluated; the image classification module is trained in the following way: Constructing a first training sample for training the image classification module; the first training sample includes: a target image containing only a target object and a preset label of the target image; at least one of the target objects is distributed on the surface of the target food; Using the first training sample to train an initial image classification module including a third model parameter to obtain an intermediate image classification module; the prediction accuracy of the image classification module is greater than the prediction accuracy of the intermediate image classification module; The target image is input into the intermediate image classification module, and the target image is calculated by the intermediate image classification module to determine the predicted distribution uniformity result of the target object in the target image and the weight of the target image; the predicted distribution uniformity result includes a first probability that the distribution of the target object in the target image predicted by the intermediate image classification module is a uniform distribution and a second probability that the distribution of the target object is a non-uniform distribution; According to the weight of each of the target images, the target images in the first training sample are divided into a first category of images and a second category of images by the intermediate image classification module; wherein the weight of the first category of images is greater than the weight of the second category of images; For each second-category image, when the first probability, the second probability, and the preset label corresponding to the second-category image satisfy the following relationship, it is determined that the preset label of the second-category image is incorrectly labeled; wherein P1 is the first probability corresponding to the second-category image, P2 is the second probability corresponding to the second-category image, P is the value of the probability corresponding to the preset label of the second-category image, and k is a preset value; Max(P1,P2)>P+k When the preset label of the second category image is incorrectly marked, the preset label of the second category image is corrected to obtain a corrected label of the second category image, and the second category image and the corrected label are used as second training samples for training the intermediate image classification module; The intermediate image classification module is trained using the second training sample until the intermediate image classification module meets a second training cutoff condition, and the trained intermediate image classification module is used as the image classification module.
2. The method according to claim 1, characterized in that: The evaluation model is obtained by training a plurality of GPUs connected in a ring; the same initial evaluation model is deployed on each GPU, and the training samples on different GPUs are different; wherein the method for training the evaluation model includes: For each GPU, a first model parameter sent by an upstream GPU of the GPU is received, where the first model parameter is obtained by the upstream GPU based on training of the current evaluation model deployed thereon; the first model parameter is fused and calculated with the model parameters corresponding to the current evaluation model of the GPU to obtain fused model parameters, and the current evaluation model including the fused model parameters is further trained using the training samples corresponding to the GPU to obtain second model parameters obtained by training the current evaluation model, the current evaluation model of the GPU is updated based on the second model parameters, and the second model parameters are sent to the downstream GPU of the GPU until the current evaluation model including the second model parameters on each GPU meets the first training cutoff condition, and the current evaluation model on any GPU is used as the evaluation model.
3. The method according to claim 2, characterized in that: The current evaluation model including the second model parameters on each GPU satisfies the first training cutoff condition including: the parameter index of the current evaluation model including the second model parameters on each GPU satisfies the preset threshold; the parameter index includes the first prediction accuracy and / or error rate of the current evaluation model; when the current evaluation model including the second model parameters on each GPU satisfies the first training cutoff condition, the second model parameters of the current evaluation model on each GPU are the same.
4. The method according to claim 2, characterized in that: The training samples are constructed in the following way: Constructing a plurality of initial training samples for training the evaluation model; Randomly selecting at least two of the initial training samples as a group of samples to be fused; For each group of samples to be fused, the initial training samples in the group of samples to be fused are subjected to image fusion processing to obtain enhanced training samples corresponding to the group of samples to be fused; All the initial training samples and all the enhanced training samples are determined as the training samples.
5. The method according to claim 1, characterized in that: The evaluation model obtains the second image by means of a region segmentation module; and the evaluation model is deployed to the server in the following manner: Converting the trained formats of the region segmentation module and the image classification module into an open neural network exchange format; Converting the region segmentation module and the image classification module in the open neural network exchange format into an accelerated format; The prediction speeds of the region segmentation module and the image classification module under the acceleration format are tested respectively. When the prediction speed of the region segmentation module is greater than a first preset speed, and the prediction speed of the image classification module is greater than a second preset speed, the region segmentation module and the image classification module are encapsulated to obtain the evaluation model, and the evaluation model is deployed to the server.
6. The method according to claim 1, characterized in that: The acquisition of the first image containing the meal to be evaluated includes: receiving a plurality of the first images of the meal to be evaluated sent by a photographing device; each meal to be evaluated corresponds to a first image less than a target number, and the first image is a picture that meets preset filtering conditions and is selected by the photographing device from a plurality of candidate images of the meal to be evaluated; the preset filtering conditions include: containing at least one of the meal to be evaluated, the clarity of at least one of the meal to be evaluated is greater than or equal to a preset clarity threshold, and the completeness of at least one of the meal to be evaluated is greater than or equal to a completeness threshold.
7. A food quality assessment device, characterized in that: Residing on the server, the device includes: An acquisition module, used for acquiring a first image containing a meal to be evaluated; An input module, used to input the first image into a pre-trained evaluation model, determine a candidate region including the food to be evaluated and part of the background information in the first image through the evaluation model, and segment a second image including the object to be evaluated from the candidate region; wherein at least one of the objects to be evaluated is distributed on the surface of the food to be evaluated; A calculation module, used to calculate the second image by using the evaluation model to obtain a distribution uniformity result of the object to be evaluated; A first determination module, configured to determine the quality of the food to be evaluated according to the distribution uniformity result of the object to be evaluated in the food to be evaluated; The evaluation model calculates the second image through the included image classification module to obtain the distribution uniformity result of the object to be evaluated; the device also includes a second training module; The second training module, when used to train the image classification module, is specifically used to: Constructing a first training sample for training the image classification module; the first training sample includes: the target image containing only the target object and a preset label of the target image; at least one of the target objects is distributed on the surface of the target food; Using the first training sample to train an initial image classification module including a third model parameter to obtain an intermediate image classification module; the prediction accuracy of the image classification module is greater than the prediction accuracy of the intermediate image classification module; The target image is input into the intermediate image classification module, and the target image is calculated by the intermediate image classification module to determine the predicted distribution uniformity result of the target object in the target image and the weight of the target image; the predicted distribution uniformity result includes a first probability that the distribution of the target object in the target image predicted by the intermediate image classification module is a uniform distribution and a second probability that the distribution of the target object is a non-uniform distribution; According to the weight of each of the target images, the target images in the first training sample are divided into a first category of images and a second category of images by the intermediate image classification module; wherein the weight of the first category of images is greater than the weight of the second category of images; For each second-category image, when the first probability, the second probability, and the preset label corresponding to the second-category image satisfy the following relationship, it is determined that the preset label of the second-category image is incorrectly labeled; wherein P1 is the first probability corresponding to the second-category image, P2 is the second probability corresponding to the second-category image, P is the value of the probability corresponding to the preset label of the second-category image, and k is a preset value; Max(P1,P2)>P+k When the preset label of the second category image is incorrectly marked, the preset label of the second category image is corrected to obtain a corrected label of the second category image, and the second category image and the corrected label are used as second training samples for training the intermediate image classification module; The intermediate image classification module is trained using the second training sample until the intermediate image classification module meets a second training cutoff condition, and the trained intermediate image classification module is used as the image classification module.
8. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the method as described in any one of claims 1 to 6 are performed.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are executed.
