Image processing method and apparatus, and electronic device

Abstract

The present disclosure discloses an image processing method and device and electronic equipment, the method comprises: a user terminal judges whether the pre-meal image input by a user meets a first preset condition, wherein the pre-meal image is an image formed after the user shoots tableware before starting eating; when the pre-meal image meets the first preset condition, the user terminal judges whether the post-meal image input by the user meets a second preset condition, wherein the post-meal image is an image formed after the user shoots the tableware after finishing eating; when the post-meal image meets the second preset condition, the user terminal sends prompt information of completing the food light plate action to the user, which can mobilize the subjective initiative of the public to practice the light plate action.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically to an image processing method, apparatus, and electronic device. Background Technology

[0002] The "Clean Plate Campaign" aims to cultivate the habit of saving food and oppose food waste, creating an atmosphere throughout society where waste is shameful and thrift is honorable. Traditional promotion of the "Clean Plate Campaign" can be done by posting public service slogans in public places or through online videos or text messages. However, these methods can only forcibly impose the spirit of "saving food" on the public, but cannot mobilize their subjective initiative to practice the campaign. Summary of the Invention

[0003] In view of this, the present disclosure provides an image processing method, apparatus, and electronic device that can mobilize the public's subjective enthusiasm for practicing the "Clean Plate Campaign".

[0004] In a first aspect, an image processing method is provided, comprising: a user terminal determining whether a pre-meal image input by a user meets a first preset condition, wherein the pre-meal image is an image formed by the user taking a picture of the tableware before starting the meal; when the pre-meal image meets the first preset condition, the user terminal determining whether a post-meal image input by the user meets a second preset condition, wherein the post-meal image is an image formed by the user taking a picture of the tableware after finishing the meal; when the post-meal image meets the second preset condition, the user terminal sending a prompt message to the user indicating completion of the "clean plate" campaign.

[0005] In one embodiment, the user terminal determines whether the pre-meal image input by the user meets the first preset condition, including: the user terminal uses a neural network model to identify the food in the tableware of the pre-meal image and obtains a first probability value that there is food in the tableware of the pre-meal image; when the first probability value is greater than or equal to a first preset probability threshold, the user terminal determines that the pre-meal image meets the first preset condition, wherein the first preset probability threshold is used to characterize the probability threshold that there is food in the tableware of the pre-meal image.

[0006] In one embodiment, the user terminal determines whether the post-meal image input by the user meets a second preset condition, including: the user terminal using a neural network model to identify food in the tableware of the post-meal image and obtain a second probability value that there is no food in the tableware of the post-meal image; the user terminal determines whether the tableware of the pre-meal image and the tableware of the post-meal image are the same tableware; when the second probability value is greater than or equal to a second preset probability threshold and the tableware of the pre-meal image and the tableware of the post-meal image are the same tableware, the user terminal determines that the post-meal image meets the second preset condition, wherein the second preset probability threshold is used to characterize the probability threshold that there is no food in the tableware of the post-meal image.

[0007] In one embodiment, the user terminal determines whether the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware. This includes: the user terminal using a neural network model to identify the appearance of the tableware in the pre-meal image and the tableware in the post-meal image, respectively, to obtain at least one first confidence score vector corresponding to the tableware appearance in the pre-meal image and at least one second confidence score vector corresponding to the tableware appearance in the post-meal image. Each tableware appearance corresponds to one first confidence score vector and one second confidence score vector. Each first confidence score vector consists of multiple first confidence scores, which are confidence scores for different preset tableware appearances in the pre-meal image. Each second confidence score vector consists of multiple second confidence scores. The system comprises multiple second confidence scores, which are confidence scores when the appearance of the tableware in the post-meal image is different from the preset tableware appearance. The user terminal determines the similarity between at least one first confidence score vector and at least one second confidence score vector. The user terminal judges whether the similarity between at least one first confidence score vector and at least one second confidence score vector meets a preset similarity threshold, wherein the preset similarity threshold is used to characterize the similarity threshold between the appearance of the tableware in the pre-meal image and the appearance of the tableware in the post-meal image. When the similarity between at least one first confidence score vector and at least one second confidence score vector meets the preset similarity threshold, the user terminal determines that the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware.

[0008] In one embodiment, when the appearance of the tableware includes color, material, and shape, at least one first confidence score vector includes a first confidence score vector corresponding to color, a first confidence score vector corresponding to material, and a first confidence score vector corresponding to shape, and at least one second confidence score vector includes a second confidence score vector corresponding to color, a second confidence score vector corresponding to material, and a second confidence score vector corresponding to shape.

[0009] In one embodiment, the user terminal determines the similarity between at least one first confidence score vector and at least one second confidence score vector, including: the user terminal using a neural network model to determine a first distance metric between the first confidence score vector corresponding to color and the second confidence score vector corresponding to color as the similarity between the two vectors; the user terminal using a neural network model to determine a second distance metric between the first confidence score vector corresponding to material and the second confidence score vector corresponding to material as the similarity between the two vectors; and the user terminal using a neural network model to determine a third distance metric between the first confidence score vector corresponding to shape and the second confidence score vector corresponding to shape as the similarity between the two vectors.

[0010] In one embodiment, the user terminal determines whether the similarity between at least one first confidence score vector and at least one second confidence score vector meets a preset similarity threshold, including: the user terminal performs a weighted summation of a first distance metric, a second distance metric, and a third distance metric to obtain a similarity score between the appearance of tableware in the pre-meal image and the appearance of tableware in the post-meal image; when the similarity score is greater than or equal to the preset similarity threshold, the user terminal determines that the similarity between at least one first confidence score vector and at least one second confidence score vector meets the preset similarity threshold.

[0011] In one embodiment, when the pre-meal image meets the first preset condition, the method further includes: the user terminal sending a prompt message to the server indicating that the pre-meal image meets the first preset condition, so that the server can perform risk detection on the pre-meal image, wherein the risk detection includes compliance risk detection and / or security risk detection.

[0012] In one embodiment, the user terminal determines whether the post-meal image input by the user meets the second preset condition, including: when the user terminal receives a prompt message from the server indicating that the risk detection of the pre-meal image has passed, the user terminal determines whether the post-meal image meets the second preset condition.

[0013] In one embodiment, when the post-meal image meets the second preset condition, the method further includes: the user terminal sending a prompt message to the server indicating that the post-meal image meets the second preset condition, so that the server can perform risk detection on the post-meal image, wherein the risk detection includes compliance risk detection and / or security risk detection.

[0014] In one embodiment, the user terminal sends a prompt message to the user indicating that the "Clean Plate Campaign" has been completed, including: when the user terminal receives a prompt message from the server indicating that the risk detection of the post-meal image has passed, the user terminal sends a prompt message to the user indicating that the "Clean Plate Campaign" has been completed.

[0015] In one embodiment, the method further includes: the user terminal performing image detection on the pre-meal image, wherein the image detection includes image angle detection for determining whether the shooting angle of the pre-meal image is tilted and / or image quality detection for determining whether the shooting quality of the pre-meal image is intact.

[0016] In one embodiment, the user terminal determines whether the pre-meal image input by the user meets the first preset condition, including: when the imaging detection of the pre-meal image passes, the user terminal determines whether the pre-meal image meets the first preset condition.

[0017] In one embodiment, the method further includes: the user terminal performing image detection on the post-meal image, wherein the image detection includes image angle detection for determining whether the shooting angle of the post-meal image is tilted and / or image quality detection for determining whether the shooting quality of the post-meal image is intact.

[0018] In one embodiment, the user terminal determines whether the post-meal image input by the user meets the second preset condition, including: when the imaging detection of the post-meal image passes, the user terminal determines whether the post-meal image meets the second preset condition.

[0019] In one embodiment, when the pre-meal image does not meet the first preset condition, the method further includes: the user terminal sending a request to the user to re-upload the pre-meal image.

[0020] In one embodiment, when the post-meal image does not meet the second preset condition, the method further includes: the user terminal sending a request to the user to re-upload the post-meal image.

[0021] Secondly, an image processing method is provided, comprising: a server receiving a prompt message from a user terminal indicating that a pre-meal image input by the user meets a first preset condition, wherein the pre-meal image is an image formed after the user takes a picture of the tableware before starting the meal; the server performing risk detection on the pre-meal image, wherein the risk detection includes compliance risk detection and / or security risk detection; the server sending a prompt message to the user terminal indicating that the risk detection of the pre-meal image has passed, so that the user terminal can determine whether a post-meal image input by the user meets a second preset condition, wherein the post-meal image is an image formed after the user takes a picture of the tableware after finishing the meal; the server receiving a prompt message from the user terminal indicating that the post-meal image meets the second preset condition; the server performing risk detection on the post-meal image; and the server sending a prompt message to the user terminal indicating that the risk detection of the post-meal image has passed, so that the user terminal, upon determining that the post-meal image meets the second preset condition, sends a prompt message to the user indicating completion of the "clean plate" campaign.

[0022] Thirdly, an image processing device is provided, comprising: a first judgment module configured to allow a user terminal to judge whether a pre-meal image input by a user meets a first preset condition, wherein the pre-meal image is an image formed after the user takes a picture of the tableware before starting the meal; a second judgment module configured to allow the user terminal to judge whether a post-meal image input by the user meets a second preset condition when the pre-meal image meets the first preset condition, wherein the post-meal image is an image formed after the user takes a picture of the tableware after finishing the meal; and a push module configured to allow the user terminal to send a prompt message to the user indicating completion of the "clean plate" campaign when the post-meal image meets the second preset condition.

[0023] In one embodiment, the first judgment module is further configured as follows: the user terminal uses a neural network model to identify the food in the tableware of the pre-meal image and obtains a first probability value that there is food in the tableware of the pre-meal image; when the first probability value is greater than or equal to a first preset probability threshold, the user terminal determines that the pre-meal image meets the first preset condition, wherein the first preset probability threshold is used to characterize the probability threshold that there is food in the tableware of the pre-meal image.

[0024] In one embodiment, the second judgment module is further configured as follows: the user terminal uses a neural network model to identify the food in the tableware of the post-meal image and obtains a second probability value that there is no food in the tableware of the post-meal image; the user terminal judges whether the tableware of the pre-meal image and the tableware of the post-meal image are the same tableware; when the second probability value is greater than or equal to a second preset probability threshold and the tableware of the pre-meal image and the tableware of the post-meal image are the same tableware, the user terminal determines that the post-meal image meets the second preset condition, wherein the second preset probability threshold is used to characterize the probability threshold that there is no food in the tableware of the post-meal image.

[0025] Fourthly, an image processing apparatus is provided, comprising: a first receiving module configured to receive, via a user terminal, a prompt message indicating that a pre-meal image input by a user meets a first preset condition, wherein the pre-meal image is an image formed by the user taking a picture of the tableware before starting the meal; a first risk detection module configured to perform risk detection on the pre-meal image, wherein the risk detection includes compliance risk detection and / or safety risk detection; a first sending module configured to send, via a user terminal, a prompt message indicating that the risk detection of the pre-meal image has passed, so that the user terminal can determine whether a post-meal image input by the user meets a second preset condition, wherein the post-meal image is an image formed by the user taking a picture of the tableware after finishing the meal; a second receiving module configured to receive, via a user terminal, a prompt message indicating that the post-meal image meets the second preset condition; a second risk detection module configured to perform risk detection on the post-meal image; and a second sending module configured to send, via a user terminal, a prompt message indicating that the risk detection of the post-meal image has passed, so that the user terminal can send a prompt message to the user indicating completion of the "clean plate" campaign when it determines that the post-meal image meets the second preset condition.

[0026] Fifthly, an electronic device is provided, including a memory and a processor, wherein executable code is stored in the memory and the processor is configured to execute the executable code to implement the method as described in the first or second aspect.

[0027] In a sixth aspect, a computer-readable storage medium is provided having executable code stored thereon, which, when executed, enables the implementation of the methods of the first or second aspect.

[0028] In a seventh aspect, a computer program product is provided, including executable code, which, when executed, enables the implementation of the methods of the first or second aspect.

[0029] This disclosure provides an image processing solution. By judging a pre-meal image taken by a user before the start of a meal based on a first preset condition, it can determine whether there is food in the tableware in the pre-meal image. Only if food is determined to be present in the tableware in the pre-meal image, does it then judge a post-meal image taken by the user after the meal based on a second preset condition. The second preset condition not only determines whether there is no food in the tableware, but also whether the tableware in the pre-meal image and the tableware in the post-meal image are the same. Only if it is determined that there is no food in the tableware in the post-meal image and that the tableware in the pre-meal image and the post-meal image are the same, does it send a prompt message to the user indicating that the "Clean Plate Campaign" has been completed. The entire image processing process is interconnected, gradually guiding users to participate in the "Clean Plate Campaign," thereby mobilizing the public's subjective enthusiasm for practicing the "Clean Plate Campaign." Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the system architecture for an application scenario of the image processing method provided in this embodiment.

[0031] Figure 2 This is a schematic flowchart of the image processing method provided in the embodiments of this disclosure.

[0032] Figure 3 This is a schematic diagram of an image processing process provided in an embodiment of the present disclosure.

[0033] Figure 4 This is a schematic flowchart of an image processing method provided in another embodiment of this disclosure.

[0034] Figure 5 This is a schematic flowchart of an image processing method provided in another embodiment of this disclosure.

[0035] Figure 6 This is a schematic flowchart of an image processing method provided in another embodiment of this disclosure.

[0036] Figure 7 This is a schematic flowchart of an image processing method provided in another embodiment of this disclosure.

[0037] Figure 8 This is a schematic flowchart of an image processing method provided in an embodiment of the present disclosure.

[0038] Figure 9 This is a schematic structural diagram of an image processing apparatus provided in one embodiment of this application.

[0039] Figure 10 This is a schematic structural diagram of an image processing apparatus provided in one embodiment of this application.

[0040] Figure 11 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0041] The technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments.

[0042] The "Clean Plate Campaign" is a public service initiative advocating for food conservation and opposing waste. Its main purpose is to encourage the public to cherish food, finish every grain of rice on their plates, and cultivate a habit of thrift and anti-waste in daily life. Traditional methods of promoting the "Clean Plate Campaign" include displaying public service slogans in public places, such as "Every grain of rice on your plate is the result of hard work." Traditional promotion of the "Clean Plate Campaign" can also be achieved through online videos or articles, such as broadcasting relevant programs on food conservation online or sharing related articles online.

[0043] However, regardless of the advocacy method, it can only forcibly impose the spirit of "saving food" and other similar initiatives on the public. Some people may see it as news that is irrelevant to them and will not subjectively practice the "Clean Plate Campaign".

[0044] Therefore, in order to fundamentally encourage people to save food and promote traditional virtues, it is necessary to mobilize the public's subjective enthusiasm for practicing the "Clean Plate Campaign".

[0045] To facilitate understanding, a brief introduction to the neural networks and some of the concepts involved in the embodiments of this disclosure will be given first.

[0046] A neural network is a computational model composed of numerous interconnected nodes (or neurons). Each node corresponds to a policy function, and the connection between any two nodes represents a weighted value for the signal passing through that connection. A neural network typically consists of multiple layers, cascaded together. The output of the i-th layer is connected to the input of the (i+1)-th layer, the output of the (i+1)-th layer is connected to the input of the (i+2)-th layer, and so on. Training samples are input into the cascaded neural network layers, and each layer outputs a result, which becomes the input of the next layer. This process continues, with multiple layers calculating the output. The predicted results of the output layers are compared with the actual target values. Based on the difference between the predicted and target values, the weight matrix and policy function of each layer are adjusted. The neural network continuously undergoes this adjustment process using training samples, refining the weights and other parameters until the predicted output matches the actual target value. This process is called the training process of the neural network. After training, a neural network model is obtained.

[0047] In deep learning, distance metrics are used to assess the similarity between data. These include Euclidean distance, Manhattan distance, cosine distance, Chebyshev distance, Hamming distance, Minkowski distance, Mahalanobis distance, and so on. For example, cosine distance measures the angle between spatial vectors to determine the similarity between two vectors.

[0048] The following is combined with Figure 1 This disclosure provides a detailed description of the system architecture for application scenarios of the image processing methods mentioned in the embodiments. For example... Figure 1 As shown in the embodiments of this disclosure, the application scenario involves a server 140 and multiple user terminals 110, 120, and 130. The user terminals 110, 120, and 130 are equipped with cameras for taking pictures of the tableware before and after the user begins and finishes eating.

[0049] User terminals 110, 120, and 130 can be mobile terminal devices such as mobile phones, game consoles, tablets, and in-vehicle computers; alternatively, user terminals 110, 120, and 130 can also be personal computers (PCs), such as laptops and desktop computers. Those skilled in the art will understand that the types of the aforementioned user terminals 110, 120, and 130 can be the same or different, and their number can be more or less. For example, there can be one of each of the aforementioned user terminals, or there can be dozens or hundreds of user terminals, or even more. This application does not limit the number or type of user terminals in its embodiments.

[0050] In one embodiment, user terminals 110, 120, and 130 may have an application for the "Clean Plate Campaign" installed. The user opens the application and uses the camera on user terminals 110, 120, and 130 to take a picture of the tableware before the meal begins, uploading the pre-meal image to the application on user terminals 110, 120, and 130. User terminals 110, 120, and 130 determine whether the pre-meal image meets a first preset condition. If the pre-meal image meets the first preset condition, the user reopens the application and uses the camera on user terminals 110, 120, and 130 to take a picture of the tableware after the meal, uploading the post-meal image to the application on user terminals 110, 120, and 130. User terminals 110, 120, and 130 determine whether the post-meal image meets a second preset condition. If the post-meal image meets the second preset condition, user terminals 110, 120, and 130 send a notification message to the user that the "Clean Plate Campaign" has been completed through their application.

[0051] In addition, user terminals 110, 120, and 130 can also be connected to server 140 via a communication network. Optionally, the communication network can be a wired network or a wireless network. Optionally, server 140 can be a single server, a combination of several servers, or a cloud computing service center.

[0052] In one embodiment, user terminals 110, 120, and 130 determine that the pre-meal image input by the user meets a first preset condition. Server 140 receives the prompt message from user terminals 110, 120, and 130 indicating that the pre-meal image input by the user meets the first preset condition, and performs risk detection on the pre-meal image. If the risk detection of the pre-meal image passes, server 140 sends a prompt message indicating that the risk detection of the pre-meal image has passed to user terminals 110, 120, and 130. Server 140 receives the prompt message from user terminals 110, 120, and 130 indicating that the post-meal image meets a second preset condition, and performs risk detection on the post-meal image. If the risk detection of the post-meal image passes, server 140 sends a prompt message indicating that the risk detection of the post-meal image has passed to user terminals 110, 120, and 130. When user terminals 110, 120, and 130 determine that the post-meal image meets the second preset condition, they send a prompt message to the user indicating that the "clean plate campaign" has been completed.

[0053] This disclosure provides an image processing solution. By judging a pre-meal image taken by a user before the start of a meal based on a first preset condition, it can determine whether there is food in the tableware in the pre-meal image. Only if food is determined to be present in the tableware in the pre-meal image, does it then judge a post-meal image taken by the user after the meal based on a second preset condition. The second preset condition not only determines whether there is no food in the tableware, but also whether the tableware in the pre-meal image and the tableware in the post-meal image are the same. Only if it is determined that there is no food in the tableware in the post-meal image and that the tableware in the pre-meal image and the post-meal image are the same, does it send a prompt message to the user indicating that the "Clean Plate Campaign" has been completed. The entire image processing process is interconnected, gradually guiding users to participate in the "Clean Plate Campaign," thereby mobilizing the public's subjective enthusiasm for practicing the "Clean Plate Campaign."

[0054] The following is combined with Figures 2 to 5 The image processing methods mentioned in the embodiments of this disclosure are described in detail.

[0055] Figure 2 This is a schematic flowchart of the image processing method provided in the embodiments of this disclosure. Figure 2 The method is by Figure 1 The user terminal 110, 120, 130, or other types of electronic devices with data processing capabilities mentioned herein shall perform the execution. For example... Figure 2 As shown, the method includes the following steps.

[0056] In step S210, the user terminal determines whether the pre-meal image input by the user meets the first preset condition.

[0057] In one embodiment, the user terminal receives a pre-meal image input by the user. The user can directly use the camera on the user terminal to take a picture of the tableware before the meal begins, and then send the pre-meal image directly to the user terminal in the background. Alternatively, the user can use other devices with shooting capabilities to take a picture of the tableware before the meal begins, and then send the pre-meal image to the user terminal via the upload button.

[0058] The embodiments disclosed herein do not specifically limit the specific form of the pre-meal image; it can be an image formed directly after being photographed, or an image formed after preprocessing following photographing.

[0059] In one embodiment, when the pre-meal image is a directly generated image after shooting, there may be some shooting flaws due to differences in each user's shooting skills. Therefore, to improve the efficiency of data processing, the user terminal does not perform the first preset condition judgment on all pre-meal images. Instead, it first performs image detection on the pre-meal images to filter out those that fail the image detection. When the image detection of the pre-meal image passes, the user terminal executes the step of judging whether the pre-meal image meets the first preset condition, thereby improving the efficiency of data processing.

[0060] Image detection includes imaging angle detection and / or imaging quality detection. Imaging angle detection can detect shooting defects such as tilted shooting angle in the pre-meal image, while imaging quality detection can detect shooting defects such as poor image quality in the pre-meal image. Poor image quality defects also include blurry images and / or incomplete images in the pre-meal image. However, it should be noted that the embodiments disclosed in this disclosure do not specifically limit the specific detection form of imaging detection. It can be at least one of imaging angle detection and imaging quality detection, or other types of detection forms. Those skilled in the art can make different choices according to actual needs.

[0061] Image detection can be accomplished using a neural network model. In other words, the pre-meal image is input into the neural network model for image detection. The neural network model can output the probability value of the pre-meal image being taken at a tilted angle, or the probability value of the pre-meal image being poorly captured. When this probability value is less than a preset threshold, it means that the pre-meal image has passed the image detection.

[0062] If you want the user terminal to make a first preset judgment on all pre-meal images to improve the user experience, you can preprocess the pre-meal images to optimize the shooting defects mentioned above.

[0063] It should be noted that the first preset condition is used to characterize the presence of food in the tableware of the pre-meal image. When the probability value of the presence of food in the tableware of the pre-meal image is greater than or equal to the preset threshold, it means that the pre-meal image meets the first preset condition.

[0064] If the pre-meal image does not meet the first preset condition, the user terminal can send a request to the user to re-upload the pre-meal image.

[0065] Step S220: When the pre-meal image meets the first preset condition, the user terminal determines whether the post-meal image input by the user meets the second preset condition.

[0066] It should be noted that the second preset condition is used to characterize that there is no food in the tableware in the post-meal image and the tableware in the pre-meal image is the same tableware as the tableware in the post-meal image. When the probability value of no food in the tableware in the pre-meal image is greater than or equal to the preset threshold and the tableware in the pre-meal image is the same tableware as the tableware in the post-meal image, it means that the post-meal image meets the second preset condition.

[0067] In one embodiment, when the pre-meal image meets the first preset condition, it indicates that the tableware in the pre-meal image input by the user actually contains food, and is not an empty plate or an image containing only a little food residue. At this time, the user terminal can receive the post-meal image input by the user. The user can directly use the camera on the user terminal to take a picture of the tableware after finishing the meal, and then send the post-meal image directly to the user terminal in the background. Alternatively, the user can use other devices with shooting capabilities to take a picture of the tableware after finishing the meal, and then send the post-meal image to the user terminal via the upload button.

[0068] The embodiments disclosed herein do not specifically limit the specific form of the post-meal image. It can be an image formed directly after shooting, or an image formed after preprocessing after shooting.

[0069] In one embodiment, when the post-meal image is an image formed directly after shooting, there may be some shooting flaws due to differences in each user's shooting skills. Therefore, to improve the efficiency of data processing, the user terminal does not perform the second preset condition judgment on all post-meal images. Instead, it first performs the aforementioned imaging detection on the post-meal images to filter out those that fail the imaging detection. When the imaging detection of a post-meal image passes, the user terminal executes the step of judging whether the post-meal image meets the second preset condition, thereby improving the efficiency of data processing.

[0070] Image detection can be performed using a neural network model. In other words, the post-meal image is input into the neural network model for image detection. The neural network model can output the probability value of the post-meal image being taken at a tilted angle, or the probability value of the post-meal image being poorly captured. When the probability value is less than a preset threshold, it means that the post-meal image has passed the image detection.

[0071] If you want the user terminal to make a second preset condition judgment on all post-meal images to improve the user experience, you can preprocess the post-meal images to optimize the shooting defects mentioned above.

[0072] In another embodiment, in addition to performing the aforementioned imaging detection on the pre-meal image to improve data processing efficiency, the user terminal can also send a prompt message to the server indicating that the pre-meal image meets a first preset condition. Upon receiving this prompt message, the server performs a risk detection on the pre-meal image to filter out images that fail the risk detection. When the risk detection of the pre-meal image passes, the server sends a prompt message indicating that the risk detection of the pre-meal image has passed to the user terminal. Only when the user terminal receives this prompt message does it proceed to the step of determining whether the post-meal image meets the second preset condition, thereby further improving data processing efficiency.

[0073] Since the server takes longer to perform risk detection than the user terminal takes to determine whether the pre-meal image meets the first preset condition, the server only performs risk detection after the user terminal determines that the pre-meal image meets the first preset condition, thereby further improving the efficiency of data processing.

[0074] Risk detection includes compliance risk detection and / or safety risk detection. Compliance risk detection can identify risks in pre-meal images that violate relevant national laws and regulations (such as consumer rights), while safety risk detection can detect related risks in pre-meal images. However, it should be noted that this disclosure does not specifically limit the specific form of risk detection; it can be at least one of compliance risk detection and safety risk detection, or other types of detection. Those skilled in the art can make different choices according to actual needs.

[0075] As described above, the timing for the user terminal to execute the step of determining whether the post-meal image meets the second preset condition includes two timings: the first timing is when the user terminal receives a prompt message from the server indicating that the risk detection of the pre-meal image has passed; the second timing is when the imaging detection of the post-meal image passes. Furthermore, the timing for the user terminal to execute the step of determining whether the post-meal image meets the second preset condition also includes a third timing, namely, when the user terminal receives a prompt message from the server indicating that the risk detection of the post-meal image has passed. This embodiment does not specifically limit the order in which these three timings are executed. The user terminal may execute the step of determining whether the post-meal image meets the second preset condition only when the first, second, and third timings are simultaneously met. Those skilled in the art can make specific selections according to actual application needs.

[0076] As can be seen from the above, the timing for the user terminal to perform the step of judging whether the pre-meal image meets the first preset condition is when the imaging detection of the pre-meal image passes. However, this embodiment of the present disclosure does not specifically limit this. The timing for the user terminal to perform the step of judging whether the pre-meal image meets the first preset condition can also be when the user terminal receives the prompt information sent by the server that the risk detection of the pre-meal image has passed. This embodiment of the present disclosure does not specifically limit the order of execution of these two timings.

[0077] In step S230, when the post-meal image meets the second preset condition, the user terminal sends a prompt message to the user indicating that the "clean plate campaign" has been completed.

[0078] In one embodiment, in addition to performing the aforementioned risk detection on the pre-meal image to improve data processing efficiency, the user terminal can also send a prompt message to the server indicating that the post-meal image meets a second preset condition. Upon receiving this prompt message, the server performs the aforementioned risk detection on the post-meal image to filter out post-meal images that fail the risk detection. When the risk detection of the post-meal image passes, the server sends a prompt message to the user terminal indicating that the risk detection of the post-meal image has passed. Only when the user terminal receives this prompt message does it execute the step of sending a prompt message to the user indicating that the "clean plate campaign" has been completed, thereby further improving data processing efficiency.

[0079] It should be noted that the embodiments disclosed herein do not specifically limit the specific implementation form of the prompt message for completing the "Clean Plate Campaign". For example, a prompt message can be pushed to the user indicating that the "Clean Plate Campaign" check-in has been successfully completed.

[0080] As can be seen from the above, the timing for the user terminal to execute the step of sending a prompt message to the user indicating that the "Clean Plate Campaign" has been completed is when the user terminal receives a prompt message from the server indicating that the risk detection of the post-meal image has passed.

[0081] If the post-meal image does not meet the second preset condition, the user terminal can send a request to the user to re-upload the post-meal image.

[0082] This disclosure provides an image processing solution. By judging a pre-meal image based on a first preset condition, it can determine whether there is food in the tableware in the pre-meal image. Only when food is determined to be present in the tableware is a second preset condition judged on the post-meal image. The second preset condition judgment can determine not only whether there is no food in the tableware in the post-meal image, but also whether the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware. Only when it is determined that there is no food in the tableware in the post-meal image and that the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware is a prompt message to the user indicating completion of the "Clean Plate Campaign" is sent. The entire image processing process is interconnected, gradually guiding users to participate in the "Clean Plate Campaign," thereby mobilizing the public's subjective enthusiasm for practicing the "Clean Plate Campaign."

[0083] Specifically, this embodiment of the disclosure involves two images: a pre-meal image and a post-meal image. The judgment processes for the preset conditions in the pre-meal and post-meal images are interconnected. First, it is determined that there is food in the tableware in the pre-meal image, and only then can it be determined that there is no food in the tableware in the post-meal image. Through this logical judgment, it is possible to accurately determine whether the user has "finished" all the food during the entire meal. However, if only one image is taken, resulting in a single target image, and the empty tableware rate in that target image is judged, this target image could be any empty plate directly uploaded by the user. Therefore, the process of the user "finishing" all the food during the entire meal is not observed. Since the user cannot perceive the process of "finishing" all the food during the meal, they cannot experience the sense of accomplishment that comes with it. Consequently, it is impossible to gradually guide users to participate in the "Clean Plate Campaign," and thus, it is impossible to mobilize the public's subjective enthusiasm for practicing the "Clean Plate Campaign." In addition, to ensure that the pre-meal and post-meal images are taken from the same tableware, the appearance of the tableware in the pre-meal and post-meal images can be compared. This ensures that the reduction of food in the tableware is the only variable in the two judgment processes under preset conditions, thereby avoiding the problem that users can complete the "clean plate" campaign by simply scanning any empty plate.

[0084] In another embodiment of this disclosure, such as Figure 3 The diagram illustrates an image processing procedure provided in an embodiment of this disclosure, including:

[0085] Execute step S310, the user uploads a pre-meal image;

[0086] When it is obvious whether the pre-meal image can pass the risk detection, proceed to step S320, and the server will conduct synchronous review.

[0087] Upon successful synchronous review, step S330 is executed, and the server directly sends a notification message to the user terminal indicating that the pre-meal image review has been approved.

[0088] If the synchronous review fails, the server directly sends a prompt message to the user terminal to re-upload the pre-meal image, so as to repeat step S310.

[0089] When it is not obvious whether the pre-meal image can pass the risk detection, step S340 is executed, and the server sends a prompt message about the pre-meal image review to the user terminal.

[0090] During the pre-meal image review, step S350 is executed, and the server reviews the image asynchronously.

[0091] If the asynchronous review fails, the server sends a prompt to the user terminal to re-upload the pre-meal image, so as to repeat step S310.

[0092] When the asynchronous review is approved, step S330 is executed, and the server sends a notification message to the user terminal indicating that the pre-meal image review has been approved.

[0093] Similarly, in step S360, the user uploads a post-meal image;

[0094] When it is obvious whether the post-meal image can pass the risk detection, proceed to step S370, and the server will conduct synchronous review.

[0095] Upon successful synchronous review, step S380 is executed, and the server directly sends a notification message to the user terminal indicating that the post-meal image review has been approved.

[0096] If the synchronous review fails, the server directly sends a prompt message to the user terminal to re-upload the post-meal image, so as to repeat step S360.

[0097] When it is not obvious whether the post-meal image can pass the risk detection, step S390 is executed, and the server sends a prompt message about the post-meal image review to the user terminal.

[0098] During post-meal image review, step S391 is executed, and the server reviews asynchronously.

[0099] If the asynchronous review fails, the server sends a prompt to the user terminal to re-upload the post-meal image, so as to repeat step S360.

[0100] When the asynchronous review is approved, step S380 is executed, and the server sends a notification message to the user terminal indicating that the post-meal image review has been approved.

[0101] When it's obvious whether the pre-meal or post-meal image will pass the risk detection, the server can quickly determine whether the risk detection is successful, and synchronous review can be used. However, when it's not obvious whether the pre-meal or post-meal image will pass the risk detection, the server needs to spend some time performing the risk detection. Therefore, in order to provide users with timely feedback on the review status, asynchronous review can be used, so that users can receive timely feedback and improve the user experience.

[0102] Furthermore, if the server takes too long to detect risks in the pre-meal or post-meal images (i.e., the review times out), and steps S330 or S380 are executed, the server first sends a notification to the user terminal that the pre-meal or post-meal images have passed the review. If the server's risk detection result is actually that the pre-meal or post-meal images have failed the review, the server can directly filter and modify the compliance and / or security risks present in the pre-meal or post-meal images, and then send the filtered and modified pre-meal or post-meal images to the user terminal. This can also improve the user experience.

[0103] The above embodiments describe the timing for the user terminal to determine whether the pre-meal image input by the user meets the first preset condition. The following will combine... Figure 4 Describe how the user terminal determines whether the pre-meal image meets the first preset condition.

[0104] In step S410, the user terminal uses a neural network model to identify the food in the tableware in the pre-meal image and obtains a first probability value that food exists in the tableware in the pre-meal image.

[0105] The neural network model is trained using images of tableware containing varying amounts of food. These images could show a full plate, a half-full plate, a plate with only a few food scraps, or an empty plate. The model identifies the food in these images, learning the probability of food presence or absence when the plate is full (e.g., 0.9 for food, 0.1 for food), half-full plate (e.g., 0.7 for food, 0.3 for food), and an empty plate (e.g., 0.1 for food, 0.9 for food). Through iterative learning, the neural network model learns the probability of food presence or absence in tableware images with varying amounts of food.

[0106] When a pre-meal image is input into a neural network model, the model identifies the food in the tableware in the pre-meal image and outputs a first probability value indicating the presence of food in the tableware.

[0107] It should be noted that the neural network model is pre-deployed in the user terminal, or it is obtained from the server when the user terminal receives the pre-meal image uploaded by the user. Optionally, the neural network model can be a Convolutional Neural Network (CNN), a Deep Neural Network (DNN), or a Recurrent Neural Network (RNN), etc., and this disclosure does not limit the type of neural network model. Optionally, the network structure of the neural network model can be designed independently according to the computer vision task, or the network structure of the neural network model can adopt at least a part of the existing network structure, such as ResNet, ResNext, or DenseNet, etc., or it can be an SWM classifier, or a linear regression classifier, etc., and this disclosure does not limit the network structure of the neural network model.

[0108] Step S420: When the first probability value is greater than or equal to the first preset probability threshold, the user terminal determines that the pre-meal image meets the first preset condition.

[0109] After the neural network model outputs the first probability value, the first probability value can be compared with the first preset probability threshold. The first preset probability threshold is used to characterize the probability threshold that there is food in the tableware of the pre-meal image. That is, when the first probability value is greater than or equal to the first preset probability threshold, it means that there is food in the tableware of the pre-meal image and the pre-meal image meets the first preset condition.

[0110] The above embodiments describe the timing for the user terminal to determine whether the post-meal image input by the user meets the second preset condition. The following will combine... Figure 5 Describe how the user terminal specifically determines whether the post-meal image input by the user meets the second preset condition.

[0111] In step S510, the user terminal uses a neural network model to identify the food in the tableware in the post-meal image and obtains a second probability value that there is no food in the tableware in the post-meal image.

[0112] The neural network model in this embodiment and Figure 4 The neural network models used in the embodiments described above are the same and will not be repeated here. For details, please refer to [link to relevant documentation]. Figure 4 The aforementioned embodiments.

[0113] When a post-meal image is input into a neural network model, the model identifies the food in the tableware in the post-meal image and outputs a second probability value indicating that there is no food in the tableware.

[0114] In step S520, the user terminal determines whether the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware.

[0115] It should be noted that the embodiments disclosed herein do not specifically limit how to determine whether the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware. The pre-meal image and the post-meal image can be input into a neural network model, and the probability value that the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware can be directly output. Then, the probability value can be compared with a preset probability threshold. Alternatively, the pre-meal image and the post-meal image can be input into a neural network model, and the similarity between the tableware in the pre-meal image and the tableware in the post-meal image can be output. Then, the similarity can be compared with a preset similarity threshold. By comparing the threshold values, it can be determined whether the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware.

[0116] In step S530, when the second probability value is greater than or equal to the second preset probability threshold and the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware, the user terminal determines that the post-meal image meets the second preset condition.

[0117] After the neural network model outputs a second probability value, this second probability value can be compared with a second preset probability threshold. The second preset probability threshold represents the probability that there is no food in the tableware in the post-meal image. In other words, if the second probability value is greater than or equal to the second preset probability threshold, it indicates that there is no food in the tableware in the post-meal image. A second probability value greater than or equal to the second preset probability threshold is equivalent to a first probability value less than the first preset probability threshold. If there is no food in the tableware in the post-meal image and the tableware in the pre-meal image is the same as that in the post-meal image, it indicates that the post-meal image meets the second preset condition.

[0118] By continuously collecting sample sets of tableware images from before and after user meals during the training of neural network models, the recognition accuracy of the neural network models can be continuously improved.

[0119] The following is combined with Figure 6 Describe how the user terminal specifically compares the appearance of tableware in pre-meal and post-meal images.

[0120] In step S610, the user terminal uses a neural network model to identify the appearance of tableware in the pre-meal image and the post-meal image, respectively, to obtain at least one first confidence score vector corresponding to the appearance of tableware in the pre-meal image and at least one second confidence score vector corresponding to the appearance of tableware in the post-meal image.

[0121] It should be noted that the neural network model mentioned in this embodiment is different from... Figure 4 or Figure 5 The neural network models mentioned in the embodiments shown may be the same neural network model or different neural network models. This disclosure does not specifically limit them.

[0122] The neural network model mentioned in this embodiment can be trained using images of tableware with different appearances. The appearance of the tableware includes at least one of color, material, and shape. This embodiment does not specifically limit the appearance of the tableware; besides color, material, and shape, the appearance can be set differently according to actual needs. The neural network model will recognize at least one of the color, material, and shape of the tableware in the tableware image. Colors can include red, orange, yellow, green, cyan, blue, purple, etc.; materials can include ceramic, plastic, glass, paper, etc.; and shapes can include circles, squares, polygons, etc.

[0123] When the appearance of tableware includes color, the pre-meal image is input into the neural network model for color recognition. The model can output the first confidence score of the color of the pre-meal image as red, orange, yellow, green, cyan, blue, and purple. For example, P11, P12, P13, P14, P15, P16, and P17. P11, P12, P13, P14, P15, P16, and P17 form the first confidence score vector (P11, P12, P13, P14, P15, P16, P17). Similarly, by inputting post-meal images into a neural network model for color recognition, the model can output second confidence scores for the colors of the post-meal images as red, orange, yellow, green, cyan, blue, and purple. For example, P21, P22, P23, P24, P25, P26, and P27 form the second confidence score vector (P21, P22, P23, P24, P25, P26, P27).

[0124] When the appearance of tableware includes its material, inputting a pre-meal image into a neural network model for material identification can output a first confidence score indicating the material as ceramic, plastic, glass, or paper. For example, P31, P32, P33, and P34, where P31, P32, P33, and P34 form the first confidence score vector (P31, P32, P33, P34). Similarly, inputting a post-meal image into the neural network model for material identification can output a second confidence score indicating the material as ceramic, plastic, glass, or paper. For example, P41, P42, P43, and P44, where P41, P42, P43, and P44 form the second confidence score vector (P41, P42, P43, P44).

[0125] When the appearance of tableware includes its shape, inputting the pre-meal image into the neural network model for shape recognition can output a first confidence score indicating that the shape of the pre-meal image is a circle, square, or polygon, for example, P51, P52, and P53. P51, P52, and P53 together form the first confidence score vector (P51, P52, P53). Similarly, inputting the post-meal image into the neural network model for shape recognition can output a second confidence score indicating that the shape of the post-meal image is a circle, square, or polygon, for example, P61, P62, and P63. P61, P62, and P63 together form the second confidence score vector (P61, P62, P63).

[0126] In other words, each tableware appearance corresponds to a first confidence score vector and a second confidence score vector. Each first confidence score vector consists of multiple first confidence scores, which are the confidence scores when the tableware appearance in the pre-meal image is different from the preset tableware appearance. Each second confidence score vector consists of multiple second confidence scores, which are the confidence scores when the tableware appearance in the post-meal image is different from the preset tableware appearance.

[0127] It should be noted that the neural network model is pre-deployed in the user terminal, or it is obtained from the server when the user terminal receives pre-meal or post-meal images uploaded by the user.

[0128] In step S620, the user terminal determines the similarity between at least one first confidence score vector and at least one second confidence score vector.

[0129] The user terminal can utilize this neural network model to determine the similarity between at least one first confidence score vector and at least one second confidence score vector. This neural network model can be a distance-metric-based model, where the user terminal uses the distance metric to determine the similarity between each first confidence score vector and its corresponding second confidence score vector. The distance metric in the neural network model is used to assess the similarity between the appearance of tableware in a pre-meal image and the appearance of tableware in a post-meal image. These metrics include Euclidean distance, Manhattan distance, cosine distance, Chebyshev distance, Hamming distance, Minkowski distance, Mahalanobis distance, and so on. For example, cosine distance measures the angle between two spatial vectors to determine their similarity.

[0130] In step S630, the user terminal determines whether the similarity between at least one first confidence score vector and at least one second confidence score vector meets a preset similarity threshold.

[0131] A preset similarity threshold is used to characterize the similarity between the appearance of tableware in the pre-meal image and the appearance of tableware in the post-meal image. By judging the similarity between each first confidence score vector and its corresponding second confidence score vector, and then comparing it with the preset similarity threshold, it is possible to determine whether the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware.

[0132] In step S640, when the similarity between at least one first confidence score vector and at least one second confidence score vector meets a preset similarity threshold, the user terminal determines that the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware.

[0133] As described above, when the appearance of tableware includes color, material, and shape, at least one first confidence score vector includes a first confidence score vector corresponding to color (P11, P12, P13, P14, P15, P16, P17), a first confidence score vector corresponding to material (P31, P32, P33, P34), and a first confidence score vector corresponding to shape (P51, P52, P53). At least one second confidence score vector includes a second confidence score vector corresponding to color (P21, P22, P23, P24, P25, P26, P27), a second confidence score vector corresponding to material (P41, P42, P43, P44), and a second confidence score vector corresponding to shape (P61, P62, P63). Figure 7 As shown, Figure 6 The step S620 shown includes the following:

[0134] In step S710, the user terminal uses a neural network model to determine the first distance metric between the first confidence score vector corresponding to the color and the second confidence score vector corresponding to the color as the similarity between the first confidence score vector corresponding to the color and the second confidence score vector corresponding to the color.

[0135] The first distance metric X between the first confidence score vector (P11, P12, P13, P14, P15, P16, P17) and the second confidence score vector (P21, P22, P23, P24, P25, P26, P27) corresponding to the color is used as the similarity between the first confidence score vector and the second confidence score vector corresponding to the color.

[0136] In step S720, the user terminal uses a neural network model to determine a second distance metric between the first confidence score vector corresponding to the material and the second confidence score vector corresponding to the material as the similarity between the first confidence score vector corresponding to the material and the second confidence score vector corresponding to the material.

[0137] The second distance metric Y between the first confidence score vector (P31, P32, P33, P34) and the second confidence score vector (P41, P42, P43, P44) corresponding to the material is used as the similarity between the first confidence score vector and the second confidence score vector corresponding to the material.

[0138] In step S730, the user terminal uses a neural network model to determine a third distance metric between the first confidence score vector corresponding to the shape and the second confidence score vector corresponding to the shape as the similarity between the first confidence score vector corresponding to the shape and the second confidence score vector corresponding to the shape.

[0139] The third distance metric Z between the first confidence score vector (P51, P52, P53) and the second confidence score vector (P61, P62, P63) corresponding to the shape is used as the similarity between the first confidence score vector and the second confidence score vector corresponding to the shape.

[0140] As described above, when the similarity between at least one first confidence score vector and at least one second confidence score vector is X, Y, and Z, the user terminal performs a weighted summation of the first distance metric X, the second distance metric Y, and the third distance metric Z to obtain a similarity score S between the appearance of the tableware in the pre-meal image and the appearance of the tableware in the post-meal image, i.e., S = a*X + b*Y + c*Z. When the similarity score S is greater than or equal to a preset similarity threshold, the user terminal determines that the similarity between at least one first confidence score vector and at least one second confidence score vector meets the preset similarity threshold, meaning that the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware.

[0141] However, it should be noted that the embodiments disclosed herein do not specifically limit the values ​​of weights a, b, and c, and those skilled in the art can make different choices according to actual needs.

[0142] Figure 8 This is a schematic flowchart of the image processing method provided in the embodiments of this disclosure. Figure 8 The method is by Figure 1 The server 140 or other type of electronic device with data processing capabilities mentioned herein shall perform the execution. For example... Figure 8 As shown, the method includes the following steps. It should be understood that the description of this method embodiment corresponds to the description of the preceding method embodiments; therefore, any parts not described in detail can be found in the preceding method embodiments.

[0143] Step S810: The server receives a prompt message from the user terminal indicating that the pre-meal image input by the user meets the first preset condition.

[0144] Step S820: The server performs risk detection on the pre-meal image;

[0145] In step S830, the server sends a notification message to the user terminal indicating that the risk detection of the pre-meal image has passed, so that the user terminal can determine whether the post-meal image input by the user meets the second preset condition.

[0146] Step S840: The server receives a notification message from the user terminal indicating that the post-meal image meets the second preset condition.

[0147] Step S850: The server performs risk detection on the post-meal images;

[0148] In step S860, the server sends a notification message to the user terminal indicating that the risk detection of the post-meal image has passed, so that when the user terminal determines that the post-meal image meets the second preset condition, it can send a notification message to the user indicating that the "clean plate campaign" has been completed.

[0149] The above text combined Figures 1 to 8 The present disclosure describes in detail the method embodiments, which are then combined with the following. Figure 9 and Figure 10 The present disclosure provides a detailed description of the apparatus embodiments. It should be understood that the descriptions of the method embodiments correspond to the descriptions of the apparatus embodiments; therefore, any parts not described in detail can be found in the foregoing method embodiments.

[0150] Figure 9 This is a schematic structural diagram of an image processing apparatus 900 provided in an embodiment of the present disclosure. Figure 9 The device 900 may include: a first judgment module 910, a second judgment module 920, and a push module 930. These modules are described in detail below.

[0151] The first judgment module 910 is configured to determine whether the pre-meal image input by the user meets the first preset condition, wherein the pre-meal image is an image formed after the user takes a picture of the tableware before starting the meal.

[0152] The second judgment module 920 is configured to determine whether the user-input post-meal image meets the second preset condition when the pre-meal image meets the first preset condition. The post-meal image is an image formed after the user takes a picture of the tableware after finishing the meal.

[0153] The push module 930 is configured to send a prompt message to the user indicating that the "clean plate campaign" has been completed when the post-meal image meets the second preset condition.

[0154] This disclosure provides an image processing solution. By judging a pre-meal image taken by a user before the start of a meal based on a first preset condition, it can determine whether there is food in the tableware in the pre-meal image. Only if food is determined to be present in the tableware, a second preset condition is applied to a post-meal image taken by the user after the meal. The second preset condition not only determines whether there is no food in the tableware, but also whether the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware. Only if it is determined that there is no food in the tableware in the post-meal image and that the tableware in the pre-meal image and the post-meal image are the same tableware, a prompt message indicating completion of the "Clean Plate Campaign" is sent to the user. The entire image processing process is interconnected, gradually guiding users to participate in the "Clean Plate Campaign," thereby mobilizing the public's subjective enthusiasm for practicing the "Clean Plate Campaign."

[0155] In one embodiment, the first judgment module 910 is further configured as follows: the user terminal uses a neural network model to identify the food in the tableware of the pre-meal image and obtains a first probability value that there is food in the tableware of the pre-meal image; when the first probability value is greater than or equal to a first preset probability threshold, the user terminal determines that the pre-meal image meets the first preset condition, wherein the first preset probability threshold is used to characterize the probability threshold that there is food in the tableware of the pre-meal image.

[0156] In one embodiment, the second judgment module 920 is further configured as follows: the user terminal uses a neural network model to identify the food in the tableware of the post-meal image and obtains a second probability value that there is no food in the tableware of the post-meal image; the user terminal judges whether the tableware of the pre-meal image and the tableware of the post-meal image are the same tableware; when the second probability value is greater than or equal to a second preset probability threshold and the tableware of the pre-meal image and the tableware of the post-meal image are the same tableware, the user terminal determines that the post-meal image meets the second preset condition, wherein the second preset probability threshold is used to characterize the probability threshold that there is no food in the tableware of the post-meal image.

[0157] In one embodiment, when the user terminal determines whether the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware, the second judgment module 920 is further configured to: use a neural network model to identify the appearance of the tableware in the pre-meal image and the tableware in the post-meal image respectively, and obtain at least one first confidence score vector corresponding to the appearance of the tableware in the pre-meal image and at least one second confidence score vector corresponding to the appearance of the tableware in the post-meal image. Each tableware appearance corresponds to one first confidence score vector and one second confidence score vector. Each first confidence score vector is composed of multiple first confidence scores, which are confidence scores when the tableware appearance in the pre-meal image is a different preset tableware appearance. Each second confidence score vector is composed of... Multiple second confidence scores are constituted, which are confidence scores when the appearance of the tableware in the post-meal image is different from the preset tableware appearance; the user terminal determines the similarity between at least one first confidence score vector and at least one second confidence score vector; the user terminal determines whether the similarity between at least one first confidence score vector and at least one second confidence score vector meets a preset similarity threshold, wherein the preset similarity threshold is used to characterize the similarity threshold between the appearance of the tableware in the pre-meal image and the appearance of the tableware in the post-meal image; when the similarity between at least one first confidence score vector and at least one second confidence score vector meets the preset similarity threshold, the user terminal determines that the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware.

[0158] In one embodiment, when the appearance of the tableware includes color, material, and shape, at least one first confidence score vector includes a first confidence score vector corresponding to color, a first confidence score vector corresponding to material, and a first confidence score vector corresponding to shape, and at least one second confidence score vector includes a second confidence score vector corresponding to color, a second confidence score vector corresponding to material, and a second confidence score vector corresponding to shape.

[0159] In one embodiment, when the user terminal determines the similarity between at least one first confidence score vector and at least one second confidence score vector, the second judgment module 920 is further configured as follows: the user terminal uses a neural network model to determine a first distance metric between the first confidence score vector corresponding to color and the second confidence score vector corresponding to color as the similarity between the two vectors; the user terminal uses a neural network model to determine a second distance metric between the first confidence score vector corresponding to material and the second confidence score vector corresponding to material as the similarity between the two vectors; and the user terminal uses a neural network model to determine a third distance metric between the first confidence score vector corresponding to shape and the second confidence score vector corresponding to shape as the similarity between the two vectors.

[0160] In one embodiment, when the user terminal determines whether the similarity between at least one first confidence score vector and at least one second confidence score vector meets a preset similarity threshold, the second judgment module 920 is further configured to: the user terminal performs a weighted summation of the first distance metric, the second distance metric, and the third distance metric to obtain a similarity score between the appearance of tableware in the pre-meal image and the appearance of tableware in the post-meal image; when the similarity score is greater than or equal to the preset similarity threshold, the user terminal determines that the similarity between at least one first confidence score vector and at least one second confidence score vector meets the preset similarity threshold.

[0161] In one embodiment, when the pre-meal image meets the first preset condition, the device 900 further includes a sending module 940, configured to send a prompt message from the user terminal to the server indicating that the pre-meal image meets the first preset condition, so that the server can perform risk detection on the pre-meal image, wherein the risk detection includes compliance risk detection and / or security risk detection.

[0162] In one embodiment, when the user terminal receives a prompt message from the server indicating that the risk detection of the pre-meal image has passed, the second judgment module 920 is configured to determine whether the post-meal image meets the second preset condition.

[0163] In one embodiment, when the post-meal image meets the second preset condition, the sending module 940 is further configured to send a prompt message from the user terminal to the server indicating that the post-meal image meets the second preset condition, so that the server can perform risk detection on the post-meal image, wherein the risk detection includes compliance risk detection and / or security risk detection.

[0164] In one embodiment, when the user terminal receives a notification message from the server indicating that the risk detection of the post-meal image has passed, the push module 930 is configured to send a notification message to the user indicating that the "Clean Plate Campaign" has been completed.

[0165] In one embodiment, the device 900 further includes a detection module 950 configured to: perform imaging detection on the pre-meal image by the user terminal, wherein the imaging detection includes imaging angle detection for determining whether the shooting angle of the pre-meal image is tilted and / or imaging quality detection for determining whether the shooting quality of the pre-meal image is intact.

[0166] In one embodiment, when the imaging detection of the pre-meal image passes, the first judgment module 910 is configured for the user terminal to judge whether the pre-meal image meets the first preset condition.

[0167] In one embodiment, the detection module 950 is further configured to: perform imaging detection on the post-meal image by the user terminal, wherein the imaging detection includes imaging angle detection for determining whether the shooting angle of the post-meal image is tilted and / or imaging quality detection for determining whether the shooting quality of the post-meal image is intact.

[0168] In one embodiment, when the imaging detection of the post-meal image passes, the second judgment module 920 is configured for the user terminal to judge whether the post-meal image meets the second preset condition.

[0169] In one embodiment, if the pre-meal image does not meet the first preset condition, the push module 930 is further configured to send a request to the user terminal to re-upload the pre-meal image.

[0170] In one embodiment, if the post-meal image does not meet the second preset condition, the push module 930 is further configured to send a request to the user terminal to re-upload the post-meal image.

[0171] Figure 10 This is a schematic structural diagram of an image processing apparatus 1000 provided in an embodiment of the present disclosure. Figure 10 The device 10 may include: a first receiving module 1010, a first risk detection module 1020, a first transmitting module 1030, a second receiving module 1040, a second risk detection module 1050, and a second transmitting module 1060. These modules are described in detail below.

[0172] The first receiving module 1010 is configured to receive a prompt message from a user terminal indicating that the pre-meal image input by the user meets the first preset condition. The pre-meal image is an image formed after the user takes a picture of the tableware before starting the meal.

[0173] The first risk detection module 1020 is configured as a server to perform risk detection on pre-meal images, including compliance risk detection and / or security risk detection.

[0174] The first sending module 1030 is configured to send a prompt message indicating that the risk detection of the pre-meal image has passed to the user terminal, so that the user terminal can determine whether the post-meal image input by the user meets the second preset condition. The post-meal image is an image of the tableware formed after the user takes a picture of the tableware after finishing the meal.

[0175] The second receiving module 1040 is configured to receive a notification message from the user terminal indicating that the post-meal image meets the second preset condition.

[0176] The second risk detection module 1050 is configured as a server to perform risk detection on post-meal images.

[0177] The second sending module 1060 is configured to send a prompt message from the server to the user terminal indicating that the risk detection of the post-meal image has passed, so that when the user terminal determines that the post-meal image meets the second preset condition, it can send a prompt message to the user indicating that the "clean plate campaign" has been completed.

[0178] Figure 11 This is a schematic diagram of the structure of an electronic device 1100 provided in one embodiment of this disclosure. The electronic device 1100 may be, for example, a computing device with computing capabilities. For instance, the electronic device 1100 may be a server. The electronic device 1100 may include a memory 1110 and a processor 1120. The memory 1110 may be used to store executable code. The processor 1120 may be used to execute the executable code stored in the memory 1110 to implement the steps in the various methods described above. In some embodiments, the electronic device 1100 may further include a network interface 1130, through which data exchange between the processor 1120 and external devices can be achieved.

[0179] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any other combination. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).

[0180] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments of this disclosure can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0181] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0182] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0183] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0184] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, include: The user terminal determines whether the pre-meal image input by the user meets the first preset condition, wherein the pre-meal image is an image formed after the user takes a picture of the tableware before starting the meal; When the pre-meal image meets the first preset condition, the user terminal determines whether the post-meal image input by the user meets the second preset condition, wherein the post-meal image is an image formed by the user taking a picture of the tableware after finishing the meal; When the post-meal image meets the second preset condition, the user terminal sends a notification message to the user indicating that the "clean plate" campaign has been completed. The user terminal determines whether the pre-meal image input by the user meets the first preset condition, including: The user terminal uses a neural network model to identify the food in the tableware in the pre-meal image and obtains a first probability value that there is food in the tableware in the pre-meal image. When the first probability value is greater than or equal to a first preset probability threshold, the user terminal determines that the pre-meal image meets the first preset condition, wherein the first preset probability threshold is used to characterize the probability threshold that food exists in the tableware of the pre-meal image. The user terminal determines whether the post-meal image input by the user meets the second preset condition, including: The user terminal uses the neural network model to identify the food in the tableware of the post-meal image and obtains a second probability value that there is no food in the tableware of the post-meal image. The user terminal determines whether the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware; When the second probability value is greater than or equal to the second preset probability threshold and the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware, the user terminal determines that the post-meal image meets the second preset condition. The second preset probability threshold is used to characterize the probability that there is no food in the tableware in the post-meal image. The user terminal determines whether the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware, including: The user terminal uses the neural network model to identify the appearance of tableware in the pre-meal image and the post-meal image, respectively, to obtain at least one first confidence score vector corresponding to the appearance of tableware in the pre-meal image and at least one second confidence score vector corresponding to the appearance of tableware in the post-meal image. Each tableware appearance corresponds to one first confidence score vector and one second confidence score vector. Each first confidence score vector consists of multiple first confidence scores, which are confidence scores when the tableware appearance in the pre-meal image is different from the preset tableware appearance. Each second confidence score vector consists of multiple second confidence scores, which are confidence scores when the tableware appearance in the post-meal image is different from the preset tableware appearance. The user terminal determines the similarity between the at least one first confidence score vector and the at least one second confidence score vector; The user terminal determines whether the similarity between the at least one first confidence score vector and the at least one second confidence score vector meets a preset similarity threshold, wherein the preset similarity threshold is used to characterize the similarity threshold between the appearance of tableware in the pre-meal image and the appearance of tableware in the post-meal image. When the similarity between at least one first confidence score vector and at least one second confidence score vector meets the preset similarity threshold, the user terminal determines that the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware. Wherein, when the appearance of the tableware includes color, material and shape, the at least one first confidence score vector includes the first confidence score vector corresponding to the color, the first confidence score vector corresponding to the material and the first confidence score vector corresponding to the shape, and the at least one second confidence score vector includes the second confidence score vector corresponding to the color, the second confidence score vector corresponding to the material and the second confidence score vector corresponding to the shape.

2. The method according to claim 1, characterized in that, The user terminal determines the similarity between the at least one first confidence score vector and the at least one second confidence score vector, including: The user terminal uses the neural network model to determine a first distance metric between the first confidence score vector corresponding to the color and the second confidence score vector corresponding to the color as the similarity between the first confidence score vector corresponding to the color and the second confidence score vector corresponding to the color. The user terminal uses the neural network model to determine a second distance metric between the first confidence score vector corresponding to the material and the second confidence score vector corresponding to the material as the similarity between the first confidence score vector corresponding to the material and the second confidence score vector corresponding to the material. The user terminal uses the neural network model to determine a third distance metric between the first confidence score vector corresponding to the shape and the second confidence score vector corresponding to the shape, as the similarity between the first confidence score vector corresponding to the shape and the second confidence score vector corresponding to the shape. Wherein, the user terminal determines whether the similarity between the at least one first confidence score vector and the at least one second confidence score vector meets a preset similarity threshold, including: The user terminal performs a weighted summation of the first distance metric, the second distance metric, and the third distance metric to obtain a similarity score between the appearance of the tableware in the pre-meal image and the appearance of the tableware in the post-meal image. When the similarity score is greater than or equal to the preset similarity threshold, the user terminal determines that the similarity between the at least one first confidence score vector and the at least one second confidence score vector satisfies the preset similarity threshold.

3. The method according to claim 1 or 2, characterized in that, When the pre-meal image meets the first preset condition, the method further includes: The user terminal sends a notification to the server indicating that the pre-meal image meets the first preset condition, so that the server can perform risk detection on the pre-meal image, wherein the risk detection includes compliance risk detection and / or security risk detection. The user terminal determines whether the post-meal image input by the user meets the second preset condition, including: When the user terminal receives a notification message from the server indicating that the risk detection of the pre-meal image has passed, the user terminal determines whether the post-meal image meets the second preset condition.

4. The method according to claim 1 or 2, characterized in that, When the post-meal image meets the second preset condition, the method further includes: The user terminal sends a notification to the server indicating that the post-meal image meets the second preset condition, so that the server can perform risk detection on the post-meal image. The risk detection includes compliance risk detection and / or security risk detection. The user terminal sends a notification message to the user indicating that the "Clean Plate Campaign" has been completed, including: When the user terminal receives a notification message from the server indicating that the risk detection of the post-meal image has passed, the user terminal sends a notification message to the user indicating that the "Clean Plate Campaign" has been completed.

5. The method according to claim 1 or 2, characterized in that, Also includes: The user terminal performs image detection on the pre-meal image, wherein the image detection includes image angle detection to determine whether the shooting angle of the pre-meal image is tilted and / or image quality detection to determine whether the shooting quality of the pre-meal image is good. The user terminal determines whether the pre-meal image input by the user meets the first preset condition, including: When the imaging detection of the pre-meal image passes, the user terminal determines whether the pre-meal image meets the first preset condition.

6. The method according to claim 1 or 2, characterized in that, Also includes: The user terminal performs image detection on the post-meal image, wherein the image detection includes image angle detection to determine whether the shooting angle of the post-meal image is tilted and / or image quality detection to determine whether the shooting quality of the post-meal image is good. The user terminal determines whether the post-meal image input by the user meets the second preset condition, including: When the post-meal image passes the imaging detection, the user terminal determines whether the post-meal image meets the second preset condition.

7. The method according to claim 1 or 2, characterized in that, When the pre-meal image does not meet the first preset condition, the method further includes: The user terminal sends a request to the user to re-upload the pre-meal image, and / or When the post-meal image does not meet the second preset condition, the method further includes: The user terminal sends a request to the user to re-upload the post-meal image.

8. An image processing method, characterized in that, include: The server receives a prompt message from the user terminal indicating that the pre-meal image input by the user meets a first preset condition. The pre-meal image is an image formed by the user taking a picture of the tableware before starting the meal. The user terminal uses a neural network model to identify the food in the tableware in the pre-meal image and obtains a first probability value indicating the presence of food in the tableware. When the first probability value is greater than or equal to a first preset probability threshold, the user terminal determines that the pre-meal image meets the first preset condition. The first preset probability threshold is used to characterize the probability threshold of the presence of food in the tableware in the pre-meal image. The server performs risk detection on the pre-meal image, wherein the risk detection includes compliance risk detection and / or security risk detection; The server sends a notification message to the user terminal indicating that the risk detection of the pre-meal image has passed, so that the user terminal can determine whether the post-meal image input by the user meets the second preset condition. The post-meal image is an image formed by the user taking a picture of the tableware after finishing the meal. The server receives a notification from the user terminal indicating that the post-meal image meets the second preset condition. The user terminal uses the neural network model to identify food in the tableware of the post-meal image, obtaining a second probability value indicating that no food is present in the tableware. The user terminal determines whether the tableware in the pre-meal image and the tableware in the post-meal image are the same. When the second probability value is greater than or equal to a second preset probability threshold and the tableware in the pre-meal image and the tableware in the post-meal image are the same, the user terminal determines that the post-meal image meets the second preset condition. The second preset probability threshold is used to characterize the... The probability threshold for the absence of food in the tableware in the post-meal image is defined as follows: The user terminal utilizes the neural network model to identify the appearance of the tableware in the pre-meal image and the post-meal image, respectively, obtaining at least one first confidence score vector corresponding to the tableware appearance in the pre-meal image and at least one second confidence score vector corresponding to the tableware appearance in the post-meal image. Each tableware appearance corresponds to one first confidence score vector and one second confidence score vector. Each first confidence score vector consists of multiple first confidence scores, which are confidence scores for different preset tableware appearances in the pre-meal image. Each second confidence score vector is composed of multiple second confidence scores, which are confidence scores for different preset tableware appearances in the post-meal image; the user terminal determines the similarity between the at least one first confidence score vector and the at least one second confidence score vector; the user terminal judges whether the similarity between the at least one first confidence score vector and the at least one second confidence score vector meets a preset similarity threshold, wherein the preset similarity threshold is used to characterize the similarity threshold between the tableware appearance in the pre-meal image and the tableware appearance in the post-meal image; in the at least one first confidence score vector When the similarity between the quantity and the at least one second confidence score vector meets the preset similarity threshold, the user terminal determines that the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware. Wherein, when the appearance of the tableware includes color, material, and shape, the at least one first confidence score vector includes the first confidence score vector corresponding to the color, the first confidence score vector corresponding to the material, and the first confidence score vector corresponding to the shape; the at least one second confidence score vector includes the second confidence score vector corresponding to the color, the second confidence score vector corresponding to the material, and the second confidence score vector corresponding to the shape. The server performs the risk detection on the post-meal images; The server sends a notification message to the user terminal indicating that the risk detection of the post-meal image has passed, so that when the user terminal determines that the post-meal image meets the second preset condition, it sends a notification message to the user indicating that the "clean plate campaign" has been completed.

9. An image processing apparatus, characterized in that, include: The first judgment module is configured to determine whether the pre-meal image input by the user meets the first preset condition, wherein the pre-meal image is an image formed after the user takes a picture of the tableware before starting the meal; The second judgment module is configured to, when the pre-meal image meets the first preset condition, determine whether the post-meal image input by the user meets the second preset condition, wherein the post-meal image is an image formed by the user taking a picture of the tableware after finishing the meal; The push module is configured to send a notification message to the user indicating that the "clean plate campaign" has been completed when the post-meal image meets the second preset condition. The first judgment module is further configured to: the user terminal uses a neural network model to identify the food in the tableware of the pre-meal image and obtain a first probability value that there is food in the tableware of the pre-meal image; When the first probability value is greater than or equal to a first preset probability threshold, the user terminal determines that the pre-meal image meets the first preset condition, wherein the first preset probability threshold is used to characterize the probability threshold that food exists in the tableware of the pre-meal image. The second judgment module is further configured as follows: the user terminal uses the neural network model to identify the food in the tableware of the post-meal image and obtains a second probability value that there is no food in the tableware of the post-meal image; The user terminal determines whether the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware; When the second probability value is greater than or equal to the second preset probability threshold and the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware, the user terminal determines that the post-meal image meets the second preset condition. The second preset probability threshold is used to characterize the probability that there is no food in the tableware in the post-meal image. Wherein, when the user terminal determines whether the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware, the second judgment module is further configured as follows: the user terminal uses the neural network model to identify the appearance of the tableware in the pre-meal image and the appearance of the tableware in the post-meal image respectively, to obtain at least one first confidence score vector corresponding to the appearance of the tableware in the pre-meal image and at least one second confidence score vector corresponding to the appearance of the tableware in the post-meal image, wherein one tableware appearance corresponds to one first confidence score vector and one second confidence score vector, each first confidence score vector is composed of multiple first confidence scores, the multiple first confidence scores being the confidence scores when the appearance of the tableware in the pre-meal image is different preset tableware appearances, and each second confidence score vector is composed of multiple second confidence scores, the multiple second confidence scores being the confidence scores when the appearance of the tableware in the post-meal image is different preset tableware appearances; The user terminal determines the similarity between the at least one first confidence score vector and the at least one second confidence score vector; The user terminal determines whether the similarity between the at least one first confidence score vector and the at least one second confidence score vector meets a preset similarity threshold, wherein the preset similarity threshold is used to characterize the similarity threshold between the appearance of tableware in the pre-meal image and the appearance of tableware in the post-meal image. When the similarity between at least one first confidence score vector and at least one second confidence score vector meets the preset similarity threshold, the user terminal determines that the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware. Wherein, when the appearance of the tableware includes color, material and shape, the at least one first confidence score vector includes the first confidence score vector corresponding to the color, the first confidence score vector corresponding to the material and the first confidence score vector corresponding to the shape, and the at least one second confidence score vector includes the second confidence score vector corresponding to the color, the second confidence score vector corresponding to the material and the second confidence score vector corresponding to the shape.

10. An image processing apparatus, characterized in that, include: The first receiving module is configured to receive a prompt message from a user terminal indicating that a pre-meal image input by the user meets a first preset condition. The pre-meal image is an image formed by the user taking a picture of the tableware before starting the meal. The user terminal uses a neural network model to identify food in the tableware in the pre-meal image and obtain a first probability value indicating the presence of food in the tableware. When the first probability value is greater than or equal to a first preset probability threshold, the user terminal determines that the pre-meal image meets the first preset condition. The first preset probability threshold is used to characterize the probability threshold of the presence of food in the tableware in the pre-meal image. The first risk detection module is configured to allow the server to perform risk detection on the pre-meal image, wherein the risk detection includes compliance risk detection and / or security risk detection; The first sending module is configured to send a prompt message indicating that the risk detection of the pre-meal image has passed to the user terminal, so that the user terminal can determine whether the post-meal image input by the user meets the second preset condition. The post-meal image is an image of the tableware formed by the user taking a picture of the tableware after finishing the meal. The second receiving module is configured to receive a prompt message from the user terminal indicating that the post-meal image meets the second preset condition. The user terminal uses the neural network model to identify food in the tableware of the post-meal image, obtaining a second probability value indicating that no food is present in the tableware. The user terminal determines whether the tableware in the pre-meal image and the tableware in the post-meal image are the same. When the second probability value is greater than or equal to a second preset probability threshold and the tableware in the pre-meal image and the tableware in the post-meal image are the same, the user terminal determines that the post-meal image meets the second preset condition. The value is used to characterize the probability threshold that there is no food in the tableware in the post-meal image. The user terminal uses the neural network model to identify the appearance of the tableware in the pre-meal image and the post-meal image, respectively, to obtain at least one first confidence score vector corresponding to the tableware appearance in the pre-meal image and at least one second confidence score vector corresponding to the tableware appearance in the post-meal image. Each tableware appearance corresponds to one first confidence score vector and one second confidence score vector. Each first confidence score vector consists of multiple first confidence scores, which represent the confidence level when the tableware appearance in the pre-meal image is a different preset tableware appearance. Each second confidence score vector consists of multiple second confidence scores, which are confidence scores for different preset tableware appearances in the post-meal image; the user terminal determines the similarity between the at least one first confidence score vector and the at least one second confidence score vector; the user terminal judges whether the similarity between the at least one first confidence score vector and the at least one second confidence score vector meets a preset similarity threshold, wherein the preset similarity threshold is used to characterize the similarity threshold between the tableware appearance in the pre-meal image and the tableware appearance in the post-meal image; in the at least one first confidence score vector... When the similarity between the score vector and the at least one second confidence score vector meets the preset similarity threshold, the user terminal determines that the tableware in the pre-meal image and the tableware in the post-meal image are the same tableware. Wherein, when the appearance of the tableware includes color, material, and shape, the at least one first confidence score vector includes the first confidence score vector corresponding to the color, the first confidence score vector corresponding to the material, and the first confidence score vector corresponding to the shape; the at least one second confidence score vector includes the second confidence score vector corresponding to the color, the second confidence score vector corresponding to the material, and the second confidence score vector corresponding to the shape. The second risk detection module is configured to allow the server to perform risk detection on the post-meal image. The second sending module is configured to send a notification message from the server to the user terminal indicating that the risk detection of the post-meal image has passed, so that when the user terminal determines that the post-meal image meets the second preset condition, it can send a notification message to the user indicating that the "clean plate campaign" has been completed.

11. An electronic device comprising a memory and a processor, the memory storing executable code, the processor being configured to execute the executable code to implement the method of any one of claims 1 to 8.

Images