Food material image acquisition method and device and electronic equipment

Abstract

The invention relates to a food material image acquisition method and device and electronic equipment, and the method comprises the steps: shooting a preset collection region in a target electric appliance based on preset camera equipment in a process of cooking a preset food material by using the target electric appliance, and obtaining a first food material image; performing food material edge detection on the first food material image to obtain first area position information of a first area where the preset food material is located; based on preset area position information corresponding to the preset acquisition area and the first area position information, determining an amplification proportion of the preset acquisition area relative to the first area; adjusting a focal length corresponding to the preset camera device based on the magnification ratio; based on the preset camera device with the adjusted focal length, shooting the preset collection area to obtain a second food material image; and based on the first region position information, cutting the second food material image to obtain a preset food material image corresponding to the preset food material. According to the embodiment of the invention, the background area proportion in the food material image can be reduced, and the food material positioning accuracy is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular, to a method, an apparatus, and an electronic device for obtaining a food ingredient image. Background Art

[0002] With the improvement of people's living standards, the requirements for cooking utensils are also getting higher and higher.

[0003] For example, it is required that the cooking utensil provides an internal food ingredient image during the cooking process. Most of the existing food ingredient images displayed by direct display of a camera are background areas. When users view the cooking situation of the food ingredients through a mobile phone or an electrical appliance screen, the experience is poor. Using the camera to automatically focus to obtain the food ingredient image, however, the camera is fixed on the electrical appliance and can only collect the food ingredient image within a fixed field of view. When the food ingredient is in the edge area, even if the focus is adjusted, most of the image will still be the background area. As a result, the existing food ingredient image acquisition solutions have problems such as an excessive proportion of the background area and poor accuracy of food ingredient positioning. Summary of the Invention

[0004] The present disclosure provides a method, an apparatus, and an electronic device for obtaining a food ingredient image, so as to at least solve the technical problems such as an excessive proportion of the background area and poor accuracy of food ingredient positioning in the related art. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided a method for obtaining a food ingredient image, including:

[0006] During the process of cooking a preset food ingredient using a target electrical appliance, based on a preset imaging device, photographing a preset collection area in the target electrical appliance to obtain a first food ingredient image, where the preset collection area includes the preset food ingredient;

[0007] Performing food ingredient edge detection on the first food ingredient image to obtain first area position information of a first area where the preset food ingredient is located;

[0008] Based on the preset area position information corresponding to the preset collection area and the first area position information, determining a magnification ratio of the preset collection area relative to the first area;

[0009] Based on the magnification ratio, adjusting the focal length corresponding to the preset imaging device;

[0010] Based on the preset imaging device with the adjusted focal length, photographing the preset collection area to obtain a second food ingredient image;

[0011] Based on the first area position information, cropping the second food ingredient image to obtain a preset food ingredient image corresponding to the preset food ingredient.

[0012] In an alternative embodiment, the preset food ingredients include a plurality of food ingredients, and the method includes:

[0013] In response to a viewing instruction for a target food ingredient among the plurality of food ingredients, obtain second region position information of a second region where the target food ingredient is located;

[0014] Based on the second region position information, crop the second food ingredient image to obtain a target food ingredient image corresponding to the target food ingredient.

[0015] In an alternative embodiment, the obtaining of the second region position information of the second region where the target food ingredient is located includes:

[0016] When a positioning operation for the target food ingredient in the second food ingredient image is detected, trigger the viewing instruction and obtain positioning position information of the target food ingredient;

[0017] Based on the positioning position information, determine the second region position information.

[0018] In an alternative embodiment, the method further includes:

[0019] Obtain preset contour indication information of the preset food ingredients;

[0020] When the preset contour indication information indicates that the preset food ingredient is a food ingredient whose contour changes during cooking, obtain first gray-scale information of the first food ingredient image and second gray-scale information of a current image, where the current image is an image captured in real time by the preset imaging device during the cooking process;

[0021] Based on the first gray-scale information and the second gray-scale information, determine gray-scale difference information between the first food ingredient image and the current image;

[0022] When the gray-scale difference information meets a preset difference condition, update the current image to the first food ingredient image, and jump to the step of performing food ingredient edge detection on the first food ingredient image to obtain first region position information of a first region where the preset food ingredient is located.

[0023] In an alternative embodiment, after capturing a second food ingredient image of the preset acquisition region by using the preset imaging device after adjusting the focal length, the method further includes:

[0024] Display the second food ingredient image on a preset interface;

[0025] After cropping the second food ingredient image based on the second region position information to obtain a target food ingredient image corresponding to the target food ingredient, the method further includes:

[0026] Update the second ingredient image in the preset interface to the target ingredient image.

[0027] In an alternative embodiment, the preset ingredient includes at least one ingredient, and the cropping of the second ingredient image based on the first region position information to obtain the preset ingredient image corresponding to the preset ingredient includes:

[0028] In response to an overall viewing instruction for the at least one ingredient, crop the second ingredient image based on the first region position information to obtain the preset ingredient image corresponding to the preset ingredient.

[0029] In an alternative embodiment, the preset region position information includes the length information of the preset acquisition region, the width information of the preset acquisition region, and the central position information of the preset acquisition region; the first region position information includes the upper left corner position information of the first region, the lower right corner position information of the first region, the horizontal position information of the first region, and the vertical position information of the first region;

[0030] The determination of the magnification ratio of the preset acquisition region relative to the first region based on the preset region position information corresponding to the preset acquisition region and the first region position information includes:

[0031] According to the central position information and the horizontal position information, determine the minimum horizontal distance between the central position corresponding to the central position information and the first region;

[0032] According to the central position information and the vertical position information, determine the minimum vertical distance between the central position and the first region;

[0033] According to the minimum horizontal distance and the length information, determine the horizontal magnification ratio of the preset acquisition region relative to the first region;

[0034] According to the minimum vertical distance and the width information, determine the vertical magnification ratio of the preset acquisition region relative to the first region;

[0035] Based on the horizontal magnification ratio and the vertical magnification ratio, determine the magnification ratio.

[0036] In an alternative embodiment, the ingredient edge detection of the first ingredient image to obtain the first region position information of the first region where the preset ingredient is located includes:

[0037] Input the first food ingredient image into an edge detection network for food ingredient edge detection to obtain the first region position information of the first region where the preset food ingredient is located. The edge detection network is a deep learning network for food ingredient edge detection.

[0038] According to a second aspect of the embodiments of the present disclosure, there is provided a food ingredient image acquisition device, including:

[0039] A first food ingredient image acquisition module, configured to, during the process of cooking a preset food ingredient using a target electrical appliance, based on a preset imaging device, capture the preset acquisition region in the target electrical appliance to obtain a first food ingredient image, where the preset acquisition region includes the preset food ingredient;

[0040] A first region position information acquisition module, configured to perform food ingredient edge detection on the first food ingredient image to obtain the first region position information of the first region where the preset food ingredient is located;

[0041] A magnification ratio acquisition module, configured to determine the magnification ratio of the preset acquisition region relative to the first region based on the preset region position information corresponding to the preset acquisition region and the first region position information;

[0042] A focal length adjustment module, configured to adjust the focal length corresponding to the preset imaging device based on the magnification ratio;

[0043] A second food ingredient image acquisition module, configured to, based on the preset imaging device with the adjusted focal length, capture the preset acquisition region to obtain a second food ingredient image;

[0044] A preset food ingredient image acquisition module, configured to crop the second food ingredient image based on the first region position information to obtain a preset food ingredient image corresponding to the preset food ingredient.

[0045] In an optional embodiment, the preset food ingredient includes multiple food ingredients, and the device includes:

[0046] A second region position information acquisition module, configured to, in response to a viewing instruction for a target food ingredient among the multiple food ingredients, obtain the second region position information of the second region where the target food ingredient is located;

[0047] A target food ingredient image acquisition module, configured to crop the second food ingredient image based on the second region position information to obtain a target food ingredient image corresponding to the target food ingredient.

[0048] In an optional embodiment, the second region position information acquisition module includes:

[0049] A positioning location information acquisition unit, configured to trigger the viewing instruction and acquire the positioning location information of the target ingredient when detecting a positioning operation for the target ingredient in the second ingredient image;

[0050] A second area location information acquisition unit, configured to determine the second area location information based on the positioning location information.

[0051] In an optional embodiment, the apparatus further includes:

[0052] A preset contour indication information acquisition module, configured to acquire the preset contour indication information of the preset ingredient;

[0053] A grayscale information acquisition module, configured to acquire the first grayscale information of the first ingredient image and the second grayscale information of the current image when the preset contour indication information indicates that the preset ingredient belongs to an ingredient whose contour changes during cooking, where the current image is an image captured in real time by the preset imaging device during the cooking process;

[0054] A grayscale difference information acquisition module, configured to determine the grayscale difference information between the first ingredient image and the current image based on the first grayscale information and the second grayscale information;

[0055] A first ingredient image update module, configured to update the current image to the first ingredient image and jump to the step of the first area location information acquisition module when the grayscale difference information meets a preset difference condition.

[0056] In an optional embodiment, after the second ingredient image acquisition module, the apparatus further includes:

[0057] A second ingredient image display module, configured to display the second ingredient image on a preset interface;

[0058] After the target ingredient image acquisition module, the apparatus further includes:

[0059] A target ingredient image display module, configured to update the second ingredient image in the preset interface to the target ingredient image.

[0060] In an optional embodiment, the preset ingredient includes at least one ingredient, and the preset ingredient image acquisition module includes:

[0061] A preset ingredient image acquisition unit, configured to crop the second ingredient image based on the first area location information in response to an overall viewing instruction for the at least one ingredient, to obtain a preset ingredient image corresponding to the preset ingredient.

[0062] In an optional embodiment, the preset area position information includes the length information of the preset acquisition area, the width information of the preset acquisition area, and the center position information of the preset acquisition area; the first area position information includes the upper left corner position information of the first area, the lower right corner position information of the first area, the horizontal position information of the first area, and the vertical position information of the first area;

[0063] The magnification ratio obtaining module includes:

[0064] A minimum horizontal distance obtaining unit, configured to determine the minimum horizontal distance between the center position corresponding to the center position information and the first area according to the center position information and the horizontal position information;

[0065] A minimum vertical distance obtaining unit, configured to determine the minimum vertical distance between the center position and the first area according to the center position information and the vertical position information;

[0066] A horizontal magnification ratio obtaining unit, configured to determine the horizontal magnification ratio of the preset acquisition area relative to the first area according to the minimum horizontal distance and the length information;

[0067] A vertical magnification ratio obtaining unit, configured to determine the vertical magnification ratio of the preset acquisition area relative to the first area according to the minimum vertical distance and the width information;

[0068] A magnification ratio obtaining unit, configured to determine the magnification ratio based on the horizontal magnification ratio and the vertical magnification ratio.

[0069] In an optional embodiment, the first area position information obtaining module includes:

[0070] A first area position information obtaining unit, configured to input the first food ingredient image into an edge detection network for food ingredient edge detection to obtain the first area position information of the first area where the preset food ingredient is located, and the edge detection network is a deep learning network for food ingredient edge detection.

[0071] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including: a processor; a memory for storing executable instructions of the processor; wherein, the processor is configured to execute the instructions to implement the method according to any one of the first aspects above.

[0072] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, enabling the electronic device to execute any one of the methods in the food ingredient image acquisition method of the embodiments of the present disclosure.

[0073] According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product including instructions which, when running on a computer, cause the computer to execute the method described in any one of the above first aspects.

[0074] The technical solutions provided by the embodiments of the present disclosure at least bring the following beneficial effects:

[0075] During the process of obtaining the food ingredient image, the regional position information of the food ingredient is obtained according to object detection, ensuring the accuracy of the recognition of the regional position information of the food ingredient. And based on the preset regional position information corresponding to the preset acquisition area and the first regional position information where the preset food ingredient is located, the magnification ratio of the target area is calculated, and the camera focal length is automatically adjusted based on the magnification ratio. It is possible to reduce the background area ratio on the premise of completely capturing the food ingredient. And according to the regional position information of the food ingredient, the background area is cropped, which can further reduce the background area, thus ensuring the accuracy of the food ingredient image acquisition, enabling the user to more effectively and timely view the cooking status of the food ingredient, and improving the user experience.

[0076] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure and do not constitute an improper limitation of the present disclosure.

[0078] Figure 1 is a flowchart of a method for obtaining a food ingredient image shown according to an exemplary embodiment;

[0079] Figure 2 is a schematic diagram of a process for obtaining a target food ingredient image shown according to an exemplary embodiment;

[0080] Figure 3 is a schematic diagram of obtaining second regional position information shown according to an exemplary embodiment;

[0081] Figure 4 is a flowchart of a method for obtaining a food ingredient image in the case where the contour of the food ingredient changes shown according to an exemplary embodiment;

[0082] Figure 5 is a flowchart of obtaining a target food ingredient image in the case where there are multiple food ingredients in the preset food ingredient shown according to an exemplary embodiment;

[0083] Figure 6 is a block diagram of a device for obtaining a food ingredient image shown according to an exemplary embodiment;

[0084] Figure 7 It is a block diagram of an electronic device for obtaining food ingredient images shown according to an exemplary embodiment. Detailed implementation manners

[0085] To enable those of ordinary skill in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0086] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned accompanying drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0087] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in the present disclosure are all information and data authorized by the user or fully authorized by all parties.

[0088] Please refer to Figure 1 , Figure 1 It is a flowchart of a method for obtaining food ingredient images shown according to an exemplary embodiment. Specifically, the method may include the following steps:

[0089] Step S101, during the process of cooking a preset food ingredient using a target electrical appliance, based on a preset imaging device, photograph a preset acquisition area in the target electrical appliance to obtain a first food ingredient image.

[0090] In a specific embodiment, the above-mentioned preset food ingredient may be all the food ingredients inside the target electrical appliance. In an optional embodiment, the above-mentioned preset food ingredient may only include one food ingredient. In an optional embodiment, the above-mentioned preset food ingredient may include multiple food ingredients.

[0091] In a specific embodiment, the above-mentioned preset imaging device may be an imaging device fixedly installed inside the target electrical appliance. Optionally, the above-mentioned preset imaging device may be a camera fixedly installed inside the target electrical appliance and shooting from a default perspective.

[0092] In a specific embodiment, the above-mentioned preset acquisition area includes the preset food ingredient.

[0093] In a specific embodiment, the above first food ingredient image is the first frame image when the target appliance starts cooking the preset food ingredient. Specifically, the above first food ingredient image can be collected by the above preset camera device at the default viewing angle. Optionally, the first food ingredient image can be displayed on the preset interface.

[0094] Step S103: Perform food ingredient edge detection on the first food ingredient image to obtain the first region position information of the first region where the preset food ingredient is located.

[0095] In a specific embodiment, the above first region is the smallest rectangular region tangent to the bounding box of the above preset food ingredient.

[0096] In a specific embodiment, the above first region position information includes the upper left corner position information of the above first region, the lower right corner position information of the above first region, the horizontal position information of the above first region, and the vertical position information of the above first region. In a specific embodiment, the above upper left corner position information can be the upper left corner coordinates of the above smallest rectangular region, the above lower right corner position information can be the lower right corner coordinates of the above smallest rectangular region, the above horizontal position information can be the horizontal coordinates of two points on the above smallest rectangular region that are on the same horizontal line as the center point of the preset acquisition region, and the above vertical position information can be the vertical coordinates of two points on the above smallest rectangular region that are on the same vertical line as the center point of the preset acquisition region.

[0097] In a specific embodiment, the above performing food ingredient edge detection on the first food ingredient image to obtain the first region position information of the first region where the preset food ingredient is located includes:

[0098] Input the first food ingredient image into an edge detection network for food ingredient edge detection to obtain the first region position information of the first region where the preset food ingredient is located.

[0099] In a specific embodiment, the above edge detection network is a deep learning network for performing food ingredient edge detection. In an alternative embodiment, the above edge detection network can be trained based on multiple sample images and the corresponding labeled region position information of each sample image. In a specific embodiment, the above sample images can be images containing food ingredients. In a specific embodiment, the above labeled region position information can be the region position information of the food ingredients in each sample image.

[0100] In a specific embodiment, based on multiple sample images and the position information of the labeled regions corresponding to each sample image, training the edge detection model to be trained for food ingredient edge detection to obtain a preset edge detection model may include: determining a current sample image from the multiple sample images, inputting the current sample image into the edge detection model to be trained for food ingredient edge detection to obtain predicted region position information; determining the food ingredient edge detection loss according to the predicted region position information and the labeled region position information; and training the edge detection model to be trained based on the food ingredient edge detection loss to obtain a preset edge detection model.

[0101] In a specific embodiment, the current sample image may be the training data (partial sample images) of the current training cycle. Specifically, the current sample image may be randomly determined from the multiple sample images, or a part of the sample images that have not participated in model training may be randomly selected from the multiple sample images as the current sample image; the position information of the labeled region corresponding to any sample image may be used to indicate the region position of the food ingredient in the sample image.

[0102] In a specific embodiment, determining the food ingredient edge detection loss according to the predicted region position information and the labeled region position information may include combining a preset loss function to determine the food ingredient edge detection loss of the predicted region position information and the labeled region position information; specifically, the food ingredient edge detection loss may characterize the performance of the current edge detection model to be trained in detecting region position information. Specifically, the preset loss function may be set in combination with actual applications.

[0103] In a specific embodiment, training the edge detection model to be trained based on the food ingredient edge detection loss to obtain a preset edge detection model may include: combining the gradient descent method and the food ingredient edge detection loss to update the model parameters of the edge detection model to be trained, and based on the updated edge detection model to be trained, repeating the above cyclic iteration operations from determining the current sample image from the multiple sample images to updating the model parameters of the edge detection model to be trained until a preset convergence condition is met, and the edge detection model to be trained corresponding to when the preset convergence condition is met is the preset edge detection model.

[0104] In a specific embodiment, the preset convergence condition may be set in combination with actual applications. For example, the number of executions of the cyclic iteration operation reaches a preset number, the food ingredient edge detection loss is less than a specified threshold, etc. Specifically, it may be set in combination with the training speed and model accuracy requirements.

[0105] Step S105, determine the magnification ratio of the preset acquisition area relative to the first area based on the preset area position information corresponding to the preset acquisition area and the first area position information.

[0106] In a specific embodiment, the above-mentioned preset area position information includes the length information of the preset acquisition area, the width information of the preset acquisition area, and the central position information of the preset acquisition area. In a specific embodiment, the length information of the above-mentioned preset acquisition area may be the length of the preset acquisition area, denoted as w; the width information of the above-mentioned preset acquisition area may be the width of the preset acquisition area, denoted as h; the central position information of the above-mentioned preset acquisition area may be the center point coordinates of the preset acquisition area.

[0107] In a specific embodiment, the determination of the magnification ratio of the preset acquisition area relative to the first area based on the preset area position information corresponding to the preset acquisition area and the first area position information includes:

[0108] According to the central position information and the horizontal position information, determine the minimum horizontal distance between the central position corresponding to the central position information and the first area;

[0109] According to the central position information and the vertical position information, determine the minimum vertical distance between the central position and the first area;

[0110] According to the minimum horizontal distance and the length information, determine the horizontal magnification ratio of the preset acquisition area relative to the first area;

[0111] According to the minimum vertical distance and the width information, determine the vertical magnification ratio of the preset acquisition area relative to the first area;

[0112] Based on the horizontal magnification ratio and the vertical magnification ratio, determine the magnification ratio.

[0113] In a specific embodiment, the determination of the minimum horizontal distance between the central position corresponding to the central position information and the first area according to the central position information and the horizontal position information may be to determine the minimum horizontal distance according to the center point coordinates of the above-mentioned preset acquisition area and the horizontal coordinates of two points on the above-mentioned minimum rectangular area that are on the same horizontal line as the center point of the preset acquisition area. Specifically, calculate the horizontal distances from two points on the minimum rectangular area where the preset food ingredient is located and on the same horizontal line as the center point of the preset acquisition area to the center point of the preset acquisition area, and take the smaller of the two as the minimum horizontal distance d1.

[0114] In a specific embodiment, the determination of the minimum vertical distance between the central position and the first area according to the central position information and the vertical position information may be to determine the minimum vertical distance according to the center point coordinates of the above-mentioned preset acquisition area and the vertical coordinates of two points on the above-mentioned minimum rectangular area that are on the same vertical line as the center point of the preset acquisition area. Specifically, calculate the vertical distances from two points on the minimum rectangular area where the preset food ingredient is located and on the same vertical line as the center point of the preset acquisition area to the center point of the preset acquisition area, and take the smaller of the two as the minimum vertical distance d2.

[0115] In a specific embodiment, the horizontal magnification ratio of the preset acquisition area relative to the first area determined according to the minimum horizontal distance and length information may be to determine the horizontal magnification ratio of the preset acquisition area relative to the first area according to the minimum horizontal distance d1 and the length w of the preset acquisition area, denoted as

[0116] In a specific embodiment, the horizontal vertical ratio of the preset acquisition area relative to the first area determined according to the minimum vertical distance and width information may be to determine the vertical magnification ratio of the preset acquisition area relative to the first area according to the minimum vertical distance d2 and the width h of the preset acquisition area, denoted as

[0117] In a specific embodiment, the magnification ratio determined based on the horizontal magnification ratio and the vertical magnification ratio may be to take the horizontal magnification ratio and the vertical magnification ratio The smaller value among them is the magnification ratio α.

[0118] Step S107: Adjust the focal length corresponding to the preset imaging device based on the magnification ratio.

[0119] In a specific embodiment, adjusting the focal length corresponding to the preset imaging device based on the magnification ratio may be to select an appropriate optical zoom level according to the calculated magnification ratio α, and each level corresponds to a different focal length range. Set the focal length of the camera to the optical zoom level that can best adapt to the display ratio of the food ingredients in the image. The larger the magnification ratio, the higher multiple optical zoom level needs to be selected to ensure that the food ingredients can be clearly and completely presented in the image.

[0120] Step S109: Take a picture of the preset acquisition area based on the preset imaging device after adjusting the focal length to obtain a second food ingredient image.

[0121] In a specific embodiment, when the preset imaging device after adjusting the focal length takes a picture of the preset acquisition area, it can focus more on the food ingredients and take a clearer picture of the food ingredients.

[0122] Specifically, taking a picture of the preset acquisition area based on the preset imaging device after adjusting the focal length to obtain a second food ingredient image may be that when taking a picture of the preset acquisition area based on the preset imaging device after adjusting the focal length, it can focus more on the preset food ingredients, so that the second food ingredient image taken can display the preset food ingredients and reduce the background area.

[0123] In a specific embodiment, after obtaining the above second food ingredient image, it is displayed on a preset interface.

[0124] Step S111: Crop the second ingredient image based on the first region position information to obtain a preset ingredient image corresponding to the preset ingredient.

[0125] In a specific embodiment, the above-mentioned preset ingredient image is an image that completely displays the preset ingredient with the background area reduced to the minimum proportion.

[0126] In a specific embodiment, the above-mentioned cropping the second ingredient image based on the first region position information to obtain a preset ingredient image corresponding to the preset ingredient can be cropping the second ingredient image based on the position information of the smallest rectangular region tangent to the bounding box of the preset ingredient to obtain an image that completely displays the preset ingredient with the background area reduced to the minimum proportion.

[0127] In a specific embodiment, the above-mentioned preset ingredient includes at least one ingredient. Cropping the second ingredient image based on the first region position information to obtain a preset ingredient image corresponding to the preset ingredient includes:

[0128] In response to an overall viewing instruction for at least one ingredient, crop the second ingredient image based on the first region position information to obtain a preset ingredient image corresponding to the preset ingredient.

[0129] In a specific embodiment, the above-mentioned overall viewing instruction can be an instruction for viewing the overall preset ingredient triggered by the program. In a specific embodiment, when the display duration of the second ingredient image reaches a preset specific duration, if the program does not receive a viewing instruction for the target ingredient, the program automatically responds to the overall viewing instruction for at least one ingredient. In a specific embodiment, the preset specific duration can be set in advance.

[0130] In the above embodiment, the automatic response of the program can ensure that the processing and analysis of the ingredient image can continue without a user instruction, thus maintaining the continuity and efficiency of the system. And automatically responding to the overall viewing instruction can provide a comprehensive perspective before the user makes a specific selection, helping the user obtain more comprehensive information and contributing to the comprehensive analysis of the overall state of the ingredient or the cooking preparation situation. When the user does not operate on a specific ingredient within the preset time, the system automatically switches to the overall viewing mode, avoiding user waiting or unnecessary operation steps and enhancing the user's operation experience and efficiency.

[0131] In a specific embodiment, as Figure 2 shown, when the above-mentioned preset ingredient includes multiple ingredients, the above method further includes:

[0132] Step S201: In response to a viewing instruction for the target ingredient among the multiple ingredients, obtain the second region position information of the second region where the target ingredient is located.

[0133] In a specific embodiment, the above viewing instruction may be a viewing instruction for a target ingredient triggered by a user. Optionally, the viewing instruction may be triggered by touch operation methods such as clicking or long-pressing on the area where the target ingredient is located in the second ingredient image.

[0134] In a specific embodiment, the above second region position information may be the bounding box position information where the target ingredient is located.

[0135] Step S203: Based on the second region position information, crop the second ingredient image to obtain a target ingredient image corresponding to the target ingredient.

[0136] In a specific embodiment, the above-mentioned cropping the second ingredient image based on the second region position information to obtain a target ingredient image corresponding to the target ingredient may be cropping the second ingredient image based on the bounding box position information of the target ingredient to obtain the target ingredient image, so that the target ingredient image can completely display the target ingredient, and the proportion of other ingredients or background regions is reduced to the lowest.

[0137] In the above embodiment, if the user selects the target ingredient, then according to the bounding box position information where the target ingredient is located, the second ingredient image is cropped to obtain the target ingredient image. Through the accurate bounding box position information, the cropping operation can precisely intercept the part of the target ingredient, avoiding background interference and irrelevant information, thereby ensuring that the obtained cropped image contains the complete ingredient content. And the cropping operation can effectively reduce the complexity of image processing and analysis. By only intercepting the part of the ingredient, computing resources and processing time can be saved.

[0138] In a specific embodiment, as Figure 3 shown, the above-mentioned obtaining the second region position information of the second region where the target ingredient is located includes:

[0139] Step S301: When a positioning operation for the target ingredient in the second ingredient image is detected, trigger a viewing instruction and obtain the positioning position information of the target ingredient.

[0140] In a specific embodiment, the above-mentioned positioning position information of the target ingredient may be the coordinate information of the operation point of the positioning operation.

[0141] Step S303: Based on the positioning position information, determine the second region position information.

[0142] In a specific embodiment, the above-mentioned determining the second region position information based on the positioning position information may be that the preset interface sends the coordinate point of the clicked target ingredient to the main control, and the main control calculates the bounding box position information where the target ingredient is located through the coordinate point.

[0143] In the above embodiments, due to the use of the actual coordinate information of the food ingredients, the position of the bounding box can more accurately reflect the true boundary of the food ingredients, accurately locate the position and range of the food ingredients in the image, and avoid the problems of blurred or excessive bounding boxes. Moreover, it has strong applicability, is not affected by the complexity or changes in the shape of the food ingredients, and can stably calculate the bounding box that matches the contour of the food ingredients. By directly using coordinate points to determine the bounding box, additional calculation and processing steps can be reduced, thereby improving the operation efficiency and processing speed.

[0144] In a specific embodiment, as Figure 4 shown, the above method further includes:

[0145] Step S401, obtaining the preset contour indication information of the preset food ingredients.

[0146] In a specific embodiment, the above preset contour indication information can be used to indicate whether the contour of the preset food ingredients will change during the cooking process. Specifically, the above preset contour indication information can be obtained from the food processing manual. Specifically, the above food processing manual records information on whether the contour of the food ingredients will change during the cooking process for different recipes.

[0147] Step S403, in the case where the preset contour indication information indicates that the preset food ingredients belong to the food ingredients whose contours change during the cooking process, obtaining the first grayscale information of the first food ingredient image and the second grayscale information of the current image.

[0148] In a specific embodiment, the above current image is an image captured in real time by a preset camera device during the cooking process.

[0149] Step S405, based on the first grayscale information and the second grayscale information, determining the grayscale difference information between the first food ingredient image and the current image.

[0150] In a specific embodiment, the above grayscale difference information can be the grayscale difference value between the first food ingredient image and the current image.

[0151] Step S407, in the case where the grayscale difference information meets the preset difference condition, updating the current image to the first food ingredient image, and jumping to the step of performing food ingredient edge detection on the first food ingredient image in the above S103 to obtain the first region position information of the first region where the preset food ingredients are located.

[0152] In a specific embodiment, the above preset difference condition may be a lower limit condition for the difference in the outline of the preset food ingredients that can be completely displayed on the preset interface. In a specific embodiment, the preset difference condition may be a preset gray-scale difference value. If the gray-scale difference value between the current image and the first food ingredient image is greater than the preset gray-scale difference value, the outline of the food ingredient in the current image cannot be completely displayed on the preset interface. On the contrary, if the gray-scale difference value between the current image and the first food ingredient image is less than or equal to the preset gray-scale difference value, the outline of the food ingredient in the current image can be completely displayed on the preset interface. In a specific embodiment, the preset difference condition can be set in advance. In a specific embodiment, the above-mentioned updating the current image to the first food ingredient image when the gray-scale difference information meets the preset difference condition may be when the gray-scale difference value between the current image and the first food ingredient image is greater than the preset gray-scale difference value, that is, when the outline of the preset food ingredient changes and cannot be completely displayed on the preset interface, updating the current image to the first food ingredient image.

[0153] In the above embodiment, by monitoring the change of the gray-scale difference value between frames, the system can respond to the change of the food ingredient area in real time, making the update of the target area more immediate and dynamic, and requiring less hardware resources and lower cost. The change of the gray-scale difference value can effectively reflect the movement or deformation of the food ingredient area, so as to accurately update the target area in the image and ensure the accurate capture of the latest position and shape of the food ingredient. And by updating the target area through the change of the gray-scale difference value, the stability of the system against factors such as light change and background interference can be enhanced, and the accuracy of food ingredient recognition and tracking can be improved.

[0154] In a specific embodiment, after the above-mentioned second food ingredient image is obtained by photographing the preset acquisition area with the preset imaging device after adjusting the focal length, the method further includes:

[0155] Display the second food ingredient image on the preset interface;

[0156] After the target food ingredient image corresponding to the target food ingredient is obtained by cropping the second food ingredient image based on the second area position information, the method further includes:

[0157] Update the second food ingredient image in the preset interface to the target food ingredient image.

[0158] In a specific embodiment, the above-mentioned displaying the second food ingredient image on the preset interface may be updating the first food ingredient image to the second food ingredient image on the preset interface.

[0159] In the above embodiments, by updating the food ingredient image in a preset interface in a timely manner, the user can see the status and changes of the food ingredients in real time during the current cooking process, enhancing the user's sense of control and participation in the food ingredient processing process, improving the interaction experience between the user and the cooking device, and making the operation more intuitive and friendly. Moreover, it can provide real-time feedback on the actual status of the food ingredients during cooking, such as the degree of doneness, color change, etc., to help the user adjust the cooking time and temperature to ensure that the food ingredients achieve the ideal cooking effect.

[0160] In an alternative embodiment, as Figure 5 shown, Figure 5 is a schematic flowchart of obtaining a target food ingredient image in the case where there are multiple food ingredients in a preset food ingredient. Specifically, it may include the following steps:

[0161] Step S501, during the process of cooking the preset food ingredient using the target appliance, based on a preset imaging device, photograph a preset acquisition area in the target appliance to obtain a first food ingredient image;

[0162] Step S503, obtain preset contour indication information of the preset food ingredient;

[0163] Step S505, in the case where the preset contour indication information indicates that the preset food ingredient is a food ingredient whose contour changes during cooking, obtain the first grayscale information of the first food ingredient image and the second grayscale information of the current image;

[0164] Step S507, based on the first grayscale information and the second grayscale information, determine the grayscale difference information between the first food ingredient image and the current image;

[0165] Step S509, in the case where the grayscale difference information meets a preset difference condition, update the current image to the first food ingredient image;

[0166] Step S511, perform food ingredient edge detection on the first food ingredient image to obtain first area position information of the first area where the preset food ingredient is located;

[0167] Step S513, based on the preset area position information corresponding to the preset acquisition area and the first area position information, determine the magnification ratio of the preset acquisition area relative to the first area;

[0168] Step S515, based on the magnification ratio, adjust the focal length corresponding to the preset imaging device;

[0169] Step S517, based on the preset imaging device with the adjusted focal length, photograph the preset acquisition area to obtain a second food ingredient image;

[0170] Step S519: When a positioning operation on the target ingredient in the second ingredient image is detected, trigger a viewing instruction and obtain the positioning position information of the target ingredient.

[0171] Step S521: Based on the positioning position information, determine the second region position information.

[0172] Step S523: Based on the second region position information, crop the second ingredient image to obtain the target ingredient image corresponding to the target ingredient.

[0173] Figure 6 It is a block diagram of an ingredient image acquisition device shown according to an exemplary embodiment. Refer to Figure 6 The device includes:

[0174] The first ingredient image acquisition module 610 is configured to, during the process of cooking a preset ingredient using a target electrical appliance, capture a first ingredient image of a preset acquisition area in the target electrical appliance based on a preset imaging device, where the preset acquisition area includes the preset ingredient.

[0175] The first region position information acquisition module 620 is configured to perform ingredient edge detection on the first ingredient image to obtain the first region position information of the first region where the preset ingredient is located.

[0176] The magnification ratio acquisition module 630 is configured to determine the magnification ratio of the preset acquisition area relative to the first region based on the preset region position information corresponding to the preset acquisition area and the first region position information.

[0177] The focal length adjustment module 640 is configured to adjust the focal length corresponding to the preset imaging device based on the magnification ratio.

[0178] The second ingredient image acquisition module 650 is configured to capture a second ingredient image of the preset acquisition area based on the preset imaging device with the adjusted focal length.

[0179] The preset ingredient image acquisition module 660 is configured to crop the second ingredient image based on the first region position information to obtain the preset ingredient image corresponding to the preset ingredient.

[0180] In an optional embodiment, the above-mentioned preset ingredient includes multiple ingredients, and the device includes:

[0181] The second region position information acquisition module is configured to, in response to a viewing instruction for the target ingredient among the multiple ingredients, obtain the second region position information of the second region where the target ingredient is located.

[0182] The target ingredient image acquisition module is configured to crop the second ingredient image based on the second region position information to obtain the target ingredient image corresponding to the target ingredient.

[0183] In an optional embodiment, the above-mentioned second area position information acquisition module includes:

[0184] A positioning position information acquisition unit: configured to trigger a viewing instruction and acquire the positioning position information of the target ingredient when a positioning operation on the target ingredient in the second ingredient image is detected;

[0185] A second area position information acquisition unit: configured to determine the second area position information based on the positioning position information.

[0186] In an optional embodiment, the above-mentioned device further includes:

[0187] A preset contour indication information acquisition module: configured to acquire the preset contour indication information of the preset ingredient;

[0188] A grayscale information acquisition module: configured to acquire the first grayscale information of the first ingredient image and the second grayscale information of the current image when the preset contour indication information indicates that the preset ingredient is an ingredient whose contour changes during cooking, and the current image is an image captured in real time by a preset imaging device during cooking;

[0189] A grayscale difference information acquisition module: configured to determine the grayscale difference information between the first ingredient image and the current image based on the first grayscale information and the second grayscale information;

[0190] A first ingredient image update module: configured to update the current image to the first ingredient image and jump to the step of the above-mentioned first area position information acquisition module 620 when the grayscale difference information meets a preset difference condition.

[0191] In an optional embodiment, after the above-mentioned second ingredient image acquisition module, the above-mentioned device further includes:

[0192] A second ingredient image display module: configured to display the second ingredient image on a preset interface;

[0193] After the above-mentioned target ingredient image acquisition module, the above-mentioned device further includes:

[0194] A target ingredient image display module: configured to update the second ingredient image in the preset interface to the target ingredient image.

[0195] In an optional embodiment, the above-mentioned preset ingredient includes at least one ingredient, and the above-mentioned preset ingredient image acquisition module 660 includes:

[0196] A preset ingredient image acquisition unit: configured to crop the second ingredient image based on the first area position information in response to an overall viewing instruction for at least one ingredient, and obtain a preset ingredient image corresponding to the preset ingredient.

[0197] In an alternative embodiment, the above-mentioned preset area position information includes the length information of the above-mentioned preset acquisition area, the width information of the above-mentioned preset acquisition area, and the central position information of the above-mentioned preset acquisition area; the above-mentioned first area position information includes the upper left corner position information of the above-mentioned first area, the lower right corner position information of the above-mentioned first area, the horizontal position information of the above-mentioned first area, and the vertical position information of the above-mentioned first area;

[0198] The above-mentioned magnification ratio obtaining module 630 includes:

[0199] Minimum horizontal distance obtaining unit: configured to determine the minimum horizontal distance between the central position corresponding to the central position information and the first area according to the central position information and the horizontal position information;

[0200] Minimum vertical distance obtaining unit: configured to determine the minimum vertical distance between the central position and the first area according to the central position information and the vertical position information;

[0201] Horizontal magnification ratio obtaining unit: configured to determine the horizontal magnification ratio of the preset acquisition area relative to the first area according to the minimum horizontal distance and the length information;

[0202] Vertical magnification ratio obtaining unit: configured to determine the vertical magnification ratio of the preset acquisition area relative to the first area according to the minimum vertical distance and the width information;

[0203] Magnification ratio obtaining unit: configured to determine the magnification ratio based on the horizontal magnification ratio and the vertical magnification ratio.

[0204] In an alternative embodiment, the above-mentioned first area position information obtaining module 620 includes:

[0205] First area position information example obtaining unit: configured to input the first food ingredient image into an edge detection network for food ingredient edge detection to obtain the first area position information of the first area where the preset food ingredient is located, and the edge detection network is a deep learning network for food ingredient edge detection.

[0206] Regarding the device in the above-mentioned embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment related to the method, and will not be elaborated here.

[0207] Figure 7 is a block diagram of an electronic device for food ingredient image acquisition shown according to an exemplary embodiment. The electronic device may be a terminal, and its internal structure diagram may be as Figure 7As shown in the figure. The electronic device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it realizes a method for obtaining food ingredient images. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the electronic device, or an external keyboard, touchpad, or mouse, etc.

[0208] Those skilled in the art can understand that Figure 7 the structure shown in the figure is only a block diagram of some structures related to the solution of the present disclosure, and does not constitute a limitation on the electronic device to which the solution of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0209] In an exemplary embodiment, there is also provided an electronic device, including: a processor; a memory for storing executable instructions of the processor; wherein, the processor is configured to execute the instructions to realize the method for obtaining food ingredient images as in the embodiments of the present disclosure.

[0210] In an exemplary embodiment, there is also provided a computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the electronic device, enabling the electronic device to execute the method for obtaining food ingredient images in the embodiments of the present disclosure.

[0211] In an exemplary embodiment, there is also provided a computer program product containing instructions, when it runs on a computer, enabling the computer to execute the method for obtaining food ingredient images in the embodiments of the present disclosure.

[0212] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0213] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0214] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A method for obtaining a food ingredient image, characterized in that, Including: During the process of cooking preset ingredients using a target appliance, based on a preset camera device, photograph a preset collection area in the target appliance to obtain a first ingredient image, where the preset collection area includes the preset ingredients; Perform ingredient edge detection on the first ingredient image to obtain first area position information of a first area where the preset ingredients are located; Based on the preset area position information corresponding to the preset collection area and the first area position information, determine a magnification ratio of the preset collection area relative to the first area; Based on the magnification ratio, adjust the focal length corresponding to the preset camera device; Based on the preset camera device after adjusting the focal length, photograph the preset collection area to obtain a second ingredient image; Based on the first area position information, crop the second ingredient image to obtain a preset ingredient image corresponding to the preset ingredients.

2. The method according to claim 1, wherein The preset ingredients include multiple ingredients, and the method includes: In response to a viewing instruction for a target ingredient among the multiple ingredients, obtain second area position information of a second area where the target ingredient is located; Based on the second area position information, crop the second ingredient image to obtain a target ingredient image corresponding to the target ingredient.

3. The method according to claim 2, wherein The obtaining of the second area position information of the second area where the target ingredient is located includes: When a positioning operation for the target ingredient in the second ingredient image is detected, trigger the viewing instruction and obtain the positioning position information of the target ingredient; Based on the positioning position information, determine the second area position information.

4. The method according to claim 1, wherein The method further includes: Obtain preset contour indication information of the preset ingredients; When the preset contour indication information indicates that the preset ingredients are ingredients whose contours change during the cooking process, obtain first gray-scale information of the first ingredient image and second gray-scale information of a current image, where the current image is an image captured in real time by the preset camera device during the cooking process; Based on the first gray-scale information and the second gray-scale information, determine gray-scale difference information between the first ingredient image and the current image; When the gray-scale difference information meets a preset difference condition, update the current image to the first ingredient image, and jump to the step of performing ingredient edge detection on the first ingredient image to obtain first area position information of a first area where the preset ingredients are located.

5. The method according to claim 1, wherein After obtaining the second ingredient image by photographing the preset collection area based on the preset camera device after adjusting the focal length, the method further includes: Display the second ingredient image on a preset interface; After cropping the second ingredient image based on the second area position information to obtain a target ingredient image corresponding to the target ingredient, the method further includes: Update the second ingredient image in the preset interface to the target ingredient image.

6. The method according to claim 1, wherein The preset ingredients include at least one ingredient, and the cropping of the second ingredient image based on the first area position information to obtain a preset ingredient image corresponding to the preset ingredients includes: In response to an overall viewing instruction for the at least one ingredient, based on the first region position information, crop the second ingredient image to obtain a preset ingredient image corresponding to the preset ingredient.

7. According to the method described in any one of claims 1 to 6, characterized in that, The preset region position information includes the length information of the preset acquisition region, the width information of the preset acquisition region, and the central position information of the preset acquisition region; the first region position information includes the upper left corner position information of the first region, the lower right corner position information of the first region, the horizontal position information of the first region, and the vertical position information of the first region; The determining the magnification ratio of the preset acquisition region relative to the first region based on the preset region position information corresponding to the preset acquisition region and the first region position information includes: According to the central position information and the horizontal position information, determine the minimum horizontal distance between the central position corresponding to the central position information and the first region; According to the central position information and the vertical position information, determine the minimum vertical distance between the central position and the first region; According to the minimum horizontal distance and the length information, determine the horizontal magnification ratio of the preset acquisition region relative to the first region; According to the minimum vertical distance and the width information, determine the vertical magnification ratio of the preset acquisition region relative to the first region; Based on the horizontal magnification ratio and the vertical magnification ratio, determine the magnification ratio.

8. According to the method as claimed in any one of claims 1 to 6, characterized in that, The obtaining the first region position information of the first region where the preset ingredient is located by performing ingredient edge detection on the first ingredient image includes: Input the first ingredient image into an edge detection network for ingredient edge detection to obtain the first region position information of the first region where the preset ingredient is located, and the edge detection network is a deep learning network for performing ingredient edge detection.

9. An apparatus for acquiring an image of a food ingredient, characterized in that, Includes: A first ingredient image acquisition module, configured to, during the process of cooking a preset ingredient using a target electrical appliance, based on a preset imaging device, capture the preset acquisition region in the target electrical appliance to obtain a first ingredient image, where the preset acquisition region includes the preset ingredient; A first region position information acquisition module, configured to perform ingredient edge detection on the first ingredient image to obtain the first region position information of the first region where the preset ingredient is located; A magnification ratio acquisition module, configured to determine the magnification ratio of the preset acquisition region relative to the first region based on the preset region position information corresponding to the preset acquisition region and the first region position information; A focal length adjustment module, configured to adjust the focal length corresponding to the preset imaging device based on the magnification ratio; A second ingredient image acquisition module, configured to, based on the preset imaging device with the adjusted focal length, capture the preset acquisition region to obtain a second ingredient image; A preset ingredient image acquisition module, configured to crop the second ingredient image based on the first region position information to obtain a preset ingredient image corresponding to the preset ingredient.

10. An electronic device, characterized in that, Includes: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the instructions to implement the food ingredient image acquisition method according to any one of claims 1 to 8.

Images