Food material placement position recognition method and device, electronic equipment and storage medium
By using a mask to segment and identify food ingredients and their placement structure within kitchen appliances, the problem of inaccurate food placement identification in existing kitchen appliances is solved, achieving higher identification accuracy and applicability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NINGBO FOTILE KITCHEN WARE CO LTD
- Filing Date
- 2026-04-15
- Publication Date
- 2026-06-12
AI Technical Summary
Existing kitchen appliances that use weight sensors to identify the placement of food are costly and lack sufficient accuracy, resulting in low recognition accuracy.
By acquiring images of the interior of the kitchen utensils, performing image segmentation, identifying the masks of the food placement components and placement structures, determining their positional relationships, and identifying the target placement layer of the food based on a preset placement layer number.
It improves the accuracy and robustness of food placement location recognition, adapts to different camera installation angles, and enhances the applicability of recognition.
Smart Images
Figure CN122200635A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of smart kitchen appliance technology, and in particular to a method, device, electronic device, and storage medium for identifying the placement position of food ingredients. Background Technology
[0002] With the advancement of human life, kitchen appliances are developing towards intelligence. For example, commonly used ovens and steamers can identify the placement of food inside and control the heating temperature and cooking time based on the shelf position of the food. Currently, the shelf position of food is usually identified using weight sensors, requiring one weight sensor to be placed on each shelf. However, this method is costly, and the accuracy of the sensors is greatly affected, making it difficult to accurately identify the placement of food and reducing its practicality. Summary of the Invention
[0003] To address the aforementioned problems in the prior art, this invention discloses a method, apparatus, electronic device, and storage medium for identifying the placement location of food ingredients, which can effectively identify the placement location of food ingredients and improve the accuracy and robustness of food ingredient placement location identification. The technical solution disclosed in this invention is as follows: According to one aspect of the disclosed embodiments of the present invention, a method for identifying the placement position of food ingredients is provided, comprising: Acquire an internal image of the kitchen utensil; the kitchen utensil includes a food placement component and at least two layers of placement structures for placing the food placement component, the food placement component being used to place food, and the placement structures being disposed on both sides inside the kitchen utensil; The internal image of the kitchenware is segmented to obtain a segmentation result; the segmentation result includes a first segmentation mask corresponding to the placement structure and a preset number of placement layers. If the segmentation result also includes a second segmentation mask corresponding to the food placement component, determine the positional relationship between the first segmentation mask corresponding to the placement structure and the second segmentation mask corresponding to the food placement component; Based on the positional relationship and the preset number of placement layers corresponding to the placement structure, the target number of placement layers for the food placement component is determined.
[0004] Optionally, determining the positional relationship between the first segmentation mask corresponding to the placement structure and the second segmentation mask corresponding to the food placement component includes: When the second segmentation mask includes multiple initial segmentation sub-masks, the target segmentation sub-mask corresponding to the food placement component is determined based on the area of the multiple initial segmentation sub-masks; the target segmentation sub-mask is the segmentation sub-mask among the multiple initial segmentation masks whose area is greater than a preset area threshold; Determine the positional relationship between the first segmentation mask corresponding to the placement structure and the target segmentation sub-mask corresponding to the food placement component.
[0005] Optionally, determining the target number of placement layers for the food placement component based on the positional relationship and the preset number of placement layers corresponding to the placement structure includes: When the positional relationship indicates that the first segmentation mask corresponding to any of the placement structures is above the second segmentation mask, a first target segmentation mask is determined from the first segmentation mask corresponding to the placement structure. The target placement layer number is determined based on the preset placement layer number corresponding to the first target segmentation mask.
[0006] Optionally, determining the target number of placement layers for the food placement component based on the positional relationship and the preset number of placement layers corresponding to the placement structure includes: When the positional relationship indicates that the first segmentation mask corresponding to each placement structure is not above the second segmentation mask, and the first segmentation mask corresponding to any placement structure is below the second segmentation mask, the upper limit placement layer number corresponding to the placement structure is determined as the target placement layer number.
[0007] Optionally, determining the positional relationship between the first segmentation mask corresponding to the placement structure and the second segmentation mask corresponding to the food placement component includes: Establish the target coordinate system inside the kitchenware; In the target coordinate system, determine the first coordinate information corresponding to the first segmentation mask and the second coordinate information corresponding to the second segmentation mask; The positional relationship is determined based on the first coordinate information and the second coordinate information.
[0008] Optionally, after determining the target number of layers for the food placement component, the method further includes: Obtain target recipe information, which includes the recommended number of layers for placing the ingredients; If the target placement layer number is different from the recommended placement layer number, a first prompt message is generated; the first prompt message is used to indicate the recommended placement layer number.
[0009] Optionally, the method further includes: If the segmentation result does not include the second segmentation mask corresponding to the food placement component, a second prompt message is generated; the second prompt message is used to indicate that the food placement component has not been placed.
[0010] According to another aspect of the disclosed embodiments of the present invention, a food placement location identification device is provided, comprising: The first acquisition module is used to acquire an image of the interior of the kitchenware; the kitchenware includes a food placement component and at least two layers of placement structures for placing the food placement component, the food placement component is used to place food, and the placement structures are disposed on both sides inside the kitchenware; An image segmentation module is used to segment the internal image of the kitchenware to obtain a segmentation result; the segmentation result includes a first segmentation mask corresponding to the placement structure and a preset number of placement layers. The positional relationship determination module is used to determine the positional relationship between the first segmentation mask corresponding to the placement structure and the second segmentation mask corresponding to the food placement component when the segmentation result also includes the second segmentation mask corresponding to the food placement component. The target placement layer determination module is used to determine the target placement layer of the food placement component based on the positional relationship and the preset placement layer corresponding to the placement structure.
[0011] According to another aspect of the embodiments disclosed in this invention, an electronic device for identifying the placement location of food ingredients is provided, including a processor and a memory, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the food ingredient placement location identification method described in any of the preceding claims.
[0012] According to another aspect of the embodiments disclosed in this invention, a computer-readable storage medium is provided, wherein at least one instruction is stored in the computer storage medium, the at least one instruction being loaded and executed by a processor to implement the food placement location identification method described in any of the preceding claims.
[0013] According to another aspect of the disclosed embodiments of the present invention, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to execute the food placement location identification method described in any of the above-mentioned embodiments of the present invention.
[0014] The food placement location identification method provided by this invention has the following technical effects: This invention acquires an image of the interior of a kitchen utensil, wherein the kitchen utensil includes a food placement component and at least two layers of placement structures for placing the food placement component. The food placement component is used to place food, and the placement structures are disposed on both sides inside the kitchen utensil. The image of the interior of the kitchen utensil is segmented to obtain a segmentation result. The segmentation result includes a first segmentation mask corresponding to the placement structure and a preset number of placement layers. If the segmentation result also includes a second segmentation mask corresponding to the food placement component, the positional relationship between the first segmentation mask corresponding to the placement structure and the second segmentation mask corresponding to the food placement component is determined. Based on the positional relationship and the preset number of placement layers corresponding to the placement structure, the target number of placement layers for the food placement component is determined, thereby effectively identifying the food placement position and improving the accuracy and robustness of food placement position recognition.
[0015] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This is a flowchart illustrating a method for identifying the placement location of food ingredients according to an exemplary embodiment; Figure 2 This is a schematic diagram illustrating a segmentation result according to an exemplary embodiment; Figure 3 This is a schematic diagram illustrating a process for determining positional relationships according to an exemplary embodiment; Figure 4 This is a schematic diagram illustrating a process for determining the number of target placement layers according to an exemplary embodiment; Figure 5 This is a block diagram illustrating a food placement location recognition device according to an exemplary embodiment; Figure 6 This is a block diagram illustrating a terminal electronic device for identifying the placement position of food ingredients according to an exemplary embodiment; Figure 7 This is a block diagram illustrating a server electronic device for identifying the placement location of food ingredients, according to an exemplary embodiment. Detailed Implementation
[0018] To enable those skilled in the art to better understand the technical solutions disclosed in this invention, the technical solutions in the disclosed embodiments will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention disclosed herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.
[0020] The following describes a method for identifying the placement location of food ingredients according to this application. Please refer to [link / reference]. Figure 1 , Figure 1 This is a flowchart illustrating a method for identifying the placement location of food ingredients according to an exemplary embodiment. This specification provides the operational steps of the method as described in the embodiments or flowchart, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual system or server product execution, the method can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiments or drawings. Specifically, as... Figure 1 As shown, the above method may include: S101: Obtain an image of the interior of the kitchenware.
[0021] In one specific embodiment, the kitchen utensil may include a food placement component and at least two layers of placement structures for placing the food placement component. The food placement component is used to place food. The number of food placement layers in the kitchen utensil can be determined by determining the number of placement layers of the food placement component. The placement structures can be located on both sides inside the kitchen utensil (side placement structures, such as side racks) to support the food placement component. The kitchen utensil may have multiple placement positions, specifically including ovens, steamers, etc. Specifically, when the kitchen utensil is an oven, the corresponding food placement component can be a baking tray or a baking rack, and the placement structure can be a grooved structure or a protruding structure corresponding to the food placement component for placing the food placement component. The placement of the food placement component on different layers is related to the food cooking operation and cooking result. By identifying and determining the number of food placement layers, it is possible to detect whether it conforms to the cooking operation of the food.
[0022] Specifically, the internal images of the kitchenware can include internal images of the back and sides of the kitchenware. The internal images of the kitchenware can be captured by a shooting device (such as a camera). The camera can be set inside the front side of the kitchenware, such as inside the door, and can capture images of the back and sides of the kitchenware.
[0023] S103: Perform image segmentation on the internal image of the kitchenware to obtain the segmentation result.
[0024] In one specific embodiment, such as Figure 2 As shown, the segmentation result can be the segmentation result of food placement components and multi-layer placement structures in the internal image of kitchenware. By identifying and segmenting the food placement components and multi-layer placement structures from the internal image of kitchenware, the first segmentation mask (segmentation result) of the placement structure and the corresponding pre-labeled placement layer number label (preset placement layer number), as well as the second segmentation mask of the food placement component (e.g., Figure 2 The area formed by the white dashed line in the middle), or no food placement component / structure was identified, that is, the corresponding segmentation result is empty.
[0025] Optionally, the image segmentation of the interior of the kitchen utensils described above, to obtain the segmentation result, may include: The image of the interior of the kitchen utensils is input into an image segmentation network for image segmentation, resulting in the segmentation of the food placement components in the image of the interior of the kitchen utensils.
[0026] In one specific embodiment, the image segmentation network can be used to perform target recognition and segmentation of food placement components and multi-layered placement structures in images of the interior of kitchen utensils. This image segmentation network is trained using images of the interior of the kitchen utensils with food placement components at different layers, pre-labeled segmentation masks corresponding to the food placement components, and pre-labeled segmentation masks and layer number labels corresponding to the placement structures. Specifically, the network structure of the image segmentation network can be set according to actual application requirements.
[0027] In practical applications, after obtaining the above segmentation results, it is necessary to handle possible anomalies, such as overlapping segmentation masks, which may lead to inaccurate identification of the number of placement layers. By effectively identifying and correcting anomalies, the robustness of the model and the success rate of identification can be improved.
[0028] S105: If the segmentation result also includes the second segmentation mask corresponding to the food placement component, determine the positional relationship between the first segmentation mask corresponding to the placement structure and the second segmentation mask corresponding to the food placement component.
[0029] In one specific embodiment, when a food placement component is detected in an image inside the food component, the segmentation result includes a second segmentation mask corresponding to the food placement component. Simultaneously, the segmentation result also includes a first segmentation mask corresponding to the placement structure. That is, when both the first and second segmentation masks exist, the positional relationship between the first and second segmentation masks is determined, and then the number of placement layers is further determined based on this positional relationship. Specifically, the positional relationship can indicate that the first segmentation mask is above the second segmentation mask, or it can indicate that the first segmentation mask is below the second segmentation mask.
[0030] Optionally, the positional relationship between the first segmentation mask corresponding to the placement structure and the second segmentation mask corresponding to the food placement component may include: Establish a target coordinate system inside the kitchenware; In the target coordinate system, determine the first coordinate information corresponding to the first segmentation mask and the second coordinate information corresponding to the second segmentation mask; The positional relationship is determined based on the first coordinate information and the second coordinate information.
[0031] In one specific embodiment, the target coordinate system can be set according to actual application requirements. For example, the origin can be taken as the vertex inside the kitchenware, and the three connecting edges between the kitchenware can be taken as the x, y, and z axes respectively. Points located within the segmentation mask area can be selected, and the above positional relationship can be determined based on the coordinate information. Specifically, the ordinates of points with the same x-coordinate located in the first and second segmentation mask areas are compared. The segmentation mask corresponding to the point with the larger ordinate is located above the segmentation mask corresponding to the point with the smaller ordinate.
[0032] In an optional embodiment, such as Figure 3 As shown, the positional relationship between the first segmentation mask corresponding to the placement structure and the second segmentation mask corresponding to the food placement component may further include: S301: When the second segmentation mask includes multiple initial segmentation sub-masks, the target segmentation sub-mask corresponding to the food placement component is determined based on the area of the multiple initial segmentation sub-masks.
[0033] In one specific embodiment, the second segmentation mask includes multiple initial segmentation sub-masks, which can indicate that the segmentation masks corresponding to the food placement components overlap. That is, during the target detection and segmentation process, due to missegmentation of lines in the food placement components (such as a grill rack), multiple masks (the aforementioned initial segmentation sub-masks) may be obtained for the food placement components in the image. The target segmentation sub-mask can be the segmentation sub-mask with an area greater than a preset area threshold among the multiple initial segmentation sub-masks. Specifically, the target segmentation mask can be the segmentation sub-mask with the largest area among the multiple initial segmentation masks, and the preset area threshold can be set according to actual application requirements.
[0034] S305: Determine the positional relationship between the first segmentation mask corresponding to the placement structure and the target segmentation sub-mask corresponding to the food placement component.
[0035] Specifically, the positional relationship can indicate that the first segmentation mask is above the target segmentation sub-mask, or it can indicate that the first segmentation mask is below the target segmentation sub-mask.
[0036] S107: Determine the target number of placement layers for the food placement component based on the preset number of placement layers corresponding to the positional relationship and placement structure.
[0037] Specifically, the target placement layer number is the current placement layer number of the identified food placement component, that is, the current placement layer number of the food inside the kitchenware.
[0038] Optional, such as Figure 4 As shown, determining the target number of placement layers for the food placement component based on the preset placement layers corresponding to the positional relationship and placement structure can include: S401: When the positional relationship indicates that the first segmentation mask corresponding to any placement structure is above the second segmentation mask, determine the first target segmentation mask from the first segmentation mask corresponding to the placement structure.
[0039] In one specific embodiment, the first target segmentation mask can be any one of a plurality of first segmentation masks located above the second segmentation mask. Specifically, the first target segmentation mask can be the segmentation mask among the plurality of first segmentation masks whose distance from the second segmentation mask is less than a preset distance threshold, such as the segmentation mask closest to the second segmentation mask. The preset distance threshold can be set according to actual application requirements.
[0040] S403: Determine the target placement layer number based on the preset placement layer number corresponding to the first target segmentation mask.
[0041] In one specific embodiment, the first segmentation mask of the placement structure corresponds to a pre-labeled placement layer number label. By subtracting the corresponding value from the preset placement layer number corresponding to the first target segmentation mask, the current placement layer number of the food placement component can be obtained. For example, if the first target segmentation mask can be the segmentation mask that is closest to the second segmentation mask among multiple first segmentation masks located above the second segmentation mask, the preset placement layer number corresponding to the first target segmentation mask is subtracted by 1, which is the current placement layer number (target placement layer number) of the food placement component.
[0042] Optionally, determining the target number of placement layers for the food placement component based on the preset placement layers corresponding to the positional relationship and placement structure may include: If the positional relationship indicates that the first segmentation mask corresponding to each placement structure is not above the second segmentation mask, and the first segmentation mask corresponding to any placement structure is below the second segmentation mask, then the upper limit placement layer number corresponding to the placement structure is determined as the target placement layer number.
[0043] Specifically, the upper limit of the placement layer can be the maximum number of layers corresponding to the placement structure in the kitchenware. The first segmentation mask corresponding to each placement structure is not above the second segmentation mask, and the first segmentation mask corresponding to any placement structure is below the second segmentation mask. That is, if there is no placement structure above the food placement piece and there is a placement structure below the food placement piece, then the food placement piece can be considered to be on the top layer. Thus, the upper limit of the placement layer corresponding to the placement structure is determined as the target placement layer. In an optional embodiment, after determining the target number of food placement layers, the method may further include: Obtain the target recipe information, which includes the recommended number of layers for placing the ingredients; If the target placement layer differs from the recommended placement layer, a first prompt message is generated.
[0044] In one specific embodiment, the first prompt message can be used to suggest the recommended placement layer. Optionally, after determining the target placement layer of the ingredient, it can be compared with the recommended placement layer of the ingredient in the corresponding recipe. If the two layer numbers are inconsistent, a prompt message can be generated and sent to the user. For example, a prompt message "Currently not the best cooking position" can be generated, and the recommended best cooking position (recommended placement layer) can be sent to the user at the same time.
[0045] In an optional embodiment, the above method may further include: If the segmentation result does not include the second segmentation mask corresponding to the food placement component, a second prompt message is generated.
[0046] In one specific embodiment, the second prompt message can be used to indicate that the food placement component is not placed. Specifically, the segmentation result does not include the segmentation mask corresponding to the food placement component; that is, during the image segmentation process of the internal image of the kitchen utensil, the food placement component in the image was not identified, and its corresponding segmentation mask was not obtained. Specifically, a prompt message can be generated based on the abnormal location result and sent to the user, for example, a prompt message such as "No food placement component placed" can be returned to the user.
[0047] As can be seen from the technical solutions provided in the embodiments of this specification above, this specification acquires an image of the interior of a kitchen utensil. The kitchen utensil includes a food placement component and at least two layers of placement structures for placing the food placement component. The food placement component is used to place food, and the placement structures are located on both sides inside the kitchen utensil. The image of the interior of the kitchen utensil is segmented to obtain a segmentation result. The segmentation result includes a first segmentation mask corresponding to the placement structure and a preset number of placement layers. If the segmentation result also includes a second segmentation mask corresponding to the food placement component, the positional relationship between the first segmentation mask corresponding to the placement structure and the second segmentation mask corresponding to the food placement component is determined. Based on the positional relationship and the preset number of placement layers corresponding to the placement structure, the target number of placement layers for the food placement component is determined, thereby effectively identifying the food placement position and improving the accuracy and robustness of food placement position recognition. Furthermore, using the relative position between the segmented food placement component and the masks of the two side placement structures for judgment can adapt to different camera installation angles, enhancing the robustness and applicability of food placement position recognition.
[0048] This invention also provides a food placement location identification device, such as... Figure 5 As shown, the device includes: The first acquisition module 510 is used to acquire an image of the interior of the kitchenware; the kitchenware includes a food placement component and at least two layers of placement structures for placing the food placement component, the food placement component is used to place food, and the placement structures are disposed on both sides inside the kitchenware; The image segmentation module 520 is used to segment the internal image of the kitchenware to obtain a segmentation result; the segmentation result includes a first segmentation mask corresponding to the placement structure and a preset number of placement layers. The positional relationship determination module 530 is used to determine the positional relationship between the first segmentation mask corresponding to the placement structure and the second segmentation mask corresponding to the food placement component when the segmentation result also includes the second segmentation mask corresponding to the food placement component. The target placement layer determination module 540 is used to determine the target placement layer of the food placement component based on the positional relationship and the preset placement layer corresponding to the placement structure.
[0049] Optionally, the positional relationship determination module 530 includes: The target segmentation sub-mask determination unit is used to determine the target segmentation sub-mask corresponding to the food placement component based on the area of the multiple initial segmentation sub-masks when the second segmentation mask includes multiple initial segmentation sub-masks; the target segmentation sub-mask is the segmentation sub-mask among the multiple initial segmentation masks whose area is greater than a preset area threshold. The first positional relationship determination unit is used to determine the positional relationship between the first segmentation mask corresponding to the placement structure and the target segmentation sub-mask corresponding to the food placement component.
[0050] Optionally, the target placement layer number determination module 540 includes: The first target segmentation mask determination unit is configured to determine a first target segmentation mask from the first segmentation mask corresponding to the placement structure when the positional relationship indicates that the first segmentation mask corresponding to any of the placement structures is located above the second segmentation mask. The first target placement layer determination unit is used to determine the target placement layer based on the preset placement layer number corresponding to the first target segmentation mask.
[0051] Optionally, the target placement layer number determination module 540 includes: The second target placement layer number determination unit is used to determine the upper limit placement layer number corresponding to the placement structure as the target placement layer number when the positional relationship indicates that the first segmentation mask corresponding to each placement structure is not located above the second segmentation mask, and the first segmentation mask corresponding to any placement structure is located below the second segmentation mask.
[0052] Optionally, the positional relationship determination module 530 includes: A coordinate system establishment unit is used to establish a target coordinate system inside the kitchenware; The coordinate information determination unit is used to determine, in the target coordinate system, the first coordinate information corresponding to the first segmentation mask and the second coordinate information corresponding to the second segmentation mask; The second positional relationship determination unit is used to determine the positional relationship based on the first coordinate information and the second coordinate information.
[0053] Optionally, the device further includes: The second acquisition module is used to acquire target recipe information, which includes the recommended number of layers for placing the ingredients; The first prompt information generation module is used to generate a first prompt information when the target placement layer number is different from the recommended placement layer number; the first prompt information is used to prompt the recommended placement layer number.
[0054] Optionally, the device further includes: The second prompt information generation module is used to generate a second prompt information when the segmentation result does not include the second segmentation mask corresponding to the food placement component; the second prompt information is used to indicate that the food placement component has not been placed.
[0055] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0056] Figure 6 This is a block diagram illustrating a terminal electronic device for identifying the placement position of food ingredients, according to an exemplary embodiment. The electronic device can be a terminal, and its internal structure diagram can be as follows: Figure 6 As shown, the electronic device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for identifying the placement position of food items. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.
[0057] Figure 7 This is a block diagram illustrating a server electronic device for identifying the placement location of food ingredients, according to an exemplary embodiment. The electronic device can be a server, and its internal structure diagram can be as follows: Figure 7 As shown, the electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for identifying the placement location of food ingredients.
[0058] Those skilled in the art will understand that Figure 6 or Figure 7The structures shown are merely block diagrams of some structures related to the disclosed solutions of this invention, and do not constitute a limitation on the electronic devices to which the disclosed solutions of this invention are applied. Specific electronic devices may include more or fewer components than those shown in the figures, or combine certain components, or have different component arrangements.
[0059] In an exemplary embodiment, an electronic device for identifying the placement location of food ingredients is also provided, including a processor and a memory, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the food ingredient placement location identification method as disclosed in the embodiments of the present invention.
[0060] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one instruction, which is loaded and executed by a processor to implement the food placement location identification method in the disclosed embodiments of the present invention.
[0061] In an exemplary embodiment, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the food placement location identification method in the disclosed embodiments of the present invention.
[0062] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.
[0063] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles disclosed herein and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.
[0064] It should be understood that the present invention is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is limited only by the appended claims.
Claims
1. A method for identifying the placement location of food ingredients, characterized in that, The method includes: Acquire an internal image of the kitchen utensil; the kitchen utensil includes a food placement component and at least two layers of placement structures for placing the food placement component, the food placement component being used to place food, and the placement structures being disposed on both sides inside the kitchen utensil; The internal image of the kitchenware is segmented to obtain a segmentation result; the segmentation result includes a first segmentation mask corresponding to the placement structure and a preset number of placement layers. If the segmentation result also includes a second segmentation mask corresponding to the food placement component, determine the positional relationship between the first segmentation mask corresponding to the placement structure and the second segmentation mask corresponding to the food placement component; Based on the positional relationship and the preset number of placement layers corresponding to the placement structure, the target number of placement layers for the food placement component is determined.
2. The method according to claim 1, characterized in that, The positional relationship between the first segmentation mask corresponding to the placement structure and the second segmentation mask corresponding to the food placement component includes: When the second segmentation mask includes multiple initial segmentation sub-masks, the target segmentation sub-mask corresponding to the food placement component is determined based on the area of the multiple initial segmentation sub-masks; the target segmentation sub-mask is the segmentation sub-mask among the multiple initial segmentation masks whose area is greater than a preset area threshold. Determine the positional relationship between the first segmentation mask corresponding to the placement structure and the target segmentation sub-mask corresponding to the food placement component.
3. The method according to claim 1, characterized in that, Determining the target number of placement layers for the food placement component based on the positional relationship and the preset number of placement layers corresponding to the placement structure includes: When the positional relationship indicates that the first segmentation mask corresponding to any of the placement structures is above the second segmentation mask, a first target segmentation mask is determined from the first segmentation mask corresponding to the placement structure. The target placement layer number is determined based on the preset placement layer number corresponding to the first target segmentation mask.
4. The method according to claim 1, characterized in that, Determining the target number of placement layers for the food placement component based on the positional relationship and the preset number of placement layers corresponding to the placement structure includes: When the positional relationship indicates that the first segmentation mask corresponding to each placement structure is not above the second segmentation mask, and the first segmentation mask corresponding to any placement structure is below the second segmentation mask, the upper limit placement layer number corresponding to the placement structure is determined as the target placement layer number.
5. The method according to claim 1, characterized in that, The positional relationship between the first segmentation mask corresponding to the placement structure and the second segmentation mask corresponding to the food placement component includes: Establish the target coordinate system inside the kitchenware; In the target coordinate system, determine the first coordinate information corresponding to the first segmentation mask and the second coordinate information corresponding to the second segmentation mask; The positional relationship is determined based on the first coordinate information and the second coordinate information.
6. The method according to any one of claims 1 to 5, characterized in that, After determining the target number of layers for the food placement component, the method further includes: Obtain target recipe information, which includes the recommended number of layers for placing the ingredients; If the target placement layer number is different from the recommended placement layer number, a first prompt message is generated; the first prompt message is used to indicate the recommended placement layer number.
7. The method according to any one of claims 1 to 5, characterized in that, The method further includes: If the segmentation result does not include the second segmentation mask corresponding to the food placement component, a second prompt message is generated; the second prompt message is used to indicate that the food placement component has not been placed.
8. A food placement location identification device, characterized in that, The device includes: The first acquisition module is used to acquire an image of the interior of the kitchenware; the kitchenware includes a food placement component and at least two layers of placement structures for placing the food placement component, the food placement component is used to place food, and the placement structures are disposed on both sides inside the kitchenware; An image segmentation module is used to segment the internal image of the kitchenware to obtain a segmentation result; the segmentation result includes a first segmentation mask corresponding to the placement structure and a preset number of placement layers. The positional relationship determination module is used to determine the positional relationship between the first segmentation mask corresponding to the placement structure and the second segmentation mask corresponding to the food placement component when the segmentation result also includes the second segmentation mask corresponding to the food placement component. The target placement layer determination module is used to determine the target placement layer of the food placement component based on the positional relationship and the preset placement layer corresponding to the placement structure.
9. An electronic device for identifying the placement position of food ingredients, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one instruction, which is loaded and executed by the processor to implement the food placement location identification method as described in any one of claims 1 to 7.
10. A computer storage medium, characterized in that, The computer storage medium stores at least one instruction, which is loaded and executed by a processor to implement the food placement location identification method as described in any one of claims 1 to 7.