Food material placement position recognition method and device
By combining image segmentation technology with the positional relationship of calibration points, the problem of inaccurate food placement recognition in existing kitchen appliances has been solved, achieving efficient and low-cost food placement recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NINGBO FOTILE KITCHEN WARE CO LTD
- Filing Date
- 2026-04-15
- Publication Date
- 2026-07-21
AI Technical Summary
In existing kitchen appliances, the method of identifying the placement of food using weight sensors is costly and lacks accuracy, resulting in inaccurate identification of food placement and low practicality.
By employing image segmentation technology, the internal image of the kitchen utensils is acquired. The positional relationship between the placement calibration point and the back calibration point is utilized, combined with the image segmentation results, to determine the first and second placement layers of the food placement component. Then, the target placement layer number is calculated, thereby improving the recognition accuracy.
No additional hardware is required, reducing costs and improving the accuracy and reliability of food placement location recognition.
Smart Images
Figure CN122435595A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of smart kitchen appliance technology, and in particular to a method and device for identifying the placement position of food ingredients. Background Technology
[0002] With the advancement of human life, kitchen appliances are developing towards intelligence. For example, commonly used ovens and steamers can identify the placement of food inside and control the heating temperature and cooking time based on the shelf position of the food. Currently, the placement of food shelves is usually identified using weight sensors, requiring one weight sensor to be placed on each shelf. However, this method is costly, and the accuracy of the sensors is greatly affected, making it difficult to accurately identify the placement of food and limiting its practicality. Summary of the Invention
[0003] To address the aforementioned problems in the prior art, this invention discloses a method and apparatus for identifying the placement position of food ingredients, which can improve the accuracy and reliability of food ingredient placement position identification. The technical solution disclosed in this invention is as follows: According to one aspect of the disclosed embodiments of the present invention, a method for identifying the placement position of food ingredients is provided, comprising: Acquire an internal image of the kitchen utensil; the kitchen utensil includes a food placement component and at least two layers of placement structures for placing the food placement component, the food placement component is used to place food, the placement structures are disposed on both sides inside the kitchen utensil, each layer of the placement structure corresponds to a placement calibration point, and the inner back side of the kitchen utensil corresponds to a back calibration point; The internal image of the kitchen utensil is segmented to obtain the segmentation result; When the segmentation result includes the segmentation mask corresponding to the food placement component, the first placement layer corresponding to the food placement component is determined according to the positional relationship between the placement calibration point and the segmentation mask, and the second placement layer corresponding to the food placement component is determined according to the positional relationship between the back calibration point and the segmentation mask. The target number of placement layers for the food placement component is determined based on the first placement layer number and the second placement layer number.
[0004] Optionally, determining the target number of placement layers for the food placement component based on the first placement layer number and the second placement layer number includes: When both the first placement layer and the second placement layer are not empty, and the first placement layer is greater than the second placement layer, determine the first ratio information of the placement calibration point located in the segmentation mask area among the placement calibration points corresponding to the first placement layer. If the first ratio information is greater than or equal to the first preset ratio threshold, the first placement layer number is determined as the target placement layer number.
[0005] Optionally, the method further includes: If the first ratio information is less than the first preset ratio threshold, the second placement layer number is determined as the target placement layer number.
[0006] Optionally, determining the target number of placement layers for the food placement component based on the first placement layer number and the second placement layer number includes: When both the first placement layer and the second placement layer are not empty, and the first placement layer is less than the second placement layer, determine the second ratio information of the back calibration point located in the segmentation mask area in the back calibration point corresponding to the second placement layer; If the second ratio information is greater than or equal to the second preset ratio threshold, the second placement layer number is determined as the target placement layer number.
[0007] Optionally, the method further includes: If the second ratio information is less than the second preset ratio threshold, the first placement layer number is determined as the target placement layer number.
[0008] Optionally, determining the target number of placement layers for the food placement component based on the first placement layer number and the second placement layer number includes: When the first placement layer is not empty and the second placement layer is empty, determine the first ratio information of the placement calibration point located in the segmentation mask area in the placement calibration point corresponding to the first placement layer; If the first ratio information is greater than or equal to the third preset ratio threshold, the first placement layer number is determined as the target placement layer number.
[0009] Optionally, the method further includes: If the first ratio information is less than the third preset ratio threshold, the abnormal location result is determined.
[0010] Optionally, determining the target number of placement layers for the food placement component based on the first placement layer number and the second placement layer number includes: When the first placement layer is empty and the second placement layer is not empty, determine the second ratio information of the back calibration point located in the segmentation mask area in the back calibration point corresponding to the second placement layer; If the second ratio information is greater than or equal to the fourth preset ratio threshold, the second placement layer number is determined as the target placement layer number.
[0011] Optionally, the method further includes: If the second ratio information is less than the fourth preset ratio threshold, an abnormal location result is determined.
[0012] According to another aspect of the disclosed embodiments of the present invention, a food placement location identification device is provided, comprising: An acquisition module is used to acquire an image of the interior of a kitchen utensil; the kitchen utensil includes a food placement component and at least two layers of placement structures for placing the food placement component, the food placement component is used to place food, the placement structures are disposed on both sides inside the kitchen utensil, each layer of the placement structure corresponds to a placement calibration point, and the inner back side of the kitchen utensil corresponds to a back calibration point; The image segmentation module is used to segment the internal image of the kitchenware to obtain the segmentation result; The layer number determination module is used to determine the first layer number corresponding to the food placement component based on the positional relationship between the placement calibration point and the partition mask when the segmentation result includes the partition mask corresponding to the food placement component; and to determine the second layer number corresponding to the food placement component based on the positional relationship between the back calibration point and the partition mask. The first target placement layer determination module is used to determine the target placement layer of the food placement component based on the first placement layer and the second placement layer.
[0013] According to another aspect of the embodiments disclosed in this invention, an electronic device for identifying the placement location of food ingredients is provided, including a processor and a memory, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the food ingredient placement location identification method described in any of the preceding claims.
[0014] According to another aspect of the embodiments disclosed in this invention, a computer-readable storage medium is provided, wherein at least one instruction is stored in the computer storage medium, the at least one instruction being loaded and executed by a processor to implement the food placement location identification method described in any of the preceding claims.
[0015] According to another aspect of the disclosed embodiments of the present invention, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to execute the food placement location identification method described in any of the above-mentioned embodiments of the present invention.
[0016] The food placement location identification method provided by this invention has the following technical effects: This invention acquires an image of the interior of a kitchen utensil, wherein the utensil includes a food placement component and at least two layers of placement structures for placing the food placement component. The food placement component is used to place food, and the placement structures are located on both sides inside the utensil. Each layer of placement structure corresponds to a placement calibration point, and the inner back of the utensil corresponds to a back calibration point. The image of the interior of the utensil is segmented to obtain a segmentation result. If the segmentation result includes a segmentation mask corresponding to the food placement component, the first placement layer corresponding to the food placement component is determined based on the positional relationship between the placement calibration point and the segmentation mask, and the second placement layer corresponding to the food placement component is determined based on the positional relationship between the back calibration point and the segmentation mask. Based on the first and second placement layers, the target placement layer of the food placement component is determined, thereby improving the accuracy and reliability of food placement position recognition.
[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is a flowchart illustrating a method for identifying the placement location of food ingredients according to an exemplary embodiment; Figure 2 This is a schematic diagram illustrating the placement of a calibration point and a back calibration point according to an exemplary embodiment; Figure 3 This is a schematic diagram illustrating a segmentation result according to an exemplary embodiment; Figure 4 This is a schematic diagram illustrating a process for determining the number of target placement layers according to an exemplary embodiment; Figure 5 This is a schematic diagram illustrating another process for determining the number of target placement layers according to an exemplary embodiment; Figure 6 This is a schematic diagram illustrating another process for determining the number of target placement layers according to an exemplary embodiment; Figure 7 This is a schematic diagram illustrating another process for determining the number of target placement layers according to an exemplary embodiment; Figure 8 This is a block diagram illustrating a food placement location recognition device according to an exemplary embodiment; Figure 9This is a block diagram illustrating a terminal electronic device for identifying the placement position of food ingredients according to an exemplary embodiment; Figure 10 This is a block diagram illustrating a server electronic device for identifying the placement location of food ingredients, according to an exemplary embodiment. Detailed Implementation
[0020] To enable those skilled in the art to better understand the technical solutions disclosed in this invention, the technical solutions in the disclosed embodiments will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention disclosed herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.
[0022] The following describes a method for identifying the placement location of food ingredients according to this application. Please refer to [link / reference]. Figure 1 , Figure 1 This is a flowchart illustrating a method for identifying the placement location of food ingredients according to an exemplary embodiment. This specification provides the operational steps of the method as described in the embodiments or flowchart, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual system or server product execution, the method can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiments or drawings. Specifically, as... Figure 1 As shown, the above method may include: S101: Obtain an image of the interior of the kitchenware.
[0023] In one specific embodiment, the kitchen appliance may include a food placement component and at least two layers of placement structures for placing the food placement component. The food placement component is used to place food. The number of food placement layers in the kitchen appliance can be determined by determining the number of placement layers of the food placement component. The placement structures can be located on both sides inside the kitchen appliance (side placement structures, such as side racks) to support the food placement component. The kitchen appliance may have multiple placement positions, specifically including ovens, steamers, etc. Specifically, when the kitchen appliance is an oven, the corresponding food placement component can be a baking tray or a baking rack, and the placement structure can be a grooved structure or a protruding structure corresponding to the food placement component for placing the food placement component.
[0024] In one specific embodiment, such as Figure 2 As shown, each layer of the structure can be used to place corresponding calibration points (e.g., Figure 2 The white dots in the middle), the inner side of the back of the kitchenware can correspond to the marking points on the back (such as...). Figure 2 The black dots in the diagram represent the placement of calibration points. Each calibration point on the back of the placement box corresponds to a different number of layers. The calibration points for each layer can include the two endpoints of the layer and a selected point in between. Specifically, the number of calibration points for each layer and the number of back calibration points can be set according to actual application requirements. For example, each layer can have 5 calibration points, and each layer can have 2 back calibration points. The food placement components are placed on different layers, which are related to the food cooking operation and the cooking result. By identifying and determining the number of layers where the food is placed, it can be detected whether it conforms to the cooking operation for that food.
[0025] Specifically, the internal images of the kitchenware can include internal images of the back and sides of the kitchenware. The internal images of the kitchenware can be captured by a shooting device (such as a camera). The camera can be set inside the front side of the kitchenware, such as inside the door, and can capture images of the back and sides of the kitchenware.
[0026] S103: Perform image segmentation on the internal image of the kitchenware to obtain the segmentation result.
[0027] Specifically, the segmentation result can be the segmentation result of the food placement component in the image inside the kitchenware. By identifying and segmenting the food placement component from the image inside the kitchenware, a segmentation mask for the food placement component can be obtained (e.g., Figure 3 The segmentation result is empty if the white dotted line area is not detected, or if no food placement component is identified.
[0028] Optionally, the image segmentation of the interior of the kitchen utensils described above, to obtain the segmentation result, may include: The image of the interior of the kitchen utensils is input into an image segmentation network for image segmentation, resulting in the segmentation of the food placement components in the image of the interior of the kitchen utensils.
[0029] In one specific embodiment, an image segmentation network can be used to perform target recognition and segmentation of food placement components in images of the interior of kitchen utensils. This image segmentation network can be trained using images of the interior of kitchen utensils with food placement components at different layers, along with pre-labeled segmentation masks corresponding to the food placement components. Specifically, the network structure of the image segmentation network can be configured according to actual application requirements.
[0030] S105: If the segmentation result includes the segmentation mask corresponding to the food placement component, determine the first placement layer corresponding to the food placement component based on the positional relationship between the placement calibration point and the segmentation mask, and determine the second placement layer corresponding to the food placement component based on the positional relationship between the back calibration point and the segmentation mask.
[0031] In one specific embodiment, when the segmentation result includes a segmentation mask corresponding to the food placement component (i.e., when the food placement component is placed inside the kitchen utensil), the number of placement layers is determined based on the positional relationship between the placement calibration point, the back calibration point, and the segmentation mask corresponding to the food placement component. Specifically, the first number of placement layers can be determined based on the proportion of placement calibration points in the same layer within the segmentation mask area. If this proportion is greater than a first preset proportion, the number of layers corresponding to the placement calibration points in that layer is determined as the first number of placement layers. The first preset proportion can be set according to actual application requirements, for example, it can be set to 50%. Similarly, the second number of placement layers can be determined based on the proportion of back calibration points in the same layer within the segmentation mask area. If this proportion is greater than a second preset proportion, the number of layers corresponding to the back calibration points in that layer is determined as the second number of placement layers. The second preset proportion can be set according to actual application requirements, for example, it can be set to 100% when the number of back calibration points in each layer is small.
[0032] In practical applications, the identification and segmentation of food ingredients may be inaccurate due to obstructions from the back or sides of the cookware. This can lead to discrepancies or inaccuracies in the layer count determined by the back or placement marking points, making it impossible to directly determine the number of layers. Therefore, after determining the number of layers using the placement and back marking points, further verification is necessary to finalize the layer count and improve its accuracy and reliability.
[0033] S107: Determine the target number of placement layers for the food placement component based on the first placement layer number and the second placement layer number.
[0034] Specifically, the target placement layer number is the current placement layer number of the identified food placement component, that is, the current placement layer number of the food inside the kitchenware.
[0035] In an optional embodiment, such as Figure 4 As shown, determining the target number of food placement layers based on the first and second placement layers can include: S401: When both the first placement layer and the second placement layer are not empty, and the first placement layer is greater than the second placement layer, determine the first proportional information of the placement calibration point located within the segmentation mask area in the placement calibration point corresponding to the first placement layer.
[0036] Specifically, if the number of placement layers determined by both the placement calibration point and the back calibration point exists, and if the number of layers corresponding to the placement calibration point is inconsistent with the number of layers corresponding to the back calibration point, and the number of layers corresponding to the placement calibration point is greater than the number of layers corresponding to the back calibration point, then the first ratio information of the first placement layer corresponding to the placement calibration point within the segmentation mask area is further determined. Specifically, the existence of a placement layer determined by the placement calibration point can be represented as the ability to obtain a definite placement layer number (non-null value) based on the positional relationship between the placement calibration point and the segmentation mask of the food placement component. The absence of a placement layer determined by the placement calibration point can be represented as the inability to obtain a definite placement layer number based on the positional relationship between the placement calibration point and the segmentation mask of the food placement component (e.g., the placement calibration point is obscured), meaning the placement layer number is null.
[0037] S403: If the first ratio information is greater than or equal to the first preset ratio threshold, the first placement layer number is determined as the target placement layer number.
[0038] Specifically, if the first ratio information is greater than or equal to the first preset ratio threshold, meaning that among the placement calibration points corresponding to the first placement layer, a sufficient number of placement calibration points are located within the segmentation mask area, it indicates that the placement layer corresponding to the placement calibration point is relatively reliable. Therefore, the placement layer corresponding to the placement calibration point (i.e., the first placement layer) is determined as the current placement layer of the food placement component. Specifically, the first preset ratio threshold can be set according to actual application requirements, for example, it can be set to 90%.
[0039] Optionally, the above method may also include: If the first ratio information is less than the first preset ratio threshold, the second placement layer number is determined as the target placement layer number.
[0040] Specifically, if the number of placement layers determined by both the placement calibration point and the back calibration point exists, and the first ratio information is less than the first preset ratio threshold, that is, there are not enough placement calibration points in the segmentation mask area among the placement calibration points corresponding to the first placement layer, it can be said that the number of placement layers corresponding to the placement calibration point is unreliable. In this case, the number of placement layers corresponding to the back calibration point (i.e., the second placement layer) is determined as the current placement layer of the food placement component.
[0041] In an optional embodiment, such as Figure 5 As shown, determining the target number of food placement layers based on the first and second placement layers can include: S501: When both the first placement layer and the second placement layer are not empty, and the first placement layer is less than the second placement layer, determine the second proportional information of the back calibration point located in the segmentation mask area in the back calibration point corresponding to the second placement layer.
[0042] In one specific embodiment, if the number of placement layers determined based on both the placement calibration point and the back calibration point exists, and if the number of layers corresponding to the placement calibration point is less than the number of layers corresponding to the back calibration point, then the second ratio information of the second placement layer corresponding to the back calibration point within the segmentation mask area is further determined.
[0043] S503: If the second ratio information is greater than or equal to the second preset ratio threshold, the second placement layer number is determined as the target placement layer number.
[0044] Specifically, if the second ratio information is greater than or equal to the second preset ratio threshold, meaning that among the back calibration points corresponding to the first placement layer, a sufficient number of back calibration points are located within the segmentation mask area, it indicates that the placement layer corresponding to the back calibration point is relatively reliable. Therefore, the placement layer corresponding to the back calibration point (i.e., the second placement layer) is determined as the current placement layer of the food placement component. Specifically, the second preset ratio threshold can be set according to actual application requirements, for example, it can be set to 30%.
[0045] Optionally, the above method may also include: If the second ratio information is less than the second preset ratio threshold, the first placement layer number is determined as the target placement layer number.
[0046] Specifically, if the number of placement layers determined by the placement calibration point and the back calibration point both exist, and the second ratio information is less than the first preset ratio threshold, that is, there are not enough back calibration points in the segmentation mask area among the back calibration points corresponding to the second placement layer, it can be considered that the number of placement layers corresponding to the back calibration point is unreliable. Then, the number of placement layers corresponding to the placement calibration point (i.e. the first placement layer) is determined as the current placement layer of the food placement component.
[0047] Optionally, if the number of placement layers determined by the placement calibration point and the back calibration point are both non-empty, and the number of layers corresponding to the placement calibration point is the same as the number of layers corresponding to the back calibration point, then the first placement layer or the second placement layer is determined as the target placement layer.
[0048] In an optional embodiment, such as Figure 6 As shown, determining the target number of food placement layers based on the first and second placement layers can include: S601: When the first placement layer is not empty and the second placement layer is empty, determine the first proportional information of the placement calibration point located in the segmentation mask area among the placement calibration points corresponding to the first placement layer.
[0049] In one specific embodiment, if the number of placement layers determined by the placement calibration point exists, but the number of placement layers determined by the back calibration point does not exist, the proportion information of the first placement layer corresponding to the placement calibration point within the segmentation mask area is further determined.
[0050] S603: If the first ratio information is greater than or equal to the third preset ratio threshold, the first placement layer number is determined as the target placement layer number.
[0051] Specifically, if the first ratio information is greater than or equal to the third preset ratio threshold, meaning that among the placement calibration points corresponding to the first placement layer, a sufficient number of placement calibration points are located within the segmentation mask area, it indicates that the placement layer corresponding to the placement calibration point is relatively reliable. Therefore, the placement layer corresponding to the placement calibration point (i.e., the first placement layer) is determined as the current placement layer of the food placement component. Specifically, the third preset ratio threshold can be set according to actual application needs, and can be set to the same as the first preset ratio threshold, for example, 90%.
[0052] Optionally, the above method may also include: If the first ratio information is less than the third preset ratio threshold, the abnormal location result is determined.
[0053] Specifically, the abnormal location result can be used to characterize an abnormal number of placement layers for the food placement component. If the number of placement layers determined by the placement calibration point exists, but the number of placement layers determined by the back calibration point does not exist, the aforementioned first ratio information is less than the third preset ratio threshold. That is, among the placement calibration points corresponding to the first placement layer, there are insufficient placement calibration points located in the segmentation mask area, and the placement layer number corresponding to the placement calibration point can be considered unreliable. In this case, the placement layer number corresponding to the placement calibration point is unreliable, the placement layer number corresponding to the back calibration point is null, and the effective placement layer number of the food placement component cannot be determined. An abnormal location result is returned. Specifically, a prompt message can be generated based on the abnormal location result and sent to the user. For example, a prompt message such as "Number of layers is zero" or "Position / layer number recognition abnormal" can be returned to the user to indicate that the effective position / layer number of the food placement component has not been identified.
[0054] In an optional embodiment, such as Figure 7 As shown, determining the target number of food placement layers based on the first and second placement layers can include: S701: When the first placement layer is empty and the second placement layer is not empty, determine the second proportional information of the back calibration point located in the segmentation mask area in the back calibration point corresponding to the second placement layer.
[0055] In one specific embodiment, if the number of placement layers determined by the placement calibration point does not exist, but the number of placement layers determined by the back calibration point does exist, the proportion information of the back calibration point corresponding to the second placement layer located within the segmentation mask area is further determined.
[0056] S703: If the second ratio information is greater than or equal to the fourth preset ratio threshold, the second placement layer number is determined as the target placement layer number.
[0057] Specifically, if the second ratio information is greater than or equal to the fourth preset ratio threshold, meaning that among the back calibration points corresponding to the second placement layer, a sufficient number of back calibration points are located within the segmentation mask area, it indicates that the placement layer corresponding to the back calibration point is relatively reliable. Therefore, the placement layer corresponding to the back calibration point (i.e., the second placement layer) is determined as the current placement layer of the food placement component. Specifically, the fourth preset ratio threshold can be set according to actual application needs, and can be set to the same as the second preset ratio threshold, for example, 30%.
[0058] Optionally, the above method may also include: If the second ratio information is less than the fourth preset ratio threshold, the abnormal location result is determined.
[0059] Specifically, if the number of placement layers determined by the placement calibration point does not exist, but the number of placement layers determined by the back calibration point does exist, the aforementioned second ratio information is less than the fourth preset ratio threshold. That is, among the back calibration points corresponding to the second placement layer, there are insufficient back calibration points located in the segmentation mask area. Therefore, the placement layer number corresponding to the back calibration point can be considered unreliable. In this case, the placement layer number corresponding to the back calibration point is unreliable, the placement layer number corresponding to the placement calibration point is null, the effective placement layer number of the food placement component cannot be determined, and an abnormal position result is returned.
[0060] Optionally, the above method may also include: If the segmentation result does not include the segmentation mask corresponding to the food placement piece, or if both the first and second placement layers are empty, determine the abnormal location result.
[0061] Specifically, the segmentation results do not include the segmentation mask corresponding to the food placement component. That is, during the image segmentation of the interior of the kitchen utensil, the food placement component was not identified in the image, and its corresponding segmentation mask was not obtained. Specifically, a prompt message can be generated based on the abnormal location result and sent to the user, for example, a message such as "No food placement component placed" can be returned to the user.
[0062] Optionally, after determining the target placement layer of the ingredients, it can be compared with the recommended placement layer of the ingredients in the corresponding recipe. If the two layer numbers are inconsistent, a prompt message can be generated and sent to the user. For example, a prompt message "This is not the best cooking position" can be generated, and the recommended best cooking position (recommended placement layer) can be sent to the user at the same time.
[0063] As can be seen from the technical solutions provided in the embodiments of this specification above, the method for acquiring an image of the interior of a kitchen utensil involves the kitchen utensil including a food placement component and at least two layers of placement structures for placing the food placement component. The food placement component is used to place food, and the placement structures are located on both sides inside the kitchen utensil. Each layer of placement structure corresponds to a placement calibration point, and the inner back of the kitchen utensil corresponds to a back calibration point. The image of the interior of the kitchen utensil is segmented to obtain a segmentation result. If the segmentation result includes a segmentation mask corresponding to the food placement component, the first placement layer corresponding to the food placement component is determined based on the positional relationship between the placement calibration point and the segmentation mask, and the second placement layer corresponding to the food placement component is determined based on the positional relationship between the back calibration point and the segmentation mask. Based on the first and second placement layers, the target placement layer of the food placement component is determined, thereby improving the accuracy and reliability of food placement position recognition. Furthermore, no additional hardware components are required, reducing costs and demonstrating strong practicality.
[0064] This invention also provides a food placement location identification device, such as... Figure 8 As shown, the device includes: The acquisition module 810 is used to acquire an image of the interior of the kitchenware; the kitchenware includes a food placement component and at least two layers of placement structures for placing the food placement component, the food placement component is used to place food, the placement structures are disposed on both sides inside the kitchenware, each layer of the placement structure corresponds to a placement calibration point, and the inner side of the back of the kitchenware corresponds to a back calibration point; The image segmentation module 820 is used to perform image segmentation on the internal image of the kitchenware to obtain the segmentation result; The layer number determination module 830 is used to determine the first layer number corresponding to the food placement component based on the positional relationship between the placement calibration point and the partition mask when the segmentation result includes the partition mask corresponding to the food placement component; and to determine the second layer number corresponding to the food placement component based on the positional relationship between the back calibration point and the partition mask. The first target placement layer determination module 840 is used to determine the target placement layer of the food placement component based on the first placement layer and the second placement layer.
[0065] Optionally, the first target placement layer determination module 840 includes: The first ratio information determination unit is used to determine the first ratio information of the placement calibration point located in the segmentation mask area among the placement calibration points corresponding to the first placement layer when both the first placement layer number and the second placement layer number are not empty and the first placement layer number is greater than the second placement layer number. The first target placement layer number determination unit is used to determine the first placement layer number as the target placement layer number when the first ratio information is greater than or equal to the first preset ratio threshold.
[0066] Optionally, the device further includes: The second target placement layer determination module is used to determine the second placement layer as the target placement layer when the first ratio information is less than the first preset ratio threshold.
[0067] Optionally, the first target placement layer determination module 840 includes: The second ratio information determination unit is used to determine the second ratio information of the back calibration point located in the segmentation mask area in the back calibration point corresponding to the second placement layer when both the first placement layer number and the second placement layer number are not empty and the first placement layer number is less than the second placement layer number. The second target placement layer determination unit is used to determine the second placement layer as the target placement layer when the second ratio information is greater than or equal to the second preset ratio threshold.
[0068] Optionally, the device further includes: The third target placement layer determination module is used to determine the first placement layer as the target placement layer when the second ratio information is less than the second preset ratio threshold.
[0069] Optionally, the first target placement layer determination module 840 includes: The third ratio information determination unit is used to determine the first ratio information of the placement calibration point located in the segmentation mask area among the placement calibration points corresponding to the first placement layer when the first placement layer is not empty and the second placement layer is empty. The third target placement layer determination unit is used to determine the first placement layer as the target placement layer when the first ratio information is greater than or equal to the third preset ratio threshold.
[0070] Optionally, the device further includes: The first abnormal location result determination module is used to determine the abnormal location result when the first ratio information is less than the third preset ratio threshold.
[0071] Optionally, the first target placement layer determination module 840 includes: The fourth ratio information determination unit is used to determine the second ratio information of the back calibration point located in the segmentation mask area in the back calibration point corresponding to the second placement layer when the first placement layer is empty and the second placement layer is not empty. The fourth target placement layer determination unit is used to determine the second placement layer as the target placement layer when the second ratio information is greater than or equal to the fourth preset ratio threshold.
[0072] Optionally, the device further includes: The second abnormal location result determination module is used to determine the abnormal location result when the second ratio information is less than the fourth preset ratio threshold.
[0073] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0074] Figure 9 This is a block diagram illustrating a terminal electronic device for identifying the placement position of food ingredients, according to an exemplary embodiment. The electronic device can be a terminal, and its internal structure diagram can be as follows: Figure 9As shown, the electronic device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for identifying the placement position of food items. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.
[0075] Figure 10 This is a block diagram illustrating a server electronic device for identifying the placement location of food ingredients, according to an exemplary embodiment. The electronic device can be a server, and its internal structure diagram can be as follows: Figure 10 As shown, the electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for identifying the placement location of food ingredients.
[0076] Those skilled in the art will understand that Figure 9 or Figure 10 The structures shown are merely block diagrams of some structures related to the disclosed solutions of this invention, and do not constitute a limitation on the electronic devices to which the disclosed solutions of this invention are applied. Specific electronic devices may include more or fewer components than those shown in the figures, or combine certain components, or have different component arrangements.
[0077] In an exemplary embodiment, an electronic device for identifying the placement location of food ingredients is also provided, including a processor and a memory, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the food ingredient placement location identification method as disclosed in the embodiments of the present invention.
[0078] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one instruction, which is loaded and executed by a processor to implement the food placement location identification method in the disclosed embodiments of the present invention.
[0079] In an exemplary embodiment, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the food placement location identification method in the disclosed embodiments of the present invention.
[0080] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.
[0081] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles disclosed herein and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.
[0082] It should be understood that the present invention is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is limited only by the appended claims.
Claims
1. A method for identifying the placement location of food ingredients, characterized in that, The method includes: Acquire an internal image of the kitchen utensil; the kitchen utensil includes a food placement component and at least two layers of placement structures for placing the food placement component, the food placement component is used to place food, the placement structures are disposed on both sides inside the kitchen utensil, each layer of the placement structure corresponds to a placement calibration point, and the inner back side of the kitchen utensil corresponds to a back calibration point; The internal image of the kitchen utensil is segmented to obtain the segmentation result; When the segmentation result includes the segmentation mask corresponding to the food placement component, the first placement layer corresponding to the food placement component is determined according to the positional relationship between the placement calibration point and the segmentation mask, and the second placement layer corresponding to the food placement component is determined according to the positional relationship between the back calibration point and the segmentation mask. The target number of placement layers for the food placement component is determined based on the first placement layer number and the second placement layer number.
2. The method according to claim 1, characterized in that, Determining the target number of placement layers for the food placement component based on the first placement layer number and the second placement layer number includes: When both the first placement layer and the second placement layer are not empty, and the first placement layer is greater than the second placement layer, determine the first ratio information of the placement calibration point located in the segmentation mask area among the placement calibration points corresponding to the first placement layer. If the first ratio information is greater than or equal to the first preset ratio threshold, the first placement layer number is determined as the target placement layer number.
3. The method according to claim 2, characterized in that, The method further includes: If the first ratio information is less than the first preset ratio threshold, the second placement layer number is determined as the target placement layer number.
4. The method according to claim 1, characterized in that, Determining the target number of placement layers for the food placement component based on the first placement layer number and the second placement layer number includes: When both the first placement layer and the second placement layer are not empty, and the first placement layer is less than the second placement layer, determine the second ratio information of the back calibration point located in the segmentation mask area in the back calibration point corresponding to the second placement layer; If the second ratio information is greater than or equal to the second preset ratio threshold, the second placement layer number is determined as the target placement layer number.
5. The method according to claim 4, characterized in that, The method further includes: If the second ratio information is less than the second preset ratio threshold, the first placement layer number is determined as the target placement layer number.
6. The method according to claim 1, characterized in that, Determining the target number of placement layers for the food placement component based on the first placement layer number and the second placement layer number includes: When the first placement layer is not empty and the second placement layer is empty, determine the first ratio information of the placement calibration point located in the segmentation mask area in the placement calibration point corresponding to the first placement layer; If the first ratio information is greater than or equal to the third preset ratio threshold, the first placement layer number is determined as the target placement layer number.
7. The method according to claim 6, characterized in that, The method further includes: If the first ratio information is less than the third preset ratio threshold, the abnormal location result is determined.
8. The method according to claim 1, characterized in that, Determining the target number of placement layers for the food placement component based on the first placement layer number and the second placement layer number includes: When the first placement layer is empty and the second placement layer is not empty, determine the second ratio information of the back calibration point located in the segmentation mask area in the back calibration point corresponding to the second placement layer; If the second ratio information is greater than or equal to the fourth preset ratio threshold, the second placement layer number is determined as the target placement layer number.
9. The method according to claim 8, characterized in that, The method further includes: If the second ratio information is less than the fourth preset ratio threshold, an abnormal location result is determined.
10. A food placement location identification device, characterized in that, The device includes: An acquisition module is used to acquire an image of the interior of a kitchen utensil; the kitchen utensil includes a food placement component and at least two layers of placement structures for placing the food placement component, the food placement component is used to place food, the placement structures are disposed on both sides inside the kitchen utensil, each layer of the placement structure corresponds to a placement calibration point, and the inner back side of the kitchen utensil corresponds to a back calibration point; The image segmentation module is used to segment the internal image of the kitchenware to obtain the segmentation result; The layer number determination module is used to determine the first layer number corresponding to the food placement component based on the positional relationship between the placement calibration point and the partition mask when the segmentation result includes the partition mask corresponding to the food placement component; and to determine the second layer number corresponding to the food placement component based on the positional relationship between the back calibration point and the partition mask. The first target placement layer determination module is used to determine the target placement layer of the food placement component based on the first placement layer and the second placement layer.