Catering content recognition method and catering content recognition system
By combining 3D scanning and image retrieval, the catering content recognition system solves the problems of complex equipment and high computational consumption in existing technologies, and achieves efficient and accurate catering content recognition and management.
Patent Information
- Application Number
- CN202011107570.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-10-16
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-10-16
AI Technical Summary
Existing technologies require complex equipment, consume high computing resources, or add extra processing steps when automatically recognizing food contents, and image recognition methods are limited by hardware performance and food shape.
A food content recognition system that combines a 3D scanner and an image retrieval device obtains distance distribution through 3D scanning, maps it to an image, sets a threshold distance to define the block to be recognized, and performs image recognition to identify food types and calculate volume, thereby reducing computational load.
It improves the efficiency and accuracy of food recognition, reduces the computational requirements of image recognition, and enables rapid analysis of catering information and management of diners.
Smart Images

Figure CN114445614B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to automatic identification of food and beverage contents, and in particular to a food and beverage content identification method and a food and beverage content identification system. Background Art
[0002] In situations where it is necessary to quickly record the content of food and beverages, such as the pricing of self-service meals and personal dietary content management, it is often necessary to manually identify and record the type and weight of food in order to further summarize it into other related information, such as meal prices and nutritional composition.
[0003] The prior art proposes several technical solutions for automatically identifying food and measuring its weight. One of the various technical solutions is to use corresponding containers to hold different foods, and set machine-readable labels on the containers, such as barcodes and RFID, for the reader to quickly read them. The type of food is obtained by reading the label, and the weight is measured by an electronic scale or other device to distinguish the type of food and the weight of the food. This method requires the installation of relatively complex equipment, and the food must also be placed in the correct container in advance, which requires a more complex pre-operation process.
[0004] Another technical solution is to use image recognition to distinguish food and determine the volume of food, and then use the food volume and food information to convert the food weight. Image recognition needs to exclude parts that do not belong to food in the complete image to be analyzed, and requires higher computing resources for image processing and image recognition, and the processing performance is limited by the hardware performance. The determination of food volume also depends on image recognition, and analyzing the volume of food through image recognition brings additional computing load. At the same time, in order to avoid misinterpretation, the three-dimensional shape of the food is also limited, resulting in the need for additional pre-processing procedures for food processing. Summary of the invention
[0005] The existing technical solutions for automatically identifying food and measuring its weight have the problem of requiring relatively complex equipment, requiring high computing resources, or adding additional pre-processing procedures for additional food processing.
[0006] Based on the above technical problems, the present invention proposes a method for identifying food content, comprising: obtaining multiple distance values on a detection area to generate a distance distribution, and retrieving an image to be analyzed that includes the detection area; mapping the distance distribution to the image to be analyzed; setting a threshold distance to define a portion of the image to be analyzed in which each distance value is less than the threshold distance as at least one block to be identified; performing image recognition on at least one block to be identified to identify the type of food located in the block to be identified; and obtaining the volume of the food based on the distance distribution.
[0007] In at least one embodiment, before obtaining a plurality of distance values, the image to be analyzed is continuously searched to analyze whether the image to be analyzed includes a detection area.
[0008] In at least one embodiment, retrieving the image to be analyzed includes retrieving the image to be analyzed using an image retrieval device, and there is an actual reference distance value between the image retrieval device and a surface of the detection area, and the threshold distance is less than or equal to the actual reference distance value.
[0009] In at least one embodiment, the length and width of the detection area occupy the number of pixels in the image to be analyzed respectively Xn×Yn, and there is a preset reference distance value between the image retrieval device and the surface of the detection area, and the actual length and actual width of the detection area at the preset reference distance value occupy the number of pixels in the image to be analyzed respectively L×W, and the relationship between the actual reference distance value and the preset reference distance value is:
[0010] z0' = z0 × (L / Xn); or
[0011] z0'=z0×(W / Yn);
[0012] Among them, z0 is the preset reference distance value, and z0' is the actual reference distance value.
[0013] In at least one embodiment, the food content recognition method further includes determining whether each distance value is within a range between an upper distance limit value and a lower distance limit value; if any distance value exceeds the range between the upper distance limit value and the lower distance limit value, re-obtaining multiple distance values and re-retrieving the image to be analyzed.
[0014] In at least one embodiment, the food and beverage content recognition method further includes identifying whether at least one to-be-recognized block conforms to a correctly placed standard container based on the distance distribution and at least one standard container information.
[0015] In at least one embodiment, the step of identifying the type of food located in the block to be identified includes: loading at least one food information; wherein the at least one food information at least includes the type of food and a sample image; comparing the image of the block to be identified and the sample image to see if they are similar, so as to identify the type of food in the block to be identified.
[0016] In at least one embodiment, the method for identifying food content further includes: loading at least one food information; wherein the at least one food information further includes a food density of the food; and multiplying the food density by the volume of the food to obtain the weight of the food.
[0017] In at least one embodiment, the catering content identification method further includes: loading at least one food information; wherein at least one food information also includes nutritional ingredients and price per unit weight; and multiplying the nutritional ingredients by the weight of the food to obtain a nutritional composition of the food; multiplying the price per unit weight by the weight of the food to obtain the price of the food; and associating the type, weight, nutritional composition and price of the food with a diner and uploading it to a management platform.
[0018] In at least one embodiment, the food content recognition method further includes analyzing whether at least one block to be recognized conforms to a correctly placed standard container; when at least one block to be recognized does not conform to a correctly placed standard container, an error prompt is generated.
[0019] The present invention also provides a catering content recognition system, comprising: a three-dimensional scanner, an image retrieval device and a data processing module.
[0020] The three-dimensional scanner is used to obtain multiple distance values on a detection area to generate a distance distribution. The image retrieval device is used to retrieve an image to be analyzed in the detection area. The data processing module is electrically connected to the three-dimensional scanner and the image retrieval device; wherein the data processing module maps the distance distribution to the image to be analyzed; the data processing module also sets a threshold distance to define the portion of the image to be analyzed where each distance value is less than the threshold distance as at least one block to be identified; and the data processing module performs image recognition on at least one block to be identified, identifies the type of a food located in at least one block to be identified, and obtains the volume of the food based on the distance distribution.
[0021] In at least one embodiment, before obtaining a plurality of distance values, the image search device continuously searches for the image to be analyzed, and the data processing module analyzes whether the image to be analyzed includes the detection area.
[0022] In at least one embodiment, there is an actual reference distance value between the image retrieval device and a surface of the detection area, and the threshold distance is less than or equal to the actual reference distance value.
[0023] In at least one embodiment, the length and width of the detection area occupy the number of pixels in the image to be analyzed respectively Xn×Yn, and there is a preset reference distance value between the image retrieval device and the surface of the detection area, and the actual length and actual width of the detection area at the preset reference distance value occupy the number of pixels in the image to be analyzed respectively L×W, and the relationship between the actual reference distance value and the preset reference distance value is:
[0024] z0' = z0 × (L / Xn); or
[0025] z0'=z0×(W / Yn);
[0026] Among them, z0 is the preset reference distance value, and z0' is the actual reference distance value.
[0027] In at least one embodiment, the data processing module determines whether each distance value is within a range between an upper distance limit value and a lower distance limit value; if any distance value exceeds the range between the upper distance limit value and the lower distance limit value, a three-dimensional scanner and an image retrieval device are used to re-acquire multiple distance values and re-retrieve the image to be analyzed.
[0028] In at least one embodiment, the data processing module loads at least one standard container information, and identifies whether at least one block to be identified conforms to a correctly placed standard container according to the distance distribution and the at least one standard container information.
[0029] In at least one embodiment, the data processing module loads the type of food and a sample image, and compares the image of the block to be identified with the sample image to see if they are similar, so as to identify the type of food in the block to be identified.
[0030] In at least one embodiment, the data processing module loads the food density of the food and multiplies the food density by the volume of the food to obtain the weight of the food.
[0031] In at least one embodiment, the data processing module loads the nutritional ingredients and the price per unit weight, multiplies the nutritional ingredients by the weight of the food to obtain a nutritional composition of the food, and multiplies the price per unit weight by the weight of the food to obtain the price of the food; and the data processing module associates the type, weight, nutritional composition and price of the food with a diner and uploads it to a management platform.
[0032] In at least one embodiment, the data processing module analyzes whether at least one block to be identified fits in a correctly placed standard container; when at least one block to be identified does not fit in a correctly placed standard container, an error prompt is generated.
[0033] The present invention combines three-dimensional scanning and image retrieval, and marks the area occupied by the standard container in the detection area of the image to be analyzed based on the correspondence between the retrieved image and the three-dimensional scanning information, and only performs image recognition on the block to be identified corresponding to the standard container, instead of performing image recognition on the entire image to be analyzed, to determine the type of food. Therefore, the present invention can reduce the computing load required for image recognition and accelerate the process of identifying food. At the same time, through the application of standard containers, the distance distribution obtained by three-dimensional scanning can be used to effectively and relatively accurately calculate the volume of food. In conjunction with the association of food information, the complete catering information can be quickly analyzed, thereby combining the diner identification information for back-end management.
[0034] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments, but is not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is a side schematic diagram of a catering content recognition system according to an embodiment of the present invention.
[0036] Figure 2 1 is a top view schematic diagram of a catering content recognition system according to an embodiment of the present invention.
[0037] Figure 3 4 is a system block diagram of a catering content recognition system according to an embodiment of the present invention.
[0038] Figure 4 Schematic diagram of mapping distance to an image to be analyzed in an embodiment of the present invention.
[0039] Figure 5 It is a schematic diagram of correcting multiple distance values in an embodiment of the present invention.
[0040] Figure 6 1 is a flowchart of a method for identifying food and beverage content according to an embodiment of the present invention.
[0041] Figure 7 It is a schematic diagram of defining at least one block to be identified according to a distance threshold in the restaurant content identification method according to an embodiment of the present invention.
[0042] Figure 8 It is an information comparison table in the food database in the catering content identification method of the embodiment of the present invention.
[0043] Fig. 9 It is a schematic diagram of obtaining the volume of food according to the distance distribution in the catering content recognition method according to an embodiment of the present invention.
[0044] Fig.10 This is a flowchart (II) of the method for identifying catering content according to an embodiment of the present invention.
[0045] Fig.11 Flowchart (III) of the method for identifying catering content according to an embodiment of the present invention.
[0046] Fig.12 Flowchart (IV) of the method for identifying catering content according to an embodiment of the present invention.
[0047] Fig.13 It is a schematic diagram of the identification and detection area in the catering content identification method according to an embodiment of the present invention.
[0048] Fig.14 It is a schematic diagram of defining at least one block to be identified in the method for identifying food and beverage content according to an embodiment of the present invention.
[0049] Fig.15It is a schematic diagram of analyzing whether at least one block to be identified conforms to a correctly placed standard container in the food and beverage content recognition method according to an embodiment of the present invention.
[0050] Reference numerals
[0051] 100: Catering content recognition system
[0052] 110:3D Scanner
[0053] 120: Image retrieval device
[0054] 130: Data processing module
[0055] 140: Bracket
[0056] 150:Container Database
[0057] 160: Food Database
[0058] 200: Food
[0059] 301, 302, 303: Standard containers
[0060] 304: Non-standard container
[0061] 400: Management Platform
[0062] A, A': detection area
[0063] B: Block to be identified
[0064] C: Image to be analyzed
[0065] D: Spacing distance
[0066] L: Actual length of detection area A
[0067] Xn: length of detection area A'
[0068] W: Actual width of detection area A
[0069] Yn: Width of detection area A'
[0070] P: Distance distribution
[0071] S: Placement table
[0072] X: Length direction
[0073] Y: Width direction
[0074] Z: height direction
[0075] Δa: Grid area
[0076] h: grid height
[0077] z, z(x,y): distance value
[0078] zmax: threshold distance
[0079] z0: preset reference distance value
[0080] z0': actual reference distance value
[0081] Step 112~Step 170: Steps DETAILED DESCRIPTION
[0082] The structural principle and working principle of the present invention are described in detail below in conjunction with the accompanying drawings:
[0083] The term "module" used in the following description may refer to an application specific integrated circuit (ASIC), an electronic circuit, a microprocessor, a chip or a circuit design that executes one or more software or firmware programs. A module is configured to perform various algorithms, transformations and / or logical processing to generate one or more signals. When a module is implemented in software, the module can be implemented in a memory as a program code that can be read by the chip or circuit design and executed by the program code.
[0084] like Figure 1 , Figure 2 as well as Figure 3 1 is a food content recognition system 100 according to an embodiment of the present invention, which is used to perform a food content recognition method to recognize the type of at least one food 200 located in a detection area A. The food content recognition system 100 includes a three-dimensional scanner 110 , an image retrieval device 120 and a data processing module 130 .
[0085] like Figure 1 , Figure 2 as well as Figure 3 As shown, the three-dimensional scanner 110 and the image retrieval device 120 are disposed on a bracket 140 , and the three-dimensional scanner 110 and the image retrieval device 120 face the placement table S, and the detection area A is located on the placement table S.
[0086] like Figure 1 , Figure 2 , Figure 3 as well as Figure 4As shown, the 3D scanner 110 is used to perform a 3D scan on the detection area A, obtain multiple distance values z(x, y) of multiple detection points on the detection area A, and generate a distance distribution P. The distance distribution P, that is, the distance values z(x, y) and the corresponding positions in the detection area A, form a point cloud, which can present the fluctuation state of the detection area A, so as to allow the data processing module 130 to determine whether there are objects placed on the detection area A, and can be used to analyze the 3D shape of the object. The 3D scanner 110 can be, but is not limited to, a stereo camera, a time of flight (TOF) camera, a lidar, or other optical or acoustic distance measuring devices, as long as it can detect multiple depth values.
[0087] like Figure 1 , Figure 2 , Figure 3 as well as Figure 4 As shown, the image search device 120 is used to search for an image C to be analyzed that includes the detection area A. In a specific embodiment, the image search device 120 is an RGB color camera, and can search for a color image C to be analyzed to facilitate image analysis. The image search device 120 is not limited to a color camera, and it is not excluded that it is a monochrome camera, so that the image C to be analyzed is a monochrome (black and white) image.
[0088] like Figure 1 , Figure 2 , Figure 3 as well as Figure 4 As shown, the data processing module 130 is electrically connected to the 3D scanner 110 and the image retrieval device 120 to receive the distance distribution P and the image to be analyzed C. The 3D scanner 110 and the image retrieval device 120 mounted on the bracket 140 are substantially located at the same height (relative to the detection area A), but are separated by a spacing distance D in the horizontal direction.
[0089] like Figure 4 As shown, the data processing module 130 can shift the plane coordinate relationship between the distance distribution P and the image to be analyzed C according to the interval distance D, and map the distance distribution P to the image to be analyzed C.
[0090] like Figure 5 As shown, in addition, the data processing module 130 also uses the pixel information of the detection area A in the image C to be analyzed and the actual size of the detection area A to calibrate the reference plane of the distance value z(x, y), that is, to obtain the actual reference distance value z0' from the image retrieval device 120 to the surface of the detection area A.
[0091] Generally speaking, the detection area A is usually rectangular and has a fixed length and width. For example, in the embodiment of the present invention, the surface of a dinner plate is used as the detection area A. A standardized dinner plate has a fixed length and width.
[0092] like Figure 5 As shown, in the standard operation state, the dinner plate is stably placed on a platform, so that there is a preset reference distance z0 between the surface of the detection area A and the image retrieval device 120. At this time, in the length direction X and the width direction Y, the actual length and the actual width of the detection area A occupy the number of pixels in the image C to be analyzed respectively.
[0093] When actually operating the food content recognition system 100 of the present invention, the user may not place the plate completely flat on the platform, but may hold the plate at a certain height. In this case, the distance between the plane of the detection area A and the image retrieval device 120 will be reduced, making the preset reference distance value z0 inapplicable, resulting in the subsequent threshold distance zmax being set incorrectly, affecting the subsequent judgment and calculation operations.
[0094] like Figure 5 As shown in FIG. 1 , when the dinner plate is raised and located at a higher height, the distance between the plane of the detection area A' and the image retrieval device 120 will be reduced to the actual reference distance value z0'. At this time, in the image C to be analyzed retrieved by the data processing module 130, the detection area A' will be enlarged, so that the number of pixels occupied by the length and width of the detection area A' in the image C to be analyzed is changed to Xn×Yn respectively.
[0095] In this case, the preset reference distance value z0 originally stored in the data processing module 130 is not applicable, and the actual reference distance value z0' needs to be obtained. At this time, the relationship between the actual reference distance value z0' and the preset reference distance value z0 can be expressed as a pixel ratio as follows:
[0096] z0' = z0 × (L / Xn); or
[0097] z0'=z0×(W / Yn).
[0098] like Figure 2 as well as Figure 3As shown, different foods 200 are respectively filled in different standard containers 301, 302, 303, and the openings of the standard containers 301, 302, 303 face upwards of the detection area A, so that the three-dimensional scanner 110 and the image retrieval device 120 can detect the internal conditions of the standard containers 301, 302, 303. In addition, each standard container 301, 302, 303 has a side wall surrounding the internal space, so that the standard containers 301, 302, 303 form an annular area in the detection area A that is higher than the detection area A. Specifically, the data processing module 130 can be built-in or connected to a container database 150, and the container database 150 stores information of the standard containers 301, 302, 303, such as the area and shape of the opening, the thickness of the side wall, and the cross-sectional area of the accommodation space. In the ideal standard containers 301, 302, 303, the cross-sectional area of the accommodation space is preferably kept consistent in the height direction Z, such as a columnar shape, which is convenient for subsequent analysis and calculation. However, it is not excluded that the accommodation space is in an inverted cone shape, that is, the cross-sectional area gradually decreases from the opening to the bottom. The relationship between the cross-sectional area and the height can be stored in the container database 150 in advance and can be quickly obtained through table lookup or formula calculation.
[0099] Generally speaking, for example, in a self-service meal, diners place the food 200 to be taken together with its standard container 301, 302, 303 on a meal plate. Therefore, the detection area A can be set to be located on the surface of a meal plate. By setting the color difference, the surface of the meal plate (i.e., the detection area A) can be contrasted with the environment, which is conducive to marking the detection area A. At the same time, the standardized meal plate can also allow the detection area A to have a fixed length (L) and width (W).
[0100] The following describes a method for identifying food and beverage content disclosed in an embodiment of the present invention.
[0101] like Figure 1 , Figure 2 , Figure 4 as well as Figure 6 As shown, first, the data processing module 130 starts and controls the three-dimensional scanner 110 and the image retrieval device 120 to work. According to the control of the data processing module 130, the three-dimensional scanner 110 performs a three-dimensional scan on the detection area A, obtains a plurality of distance values z(x, y) on the detection area A, and generates a distance distribution P, as shown in step Step 112. The image retrieval device 120 retrieves an image C to be analyzed including the detection area A, as shown in step Step 114. The aforementioned steps Step 112 and Step 114 can be performed simultaneously or sequentially.
[0102] like Figure 4 as well as Figure 6As shown, the data processing module 130 then maps the distance distribution P to the image to be analyzed C, as shown in step 120. This mapping step is used to make each distance value z(x, y) have one or a group of corresponding pixels on the image to be analyzed C, that is, each distance value z(x, y) has a corresponding coordinate value (x, y) on the image to be analyzed C.
[0103] like Figure 6 as well as Figure 7 As shown, a threshold distance zmax is set in the data processing module 130. According to the threshold distance zmax, the data processing module 130 defines the portion of the image C to be analyzed whose distance value z(x, y) is less than the threshold distance zmax as one or more blocks B to be identified, as shown in step 130.
[0104] like Figure 7 As shown, the threshold distance zmax can be set to fall on the surface of the detection area A, that is, the actual reference distance value z0' is set as the threshold distance zmax, or a value slightly smaller than the actual reference distance value z0' is set as the threshold distance zmax. Each standard container 301, 302, 303 has a side wall surrounding its internal space, so that the standard container 301, 302, 303 forms an annular area higher than the surface of the detection area A in the detection area A; even the bottom of the internal space of the empty standard container 301, 302, 303 will be higher than the surface of the detection area A due to the physical thickness of the standard container 301, 302, 303; therefore, as long as the height is higher than zero, that is, the pixel corresponding to the distance value z is less than the threshold distance zmax, can be regarded as the area covered by the standard container 301, 302, 303. Therefore, the data processing module 130 can separate the part occupied by the relevant pixels in the image C to be analyzed as the block B to be identified, instead of analyzing the entire image C to be analyzed.
[0105] like Figure 6 as well as Figure 7 As shown, next, the data processing module 130 performs image recognition on the block B to be identified, and identifies the type of a food 200 located in the block B to be identified, as shown in step 140 .
[0106] like Figure 6 as well as Figure 7As shown, in step 130, the data processing module 130 has segmented the block B to be identified where food and beverage may be placed, and excluded the pixel areas that are not related to the food 200 and the standard containers 301, 302, 303. Therefore, in step 140, the data processing module 130 only needs to perform image recognition on the block B to be identified without performing image recognition on the entire image C to be analyzed. The data processing module 130 needs to consume less computing resources, which can speed up the image recognition efficiency and effectively reduce the probability of recognition errors.
[0107] like Figure 3 as well as Figure 8 As shown, the data processing module 130 is built-in or externally connected to a food database 160. The food database 160 stores a variety of food information. The food information includes information such as the type of food 200, sample image, food density, nutritional ingredients, unit weight price (e.g., yuan / 100 grams), etc. By comparing the image of the block B to be identified and the sample image to see if they are similar, the data processing module 130 can identify the type of food 200 in the block B to be identified.
[0108] like Figure 6 As shown, according to the distance distribution P in the block B to be identified, the data processing module 130 obtains the volume of the food 200, as shown in step 150.
[0109] like Fig. 9 As shown, a specific embodiment of obtaining the volume of the food 200 is described as follows. The block B to be identified can be cut into multiple grid areas, each grid area corresponds to a distance value z(x, y). Based on the standard container information and the actual reference distance value z0' of the surface of the detection area A (i.e., the surface of the plate), the data processing module 130 can convert the grid height h of the food relative to the bottom of the internal space of the standard container 301, 302, 303 in each grid area. The grid height h multiplied by the grid area area Δa is the volume of the local food 200 in the grid area.
[0110] Finally, the data processing module 130 adds up the volumes of the food 200 in all grid areas, which is the volume of the food 200 in the standard containers 301, 302, and 303, as shown in the diagonal line portion of the figure. As mentioned above, the cross-sectional area of the storage space is preferably maintained consistent in the height direction Z, such as a columnar shape, which is beneficial for subsequent analysis and calculation. However, it is not excluded that the storage space is in an inverted cone shape; in this case, according to the standard container information, the grid height h can be modified according to the slope condition, such as Fig. 9The grid height h at the edge is deducted from the average height of the slope. The standard container information may include the three-dimensional shape of the standard containers 301, 302, 303 and the accommodating space, so the grid height h can be corrected according to the three-dimensional shape to obtain the correct volume of the food 200. Figure 5 The corresponding description of obtaining the actual reference distance value z0' is to avoid erroneously adopting the preset reference distance value z0, which may cause the grid height h to be erroneously increased, resulting in an erroneous calculation of the volume of the food 200.
[0111] like Figure 6 as well as Figure 8 As shown, finally, the weight of the food 200 is calculated based on the food information loaded into the food database 160 and the volume of the food 200, as shown in step 160. As mentioned above, the food information also includes information such as food density, nutritional components, and unit weight price, so the data processing module 130 multiplies the food density by the volume of the food 200 to obtain the weight of the food 200. In addition, the nutritional components in the composition are multiplied by the weight to obtain the nutritional composition of the food 200. Similarly, the price of the food 200 can be obtained by multiplying the unit weight price by the weight.
[0112] The aforementioned dining information, including the type, weight, nutritional composition and price of the food 200, can be associated with a specific diner. The data processing module 130 associates the diner identification information with the dining content and uploads it to a management platform 400 to perform the dining management or dining expense management of the diner, as shown in step 170. The identification method of the specific diner can be biometric identification, reading RFID, entering an account number and password, etc. The identification of the diner can be performed before the dining content identification method is executed, and can be performed after step 160.
[0113] The above-mentioned method for identifying food content is performed under an ideal state. In actual application scenarios, different problems may be encountered and need to be solved. The following further describes the variation examples of the above-mentioned method for identifying food content.
[0114] like Fig.10 , Fig.11 as well as Fig.12As shown, first, the start of the catering content recognition method can be manually started or automatically started. In the case of automatic start, the data processing module 130 starts and controls the three-dimensional scanner 110 and the image retrieval device 120 to work, and continuously detects the placement table S. The image retrieval device 120 continuously retrieves the image C to be analyzed, and the data processing module 130 continuously analyzes whether the image C to be analyzed contains the detection area A, as shown in step Step116. When it is detected that the image C to be analyzed contains the detection area A, the data processing module 130 executes the mapping distance distribution P to the image C to be analyzed, as shown in step Step120, and continues to execute step Step122.
[0115] like Fig.13 As shown, the detection area A is the surface of the dinner plate, which has a fixed shape and a specific color when viewed from above. Therefore, the data processing module 130 can determine whether the detection area A (i.e., the dinner plate) appears on the placement table S based on the shape and color, and automatically start the subsequent recognition process.
[0116] like Figure 5 as well as Fig.10 As shown, then, the data processing module 130 uses the pixel information in the image of the detection area A and the actual size of the detection area A to obtain the actual reference distance value z0', as shown in step Step 122. The relationship between the actual reference distance value z0' and the preset reference distance value z0 can be expressed as a pixel ratio as follows:
[0117] z0' = z0 × (L / Xn); or
[0118] z0'=z0×(W / Yn).
[0119] like Fig.10 As shown, the data processing module 130 analyzes whether the distribution of the distance value z(x, y) is reasonable based on the actual reference distance value z0', as shown in step Step 124. For example, based on the thickness of the dinner plate, the height of the standard containers 301, 302, 303, and the possible filling amount of food, each distance value z(x, y) should be within a range between an upper distance limit value and a lower distance limit value. If one of the distance values z(x, y) exceeds the range between the upper distance limit value and the lower distance limit value, the distance value z(x, y) is unreasonable, and steps Step 112 and Step 114 need to be re-executed.
[0120] The one or more to-be-identified blocks B found in step 130 may be standard containers 301, 302, 303. The openings of the standard containers 301, 302, 303 face upwards of the detection area A and have a fixed top-view shape.
[0121] like Fig.14 As shown, at this time, the data processing module 130 can remove the part other than one or more to-be-identified blocks B in the to-be-analyzed image C according to the detection area A identified in step 116 for use as a subsequent image.
[0122] like Fig.11 As shown, the data processing module 130 loads the information of the standard containers 301, 302, 303, such as the area and shape of the opening, from the container database 150, and analyzes whether the block B to be identified conforms to the area and shape of the opening of the standard containers 301, 302, 303, so as to determine whether each block B to be identified corresponds to the correctly placed standard container 301, 302, 303, as shown in step Step 132.
[0123] like Fig.15 As shown, incorrect placement includes: the standard containers 301, 302, 303 interfere with each other and overlap, so that the standard containers 301, 302, 303 are collectively defined as a single block B to be identified, resulting in the top view shape not conforming to the opening shape of any standard container 301, 302, 303; the standard containers 301, 302, 303 are tilted, resulting in the top view shape not conforming to the opening shape of any standard container 301, 302, 303; the placement of the standard containers 301, 302, 303 exceeds the detection area A, resulting in the shape of the part located in the detection area A not conforming to the opening shape of the standard containers 301, 302, 303; or, the container placed in the detection area A is an incorrect non-standard container 304.
[0124] like Fig.11 as well as Fig.15 As shown, when one or more blocks B to be identified do not conform to the opening shape of any standard container 301, 302, 303, the data processing module 130 determines that the standard container 301, 302, 303 corresponding to at least one block B to be identified is not placed correctly, then the data processing module 130 generates an error prompt and drives a display device or a warning device to issue an error prompt, as shown in step Step 134.
[0125] like Fig.11 As shown, if each block B to be identified corresponds to a correctly placed standard container 301, 302, 303, the data processing module 130 executes step 140 to identify the type of a food 200 located in the block B to be identified. When the type of the food 200 is successfully identified, the volume and weight of the food 200 are continuously calculated, as shown in steps 142, 150, and 160, and based on the identification of the diner, the diner identification information and the food content are associated, and the data is uploaded to the management platform 400, as shown in step 170.
[0126] like Fig.11 as well as Fig.12 As shown, part of the food 200 may be contained in a specific standard container 301, 302, 303. Therefore, if it is determined in step 142 that the type of the food 200 cannot be identified, the data processing module 130 analyzes whether the type of the food 200 can be determined from the types of the standard containers 301, 302, 303 based on the standard container information, as shown in step 144. If the type of the food 200 can be determined from the types of the standard containers 301, 302, 303, the data processing module 130 continues to execute steps 150 and 160. If the type of the food 200 cannot be determined from the types of the standard containers 301, 302, 303, the data processing module 130 executes step 134 and generates an error prompt. Step 144 can also be omitted. When it is determined in step 142 that the type of the food 200 cannot be identified, the data processing module 130 directly executes step 134.
[0127] The present invention combines three-dimensional scanning and image retrieval, and based on the correspondence between the retrieved image and the three-dimensional scanning information, marks the area occupied by the standard container in the detection area of the image to be analyzed, and only performs image recognition on the block to be identified corresponding to the standard container, instead of performing image recognition on the entire image to be analyzed, to determine the type of food. Therefore, the present invention can reduce the computational load required for image recognition and accelerate the process of identifying food. At the same time, through the application of standard containers, the distance distribution obtained by three-dimensional scanning can be used to effectively and relatively accurately calculate the volume of food. In conjunction with the association of food information, the complete catering information can be quickly analyzed, thereby combining the diner identification information for back-end management.
[0128] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.
Claims
1. A method for identifying catering content, It is characterized in that Include: Obtaining a plurality of distance values on a detection area to generate a distance distribution, and retrieving an image to be analyzed including the detection area; Mapping the distance distribution to the image to be analyzed; Setting a threshold distance to define each portion of the image to be analyzed whose distance value is smaller than the threshold distance as at least one block to be identified; Performing image recognition on the at least one block to be identified to identify the type of food located in the block to be identified; as well as According to the distance distribution, the volume of the food is obtained; Wherein, retrieving the image to be analyzed includes: retrieving the image to be analyzed by an image retrieval device, wherein there is an actual reference distance value between the image retrieval device and a surface of the detection area, and the threshold distance is less than or equal to the actual reference distance value; The number of pixels occupied by a length and a width of the detection area in the image to be analyzed is Xn×Yn respectively, and there is a preset reference distance value between the image retrieval device and the surface of the detection area. The number of pixels occupied by an actual length and an actual width of the detection area in the image to be analyzed under the preset reference distance value is L×W respectively. The relationship between the actual reference distance value and the preset reference distance value is: z0' = z0×(L / Xn); or z0' = z0×(W / Yn); Wherein, z0 is the preset reference distance value, and z0' is the actual reference distance value.
2. The method for identifying food and beverage content according to claim 1, It is characterized in that in, Before obtaining the plurality of distance values, the image to be analyzed is continuously searched to analyze whether the image to be analyzed includes the detection area.
3. The method for identifying food and beverage content according to claim 1, It is characterized in that The method further includes determining whether each of the distance values is within a range between an upper distance limit and a lower distance limit; if any of the distance values exceeds the range between the upper distance limit and the lower distance limit, re-obtaining the plurality of distance values and re-searching the image to be analyzed.
4. The method for identifying food and beverage content according to claim 1, It is characterized in that The method further includes identifying whether the at least one block to be identified conforms to a correctly placed standard container according to the distance distribution and at least one standard container information.
5. The method for identifying food and beverage content according to claim 1, It is characterized in that in, The step of identifying the type of the food located in the to-be-identified block includes: Loading at least one piece of food information; wherein the at least one piece of food information at least includes the type of the food and a sample image; The image of the block to be identified is compared with the sample image to determine whether they are similar, so as to identify the type of food in the block to be identified.
6. The method for identifying food and beverage content according to claim 5, It is characterized in that Also includes: Loading the at least one food information; wherein the at least one food information further includes the food density of the food; and Multiply the food density by the volume of the food to get the weight of the food.
7. The method for identifying food and beverage content according to claim 6, It is characterized in that Also includes: Loading the at least one food information; wherein the at least one food information further includes nutritional ingredients and price per unit weight; multiplying the nutrient content by the weight of the food to obtain a nutrient composition of the food; multiplying the unit weight price by the weight of the food to obtain the price of the food; and The type, weight, nutritional composition and price of the food are associated with a diner and uploaded to a management platform.
8. The method for identifying food and beverage content according to claim 1, It is characterized in that The method further includes analyzing whether the at least one block to be identified conforms to a correctly placed standard container; when the at least one block to be identified does not conform to the correctly placed standard container, an error prompt is generated.
9. A catering content recognition system, It is characterized in that Include: a three-dimensional scanner for acquiring a plurality of distance values on a detection area to generate a distance distribution; An image retrieval device, used to retrieve an image to be analyzed in the detection area; as well as a data processing module electrically connected to the three-dimensional scanner and the image retrieval device; The data processing module maps the distance distribution to the image to be analyzed; the data processing module sets a threshold distance to define each portion of the image to be analyzed whose distance value is less than the threshold distance as at least one block to be identified; and the data processing module performs image recognition on the at least one block to be identified, identifies the type of a food located in the at least one block to be identified, and obtains the volume of the food according to the distance distribution; There is an actual reference distance value between the image retrieval device and a surface of the detection area, and the threshold distance is less than or equal to the actual reference distance value; The length and width of the detection area occupy the number of pixels in the image to be analyzed, respectively, Xn×Yn, and there is a preset reference distance value between the image retrieval device and the surface of the detection area. The actual length and the actual width of the detection area at the preset reference distance value occupy the number of pixels in the image to be analyzed, respectively, L×W. The relationship between the actual reference distance value and the preset reference distance value is: z0' = z0×(L / Xn); or z0' = z0×(W / Yn); Wherein, z0 is the preset reference distance value, and z0' is the actual reference distance value.
10. The food and beverage content recognition system according to claim 9, It is characterized in that in, Before obtaining the plurality of distance values, the image retrieval device continuously retrieves the image to be analyzed, and the data processing module analyzes whether the image to be analyzed includes the detection area.
11. The food and beverage content recognition system according to claim 9, It is characterized in that in, The data processing module determines whether each of the distance values is within a range between an upper distance limit and a lower distance limit; if any of the distance values exceeds the range between the upper distance limit and the lower distance limit, the three-dimensional scanner and the image retrieval device are used to re-acquire the multiple distance values and re-retrieve the image to be analyzed.
12. The food and beverage content recognition system according to claim 9, It is characterized in that in, The data processing module loads at least one standard container information, and identifies whether the at least one block to be identified conforms to a correctly placed standard container according to the distance distribution and the at least one standard container information.
13. The food and beverage content recognition system according to claim 9, It is characterized in that in, The data processing module loads the type of food and a sample image, and compares the image of the block to be identified with the sample image to see if they are similar, so as to identify the type of food in the block to be identified.
14. The food and beverage content recognition system according to claim 13, It is characterized in that in, The data processing module loads the food density of the food, and multiplies the food density by the volume of the food to obtain the weight of the food.
15. The food and beverage content recognition system according to claim 14, It is characterized in that in, The data processing module loads the nutritional components and the unit weight price, multiplies the nutritional components by the weight of the food to obtain a nutritional composition of the food, and multiplies the unit weight price by the weight of the food to obtain the price of the food; The data processing module associates the type, weight, nutritional composition and price of the food with a diner and uploads it to a management platform.
16. The food and beverage content recognition system according to claim 9, It is characterized in that in, The data processing module analyzes whether the at least one block to be identified conforms to a correctly placed standard container; when the at least one block to be identified does not conform to the correctly placed standard container, an error prompt is generated.
