A food material three-dimensional information acquisition system, method, device and electronic equipment

Abstract

The application provides a food material three-dimensional information acquisition system, method, device and electronic equipment. The method comprises the following steps: acquiring a first image collected by a first image collection device and a second image collected by a second image collection device; determining a complete image of a target food material based on the first image and the second image; the complete image comprises all appearance regions of the target food material except a region in contact with a table top; determining a first projection point of a space point corresponding to each pixel point of the target food material on the table top, a first projection point coordinate of the first projection point and / or a second projection point and a second projection point coordinate of the second projection point based on the complete image; determining height information of each space point from the table top based on the first projection point coordinate and / or the second projection point coordinate of each space point; combining the height information with the first projection point coordinate or the second projection point coordinate to form a space coordinate of each space point; and determining three-dimensional space information of the target food material based on the space coordinates of each space point. The application improves the accuracy of food material space information acquisition.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional data processing technology, and more specifically, to a system, method, apparatus, and electronic device for acquiring three-dimensional information about food ingredients. Background Technology

[0002] With the development of smart kitchens, more and more smart kitchen appliances are being developed, such as devices that automatically generate electronic recipes based on ingredient information.

[0003] Currently, existing technologies mainly use photography to acquire 2D images, obtain the appearance information of ingredients from the 2D images, and then use the appearance information of ingredients to generate electronic recipes.

[0004] However, 2D images of ingredients can only provide local two-dimensional image information, which results in insufficient accuracy in the obtained appearance information of the ingredients, and consequently, insufficient accuracy in the cooking parameters in the generated electronic recipes. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a three-dimensional information acquisition system, method, device and electronic device for food ingredients, so as to improve the accuracy of acquiring the appearance information of food ingredients.

[0006] In a first aspect, a three-dimensional information acquisition system for food ingredients is provided, comprising: a table for placing food ingredients and an information acquisition device located above the table, the information acquisition device comprising a first image acquisition device, a second image acquisition device and a processor;

[0007] The first image acquisition device is centrally located below the information acquisition device to capture images of the area where the table is located. The first image acquisition device has a scaling function to scale the two-dimensional coordinate system of the plane below the first image acquisition device.

[0008] The second image acquisition device is movably mounted below the information acquisition device via a track surrounding the first image acquisition device to capture images of the area where the table is located. The second image acquisition device and the first image acquisition device are on the same plane, which is parallel to the table.

[0009] The processor is used to determine the spatial three-dimensional information of the food based on the first image data and the second image data. The first image data is obtained by the first image acquisition device, and the second image data is obtained by the second image acquisition device.

[0010] Optionally, the countertop is a cooktop, and the information collection device is a range hood; or, the countertop is a tray inside an oven, and the information collection device is an oven.

[0011] Secondly, a method for acquiring three-dimensional information of food ingredients based on the food ingredient three-dimensional information acquisition system of the first aspect is provided, the method comprising:

[0012] Acquire a first image acquired by a first image acquisition device and a second image acquired by a second image acquisition device;

[0013] A complete image of the target ingredient is determined based on the first image and the second image; the complete image includes all the appearance areas of the target ingredient except for the area in contact with the countertop.

[0014] Based on the complete image, determine the first projection point and its coordinates and / or the second projection point and its coordinates on the table surface corresponding to each pixel of the target food ingredient; wherein, the first projection point is the projection point projected onto the table surface with the first image acquisition device as the projection center; the second projection point is the projection point projected onto the table surface with the second image acquisition device as the projection center, and the coordinates of the first projection point and the coordinates of the second projection point are both two-dimensional coordinates in a two-dimensional coordinate system scaled by the first image acquisition device on the table surface below it.

[0015] The height information of each spatial point from the platform is determined based on the first projection point coordinates and / or the second projection point coordinates; and the height information is combined with the first projection point coordinates or the second projection point coordinates to form the spatial coordinates of the spatial point.

[0016] The spatial three-dimensional information of the target ingredient is determined based on the spatial coordinates of each spatial point.

[0017] Optionally, determining the height information of each spatial point from the platform based on the coordinates of its first or second projection point includes:

[0018] The preset parameters of the food ingredient 3D information acquisition system include the preset height information of the first image acquisition device from the table surface and the preset distance between the first image acquisition device and the second image acquisition device.

[0019] Determine the vertical projection point of each spatial point on the platform and the vertical projection coordinates of the vertical projection point; the vertical projection point is the projection with the spatial point as the projection center, and the vertical projection coordinates are the two-dimensional coordinates in the two-dimensional coordinate system scaled by the first image acquisition device on the platform below it.

[0020] Based on preset parameters, the vertical projection coordinates of each spatial point, and one of the coordinates of the first and second projection points, trigonometric functions are used to calculate the height information of each spatial point from the platform.

[0021] Optionally, determining the height information of each spatial point from the platform based on the coordinates of its first and second projection points includes:

[0022] Based on the coordinates of the first and second projection points of each spatial point, the height information of each spatial point from the platform is queried in the pre-constructed three-dimensional database. The three-dimensional database stores the height information of each segmented spatial point obtained by three-dimensional segmentation of the space below the first image acquisition device, as well as the coordinates of the first and second projection points on the platform.

[0023] Optionally, searching for the height information of each spatial point from the platform in a pre-built 3D database based on the coordinates of the first or second projection point of each spatial point includes:

[0024] Based on the coordinates of the second projection point of each spatial point, the coordinates of the first projection point of the corresponding spatial point are found in the pre-built 3D database;

[0025] If there are two or more first projection point coordinates, then the first projection point coordinates closest to the origin of the two-dimensional coordinate system on the platform shall be the target first projection point coordinates.

[0026] The height information of the spatial point is determined based on the coordinates of the second projection point and the first projection point of the target.

[0027] Optionally, after calculating the height information of each spatial point, the method further includes:

[0028] Based on the coordinates of the first projection point, the coordinates of the second projection point, and the preset height information of the first image acquisition device from the platform, trigonometric functions are used to calculate the first angle between the line connecting the first projection point and the origin of the two-dimensional coordinate system and the X-axis of the two-dimensional coordinate system, and the second angle between the first image acquisition device and the first projection point and the origin.

[0029] The spatial coordinates of each point are formed based on the first included angle, the second included angle, and the height information.

[0030] Alternatively, based on the coordinates of the second projection point, the preset height information of the first image acquisition device from the table surface, and the distance between the first image acquisition device and the second image acquisition device, trigonometric functions can be used to calculate the third angle between the line connecting the second projection point and the perpendicular projection point of the second image acquisition device and the X-axis of the two-dimensional coordinate system, and the fourth angle between the line connecting the second image acquisition device and the second projection point and the perpendicular line of the second image acquisition device.

[0031] The spatial coordinates of each point are formed based on the third included angle, the fourth included angle, and the height information.

[0032] Secondly, a device for acquiring three-dimensional information of food ingredients is provided, the device comprising:

[0033] The acquisition unit is used to acquire a first image acquired by a first image acquisition device and a second image acquired by a second image acquisition device.

[0034] The first determining unit is used to determine a complete image of the target food ingredient based on the first image and the second image; the complete image includes all the appearance areas of the target food ingredient except for the area in contact with the countertop.

[0035] The second determining unit is used to determine, based on the complete image, the first projection point and its coordinates and / or the second projection point and its coordinates on the table surface corresponding to each pixel of the target food ingredient; wherein, the first projection point is the projection point projected onto the table surface with the first image acquisition device as the projection center; the second projection point is the projection point projected onto the table surface with the second image acquisition device as the projection center, and the coordinates of the first projection point and the coordinates of the second projection point are both two-dimensional coordinates in a two-dimensional coordinate system scaled by the first image acquisition device on the table surface below it.

[0036] The third determining unit is used to determine the height information of each spatial point from the platform based on the first projection point coordinates and / or the second projection point coordinates; and to combine the height information with the first projection point coordinates or the second projection point coordinates to form the spatial coordinates of the spatial point.

[0037] The fourth determining unit is used to determine the spatial three-dimensional information of the target food ingredient based on the spatial coordinates of each spatial point.

[0038] Fourthly, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0039] Memory, used to store computer programs;

[0040] When a processor executes a program stored in memory, it implements any of the steps described in the first aspect.

[0041] Fourthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the steps of any of the methods described in the first aspect.

[0042] This invention provides a three-dimensional information acquisition system, method, apparatus, and electronic device for food ingredients. The method utilizes the scaling function of a first image acquisition device to scale a two-dimensional coordinate system on a lower plane. Based on this two-dimensional coordinate system, the projected coordinates of each point of the target food ingredient on the two-dimensional coordinate system are obtained, and the height of each point of the target food ingredient is determined based on the projected coordinates. Based on the height and projected coordinates, the spatial coordinate data of each point of the target food ingredient is obtained. Furthermore, a second image acquisition device works in conjunction with the first image acquisition device to obtain the complete spatial coordinate data of the target food ingredient, thereby improving the accuracy of acquiring the spatial information of the target food ingredient.

[0043] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A schematic diagram of the food ingredient three-dimensional information acquisition system provided in an embodiment of the present invention is shown;

[0046] Figure 2 A flowchart of the method for obtaining three-dimensional information of food ingredients provided in an embodiment of the present invention is shown;

[0047] Figure 3 This shows a schematic diagram of the projection of a spatial point provided in an embodiment of the present invention;

[0048] Figure 4 This diagram illustrates the grid division on the platform provided in an embodiment of the present invention.

[0049] Figure 5 A schematic diagram of the space division above the countertop provided in an embodiment of the present invention is shown;

[0050] Figure 6 A schematic diagram of the structure of the food ingredient three-dimensional information acquisition device provided in an embodiment of the present invention is shown;

[0051] Figure 7 A schematic diagram of the structure of the electronic device provided in an embodiment of the present invention is shown. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0053] Considering that current methods for obtaining 2D images of ingredients can only provide localized two-dimensional image information, resulting in inaccurate ingredient information and consequently, inaccurate cooking parameters in generated electronic recipes, this invention provides a method for acquiring 3D ingredient information. This method is implemented based on a 3D ingredient information acquisition system, which includes: a platform for placing ingredients and an information acquisition device located above the platform. The information acquisition device includes a first image acquisition device, a second image acquisition device, and a processor.

[0054] The first image acquisition device is centrally located below the information acquisition device to capture images of the area where the table is located. The first image acquisition device has a scaling function to scale the two-dimensional coordinate system of the plane below the first image acquisition device.

[0055] In this embodiment of the invention, the angle and position of the first image acquisition device are fixed.

[0056] The second image acquisition device is movably mounted below the information acquisition device via a track surrounding the first image acquisition device to capture images of the area where the table is located. The second image acquisition device and the first image acquisition device are on the same plane, which is parallel to the table.

[0057] The track of the second image acquisition device can be circular, elliptical, rectangular, or other shapes. Once the shape of the track is determined, the position and angle of the second image acquisition device relative to the first image acquisition device are also fixed.

[0058] The processor is used to determine the spatial three-dimensional information of the food based on the first image data and the second image data. The first image data is obtained by the first image acquisition device, and the second image data is obtained by the second image acquisition device.

[0059] This invention uses two image acquisition devices to collect image information of ingredients, which can obtain relatively complete image data, thereby constructing more accurate three-dimensional spatial information of ingredients. Combined with the spatial three-dimensional information of ingredients and other information such as weight, it can help generate a more accurate electronic recipe.

[0060] In one specific embodiment, such as Figure 1 As shown, the countertop is the stove countertop, and the information collection device is the range hood. In this... Figure 1 In the diagram, A represents the first image acquisition device, B represents the second image acquisition device, C represents the stove (which can be an induction cooker stove), and D represents the track.

[0061] In another embodiment, the information acquisition device may also be an oven, with the first image acquisition device and the second image acquisition device installed on the top inside the oven, and the work surface being a tray inside the oven.

[0062] The process of acquiring three-dimensional information of food ingredients based on this system will be described below through an example.

[0063] This invention provides a method for acquiring three-dimensional information of food ingredients, such as... Figure 2 As shown, the method includes the following steps:

[0064] Step S201: Acquire the first image acquired by the first image acquisition device and the second image acquired by the second image acquisition device.

[0065] In this step, the first image acquired by the first image acquisition device mainly consists of images of the top and sides of the target ingredient. Because the shape and size of the target ingredient vary, and the acquisition angle of the first image acquisition device is fixed, the first image cannot completely cover the entire appearance area of ​​the target ingredient. For example, when the target ingredient is large, only a portion of the side can be captured; or, if the side of the target ingredient has a concave area, the first image acquisition device cannot capture that concave area.

[0066] The second image acquisition device is rotated, allowing it to capture images from multiple angles, ensuring that the appearance of the target food can be captured.

[0067] Step S202: Determine a complete image of the target ingredient based on the first image and the second image; the complete image includes all the appearance areas of the target ingredient except for the area in contact with the countertop.

[0068] In this step, the first image and the second image can be stitched together to obtain the complete appearance area of ​​the target ingredient.

[0069] In one example, the splicing process is as follows:

[0070] Step 1: First, use computer vision algorithms (such as SIFT, SURF, ORB, etc.) to detect feature points on each image.

[0071] These feature points are typically key points that are invariant to scale and rotation changes.

[0072] Step 2: Calculate a feature descriptor for each feature point, and use the feature descriptors to match the feature points in the two images.

[0073] This step is to find the correspondence between the two images, that is, which points in the two images represent the same physical location.

[0074] Step 3: Based on the matched feature point pairs, calculate a transformation matrix that can map points on the second image to corresponding points on the first image.

[0075] Step 4: Use the transformation matrix calculated above to deform the second image at various angles so that it can be aligned with the first image in the overlapping area. Then stitch the deformed first images and second images together.

[0076] The above method can be used to obtain a complete image of the target ingredient.

[0077] Step S203: Based on the complete image, determine the first projection point and its coordinates and / or the second projection point and its coordinates on the table surface corresponding to each pixel of the target food.

[0078] The first projection point is the projection point projected onto the table surface with the first image acquisition device as the projection center; the second projection point is the projection point projected onto the table surface with the second image acquisition device as the projection center. The coordinates of the first projection point and the coordinates of the second projection point are both two-dimensional coordinates in a two-dimensional coordinate system scaled by the first image acquisition device on the table surface below it.

[0079] In a specific example, such as Figure 3 As shown, the first projection point of spatial point T is Tca, and the second projection point is Tcb.

[0080] Step S204: Determine the height information of each spatial point from the platform based on the first projection point coordinates and / or the second projection point coordinates; and combine the height information with the first projection point coordinates or the second projection point coordinates to form the spatial coordinates of the spatial point.

[0081] In this step, such as Figure 3 As shown, the height of a spatial point is LT. The position of the spatial point T in space can be determined by the coordinates Tca(x, y) of Tca and LT. Alternatively, the position of the spatial point T in space can be determined by the coordinates Tcb(x, y) of Tcb and LT.

[0082] Step S205: Determine the spatial three-dimensional information of the target ingredient based on the spatial coordinates of each spatial point.

[0083] The spatial three-dimensional information of the target ingredient can be obtained by using the spatial coordinates of each spatial point.

[0084] This invention utilizes the scaling function of the first image acquisition device to scale a two-dimensional coordinate system on the lower plane. Based on this two-dimensional coordinate system, the projected coordinates of each point of the target ingredient on the two-dimensional coordinate system are obtained, and the height of each point of the target ingredient is determined based on the projected coordinates, thereby obtaining the spatial coordinate data of each point of the target ingredient. Furthermore, the second image acquisition device works in conjunction with the first image acquisition device to obtain the complete spatial coordinate data of the target ingredient, thereby improving the modeling accuracy of the target ingredient.

[0085] Based on the above embodiments, determining the height information of each spatial point from the platform based on the coordinates of the first or second projection point includes:

[0086] Step S204A1: Obtain the preset parameters of the food ingredient 3D information acquisition system. The preset parameters include the preset height information of the first image acquisition device from the table surface and the preset distance between the first image acquisition device and the second image acquisition device.

[0087] In one example, such as Figure 3 As shown, the first image acquisition device is A, and its preset height from the table surface C is LA. The second image acquisition device is B, and the preset distance between A and B is Lab.

[0088] Step S204A2: Determine the vertical projection point of each spatial point on the platform and the vertical projection coordinates of the vertical projection point; the vertical projection point is the projection with the spatial point as the projection center, and the vertical projection coordinates are two-dimensional coordinates in a two-dimensional coordinate system scaled by the first image acquisition device on the platform below it.

[0089] Step S204A3: Based on preset parameters, the vertical projection coordinates of each spatial point, and one of the coordinates of the first projection point and the second projection point, calculate the height information of each spatial point from the platform using trigonometric functions.

[0090] In this embodiment of the invention, the calculation process of the height information of each spatial point from the platform is described in detail, taking preset parameters, the vertical projection coordinates of each spatial point and the coordinates of the second projection point as examples.

[0091] like Figure 3 As shown, the distance BB0 from the platform of the second image acquisition device B is the same as LA. For a spatial point T, the coordinates of its second projection point are Tcb(x, y), and the height from the platform is LT. The distance from point Tcb to B0 is calculated based on Tcb(x, y). In the right triangle B0BTcb, the length of BTcb is calculated based on the length of B0Tcb and the length of BB0. ∠Bz is calculated using trigonometric functions. In the right triangle TT0Tcb, ∠T0TTcb is the same as ∠Bz. The distance from the perpendicular projection point T0 of point T to B0 can be calculated based on the coordinates of T0, and thus the length from T0 to Tcb can be obtained. Finally, the length of LT can be calculated.

[0092] In another calculation method, the principle of similar triangles can be used. Right triangle TT0Tcb is similar to right triangle BB0Tcb. Given the lengths of B0Tcb, T0Tcb, and BB0, the length of LT can be calculated.

[0093] Similarly, using the preset parameters, the vertical projection coordinates of each spatial point, and the coordinates of the first projection point, the height LT of point T can also be calculated using the above calculation method, which will not be elaborated here.

[0094] In this embodiment, the three-dimensional spatial information of the target food ingredient is obtained through real-time calculation based on the image data collected by the first image acquisition device and the second image acquisition device. This method can obtain almost all the spatial point information of the target food ingredient, and a high-precision target food ingredient model can be constructed using this spatial point information.

[0095] Based on the above embodiments, determining the height information of each spatial point from the platform based on the coordinates of the first and second projection points includes:

[0096] Step S204B: Based on the coordinates of the first and second projection points of each spatial point, query the height information of each spatial point from the platform in the pre-constructed three-dimensional database; the three-dimensional database stores the height information of each segmented spatial point obtained by three-dimensional segmentation of the space below the first image acquisition device, as well as the coordinates of the first and second projection points on the platform.

[0097] To obtain the height information of spatial points using this query method, the spatial area below the first image acquisition device is first divided into three dimensions to obtain several spatial modules. Based on the division scale data, the spatial coordinate data of each spatial module can be directly obtained and stored in the three-dimensional database. When actually acquiring images of the target food, the data can be directly searched. This method is efficient, but because the number of spatial points is limited, the accuracy is lower than that of the above embodiment. To improve the accuracy, the segmentation accuracy can be improved during the three-dimensional segmentation of the space.

[0098] The process of dividing three-dimensional space is explained in detail below:

[0099] The first step is to divide the plane on the platform according to the two-dimensional coordinate system scaled by the first image acquisition device, such as... Figure 4 As shown, the X-axis is divided into 2n parts and the Y-axis is divided into 2m parts, resulting in a total of 2n*2m spatial modules.

[0100] The scale of the two-dimensional coordinate system determines the accuracy of the three-dimensional spatial division. The scale of the two-dimensional coordinate system can be set according to the size of the platform. The larger the area of ​​the platform, the larger the scale of the two-dimensional coordinate system can be, and vice versa.

[0101] The second step involves dividing the Z-axis into p planes at equal intervals, named C1, C2, ..., Cp-1, Cp. Each plane is further divided into 2n*2m spatial modules, resulting in a total of 2n*2m*p spatial modules. The height of each spatial module relative to the platform can then be determined based on the Z-axis division scale.

[0102] Similarly, the interval of the Z-axis division determines the accuracy of the three-dimensional segmentation. The interval can be set according to the relative distance between the first image acquisition device and the platform. The larger the relative distance, the larger the interval can be set, and the smaller the relative distance, the smaller the interval can be set.

[0103] After obtaining the various spatial modules, it is necessary to acquire the spatial coordinate data of each spatial module and store it in the 3D database.

[0104] The second image acquisition device rotates 360 degrees, which can be evenly divided into q acquisition angles. At each acquisition angle, a partial side image of the target food is acquired once. Then, these 2n*2m*p spatial modules can yield 2n*2m*p*q second projection point coordinates. The spatial coordinates of each second projection point are then combined with the coordinates of the first projection point and the height information to form the spatial coordinates of each spatial module.

[0105] After constructing the 3D database, the corresponding height information can be found based on the coordinates of two projection points. This is because if height information is found based on the coordinates of a projection point in only one direction, then two or even more spatial points might correspond to the same projection point coordinates, such as... Figure 3 As shown, with the first image acquisition device A as the projection center, spatial points T and T1 are projected. Points T and T1 are exactly on the same projection direction, and the first projection point of points T1 and T is the same, both being Tca. Therefore, their corresponding first projection coordinates Tca(x, y) are also the same. If only the first projection point coordinates Tca(x, y) are used to find the height information, the height LT of point T and the height LT1 of point T1 will be found simultaneously in the 3D database, making it impossible to determine the specific spatial point and its height information. However, by combining the second projection coordinates of the second image acquisition device B, it is possible to determine whether the spatial point corresponding to the first projection coordinates Tca(x, y) is T or T1. For example, if the second projection coordinates are Tcb(x, y), then the corresponding spatial point is T, and the found height information is LT. If the second projection coordinates are Tcb1(x, y), then the corresponding spatial point is T1, and the found height is LT1.

[0106] In areas of the target food that can be captured by both the first and second image acquisition devices, the first and second projection coordinates of each point in the area can be obtained simultaneously. However, for areas that cannot be captured by the first image acquisition device, such as side recessed areas, or areas that cannot be captured by the second image acquisition device, such as top recessed areas of the target food, only the first or second projection coordinates of each point in the area can be obtained. In the case of only having the first and second projection coordinates, how to find the specific height information in the three-dimensional database is explained in the following embodiments.

[0107] Based on the above embodiments, searching for the height information of each spatial point from the platform in a pre-built 3D database based on the coordinates of the first or second projection point of each spatial point includes:

[0108] Step S104C1: Based on the coordinates of the second projection point of each spatial point, search for the coordinates of the first projection point of the corresponding spatial point in the pre-built 3D database.

[0109] Step S104C2: If there are two or more first projection point coordinates, then the first projection point coordinates closest to the origin of the two-dimensional coordinate system on the platform shall be the target first projection point coordinates.

[0110] Continuing from the previous example, if the first projection point coordinates of T1 and T are the same but the heights are different, then we can select the first projection point coordinates of the point closest to the origin, because the closer it is to the origin, the deeper it is, and the closer it is to the point in the concave area.

[0111] Step S104C3: Determine the height information of the spatial point based on the coordinates of the second projection point and the first projection point of the target.

[0112] In another implementation, the average of the multiple height information values ​​found can be used as the final height information.

[0113] Based on the above embodiments, after calculating the height information of each spatial point, the method further includes:

[0114] Based on the coordinates of the first projection point, the coordinates of the second projection point, and the preset height information of the first image acquisition device from the platform, trigonometric functions are used to calculate the first angle between the line connecting the first projection point and the origin of the two-dimensional coordinate system and the X-axis of the two-dimensional coordinate system, and the second angle between the first image acquisition device and the first projection point and the origin.

[0115] The spatial coordinates of each point are formed based on the first included angle, the second included angle, and the height information.

[0116] Alternatively, based on the coordinates of the second projection point, the preset height information of the first image acquisition device from the table surface, and the distance between the first image acquisition device and the second image acquisition device, trigonometric functions can be used to calculate the third angle between the line connecting the second projection point and the perpendicular projection point of the second image acquisition device and the X-axis of the two-dimensional coordinate system, and the fourth angle between the line connecting the second image acquisition device and the second projection point and the perpendicular line of the second image acquisition device.

[0117] The spatial coordinates of each point are formed based on the third included angle, the fourth included angle, and the height information.

[0118] In a specific example, such as Figure 6 As shown, for example, (LT, ∠A, ∠Az) can be used as the spatial coordinates of point T, or (LT, ∠B, ∠Bz) can be used as the spatial coordinates of point T.

[0119] Based on the same inventive concept, a device for acquiring three-dimensional information about food ingredients is provided, such as... Figure 6 As shown, the device includes:

[0120] The acquisition unit 601 is used to acquire a first image acquired by the first image acquisition device and a second image acquired by the second image acquisition device.

[0121] The first determining unit 602 is used to determine a complete image of the target food ingredient based on the first image and the second image; the complete image includes all the appearance areas of the target food ingredient except for the area in contact with the countertop.

[0122] The second determining unit 603 is used to determine, based on the complete image, the first projection point and its coordinates and / or the second projection point and its coordinates on the table surface corresponding to each pixel of the target food ingredient; wherein, the first projection point is the projection point projected onto the table surface with the first image acquisition device as the projection center; the second projection point is the projection point projected onto the table surface with the second image acquisition device as the projection center, and the coordinates of the first projection point and the coordinates of the second projection point are both two-dimensional coordinates in a two-dimensional coordinate system scaled by the first image acquisition device on the table surface below it.

[0123] The third determining unit 604 is used to determine the height information of each spatial point from the platform based on the first projection point coordinates and / or the second projection point coordinates; and to combine the height information with the first projection point coordinates or the second projection point coordinates to form the spatial coordinates of the spatial point.

[0124] The fourth determining unit 605 is used to determine the spatial three-dimensional information of the target food ingredient based on the spatial coordinates of each spatial point.

[0125] Based on the same technical concept, embodiments of the present invention also provide an electronic device, such as... Figure 7As shown, it includes a processor 701, a communication interface 702, a memory 703, and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0126] Memory 703 is used to store computer programs;

[0127] The processor 701 is used to execute the program stored in the memory 703 to implement the steps of the method for acquiring three-dimensional information of food ingredients.

[0128] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0129] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0130] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0131] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0132] The computer program product for the method of acquiring three-dimensional information of food ingredients provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0133] The device for acquiring three-dimensional information of food ingredients provided in this embodiment of the invention can be specific hardware on a device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in this embodiment of the invention are the same as those in the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiments can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0134] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0135] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0136] In addition, the functional units in the embodiments provided by the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0137] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0138] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0139] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. All should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A three-dimensional information acquisition system for food ingredients, characterized in that, include: A countertop for placing food ingredients and an information acquisition device located above the countertop, the information acquisition device including a first image acquisition device, a second image acquisition device and a processor; The first image acquisition device is centrally located below the information acquisition device to capture images of the area where the table is located. The first image acquisition device has a scaling function to scale the two-dimensional coordinate system of the plane below the first image acquisition device. The second image acquisition device is movably mounted below the information acquisition device via a track surrounding the first image acquisition device to capture images of the area where the table is located. The second image acquisition device and the first image acquisition device are on the same plane, which is parallel to the table. The processor is used to determine the spatial three-dimensional information of the food ingredient based on the first image data and the second image data. The first image data is obtained by the first image acquisition device, and the second image data is obtained by the second image acquisition device.

2. The system according to claim 1, characterized in that, The countertop is a cooktop, and the information collection device is a range hood; or, the countertop is a tray inside an oven, and the information collection device is an oven.

3. A method for acquiring three-dimensional information of food ingredients based on the three-dimensional information acquisition system for food ingredients according to claim 1, characterized in that, The method includes: Acquire a first image acquired by a first image acquisition device and a second image acquired by a second image acquisition device; A complete image of the target ingredient is determined based on the first image and the second image; the complete image includes all the appearance areas of the target ingredient except for the area in contact with the countertop. Based on the complete image, determine the first projection point and its coordinates and / or the second projection point and its coordinates on the table surface corresponding to each pixel of the target food ingredient; wherein, the first projection point is the projection point projected onto the table surface with the first image acquisition device as the projection center; the second projection point is the projection point projected onto the table surface with the second image acquisition device as the projection center, and the coordinates of the first projection point and the coordinates of the second projection point are both two-dimensional coordinates in a two-dimensional coordinate system scaled by the first image acquisition device on the table surface below it. The height information of each spatial point from the platform is determined based on the first projection point coordinates and / or the second projection point coordinates; and the height information is combined with the first projection point coordinates or the second projection point coordinates to form the spatial coordinates of the spatial point. The spatial three-dimensional information of the target food ingredient is determined based on the spatial coordinates of each spatial point.

4. The method according to claim 3, characterized in that, The information determining the height of each spatial point from the platform based on the coordinates of the first or second projection point includes: The preset parameters of the food ingredient 3D information acquisition system include the preset height information of the first image acquisition device from the table surface and the preset distance between the first image acquisition device and the second image acquisition device. Determine the vertical projection point of each spatial point on the table surface and the vertical projection coordinates of the vertical projection point; the vertical projection point is the projection with the spatial point as the projection center, and the vertical projection coordinates are two-dimensional coordinates in a two-dimensional coordinate system scaled by the first image acquisition device on the table surface below it. Based on the preset parameters, the vertical projection coordinates of each spatial point, and one of the coordinates of the first projection point and the second projection point, the height information of each spatial point from the platform is calculated using trigonometric functions.

5. The method according to claim 3, characterized in that, The height information of each spatial point relative to the platform is determined based on the coordinates of the first and second projection points, including: Based on the coordinates of the first and second projection points of each spatial point, the height information of each spatial point from the platform is queried in the pre-constructed three-dimensional database; the three-dimensional database stores the height information of each segmented spatial point obtained by three-dimensionally segmenting the space below the first image acquisition device, as well as the coordinates of the first and second projection points on the platform.

6. The method according to claim 3, characterized in that, Based on the coordinates of the first or second projection point of each spatial point, the height information of each spatial point from the platform is retrieved from the pre-built 3D database, including: Based on the coordinates of the second projection point of each spatial point, the coordinates of the first projection point of the corresponding spatial point are found in the pre-built 3D database; If there are two or more first projection point coordinates, then the first projection point coordinates closest to the origin of the two-dimensional coordinate system on the platform shall be the target first projection point coordinates. The height information of the spatial point is determined based on the coordinates of the second projection point and the first projection point of the target.

7. The method according to claim 3, characterized in that, After calculating the height information of each spatial point, the method further includes: Based on the coordinates of the first projection point, the coordinates of the second projection point, and the preset height information of the first image acquisition device from the platform, trigonometric functions are used to calculate the first angle between the line connecting the first projection point and the origin of the two-dimensional coordinate system and the X-axis of the two-dimensional coordinate system, and the second angle between the first image acquisition device and the first projection point and the origin. The spatial coordinates of each spatial point are formed based on the first included angle, the second included angle, and the height information. Alternatively, based on the coordinates of the second projection point, the preset height information of the first image acquisition device from the table surface, and the distance between the first image acquisition device and the second image acquisition device, trigonometric functions can be used to calculate the third angle between the line connecting the second projection point and the perpendicular projection point of the second image acquisition device and the X-axis of the two-dimensional coordinate system, and the fourth angle between the line connecting the second image acquisition device and the second projection point and the perpendicular line of the second image acquisition device. The spatial coordinates of each point are formed based on the third included angle, the fourth included angle, and the height information.

8. A device for acquiring three-dimensional information about food ingredients, characterized in that, The device includes: The acquisition unit is used to acquire a first image acquired by a first image acquisition device and a second image acquired by a second image acquisition device. The first determining unit is used to determine a complete image of the target food ingredient based on the first image and the second image; the complete image includes all the appearance areas of the target food ingredient except for the area in contact with the countertop. The second determining unit is used to determine, based on the complete image, the first projection point and its coordinates, and / or the second projection point and its coordinates, on the table surface corresponding to each pixel of the target food ingredient; wherein, the first projection point is a projection point projected onto the table surface with the first image acquisition device as the projection center; the second projection point is a projection point projected onto the table surface with the second image acquisition device as the projection center, and the coordinates of the first projection point and the coordinates of the second projection point are both two-dimensional coordinates in a two-dimensional coordinate system scaled by the first image acquisition device on the table surface below it. The third determining unit is used to determine the height information of each spatial point from the platform based on the first projection point coordinates and / or the second projection point coordinates; and to combine the height information with the first projection point coordinates or the second projection point coordinates to form the spatial coordinates of the spatial point. The fourth determining unit is used to determine the spatial three-dimensional information of the target food ingredient based on the spatial coordinates of each spatial point.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 3-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 3-7.

Images