Food volume and weight detector and food volume and weight detection method

Abstract

The embodiment of the invention discloses a food volume and weight detector and a food volume and weight detection method. According to one specific embodiment, the food volume and weight detector comprises a detector main body, a food bearing part, an image acquisition device, a weight detection assembly and a controller, the detector main body comprises a horizontal detector main body and a vertical detector main body; the food bearing part comprises a transparent rotating disc and a rotating power part; the rotating power part comprises a motor and an encoder; the motor is used for driving the transparent rotating disc to rotate automatically; the image acquisition device comprises an upper detection camera assembly and a lower detection camera assembly; the upper detection camera assembly comprises a left upper detection camera and a right upper detection camera, and the lower detection camera assembly comprises a left upper detection camera and a right upper detection camera; the image acquisition device further comprises a light source assembly which comprises an upper light source and a lower light source. According to the embodiment, the accuracy of food volume detection can be improved on the basis of ensuring the appearance and taste of the food.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical field of food volume and weight detection, and particularly to a food volume and weight detector and a food volume and weight detection method. Background Art

[0002] For some processed foods, such as bread, volume is a crucial quality indicator. It not only impacts the aesthetic appearance of the food but also directly influences its taste and internal structure. Therefore, food volume testing is crucial for determining its consistency and stability. Currently, food volume testing relies on visual inspection or the use of measuring tools such as rulers and measuring cups.

[0003] However, the inventors have discovered that when using the above-mentioned food volume detection method to detect food volume, the following technical problems often arise:

[0004] The accuracy and consistency of food volume detection are low when it is detected by manual visual inspection. However, when food is detected by measuring tools such as rulers and measuring cups, it is easy to cause damage to the food and affect its appearance and taste.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0006] The content of this disclosure is used to briefly introduce concepts that will be described in detail in the detailed description section below. The content of this disclosure is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0007] Some embodiments of the present disclosure provide a food volume weight detector and a food volume weight detection method to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In the first aspect, some embodiments of the present disclosure provide a food volume weight detector, which includes: a detector body, a food carrier, an image acquisition device, a weight detection component and a controller; the above-mentioned detector body includes a horizontal detector body and a vertical detector body; the above-mentioned food carrier includes a transparent turntable and a rotating power part, the above-mentioned transparent turntable is used to carry food, the above-mentioned rotating power part includes a motor and an encoder, the above-mentioned motor is used to drive the above-mentioned transparent turntable to rotate, and the above-mentioned transparent turntable and the above-mentioned rotating power part are arranged above the above-mentioned horizontal detector body; the above-mentioned image acquisition device includes an upper detection camera component and a lower detection camera component, the above-mentioned upper detection camera component is located above the above-mentioned food carrier, and the above-mentioned lower detection camera component is located above the above-mentioned food carrier. Below the above-mentioned food carrier, the above-mentioned upper detection camera assembly is arranged at the upper end of the above-mentioned vertical detection instrument body, and the above-mentioned lower detection camera assembly is arranged at the lower end of the above-mentioned vertical detection instrument body; the above-mentioned upper detection camera assembly includes an upper left detection camera and an upper right detection camera, and the above-mentioned lower detection camera assembly includes an upper left detection camera and an upper right detection camera; the above-mentioned image acquisition device also includes a light source assembly, and the above-mentioned light source assembly includes an upper light source and a lower light source, the above-mentioned upper light source is located between the above-mentioned upper left detection camera and the above-mentioned upper right detection camera, and the above-mentioned lower light source is located between the above-mentioned upper left detection camera and the above-mentioned upper right detection camera; the above-mentioned weight detection assembly is located below the above-mentioned transparent turntable; the above-mentioned rotating power part, the above-mentioned image acquisition device and the above-mentioned weight detection assembly are all communicatively connected to the above-mentioned controller.

[0009] In a second aspect, some embodiments of the present disclosure provide a food volume weight detection method, which is applied to the food volume weight detector as described in the first aspect, and the food volume weight detection method includes: obtaining food weight information collected by the above-mentioned weight detection component; performing the following food local point cloud data collection steps through the above-mentioned food carrier and the above-mentioned image acquisition device: controlling the above-mentioned transparent turntable to rotate a preset angle through the above-mentioned rotating power component; obtaining the angular position information obtained by the above-mentioned encoder; controlling the above-mentioned light source component to project a structured stripe pattern; obtaining the upper food stripe image group collected by the above-mentioned upper detection camera component and the lower food stripe image group collected by the above-mentioned lower detection camera component; generating food local point cloud data corresponding to the angular position information based on the above-mentioned upper food stripe image group and the above-mentioned lower food stripe image group; in response to determining that the above-mentioned transparent turntable rotates one circle, generating three-dimensional food point cloud data based on the obtained each angular position information and the corresponding each food local point cloud data; performing voxel processing on the above-mentioned three-dimensional food point cloud data to obtain food volume information.

[0010] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the food volume and weight detectors of some embodiments of the present disclosure can improve the accuracy of food volume detection while ensuring the appearance and taste of the food. Specifically, the reasons for the low accuracy and consistency of volume detection or the impact on the appearance and taste of the food are: the accuracy and consistency of volume detection are low when measuring food volume through manual visual inspection, and the use of measuring tools such as rulers and measuring cups to detect food can easily cause damage to the food, affecting its appearance and taste. Based on this, some embodiments of the food volume weight detector disclosed herein include a detector body, a food carrier, an image acquisition device, a weight detection component and a controller; the above-mentioned detector body includes a horizontal detector body and a vertical detector body; the above-mentioned food carrier includes a transparent turntable and a rotating power part, the above-mentioned transparent turntable is used to carry food, the above-mentioned rotating power part includes a motor and an encoder, the above-mentioned motor is used to drive the above-mentioned transparent turntable to rotate, and the above-mentioned transparent turntable and the above-mentioned rotating power part are arranged above the above-mentioned horizontal detector body; the above-mentioned image acquisition device includes an upper detection camera component and a lower detection camera component, the above-mentioned upper detection camera component is located above the above-mentioned food carrier, and the above-mentioned lower detection camera component is located above the above-mentioned food carrier. The upper detection camera assembly is disposed at the upper end of the vertical detector body, and the lower detection camera assembly is disposed at the lower end of the vertical detector body. The upper detection camera assembly includes an upper left detection camera and an upper right detection camera, and the lower detection camera assembly includes an upper left detection camera and an upper right detection camera. The image acquisition device also includes a light source assembly, which includes an upper light source and a lower light source. The upper light source is located between the upper left detection camera and the upper right detection camera, and the lower light source is located between the upper left detection camera and the upper right detection camera. The weight detection assembly is located below the transparent turntable. The rotating power member, the image acquisition device, and the weight detection assembly are all communicatively connected to the controller. Because the transparent turntable drives the food to rotate, and the image acquisition device can capture images of the food without contact to determine the volume of the food, damage to the food can be avoided and its appearance and taste can be guaranteed. In addition, because the upper detection camera assembly and the lower detection camera assembly simultaneously capture images of the food from above and below, a more accurate three-dimensional point cloud image of the food is generated for determining the food volume, thereby improving the accuracy of food volume detection. Therefore, the food volume and weight detector of some embodiments of the present disclosure can improve the accuracy of food volume detection while ensuring the appearance and taste of the food. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0012] Figure 1 is a front cross-sectional view of some embodiments of a food volume and weight detector according to the present disclosure;

[0013] Figure 2 is a side view of some embodiments of a food volume and weight detector according to the present disclosure;

[0014] Figure 3 is a rear view of some embodiments of the food volume and weight detector according to the present disclosure;

[0015] Figure 4 4 is a flow chart of some embodiments of the food volume and weight detection method according to the present disclosure. DETAILED DESCRIPTION

[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0017] In the description of this disclosure, it should be noted that, unless otherwise expressly specified or limited, the terms "disposed" and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in this disclosure based on the specific circumstances.

[0018] It should also be noted that, for ease of description, only the parts related to the relevant disclosure are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0019] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] Figure 1 is a front cross-sectional view of some embodiments of a food volume and weight detector according to the present disclosure. Figure 1 The device comprises a detector body 1, food 2, a food carrier 3, an image acquisition device 4, and a weight detection assembly 5. The detector body 1 comprises a horizontal detector body 11 and a vertical detector body 12. The food carrier 3 comprises a transparent turntable 31 and a rotating power member 32. The image acquisition device 4 comprises an upper detection camera assembly 41, a lower detection camera assembly 42, an upper light source 43, and a lower light source 44.

[0024] Figure 2 is a side view of some embodiments of a food volume and weight detector according to the present disclosure. Figure 2 The apparatus comprises a detector body 1 , food 2 and a display 6 . The detector body 1 comprises a horizontal detector body 11 and a vertical detector body 12 .

[0025] Figure 3 is a rear view of some embodiments of the food volume and weight detector according to the present disclosure. Figure 3 The detector body 1 includes a detector body 1, a power switch 7, a power line socket 8 and a detector switch 9. The detector body 1 includes a horizontal detector body 11 and a vertical detector body 12.

[0026] In some embodiments, the food volumetric weight detector may include a detector body 1, a food carrier 3, an image acquisition device 4, a weight detection assembly 5, and a controller. The controller may be a central processing unit (CPU). The detector body 1 may include a horizontal detector body 11 and a vertical detector body 12. The horizontal detector body 11 may be a horizontally positioned bracket for mounting the food carrier 3, the weight detection assembly 5, and the controller. The vertical detector body 12 may be a vertically positioned bracket for mounting the image acquisition device 4.

[0027] In some embodiments, the food carrier 3 may include a transparent turntable 31 and a rotating power member 32. The transparent turntable 31 may be used to carry food (e.g., food 2). The rotating power member 32 may include a motor and an encoder. The encoder may be used to trigger the image acquisition device 4 and collect angular position information of the food. The motor may be used to drive the transparent turntable 31 to rotate. The transparent turntable 31 and the rotating power member 32 may be disposed above the level detector body 11. The transparent turntable 31 may be rotatably disposed on the level detector body 11.

[0028] In some embodiments, the image acquisition device 4 may include an upper detection camera assembly 41 and a lower detection camera assembly 42. The upper detection camera assembly 41 may be a binocular camera positioned above the food carrier 3. The lower detection camera assembly 42 may be a binocular camera positioned below the food carrier 3. Both the upper detection camera assembly 41 and the lower detection camera assembly 42 direct images toward the center of the transparent turntable 31. The upper detection camera assembly 41 may be positioned above the food carrier 3. The lower detection camera assembly 42 may be positioned below the food carrier 3. The upper detection camera assembly 41 may be positioned at the upper end of the vertical detection instrument body 12. The lower detection camera assembly 42 may be positioned at the lower end of the vertical detection instrument body 12. The upper detection camera assembly 41 may include an upper left detection camera and an upper right detection camera. The lower detection camera assembly 42 may include an upper left detection camera and an upper right detection camera. The image acquisition device 4 may also include a light source assembly. The light source assembly may include an upper light source 43 and a lower light source 44. The upper light source 43 may be located between the upper left detection camera and the upper right detection camera. The lower light source 44 may be located between the upper left detection camera and the upper right detection camera. Both the upper light source 43 and the lower light source 44 may be optical machines for projecting stripe patterns.

[0029] In some embodiments, the weight detection component 5 may be located below the transparent turntable 31 . The weight detection component 5 may be a weighing sensor. Specifically, the weight detection component 5 may be located below the side of the transparent turntable 31 .

[0030] In some embodiments, the rotating power member 32 , the image acquisition device 4 , and the weight detection component 5 may all be communicatively connected to the controller.

[0031] Optionally, a power supply box and a detector switch 9 may be provided on the back of the level detector body 11. The power supply box may include a power switch 7 and a power cord socket 8. The power switch 7 may be used to control the power connection or disconnection of the food volume weight detector. The detector switch 9 may be used to control the startup or shutdown of the food volume weight detector. The detector switch 9 may be communicatively connected to the controller.

[0032] Optionally, a display 6 may be provided on the side of the level detector body 11. The display 6 may be used to display various information about the food. The display 6 may be in communication with the controller.

[0033] Optionally, the food volumetric weight detector may further include a manipulator. The manipulator may be located on the side of the level detector body 11. The weight detection assembly 5 may be located below the edge of the transparent turntable 31. The manipulator may be used to move the weighed food to the center of the transparent turntable 31.

[0034] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the food volume and weight detectors of some embodiments of the present disclosure can improve the accuracy of food volume detection while ensuring the appearance and taste of the food. Specifically, the reasons for the low accuracy and consistency of volume detection or the impact on the appearance and taste of the food are: the accuracy and consistency of volume detection are low when measuring food volume through manual visual inspection, and the use of measuring tools such as rulers and measuring cups to detect food can easily cause damage to the food, affecting its appearance and taste. Based on this, some embodiments of the food volume weight detector disclosed herein include a detector body, a food carrier, an image acquisition device, a weight detection component and a controller; the above-mentioned detector body includes a horizontal detector body and a vertical detector body; the above-mentioned food carrier includes a transparent turntable and a rotating power part, the above-mentioned transparent turntable is used to carry food, the above-mentioned rotating power part includes a motor and an encoder, the above-mentioned motor is used to drive the above-mentioned transparent turntable to rotate, and the above-mentioned transparent turntable and the above-mentioned rotating power part are arranged above the above-mentioned horizontal detector body; the above-mentioned image acquisition device includes an upper detection camera component and a lower detection camera component, the above-mentioned upper detection camera component is located above the above-mentioned food carrier, and the above-mentioned lower detection camera component is located above the above-mentioned food carrier. The upper detection camera assembly is disposed at the upper end of the vertical detector body, and the lower detection camera assembly is disposed at the lower end of the vertical detector body. The upper detection camera assembly includes an upper left detection camera and an upper right detection camera, and the lower detection camera assembly includes an upper left detection camera and an upper right detection camera. The image acquisition device also includes a light source assembly, which includes an upper light source and a lower light source. The upper light source is located between the upper left detection camera and the upper right detection camera, and the lower light source is located between the upper left detection camera and the upper right detection camera. The weight detection assembly is located below the transparent turntable. The rotating power member, the image acquisition device, and the weight detection assembly are all communicatively connected to the controller. Because the transparent turntable drives the food to rotate, and the image acquisition device can capture images of the food without contact to determine the volume of the food, damage to the food can be avoided and its appearance and taste can be guaranteed. In addition, because the upper detection camera assembly and the lower detection camera assembly simultaneously capture images of the food from above and below, a more accurate three-dimensional point cloud image of the food is generated for determining the food volume, thereby improving the accuracy of food volume detection. Therefore, the food volume and weight detector of some embodiments of the present disclosure can improve the accuracy of food volume detection while ensuring the appearance and taste of the food.

[0035] Figure 4 FIG4 is a flow chart of some embodiments of the food volume weight detection method according to the present disclosure. FIG4 shows a flow chart of some embodiments of the food volume weight detection method according to the present disclosure, wherein the food volume weight detection method comprises the following steps:

[0036] Step 401: Obtain food weight information collected by the weight detection component.

[0037] In some embodiments, an execution subject (eg, a food volume weight detector) may obtain the food weight information collected by the weight detection component, wherein the food weight information may be information used to indicate the weight of the food.

[0038] Step 402: Using the food carrier and the image acquisition device, perform the following steps to acquire local point cloud data of the food:

[0039] Step 4021: Control the transparent turntable to rotate to a preset angle by rotating the power member.

[0040] In some embodiments, the executive entity can control the transparent turntable to rotate by a preset angle by rotating a power member. The preset angle can be a pre-set rotation angle of the transparent turntable. For example, the preset angle can be 45°. In practice, the executive entity can drive the transparent turntable to rotate by the preset angle by rotating the power member.

[0041] Step 4022: Obtain the angular position information obtained by the encoder.

[0042] In some embodiments, the execution entity may obtain the angular position information obtained by the encoder.

[0043] Step 4023: Control the light source assembly to project the structured stripe pattern.

[0044] In some embodiments, the execution entity may control the light source assembly to project a structured stripe pattern. The structured stripe pattern may be an optically encoded pattern with specific geometrical arrangements. In practice, the execution entity may control the light source assembly to project the structured stripe pattern onto the surface of food. Upon projection onto the food surface, the structured stripe pattern may be deformed.

[0045] Step 4024: Acquire the upper food streak image group captured by the upper detection camera assembly and the lower food streak image group captured by the lower detection camera assembly.

[0046] In some embodiments, the execution entity may obtain an upper food streak image group captured by the upper detection camera assembly and a lower food streak image group captured by the lower detection camera assembly.

[0047] Step 4025: Generate food local point cloud data corresponding to angular position information based on the upper food stripe image group and the lower food stripe image group.

[0048] In some embodiments, the execution entity may generate food local point cloud data corresponding to angular position information based on the upper food stripe image group and the lower food stripe image group.

[0049] In some optional implementations of some embodiments, based on the upper food stripe image group and the lower food stripe image group, the execution entity may generate food local point cloud data corresponding to the angular position information through the following steps:

[0050] The first step is to perform distortion correction processing on each upper food streak image in the upper food streak image group to obtain a corrected upper food streak image group. In practice, the execution entity may remove radial and tangential distortion based on calibration parameters of the upper detection camera assembly to perform distortion correction processing on each upper food streak image in the upper food streak image group to obtain a corrected upper food streak image group.

[0051] The second step is to perform food main body extraction processing on the corrected upper food stripe image group to obtain an upper food main body stripe region image group. In practice, the execution entity can perform food main body extraction processing on the corrected upper food stripe image group using a background segmentation algorithm to obtain an upper food main body stripe region image group. For example, the background segmentation algorithm can be the HSV (Hue, Saturation, Value) color model.

[0052] The third step is to perform epipolar correction on the image group of the upper food main body stripe area to obtain the corrected image group of the upper food main body stripe area. In practice, the execution subject can perform epipolar correction on the image group of the upper food main body stripe area by plane reprojecting the image group of the upper food main body stripe area onto a coplanar row-aligned coordinate system to obtain the corrected image group of the upper food main body stripe area.

[0053] The fourth step is to perform stereo matching on the corrected image set of the striped area of the upper food body to obtain an initial disparity map of the upper food body. In practice, the execution entity may first construct a matching cost based on a Census transform or normalized cross-correlation. Then, the execution entity may use a semi-global matching algorithm based on the matching cost to perform stereo matching on the corrected image set of the striped area of the upper food body to obtain an initial disparity map of the upper food body.

[0054] In the fifth step, sub-pixel optimization is performed on the initial disparity map of the upper food to obtain an optimized initial disparity map of the upper food.

[0055] In the sixth step, bilateral filtering is performed on the optimized initial disparity map of the upper food to obtain a denoised initial disparity map of the upper food.

[0056] In the seventh step, triangulation is performed on the denoised initial disparity map of the upper food to obtain the point cloud data of the upper food.

[0057] In the eighth step, a model for generating corrected lower food point cloud data corresponding to the lower food streak image group is determined based on the refractive index and thickness of the transparent turntable. The corrected lower food point cloud data generation model can be a machine learning model that takes the lower food streak image group as input and outputs corrected lower food point cloud data. The machine learning model can include an input layer, a depth mapping adjustment layer, a refractive error compensation layer, a glass grid deformation layer, and an output layer. The input layer can be used to perform feature extraction on the lower food streak image group to obtain a lower food streak image feature group. The depth mapping adjustment layer can be used to perform depth adjustment on the lower food streak image feature group using a depth mapping relationship determined by the refractive index and thickness of the transparent turntable to obtain an adjusted lower food streak image feature group. The refractive error compensation layer can be used to compensate the adjusted lower food streak image feature group using a pre-calibrated polynomial model to obtain a compensated lower food streak image feature group. The glass grid deformation layer can be used to deform the compensated lower food streak image feature group by deforming the fine mesh of the transparent turntable to obtain the corrected lower food point cloud data. The output layer is used to output the corrected point cloud data for the food below. In practice, the execution entity may select a corresponding corrected point cloud data generation model for the food below from a preset set of corrected point cloud data generation model matching information based on the refractive index and thickness of the transparent turntable. Each piece of corrected point cloud data generation model matching information in the preset corrected point cloud data generation model matching information set includes a preset corrected point cloud data generation model and a corresponding refractive index and thickness.

[0058] In step 9, the refraction error of the lower food stripe image group is corrected using the lower food point cloud correction data generation model to obtain the lower food point cloud correction data. In practice, the execution entity may input the lower food stripe image group into the lower food point cloud correction data generation model to obtain the lower food point cloud correction data.

[0059] In the tenth step, coordinate unification processing is performed on the upper food point cloud data and the lower food point cloud correction data to obtain unified food point cloud data. In practice, the execution entity can unify the upper food point cloud data and the lower food point cloud correction data into a global coordinate system with the center of the turntable as the origin to obtain unified food point cloud data.

[0060] In the eleventh step, the unified food point cloud data is completed to obtain local food point cloud data. In practice, the execution entity may use voxel grid filtering to remove redundant points from the unified food point cloud data to obtain filtered unified food point cloud data. The execution entity may then perform point cloud completion on the filtered unified food point cloud data using a Poisson reconstruction algorithm to obtain local food point cloud data.

[0061] Step 403 : In response to determining that the transparent turntable rotates one circle, three-dimensional food point cloud data is generated according to the acquired angular position information and the corresponding local point cloud data of each food.

[0062] In some embodiments, the execution entity may generate three-dimensional food point cloud data based on the acquired angular position information and the corresponding local point cloud data of each food in response to determining that the transparent turntable rotates one circle.

[0063] In some optional implementations of some embodiments, based on the acquired position information at various angles and the corresponding local point cloud data of each food, the execution entity may generate three-dimensional food point cloud data by the following steps:

[0064] The first step is to perform initial registration on the food local point cloud data based on the aforementioned angular position information to obtain a coarsely aligned food local point cloud dataset. In practice, the execution entity can apply an initial rotation transformation to the food local point cloud data based on the angular position information provided by the encoder to pre-align it to the global coordinate system, thereby obtaining a coarsely aligned food local point cloud dataset.

[0065] The second step is to perform feature extraction on each coarsely aligned food local point cloud data set in the coarsely aligned food local point cloud dataset to obtain feature data of each food local point cloud. In practice, the execution entity may perform feature extraction on each coarsely aligned food local point cloud data set in the coarsely aligned food local point cloud dataset using a fast point feature histogram to obtain feature data of each food local point cloud.

[0066] The third step is to perform adjacent point feature matching on each of the coarsely aligned food local point cloud feature data to obtain a food local point cloud matching dataset. In practice, the execution entity may perform adjacent point feature matching on each of the coarsely aligned food local point cloud feature data based on feature similarity to obtain a food local point cloud matching dataset.

[0067] The fourth step is to optimize the registration of the above-mentioned roughly aligned food local point cloud feature matching dataset through the iterative nearest point algorithm to obtain the food local point cloud transformation matrix.

[0068] In the fifth step, based on the aforementioned food local point cloud transformation matrix, the point cloud fusion processing is performed on the individual food local point cloud data to obtain three-dimensional food point cloud data. In practice, the execution entity may first transform the aforementioned food local point cloud transformation matrix into a global coordinate system to obtain unified three-dimensional food point cloud data. Then, the execution entity may perform voxel filtering to remove duplicates on the unified three-dimensional food point cloud data to obtain filtered unified three-dimensional food point cloud data. Finally, the execution entity may perform Poisson surface reconstruction on the filtered unified three-dimensional food point cloud data based on a Poisson reconstruction algorithm to obtain three-dimensional food point cloud data.

[0069] Step 404 : voxelize the three-dimensional food point cloud data to obtain food volume information.

[0070] In some embodiments, the execution entity may perform voxel processing on the three-dimensional food point cloud data to obtain food volume information.

[0071] Optionally, the food volume and weight detector further includes a manipulator. Prior to performing the following step of collecting local point cloud data of the food using the food carrier and the image acquisition device, the executing body may further use the manipulator to move the food to the center of the transparent turntable.

[0072] In the process of adopting technical solutions to solve the above technical problems, the following technical problem 2 is often accompanied: some frozen or puffed foods are easily affected by the humidity temperature during the detection process, resulting in inaccurate volume detection. In response to the above technical problem 2, the conventional solution is generally to set a humidity temperature regulator in the food volume and weight detector to uniformly regulate the humidity temperature in the detector so that the humidity temperature is maintained at a uniform level. However, the above conventional solution still has the following problems: different foods have different suitable humidity temperatures, and providing a uniform humidity temperature may cause damage to the food, and continuously regulating the humidity temperature consumes a lot of energy and wastes resources.

[0073] Considering the problems of the above conventional solutions, facing the second technical problem mentioned above: some frozen or puffed foods are easily affected by humidity and temperature during the detection process, resulting in inaccurate volume detection. In combination with the current state of technology, we can decide to adopt the following solution:

[0074] Optionally, the food volume and weight detector may further include an infrared temperature probe and an infrared humidity probe. The infrared temperature probe and the infrared humidity probe may be arranged at the upper end of the vertical detector body.

[0075] Optionally, the food volume weight detector may further include a food type selection button. The food type selection button may be disposed on the side of the level detector body. As an example, the food type selection button may be a button disposed on the side of the level detector body. The food type selection button may also be a virtual button displayed on a display. The food type selection button may be a button for selecting the type of food to be detected. For example, the food type selection button may be a button for representing frozen food or a button for representing puffed food.

[0076] In some optional implementations of some embodiments, the execution entity may perform voxel processing on the three-dimensional food point cloud data to obtain food volume information through the following steps:

[0077] The first step is to detect the temperature of the environment in which the food is located through the above-mentioned infrared temperature measuring probe to obtain the ambient temperature information.

[0078] The second step is to detect the humidity of the environment in which the food is located through the above-mentioned infrared humidity probe to obtain environmental humidity information.

[0079] The third step is to receive the food type information sent by the food type selection button, wherein the food type information can be a frozen food type or a puffed food type.

[0080] The fourth step is to perform initial voxelization processing on the above three-dimensional food point cloud data to obtain initial food volume information.

[0081] In a fifth step, the ambient temperature information, ambient humidity information, food type information, and initial food volume information are input into a pre-trained model for generating corrected food volume information to obtain corrected food volume information. The corrected food volume information generation model may be a pre-trained machine learning model that takes ambient temperature information, ambient humidity information, food type information, and initial food volume information as input and outputs corrected food volume information. The corrected food volume information generation model may include an input layer, a frozen food wet temperature correction layer, a puffed food wet temperature correction layer, and an output layer. The input layer may be a feature extraction layer that extracts features from the ambient temperature information, ambient humidity information, and initial food volume information to obtain ambient temperature feature information, ambient humidity feature information, and initial food volume feature information. The feature extraction layer may include a convolutional layer, an activation layer, a pooling layer, and a batch normalization layer. The frozen food wet temperature correction layer can be a first neural network model that generates the corrected frozen food volume information by performing linear regression on ambient temperature characteristic information, ambient humidity characteristic information, and initial food volume characteristic information in response to determining food type information to characterize the frozen food. The first neural network model can include a fully connected layer (Dense), an activation function (ReLU / Sigmoid), and a regression layer. The puffed food wet temperature correction layer can be a second neural network model that generates the corrected puffed food volume information by performing linear regression on ambient temperature characteristic information, ambient humidity characteristic information, and initial food volume characteristic information in response to determining food type information to characterize the puffed food. The second neural network model can include a convolutional layer, a pooling layer, and a fully connected layer. The output layer can be used to determine and output the corrected frozen food volume information or the corrected puffed food volume information as the corrected food volume information.

[0082] In the sixth step, the modified food volume information generation model is determined as the food volume information.

[0083] The above-mentioned content regarding correcting food volume based on humidity and temperature is an inventive feature of an embodiment of the present disclosure, addressing Technical Problem 2: "Some frozen or puffed foods are easily affected by humidity and temperature during the detection process, resulting in inaccurate food volume detection." The reasons for inaccurate food volume detection are as follows: Some frozen or puffed foods are easily affected by humidity and temperature during the detection process. If these factors are addressed, the accuracy of food volume detection can be improved. To achieve this, the food volume and weight detector of the present disclosure also includes an infrared temperature probe and an infrared humidity probe. The food volume and weight detector also includes a food type selection button. The food volume and weight detection method of the present disclosure also includes detecting the temperature of the food environment using the infrared temperature probe to obtain ambient temperature information. Detecting the humidity of the food environment using the infrared humidity probe to obtain ambient humidity information. Receiving food type information transmitted by the food type selection button. The food type information can be either frozen food type or puffed food type. Initial voxelization is performed on the three-dimensional food point cloud data to obtain initial food volume information. The ambient temperature information, ambient humidity information, food type information, and initial food volume information are input into a pre-trained corrected food volume information generation model to obtain corrected food volume information. The corrected food volume information generation model is then used to determine the food volume information. Thus, by detecting the ambient humidity temperature and establishing a corrected food volume information generation model based on the correspondence between humidity temperature and volume, the food volume is corrected, thereby reducing volume measurement errors caused by environmental factors. Furthermore, by determining the food type and using different correction models to correct the food volume, volume measurement errors caused by environmental factors can be further reduced, thereby improving the accuracy of food volume detection.

[0084] In the process of adopting technical solutions to solve the above technical problems, the following technical problem often arises: some reflective foods will cause the structured light pattern to reflect as a mirror, and the camera cannot capture the complete stripes, resulting in low volume detection accuracy. In response to the above technical problem three, the conventional solution is generally to use a dual-wavelength structured light projector and use a spectral filter to distinguish between reflected and diffuse reflection signals. However, the above conventional solution still has the following problems: the cost is high, and for foods with obvious reflectivity, the completeness of the stripes captured by the camera is still poor, and the volume detection accuracy is still low.

[0085] Considering the problems with the conventional solutions mentioned above, facing the third technical problem mentioned above: some reflective foods will cause the structured light pattern to reflect as a mirror, and the camera cannot capture the complete stripes, resulting in low volume detection accuracy. In combination with the current state of technology, we can decide to adopt the following solution:

[0086] Optionally, both the upper detection camera assembly and the lower detection camera assembly may be equipped with an adjustable polarizer. The angle of the adjustable polarizer is adjustable. The adjustable polarizer may correspond to various polarization angles. For example, the various polarization angles may include 0°, 45°, and 90°.

[0087] Optionally, the above-mentioned food volume weight detector may further include a food special type selection button. The above-mentioned food special type selection button may be arranged on the side of the above-mentioned level detector body. As an example, the above-mentioned food special type selection button may be a button arranged on the side of the above-mentioned level detector body. The above-mentioned food special type selection button may also be a virtual button displayed on the display. The above-mentioned food special type selection button may be a button for selecting the type of food to be detected. For example, the above-mentioned food special type selection button may be a button for representing strongly reflective food (such as chocolate), a button for representing diffusely reflective food (such as bread), and a button for representing mixed reflective food (such as candied fruit).

[0088] In some optional implementations of some embodiments, the execution subject may obtain the upper food streak image group captured by the upper detection camera assembly and the lower food streak image group captured by the lower detection camera assembly by performing the following steps:

[0089] The first step is obtaining a set of upper food streak image groups captured by the upper detection camera assembly and a set of lower food streak image groups captured by the lower detection camera assembly at different polarization angles. The upper food streak image groups include a first upper food streak image group, a second upper food streak image group, and a third upper food streak image group. Each upper food streak image group in the upper food streak image group corresponds to a different polarization angle. The first upper food streak image group may be an upper food streak image group at a polarization angle of 0°. The second upper food streak image group may be an upper food streak image group at a polarization angle of 45°. The third upper food streak image group may be an upper food streak image group at a polarization angle of 90°. The lower food streak image group includes a first lower food streak image group, a second lower food streak image group, and a third lower food streak image group. Each lower food streak image group in the lower food streak image group corresponds to a different polarization angle. The first lower food streak image group may be an upper food streak image group at a polarization angle of 0°. The second lower food streak image group may be an lower food streak image group at a polarization angle of 45°. The third lower food stripe image group may be a lower food stripe image group at a polarization angle of 90°.

[0090] The second step is to receive the food type information sent by the food special type selection button, wherein the food special type information can be a strong reflective food type, a diffuse reflective food type or a mixed reflective food type.

[0091] In the third step, in response to determining that the above-mentioned special food type information is a strong reflective food type, a diffuse reflective food type, or a mixed reflective food type, a fusion weight group is determined based on the above-mentioned special food type information. Different special food type information corresponds to different weight groups. For example, the weights of 0° polarization, 45° polarization, and 90° polarization corresponding to the strong reflective food type can be 0.301, 0.303, and 0.396. The weights of 0° polarization, 45° polarization, and 90° polarization corresponding to the diffuse reflective food type can be 0.333, 0.333, and 0.333. The weights of 0° polarization, 45° polarization, and 90° polarization corresponding to the mixed reflective food type can be 0.183, 0.408, and 0.409.

[0092] In the fourth step, based on the aforementioned fusion weight set, weighted fusion is performed on each of the upper food streak image groups in the upper food streak image group set to obtain a fused upper food streak image group. In practice, the execution entity may first determine the product of each of the upper food streak image groups and the weight corresponding to the uniform polarization angle. For example, the product of the upper food streak image group corresponding to 0° polarization and the weight for 0° polarization. The execution entity may then determine the sum of these products as the fused upper food streak image group.

[0093] In the fifth step, based on the aforementioned fusion weight set, weighted fusion is performed on each lower food stripe image group in the lower food stripe image group set to obtain a fused lower food stripe image group. In practice, the execution entity may first determine the product of each lower food stripe image group and the weight corresponding to the uniform polarization angle. For example, the product of the lower food stripe image group corresponding to 0° polarization and the weight for 0° polarization. The execution entity may then determine the sum of these products as the fused lower food stripe image group.

[0094] Step 6: Determine the fused upper food stripe image group as the upper food stripe image group.

[0095] In the seventh step, the fused lower food stripe image group is determined as the lower food stripe image group.

[0096] The above-mentioned content on the decentralized image fusion processing is an inventive point of the embodiment of the present disclosure, which solves the third technical problem "some reflective foods will cause the structured light pattern to undergo mirror reflection, and the camera will not be able to capture the complete stripes, resulting in low volume detection accuracy". The reasons for the low volume detection accuracy are as follows: some reflective foods will cause the structured light pattern to undergo mirror reflection, and the camera will not be able to capture the complete stripes. If the above factors are solved, the volume detection accuracy can be improved. In order to achieve this effect, the food volume weight detection method disclosed in the present disclosure can collect images with different polarization angles by setting an adjustable polarizer, and eliminate the reflective noise through algorithm fusion. Different fusion weights are used to fuse the images for foods with different reflective properties, thereby improving the integrity of the stripes captured by the camera, and thus improving the volume detection accuracy.

[0097] The above-mentioned embodiments of the present disclosure have the following beneficial effects: the food volume and weight detection methods of some embodiments of the present disclosure can improve the accuracy of food volume detection while ensuring the appearance and taste of the food. Specifically, the reasons for the low accuracy and consistency of volume detection or the impact on the appearance and taste of the food are: the accuracy and consistency of volume detection are low when measuring food volume through manual visual inspection, and the use of measuring tools such as rulers and measuring cups to detect food can easily cause damage to the food, affecting its appearance and taste. Based on this, some embodiments of the present disclosure include a method for detecting food volume and weight, which includes obtaining food weight information collected by the weight detection assembly; performing the following steps of collecting local food point cloud data using the food carrier and the image acquisition device: controlling the transparent turntable to rotate a preset angle using the rotary power member; obtaining angular position information obtained by the encoder; controlling the light source assembly to project a structured stripe pattern; obtaining an upper food stripe image group collected by the upper detection camera assembly and a lower food stripe image group collected by the lower detection camera assembly; generating local food point cloud data corresponding to the angular position information based on the upper food stripe image group and the lower food stripe image group; generating three-dimensional food point cloud data based on each acquired angular position information and each corresponding local food point cloud data in response to determining that the transparent turntable has rotated one full rotation; and performing voxelization processing on the three-dimensional food point cloud data to obtain food volume information. Because the transparent turntable drives the food to rotate, and the image acquisition device can non-contactly capture an overall image of the food to determine the food volume, food damage can be avoided, ensuring its appearance and taste. Because the upper and lower detection camera assemblies simultaneously capture images from above and below the food, a more accurate three-dimensional point cloud image of the food is generated for determining the food's volume, thereby improving the accuracy of food volume detection. Therefore, the food volume and weight detection methods of some embodiments of the present disclosure can improve the accuracy of food volume detection while ensuring the appearance and taste of the food.

[0098] The above description is only an illustration of some preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A food volume weight detector, characterized in that: include: Detector body, food carrier, image acquisition device, weight detection component and controller; The detector body includes a horizontal detector body and a vertical detector body; The food carrier includes a transparent turntable and a rotating power member, wherein the transparent turntable is used to carry food, and the rotating power member includes a motor and an encoder, wherein the motor is used to drive the transparent turntable to rotate. The transparent turntable and the rotating power member are arranged above the level detector body; The image acquisition device includes an upper detection camera assembly and a lower detection camera assembly, wherein the upper detection camera assembly is located above the food carrier, and the lower detection camera assembly is located below the food carrier. The upper detection camera assembly is arranged at the upper end of the vertical detector body, and the lower detection camera assembly is arranged at the lower end of the vertical detector body; The upper detection camera assembly includes an upper left detection camera and an upper right detection camera, and the lower detection camera assembly includes an upper left detection camera and an upper right detection camera; The image acquisition device further includes a light source assembly, the light source assembly including an upper light source and a lower light source, the upper light source is located between the upper left detection camera and the upper right detection camera, and the lower light source is located between the upper left detection camera and the upper right detection camera; The weight detection component is located below the transparent turntable; The rotating power member, the image acquisition device and the weight detection component are all in communication connection with the controller.

2. The food volume weight detector according to claim 1, characterized in that: A power supply box and a detector switch are provided on the back of the level detector body; A power switch and a power cord socket are provided in the power box. The power switch is used to control the power connection or disconnection of the food volume weight detector. The detector switch is used to control the start or shutdown of the food volume weight detector. The detector switch is communicatively connected to the controller.

3. The food volume weight detector according to claim 1, characterized in that: A display is provided on the side of the level detector body, and the display is used to display various information of the food. The display is in communication connection with the controller.

4. The food volume weight detector according to claim 1, characterized in that: The food volume and weight detector further includes a manipulator, which is located on the side of the level detector body; The weight detection component is located below the edge of the transparent turntable; The manipulator is used to move the weighed food to the center of the transparent turntable.

5. A method for detecting food volume and weight, applied to the food volume and weight detector according to any one of claims 1 to 4, characterized in that: The food volume weight detector includes a detector body, a food carrier, an image acquisition device, a weight detection component and a controller, wherein the food carrier includes a transparent turntable and a rotating power member, the rotating power member includes a motor and an encoder, and the image acquisition device includes an upper detection camera component, a lower detection camera component and a light source component; and The method comprises: Obtaining food weight information collected by the weight detection component; The following steps of collecting local point cloud data of food are performed by the food carrier and the image acquisition device: The transparent turntable is controlled to rotate by a preset angle by the rotating power member; Obtaining angular position information obtained by the encoder; controlling the light source assembly to project a structured stripe pattern; Acquire an upper food streak image group captured by the upper detection camera assembly and a lower food streak image group captured by the lower detection camera assembly; generating food local point cloud data corresponding to angular position information according to the upper food stripe image group and the lower food stripe image group; In response to determining that the transparent turntable rotates one circle, generating three-dimensional food point cloud data according to the acquired angular position information and the corresponding local point cloud data of each food; The three-dimensional food point cloud data is voxelized to obtain food volume information.

6. The food volume weight detection method according to claim 5, characterized in that: The food volume and weight detector further includes a manipulator; and Before performing the following step of collecting local point cloud data of food using the food carrier and the image acquisition device, the method further includes: The food is moved to the center of the transparent turntable by a robot.

7. The method for detecting food volume and weight according to claim 5, wherein: Generating food local point cloud data corresponding to angular position information according to the upper food stripe image group and the lower food stripe image group includes: performing distortion correction processing on each upper food stripe image in the upper food stripe image group to obtain a corrected upper food stripe image group; performing food main body extraction processing on the corrected upper food stripe image group to obtain an upper food main body stripe area image group; performing epipolar correction processing on the upper food main body stripe region image group to obtain a corrected upper food main body stripe region image group; Performing stereo matching processing on the corrected upper food main body stripe area image group to obtain an initial disparity map of the upper food; performing sub-pixel optimization on the initial disparity map of the upper food to obtain an optimized initial disparity map of the upper food; Performing bilateral filtering denoising on the optimized initial disparity map of the upper food to obtain a denoised initial disparity map of the upper food; Performing triangulation on the denoised initial disparity map of the upper food to obtain point cloud data of the upper food; determining a lower food point cloud correction data generation model corresponding to the lower food stripe image group according to the refractive index and thickness of the transparent turntable; Performing refraction error correction on the lower food stripe image group using the lower food point cloud correction data generation model to obtain lower food point cloud correction data; Performing coordinate unification processing on the upper food point cloud data and the lower food point cloud correction data to obtain unified food point cloud data; The unified food point cloud data is completed to obtain food local point cloud data.

8. The method for detecting food volume and weight according to claim 5, wherein: Generating three-dimensional food point cloud data based on the acquired position information at each angle and the corresponding local point cloud data of each food includes: Performing initial registration on the local point cloud data of each food item according to the position information of each angle to obtain a coarsely aligned local point cloud data set of the food item; Performing feature extraction on each coarsely aligned food local point cloud data in the coarsely aligned food local point cloud data set to obtain feature data of each food local point cloud; Performing adjacent point feature matching processing on each of the roughly aligned food local point cloud feature data to obtain a food local point cloud matching dataset; Performing registration optimization on the coarsely aligned food local point cloud feature matching dataset by an iterative closest point algorithm to obtain a food local point cloud transformation matrix; According to the food local point cloud transformation matrix, point cloud fusion processing is performed on each food local point cloud data to obtain three-dimensional food point cloud data.

Images