Method and device for determining visual angle of camera for shooting liner area, refrigerator and computer readable storage medium

Abstract

The invention relates to the technical field of refrigeration equipment, and discloses a method for determining the visual angle of a camera for shooting an inner container area, and the method comprises the steps: determining a to-be-recognized area, a maximum recognition area corresponding to the to-be-recognized area, and the lens position of the center of a camera lens; determining a central point of a first side surface and a central point of a second side surface of the to-be-identified area as a first reference position and a second reference position respectively; the set direction comprises a horizontal direction or a vertical direction; determining a central point of a third side surface and a central point of a fourth side surface of the maximum identification area as a third reference position and a fourth reference position respectively; according to the lens position, the first reference position and the second reference position, determining a minimum visual angle of the camera along a set direction; and determining the maximum visual angle of the camera along the set direction according to the lens position, the third reference position and the fourth reference position. According to the method, the food material image recognition rate in the liner area can be improved.

Description

Technical Field

[0001] This application relates to the field of refrigeration equipment technology, such as a method and apparatus for determining the camera angle for photographing the inner liner area, a refrigerator, and a computer-readable storage medium. Background Technology

[0002] Currently, with the rapid development of science and technology, refrigerators with refrigeration, freezing, and food management functions are widely used in the field of smart home appliances. To better manage food, refrigerators are equipped with cameras on the top of the refrigerator compartment to identify the food storage conditions. However, the drawers in the refrigerator compartment are arranged from top to bottom, and users may pull out multiple drawers at the same time. In this case, the camera installed on the top of the refrigerator compartment can only capture images of the food in the topmost drawer that has been pulled out, and cannot capture images of the food in other drawers. The shooting range is relatively limited, which is not conducive to food management.

[0003] To expand the imaging range of food inside a refrigerator's crisper compartment, a smart refrigerator has been disclosed, comprising: a crisper compartment with shelves dividing it into multiple storage spaces, each equipped with a temperature sensor; a camera mounted on the crisper compartment door for photographing the food inside; a crisper air duct connected to multiple storage spaces via multiple sets of air outlets on its front cover, with each storage space corresponding to a set of air outlets; a damper assembly for opening or closing one or more sets of air outlets; and a control unit connected to both the camera and the damper assembly. The control unit determines the set temperature for each storage space based on image information captured by the camera and compares the real-time temperature value detected by the temperature sensor with the set temperature value to control the opening and closing of one or more sets of air outlets in the damper assembly. The crisper door rotates to a set angle between 0° and 90° on the plane of the crisper compartment opening. When the angle is set between 20° and 40°, the distortion at both ends of the captured image is relatively small.

[0004] In the process of implementing the embodiments of this disclosure, at least the following problems were found in the related art:

[0005] For refrigerators with different structures, the size and pixel count of the food images captured by the cameras in these technologies vary. If the food images are too small or have low pixel counts, it will hinder food image recognition. Therefore, when there are differences in the internal structure of the refrigerator liner, these technologies cannot obtain food images of suitable size and high pixel count, affecting the food image recognition rate.

[0006] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0008] This disclosure provides a method, apparatus, refrigerator, and computer-readable storage medium for determining the camera angle of the inner liner area, so as to ensure the pixel and size of the captured food images even when there are differences in the inner liner structure, thereby improving the recognition rate of food images within the inner liner area.

[0009] In some embodiments, the method includes: a refrigerator comprising: a cabinet; a door rotatably mounted on the front side of the cabinet via a door rotation axis, with a bracket rotation axis disposed at the end of the door away from the door rotation axis; a camera bracket rotatably mounted on the door via a bracket rotation axis; and a camera mounted on the camera bracket; the method includes: determining a region to be identified, a maximum recognition region corresponding to the region to be identified, and a lens position at the center of the camera lens; determining the center point of a first side and the center point of a second side of the region to be identified as a first reference position and a second reference position, respectively; the first side represents the side furthest from the door rotation axis along a predetermined associated direction, and the second side represents the side furthest from the door rotation axis along a predetermined associated direction. The side closest to the pivot point; wherein, the set direction includes a horizontal direction or a vertical direction, the horizontal direction is associated with the thickness direction of the box, and the vertical direction is associated with the width direction of the box; the center points of the third side and the fourth side, which determine the maximum recognition area, are respectively the third reference position and the fourth reference position; the third side represents the side that is farthest from the pivot point of the door along the set direction, and the fourth side represents the side that is closest to the pivot point of the door along the set direction; the minimum viewing angle of the camera along the set direction is determined according to the lens position and the first reference position and the second reference position; the maximum viewing angle of the camera along the set direction is determined according to the lens position and the third reference position and the fourth reference position.

[0010] In some embodiments, determining the minimum viewing angle of the camera along a set direction based on the lens position and a first reference position and a second reference position includes: determining a first viewing angle boundary line based on the lens position and the first reference position; determining a second viewing angle boundary line based on the lens position and the second reference position; and determining the angle between the first viewing angle boundary line and the second viewing angle boundary line as the minimum viewing angle of the camera along the set direction. Determining the maximum viewing angle of the camera along the set direction based on the lens position and a third reference position and a fourth reference position includes: determining a third viewing angle boundary line based on the lens position and the third reference position; determining a fourth viewing angle boundary line based on the lens position and the fourth reference position; and determining the angle between the third viewing angle boundary line and the fourth viewing angle boundary line as the maximum viewing angle of the camera along the set direction.

[0011] In some embodiments, determining the angle between the first viewpoint boundary line and the second viewpoint boundary line as the minimum viewing angle of the camera along a set direction includes: calculating Where A, B, and C represent the lens coordinates of the lens position, the first coordinate of the first reference position, and the second coordinate of the second reference position, respectively, FOV. min This indicates the minimum field of view of the camera along a set direction.

[0012] In some embodiments, determining the angle between the third-view boundary line and the fourth-view boundary line as the maximum field of view of the camera along a set direction includes: calculating Where B' and C' represent the third coordinate of the third reference position and the fourth coordinate of the fourth reference position, respectively, FOV max This indicates the maximum field of view of the camera along the set direction.

[0013] In some embodiments, determining the center points of the first and second sides of the area to be identified as the first and second reference positions, respectively, includes: constructing a three-dimensional coordinate system with the intersection of the door rotation axis and a set straight line as the origin, the width direction of the box towards the area to be identified as the X-axis, the thickness direction of the box away from the box as the Y-axis, and the gravity direction as the Z-axis; defining the straight line as the line passing through the center points of the first and second sides; defining the plane as the plane along the horizontal direction and coplanar with the lens position; and obtaining the box width W. box and the maximum width W of the region to be identified along the X-axis ROI ; Obtain the box thickness D box And the maximum thickness D of the region to be identified along the Y-axis ROI When the direction is set to horizontal, according to (0.5W) box +0.5W ROI ,0,D box -0.5D ROI ), determine the first coordinate B of the first reference position, according to (0.5W box -0.5WROI ,0,D box -0.5D ROI ), determine the second coordinate C of the second reference position; when the direction is set to vertical, according to (0.5W box ,0,D box -D ROI ), determine the first coordinate B of the first reference position, according to (0.5W box ,0,D box ), determine the second coordinate C of the second reference position.

[0014] In some embodiments, determining the center points of the third and fourth sides of the maximum recognition area as the third and fourth reference positions, respectively, includes: obtaining the ratio coefficient k between the maximum recognition area and the area to be recognized; when the setting direction is horizontal, according to (0.5W) box / k+0.5W ROI / k,0,D box -0.5D ROI ), determine the third coordinate B' of the third reference position, according to (0.5W box / k-0.5W ROI / k,0,D box -0.5D ROI ), determine the fourth coordinate C' of the fourth reference position; when the direction is set to vertical, according to (0.5W box / k,0,D box -D ROI ), determine the third coordinate B' of the third reference position, according to (0.5W box / k,0,D box ), determine the fourth coordinate C' of the fourth reference position.

[0015] In some embodiments, determining the lens position of the camera lens center includes: obtaining the maximum opening angle α of the door and the real-time installation tilt angle β between the camera bracket and the door; obtaining the distance x between the camera lens center and the door rotation axis and the distance y between the bracket rotation axis and the camera lens center; and determining the lens coordinate A of the lens position of the camera lens center according to (xcosα+ycos(β-180°+α),xsinα+ysin(β-180°+α),0).

[0016] In some embodiments, the apparatus includes a processor and a memory storing program instructions, the processor being configured to, when executing the program instructions, perform the method described above for determining the camera angle for capturing an inner liner area.

[0017] In some embodiments, the refrigerator includes: a cabinet; a door rotatably mounted on the front side of the cabinet via a door rotation axis, with a bracket rotation axis disposed at the end of the door away from the door rotation axis; a camera bracket rotatably mounted on the door via a bracket rotation axis; a camera mounted on the camera bracket; and a device for determining the camera angle for capturing the inner liner area as described above, mounted on the door.

[0018] In some embodiments, the computer-readable storage medium stores program instructions that, when executed, cause a computer to perform the method described above for determining the camera angle for capturing the inner liner area.

[0019] The method, apparatus, refrigerator, and computer-readable storage medium for determining the camera angle for capturing the inner liner area provided in this disclosure can achieve the following technical effects:

[0020] This embodiment of the invention can determine the minimum field of view and the minimum angle of view corresponding to the minimum viewing angle based on the lens position and the first and second reference positions to ensure that the area to be identified is covered as much as possible within the minimum field of view. Similarly, it can determine the maximum field of view and the maximum angle of view corresponding to the maximum viewing angle based on the lens position and the third and fourth reference positions to ensure that the area to be identified is covered as much as possible within the maximum field of view, while simultaneously obtaining a recognition target of suitable size and high pixel count from the area to be identified. Thus, this embodiment of the invention, by combining the inner liner structure, the area to be identified, and the maximum recognition area to determine the minimum and maximum viewing angles of the camera in the set direction, can ensure that the area to be identified is basically covered within the field of view captured by the camera, and also ensure that a recognition target of suitable image size and high pixel count is obtained, thereby improving the recognition rate of food images within the inner liner area.

[0021] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0022] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein:

[0023] Figure 1 This is a top view of a refrigerator body and door provided in an embodiment of this disclosure;

[0024] Figure 2 This is a front view of a refrigerator body provided in an embodiment of this disclosure;

[0025] Figure 3 This is a schematic diagram of the region to be identified and the maximum identification region provided in the embodiments of this disclosure;

[0026] Figure 4 This is a top view of a camera from a horizontal perspective, provided in an embodiment of this disclosure;

[0027] Figure 5 This is a top view of a camera from a vertical perspective, provided in an embodiment of this disclosure;

[0028] Figure 6 This is a schematic diagram of a method for determining the camera angle for capturing an inner liner area, provided in an embodiment of this disclosure;

[0029] Figure 7 This is a schematic diagram of another method for determining the camera angle for capturing the inner liner area provided in an embodiment of this disclosure;

[0030] Figure 8 This is a schematic diagram of another method for determining the camera angle for capturing the inner liner area provided in an embodiment of this disclosure;

[0031] Figure 9 This is a schematic diagram of another method for determining the camera angle for capturing the inner liner area provided in an embodiment of this disclosure;

[0032] Figure 10 This is a schematic diagram of another device provided in this disclosure for determining the camera angle for capturing the inner liner area.

[0033] Figure label:

[0034] 10: Cabinet body; 20: Door; 30: Camera bracket;

[0035] 101: Inner liner; 201: Camera;

[0036] 20a: Door body rotation axis; 20b: Support frame rotation axis;

[0037] 300: A device for determining the camera's viewing angle for photographing the inner liner area;

[0038] 301: Processor; 302: Memory;

[0039] 303: Communication interface; 304: Bus. Detailed Implementation

[0040] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0041] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0042] Unless otherwise stated, the term "multiple" means two or more.

[0043] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0044] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0045] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0046] Combination Figure 1 As shown, this embodiment of the present disclosure provides a refrigerator, including a cabinet 10, a door 20, a camera bracket 30, and a camera 201. The door 20 is mounted on the front side of the cabinet 10 via a door rotation axis 20a. A bracket rotation axis 20b is provided at the end of the door 20 away from the door rotation axis 20a. The camera bracket 30 is rotatably mounted on the door 40 via the bracket rotation axis 20b. The camera 201 is mounted on the camera bracket 30.

[0047] The cabinet 10 is equipped with an inner liner 101, which includes one or more shelves and a refrigerator compartment located below the shelves. The refrigerator compartment has one or more drawers. These drawers can be pulled out to their maximum position along the thickness of the cabinet.

[0048] Optionally, there are two doors 20. The two doors 20 are symmetrically arranged on the front side of the housing 10 along the central axis of the housing thickness. The bracket rotation shaft 20b is installed at the end of either door 20 furthest from the door rotation shaft 20a. This allows the inner liner area to be identified by a camera installed on either door.

[0049] Combination Figure 3 As shown, when camera 201 takes a picture, camera 201 is associated with the area to be identified. The area to be identified refers to the area that camera 201 needs to identify when taking a picture. The area to be identified includes the inner liner area and the surrounding area of ​​the inner liner area. The inner liner area includes the shelf area and the refrigerator compartment area located below the shelf area. The shelf area includes one or more shelves, and the refrigerator compartment area includes one or more drawers. The maximum thickness of the area to be identified along the thickness direction of the cabinet is equal to the thickness of the inner liner area along the cabinet direction. It should be noted that the surrounding area of ​​the inner liner area can be an area extending beyond the inner liner area along the width direction of the cabinet.

[0050] The camera 201 is also associated with a maximum recognition area. The maximum recognition area refers to the recognition area obtained by magnifying the area to be recognized along the width of the housing, with the area to be recognized as the reference area. Furthermore, the ratio of the area to be recognized to the area of ​​the maximum recognition area is greater than the area ratio and less than 1. Specifically, the area ratio is greater than or equal to 0.7. Preferably, the area ratio is 0.7.

[0051] In one specific embodiment, combined with Figure 1 and Figure 2 As shown, the thickness of the housing 10 is H. box And the box width W box The maximum width of the area to be identified along the width direction of the box is W. ROI The maximum thickness of the area to be identified along the thickness direction of the box is D. ROI The maximum thickness of the region to be identified along the vertical direction is H. ROI .

[0052] Combination Figure 1 As shown, the maximum opening angle of the door 20 is α, and the angle between the camera bracket 30 and the door 20 is the real-time installation tilt angle β. The distance between the door rotation axis 20a and the bracket rotation axis 20b is x. The distance between the bracket rotation axis 20b and the center of the camera lens is y.

[0053] In another specific embodiment, to ensure the area recognition function of the camera, the viewing angle of the camera in the horizontal or vertical direction should not be too large. If the viewing angle of the camera in the horizontal or vertical direction is too large, the target to be recognized in the area to be recognized will be too small, resulting in a low pixel ratio of the target and a decrease in the recognition rate of the camera. To this end, this embodiment configures a ratio coefficient k between the maximum recognition area and the area to be recognized, and calculates the maximum viewing angle of the camera in the horizontal or vertical direction based on the ratio coefficient k, thereby ensuring the recognition rate of the camera.

[0054] It should be noted that the scaling factor k represents the maximum width W of the area to be identified along the width direction of the box. ROI The ratio of the maximum width of the maximum recognition area along the width direction of the box, or the maximum thickness H of the area to be recognized along the vertical direction. ROI The ratio of the maximum thickness of the maximum recognition area in the vertical direction to the maximum thickness of the maximum recognition area. The scaling factor k is determined by the area ratio. As an example, when the area ratio is 0.7, k is greater than or equal to 0.7 and less than 1.

[0055] Based on the above refrigerator structural configuration, combined with Figure 6 As shown, this disclosure provides a method for determining the camera angle for capturing an inner liner area, including:

[0056] S01, the refrigerator determines the area to be identified, the maximum identification area corresponding to the area to be identified, and the lens position of the center of the camera lens.

[0057] S02, the refrigerator determines the center points of the first and second sides of the area to be identified as the first and second reference positions, respectively. The first side represents the side furthest from the door rotation axis along a set direction, and the second side represents the side closest to the door rotation axis along a set direction. The set direction includes a horizontal or vertical direction; the horizontal direction is associated with the thickness of the refrigerator body, and the vertical direction is associated with the width of the refrigerator body.

[0058] S03, the refrigerator determines the center points of the third and fourth sides of the largest recognition area as the third reference position and the fourth reference position, respectively. The third side represents the side that is furthest from the door rotation axis along the set direction association direction, and the fourth side represents the side that is closest to the door rotation axis along the set direction association direction.

[0059] S04, the refrigerator determines the minimum viewing angle of the camera along the set direction based on the lens position and the first reference position and the second reference position.

[0060] S05, the refrigerator determines the maximum viewing angle of the camera along the set direction based on the lens position and the third and fourth reference positions.

[0061] The method for determining the camera viewing angle of the inner liner area provided in this disclosure involves determining the area to be identified, the corresponding maximum identification area, and the lens position. Then, the center points of the first and second sides of the area to be identified are determined as first and second reference positions, respectively. Similarly, the center points of the third and fourth sides of the maximum identification area are determined as third and fourth reference positions, respectively. Subsequently, based on the lens position and the first and second reference positions, the minimum field of view and the minimum viewing angle are determined to ensure that the area to be identified is covered as much as possible within the minimum field of view. Similarly, based on the lens position and the third and fourth reference positions, the maximum field of view and the maximum viewing angle are determined to ensure that the area to be identified is covered as much as possible within the maximum field of view, while simultaneously obtaining a suitable-sized and high-pixel identification target through the area to be identified. Thus, by combining the inner liner structure, the area to be identified, and the maximum identification area to determine the minimum and maximum viewing angles of the camera in the set direction, this method ensures that the area to be identified is basically covered within the camera's field of view, while also ensuring that a suitable-sized and high-pixel identification target is obtained, thereby improving the recognition rate of food images within the inner liner area.

[0062] In this embodiment of the disclosure, Figure 4 This is a top view of a camera from a horizontal perspective, provided in an embodiment of this disclosure.

[0063] Where A represents the center position of the camera lens. B1 represents the first reference position of the area to be identified, and C1 represents the second reference position of the area to be identified. The first reference position is the center point of the first side of the area to be identified, and the second reference position is the center point of the second side of the area to be identified. The first side represents the side that is furthest from the door rotation axis 20a in the thickness direction of the housing, and the second side represents the side that is closest to the door rotation axis 20a in the thickness direction of the housing.

[0064] B1' represents the third reference position of the maximum recognition area, and C1' represents the fourth reference position of the maximum recognition area. The third reference position is the center point of the third side, and the fourth reference position is the center point of the fourth side of the area to be recognized. The third side represents the side that is furthest from the door rotation axis 20a in the thickness direction of the enclosure, and the fourth side represents the side that is closest to the door rotation axis 20a in the thickness direction of the enclosure.

[0065] In this embodiment of the disclosure, FOV min FOV represents the minimum field of view of a camera along a set direction. max HFOV indicates the maximum field of view of the camera along a set direction. min and HFOVmax VFOV represents the minimum and maximum field of view of the camera along the horizontal direction. min and VFOV max These represent the minimum and maximum angles of view of the camera along the vertical direction, respectively.

[0066] Among them, FOV min HFOV min Or VFOV min FOV max HFOV max Or VFOV max .

[0067] Example 1: Combining Figure 4 As shown, when the set direction is horizontal, the refrigerator determines the minimum viewing angle of the camera along the set direction based on the lens position and the first and second reference positions, including:

[0068] The refrigerator determines the first viewpoint boundary line AB1 based on the lens position A and the first reference position B1.

[0069] The refrigerator determines the second viewpoint boundary line AC1 based on the lens position A and the second reference position C1.

[0070] The angle between the first-view boundary line AB1 and the second-view boundary line AC1 is determined by the refrigerator as the minimum field of view (HFOV) of the camera in the horizontal direction. min .

[0071] Figure 5 This is a top view of a camera from a vertical perspective, provided in an embodiment of this disclosure.

[0072] Where A represents the center position of the camera lens. B2 represents the first reference position of the area to be identified, and C2 represents the second reference position of the area to be identified. The first reference position is the center point of the first side of the area to be identified, and the second reference position is the center point of the second side of the area to be identified.

[0073] B2' represents the third reference position of the maximum recognition area, and C2' represents the fourth reference position of the maximum recognition area. The third reference position is the center point of the third side of the maximum recognition area, and the fourth reference position is the center point of the fourth side of the maximum recognition area.

[0074] Example 2: Combination Figure 5 As shown, when the set direction is vertical, the refrigerator determines the minimum viewing angle of the camera along the set direction based on the lens position and the first and second reference positions, including:

[0075] The refrigerator determines the first viewpoint boundary line AB2 based on the lens position A and the first reference position B2.

[0076] The refrigerator determines the second viewpoint boundary line AC2 based on the lens position A and the second reference position C2.

[0077] The angle between the first-view boundary line AB2 and the second-view boundary line AC2 is determined by the refrigerator as the minimum field of view (VFOV) of the camera in the vertical direction. min .

[0078] Based on the above embodiments one and two, combined with Figure 7 As shown, the refrigerator determines the minimum viewing angle of the camera along a set direction based on the lens position and the first and second reference positions, including:

[0079] S11, the refrigerator determines the boundary line of the first viewpoint based on the position of the lens and the first reference position.

[0080] S12, the refrigerator determines the boundary line of the second viewpoint based on the position of the lens and the second reference position.

[0081] S13, the refrigerator determines the angle between the boundary line of the first view and the boundary line of the second view as the minimum viewing angle of the camera along the set direction.

[0082] Thus, in this embodiment, a first viewing angle boundary line is determined based on the lens position and a first reference position, and a second viewing angle boundary line is determined based on the lens position and a second reference position. The angle formed by the first and second viewing angle boundary lines is the minimum viewing angle of the camera along a set direction, and the first field of view corresponding to the minimum viewing angle is the minimum field of view. Therefore, the first field of view can cover most of the area to be identified, which is beneficial for improving the recognition rate of food images within the inner lining area.

[0083] Example 3: Combination Figure 8 As shown, when the set direction is horizontal, the refrigerator determines the maximum viewing angle of the camera along the set direction based on the lens position and the third and fourth reference positions, including:

[0084] The refrigerator determines the third-view boundary line AB1' based on the lens position A and the third reference position B1'.

[0085] The refrigerator determines the fourth viewpoint boundary line AC1' based on the lens position A and the fourth reference position C1'.

[0086] The angle between the third-view boundary line AB1' and the fourth-view boundary line AC1' is determined by the refrigerator as the maximum horizontal field of view (HFOV) of the camera. max .

[0087] Example 4: Combination Figure 5As shown, when the set direction is vertical, the refrigerator determines the maximum viewing angle of the camera along the set direction based on the lens position and the third and fourth reference positions, including:

[0088] The refrigerator determines the third-view boundary line AB2' based on the lens position A and the third reference position B2'.

[0089] The refrigerator determines the fourth viewpoint boundary line AC2' based on the lens position A and the fourth reference position C2'.

[0090] The angle between the third-view boundary line AB2' and the fourth-view boundary line AC2' is determined by the refrigerator as the maximum field of view (VFOV) of the camera in the vertical direction. max .

[0091] Based on the above embodiments three and four, combined with Figure 8 As shown, the refrigerator determines the maximum viewing angle of the camera along a set direction based on the lens position and the third and fourth reference positions, including:

[0092] S14, the refrigerator determines the boundary line of the third perspective based on the position of the lens and the third reference position.

[0093] S15, the refrigerator determines the boundary line of the fourth perspective based on the position of the lens and the fourth reference position.

[0094] S16, the refrigerator determines the angle between the boundary line of the third view and the boundary line of the fourth view as the maximum field of view of the camera along the set direction.

[0095] Thus, in this embodiment, a third viewpoint boundary line is determined based on the lens position and a third reference position, and a fourth viewpoint boundary line is determined based on the lens position and a fourth reference position. The angle formed by the third and fourth viewpoint boundary lines is the maximum viewing angle of the camera along a set direction. Simultaneously, the second field of view corresponding to the maximum viewing angle is the maximum field of view. Thus, the second field of view can essentially cover the area to be identified. Furthermore, this embodiment can obtain a target of suitable size and high pixel count through the maximum identification area.

[0096] Optionally, the refrigerator determines the angle between the boundary line of the first viewpoint and the boundary line of the second viewpoint as the minimum viewing angle of the camera along a set direction, including:

[0097] calculate

[0098] Where A, B, and C represent the lens coordinates of the lens position, the first coordinate of the first reference position, and the second coordinate of the second reference position, respectively, FOV. min This indicates the minimum field of view of the camera along a set direction.

[0099] It should be noted that, when the direction is set to horizontal, B and C are B1 and C1 in Example 1, respectively, and the FOV is... min That is, HFOV min With the direction set as vertical, B and C are B2 and C2 in Example 2, respectively, and FOV min That is, VFOV min .

[0100] Thus, after determining the lens coordinates of the lens position, the first coordinates of the first reference position, and the second coordinates of the second reference position, the embodiments of this disclosure can determine... and The coordinates of each element are determined, and then, based on the law of cosines, the angle between the boundary lines of the first and second viewpoints can be calculated, thus calculating the minimum viewing angle of the camera along the set direction. In this embodiment, the minimum viewing angle of the camera along the set direction can be accurately calculated based on the law of cosines, further improving the recognition rate of food images within the inner liner area.

[0101] In a specific implementation, combined with Figure 4 As shown, the minimum field of view (HFOV) of the camera along the horizontal direction. min for:

[0102]

[0103] Combination Figure 5 As shown, the minimum field of view (VFOV) of the camera along the vertical direction. min for:

[0104]

[0105] Optionally, the refrigerator determines the angle between the third-view boundary line and the fourth-view boundary line as the maximum field of view of the camera along a set direction, including:

[0106] calculate

[0107] Where B' and C' represent the third coordinate of the third reference position and the fourth coordinate of the fourth reference position, respectively, FOV max This indicates the maximum field of view of the camera along the set direction.

[0108] It should be noted that, when the direction is set to horizontal, B' and C' are B1' and C1' in Embodiment 3, respectively, and the FOV... max That is, HFOV max When the direction is set to vertical, B' and C' are B2' and C2' in Example 4, respectively, and FOV max That is, VFOV max .

[0109] Thus, after determining the lens coordinates of the lens position, the third coordinates of the third reference position, and the fourth coordinates of the fourth reference position, the embodiments of this disclosure can determine... and The coordinates of each element are determined, and then, based on the law of cosines, the angle between the boundary lines of the third and fourth perspectives can be determined, thus calculating the maximum viewing angle of the camera along the set direction. In this embodiment, the maximum viewing angle of the camera along the set direction can be accurately calculated based on the law of cosines, further improving the recognition rate of food images within the inner liner area.

[0110] In a specific implementation, combined with Figure 4 As shown, the maximum field of view (HFOV) of the camera along the horizontal direction. max for:

[0111]

[0112] Combination Figure 5 As shown, the maximum field of view (VFOV) of the camera along the vertical direction. max for:

[0113]

[0114] Optionally, combined Figure 9 As shown, the refrigerator determines the center point of the first side and the center point of the second side of the area to be identified as the first reference position and the second reference position, respectively, including:

[0115] S21, the refrigerator constructs a three-dimensional coordinate system with the intersection of the door rotation axis and the set straight line as the origin, the width direction of the refrigerator body towards the area to be identified as the X-axis, the thickness direction of the refrigerator body away from the refrigerator body as the Y-axis, and the direction of gravity as the Z-axis. The set straight line represents a straight line passing through the center point of the first side and the center point of the second side.

[0116] Combination Figure 1 and Figure 2 , Figure 9 As shown in this embodiment, a three-dimensional coordinate system is constructed with the intersection of the door's rotation axis and a set straight line as the origin, the width direction of the box towards the area to be identified as the X-axis, the thickness direction of the box away from the box as the Y-axis, and the gravity direction as the Z-axis. By constructing this three-dimensional coordinate system with the width direction of the box towards the area to be identified as the X-axis, the thickness direction of the box away from the box as the Y-axis, and the gravity direction as the Z-axis, the ordinates of the first and third reference positions are both zero, and the ordinates of the second and fourth reference positions are also zero, thereby reducing... and This reduces computational complexity while ensuring the accuracy of viewpoint calculations.

[0117] S22, the refrigerator obtains its cabinet width W. box and the maximum width W of the region to be identified along the X-axis ROI .

[0118] S23, the refrigerator obtains the cabinet thickness D. box And the maximum thickness D of the region to be identified along the Y-axis ROI .

[0119] S24, when the refrigerator is set to horizontal, according to (0.5W) box +0.5W ROI ,0,D box -0.5D ROI ), determine the first coordinate B of the first reference position, according to (0.5W box -0.5W ROI ,0,D box -0.5D ROI ), determine the second coordinate C of the second reference position.

[0120] Combination Figure 4 As shown, in this step, the first coordinate B includes either B1 or B2, and the second coordinate C includes either C1 or C2. To ensure that the area to be identified is basically covered within the field of view of the camera, when the direction is set to horizontal, the horizontal coordinate of the first coordinate B1 of the first reference position is W. box With W ROI The average of the sum values, and the x-coordinate of the second coordinate C1 of the second reference position is W. box With W ROI The average of the differences; simultaneously, due to the center point of the first side of the region to be identified, the vertical coordinates of the first coordinate B1 and the second coordinate C1 are both W. box With 0.5W ROI The difference.

[0121] S25, when the refrigerator is set to the vertical direction, according to (0.5W) box ,0,D box ), determine the first coordinate B of the first reference position, according to (0.5W box ,0,D box -D ROI ), determine the second coordinate C of the second reference position.

[0122] Combination Figure 5 As shown, in this step, to ensure that the area to be identified is basically covered within the field of view of the camera, when the direction is set to vertical, the horizontal coordinates of the first coordinate B2 of the first reference position and the second coordinate C2 of the second reference position are both 0.5W. box And the vertical coordinate of the first coordinate B2 of the second reference position is D. boxThe vertical coordinate of the second coordinate C2 is D. box With D ROI The difference.

[0123] Thus, by constructing a three-dimensional coordinate system in the manner described above and determining the first coordinate B and the second coordinate C accordingly, the present disclosure embodiment can reduce... and This reduces computational complexity, lowers the computational difficulty of the minimum perspective, and improves the accuracy of the minimum perspective calculation.

[0124] Optionally, combined Figure 3 As shown, the refrigerator determines the center points of the third and fourth sides of the maximum recognition area as the third and fourth reference positions, respectively, including:

[0125] The refrigerator obtains the ratio coefficient k between the maximum recognition area and the area to be recognized.

[0126] When the refrigerator is set to horizontal, according to (0.5W) box / k+0.5W ROI / k,0,D box -0.5D ROI ), determine the third coordinate B' of the third reference position, according to (0.5W box / k-0.5W ROI / k,0,D box -0.5D ROI ), determine the fourth coordinate C' of the fourth reference position.

[0127] When the refrigerator is set to vertical, according to (0.5W) box / k,0,D box -D ROI ), determine the third coordinate B' of the third reference position, according to (0.5W box / k,0,D box ), determine the fourth coordinate C' of the fourth reference position.

[0128] In this step, when the area to be identified is magnified by a scaling factor k to obtain the maximum recognition area, the W in the abscissas of the third coordinate B' and the fourth coordinate C' is... box and W ROI All updates are based on the scaling factor k. Combined with... Figure 4 and Figure 5 As shown, the third coordinate B' includes either B1' or B2', and the fourth coordinate C' includes either C1' or C2'.

[0129] Thus, by constructing a three-dimensional coordinate system in the manner described above and determining the third coordinate B' and the fourth coordinate C' accordingly, the present disclosure embodiment can reduce... and This reduces computational complexity, lowers the computational difficulty of the maximum field of view, and improves the accuracy of the maximum field of view calculation.

[0130] Optionally, combined Figure 1 As shown, the refrigerator determines the lens position at the center of the camera lens, including:

[0131] The refrigerator obtains the maximum door opening angle α and the real-time installation tilt angle β between the camera bracket and the door.

[0132] The refrigerator obtains the distance x between the center of the camera lens and the rotation axis of the door, and the distance y between the rotation axis of the bracket and the center of the camera lens.

[0133] The refrigerator determines the lens coordinates A of the center of the camera lens based on (xcosα+ycos(β-180°+α),xsinα+ysin(β-180°+α),0).

[0134] Thus, in this embodiment, after determining the maximum opening angle α of the door, the distance x between the center of the camera lens and the rotation axis of the door, the distance y between the rotation axis of the bracket and the center of the camera lens, and the real-time installation tilt angle β between the camera bracket and the door, the projection of the x-axis to the X-axis can be obtained through xcosα, and the projection of the y-axis to the X-axis can be obtained through ycos(β-180°+α). The abscissa of lens coordinate A can be obtained by summing the two X-axis projections. Simultaneously, the projection of the x-axis to the Y-axis can be obtained through xsinα, and the projection of the y-axis to the Y-axis can be obtained through ysin(β-180°+α). The ordinate of lens coordinate A can be obtained by summing the two Y-axis projections, while the ordinate of lens coordinate A is zero. Therefore, this embodiment can determine lens coordinate A based on the three structural parameters: the maximum opening angle α of the door, the real-time installation tilt angle β between the camera bracket and the door, and the distance x between the center of the camera lens and the rotation axis of the door, ensuring the accuracy of lens position calculation and improving the accuracy of minimum and maximum viewing angle calculations.

[0135] Combination Figure 10 As shown, this disclosure provides an apparatus 300 for determining the camera angle for capturing an inner liner area, including a processor 301 and a memory 302. Optionally, the apparatus 300 may further include a communication interface 303 and a bus 304. The processor 301, communication interface 303, and memory 302 can communicate with each other via the bus 304. The communication interface 303 can be used for information transmission. The processor 301 can call logical instructions in the memory 302 to execute the method for determining the camera angle for capturing an inner liner area described in the above embodiment.

[0136] Furthermore, the logic instructions in the aforementioned memory 302 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0137] The memory 302, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 301 executes functional applications and data processing by running the program instructions / modules stored in the memory 302, that is, it implements the method for determining the camera angle for capturing the inner lining area in the above embodiments.

[0138] The memory 302 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 302 may include high-speed random access memory and may also include non-volatile memory.

[0139] This disclosure provides a refrigerator, including a cabinet, a door, a camera bracket, a camera, and the aforementioned device 300 for determining the camera's mounting angle for capturing images of the inner liner area. The door is rotatably mounted on the front side of the cabinet via a door rotation axis, with a bracket rotation axis located at the end of the door away from the door rotation axis. The camera bracket is rotatably mounted on the door via the bracket rotation axis. The camera is mounted on the camera bracket. The device 300 for determining the camera's mounting angle for capturing images of the inner liner area is mounted on the door. The mounting relationship described herein is not limited to placement inside the door, but also includes installation and connection with other components of the refrigerator, including but not limited to physical connections, electrical connections, or signal transmission connections. Those skilled in the art will understand that the device 300 for determining the camera's mounting angle for capturing images of the inner liner area can be adapted to feasible product bodies to achieve other feasible embodiments.

[0140] This disclosure provides a computer-readable storage medium storing computer-executable instructions configured to perform the above-described method for determining the camera angle for capturing an inner liner area.

[0141] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., and other media capable of storing program code.

[0142] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in this application means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0143] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0144] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0145] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A method for determining the camera angle for capturing the inner liner area of ​​a refrigerator, characterized in that the refrigerator... include: Box; The door is rotatably mounted on the front side of the box via a door rotation axis, and a bracket rotation axis is provided at the end of the door away from the door rotation axis; The camera bracket can be rotatably mounted on the door via a bracket rotation axis; The camera is mounted on a camera bracket; the method includes: Determine the area to be identified, the maximum recognition area corresponding to the area to be identified, and the lens position of the center of the camera lens; The center point of the first side and the center point of the second side of the area to be identified are determined as the first reference position and the second reference position, respectively. The first side represents the side that is farthest from the door rotation axis along the set direction associated with the direction, and the second side represents the side that is closest to the door rotation axis along the set direction associated with the direction. The set direction includes the horizontal direction or the vertical direction. The horizontal direction is associated with the thickness direction of the box, and the vertical direction is associated with the width direction of the box. The center points of the third and fourth sides of the maximum recognition area are determined as the third reference position and the fourth reference position, respectively; the third side represents the side that is farthest from the door rotation axis along the set direction association direction, and the fourth side represents the side that is closest to the door rotation axis along the set direction association direction. Based on the lens position and the first and second reference positions, determine the minimum angle of view of the camera along the set direction; Based on the lens position and the third and fourth reference positions, determine the maximum viewing angle of the camera along the set direction.

2. The method according to claim 1, characterized in that, Based on the lens position and the first and second reference positions, determine the minimum viewing angle of the camera along a set direction, including: Determine the boundary line of the first viewpoint based on the camera position and the first reference position; Determine the boundary line of the second viewpoint based on the camera position and the second reference position; The angle between the boundary line of the first viewpoint and the boundary line of the second viewpoint is determined as the minimum field of view of the camera along the set direction. Based on the lens position and the third and fourth reference positions, determine the maximum viewing angle of the camera along the set direction, including: Determine the boundary line of the third-view perspective based on the camera position and the third reference position; Determine the boundary line of the fourth perspective based on the camera position and the fourth reference position; The angle between the boundary line of the third view and the boundary line of the fourth view is determined as the maximum field of view of the camera along the set direction.

3. The method according to claim 2, characterized in that, Determine the angle between the boundary line of the first viewpoint and the boundary line of the second viewpoint as the minimum field of view of the camera along a set direction, including: calculate Where A, B, and C represent the lens coordinates of the lens position, the first coordinate of the first reference position, and the second coordinate of the second reference position, respectively, FOV. min This indicates the minimum field of view of the camera along a set direction.

4. The method according to claim 3, characterized in that, The angle between the third-view boundary line and the fourth-view boundary line is determined as the maximum field of view of the camera along the set direction, including: calculate Where B' and C' represent the third coordinate of the third reference position and the fourth coordinate of the fourth reference position, respectively, FOV max This indicates the maximum field of view of the camera along the set direction.

5. The method according to claim 4, characterized in that, Determine the center points of the first and second sides of the region to be identified as the first and second reference positions, respectively, including: A three-dimensional coordinate system is constructed with the intersection of the door's rotation axis and the set straight line as the origin, the width direction of the box towards the area to be identified as the X-axis, the thickness direction of the box away from the box as the Y-axis, and the direction of gravity as the Z-axis; the set straight line represents the straight line passing through the center point of the first side and the center point of the second side. Obtain the box width W box and the maximum width W of the region to be identified along the X-axis ROI ; Obtain the box thickness D box And the maximum thickness D of the region to be identified along the Y-axis ROI ; When the direction is set to horizontal, according to (0.5W) box +0.5W ROI ,0,D box -0.5D ROI ), determine the first coordinate B of the first reference position, according to (0.5W box -0.5W ROI ,0,D box -0.5D ROI ), determine the second coordinate C of the second reference position; When the direction is set to vertical, according to (0.5W) box ,0,D box -D ROI ), determine the first coordinate B of the first reference position, according to (0.5W box ,0,D box ), determine the second coordinate C of the second reference position.

6. The method according to claim 5, characterized in that, The center points of the third and fourth sides of the largest recognition region are determined as the third and fourth reference positions, respectively, including: Obtain the ratio k between the maximum recognition area and the area to be recognized; When the direction is set to horizontal, according to (0.5W) box / k+0.5W ROI / k,0,D box -0.5D ROI ), determine the third coordinate B' of the third reference position, according to (0.5W box / k-0.5W ROI / k,0,D box -0.5D ROI ), determine the fourth coordinate C' of the fourth reference position; When the direction is set to vertical, according to (0.5W) box / k,0,D box -D ROI ), determine the third coordinate B' of the third reference position, according to (0.5W box / k,0,D box ), determine the fourth coordinate C' of the fourth reference position.

7. The method according to claim 5, characterized in that, Determine the center position of the camera lens, including: Obtain the maximum opening angle α of the door and the real-time installation tilt angle β between the camera bracket and the door; Obtain the distance x between the center of the camera lens and the rotation axis of the door, and the distance y between the rotation axis of the bracket and the center of the camera lens; Based on (xcosα+ycos(β-180°+α),xsinα+ysin(β-180°+α),0), determine the lens coordinates A of the lens position at the center of the camera lens.

8. An apparatus for determining the camera angle for capturing an inner liner area, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to, when running the program instructions, execute the method for determining the camera angle for capturing the inner liner area as described in any one of claims 1 to 7.

9. A refrigerator, characterized in that, include: Box; The door is rotatably mounted on the front side of the box via a door rotation axis, and a bracket rotation axis is provided at the end of the door away from the door rotation axis; The camera bracket can be rotatably mounted on the door via a bracket rotation axis; The camera is mounted on a camera bracket; The device for determining the camera angle for capturing the inner liner area as described in claim 8 is installed on the door body.

10. A computer-readable storage medium storing program instructions, characterized in that, When the program instructions are executed, they cause the computer to perform the method for determining the camera angle for capturing the inner liner area as described in any one of claims 1 to 7.

Images