Camera visual angle regulation and control method and device for inner container area, refrigerator and computer readable storage medium

Abstract

The invention relates to the technical field of refrigeration equipment, and discloses a camera visual angle regulation and control method for an inner container area, and the method comprises the steps: determining a current to-be-recognized area, the area offset in the horizontal direction, and the lens position of the center of a camera lens; determining a central point of a first side surface and a central point of a second side surface of the current to-be-identified area as a first reference position and a second reference position respectively; determining a current view angle and a target view angle of the camera in the horizontal direction according to the lens position, the first reference position, the second reference position and the area offset; and determining the visual angle offset of the camera according to the visual angle difference between the current visual angle and the target visual angle. According to the method, the relatively complete food material area of the liner area can be obtained through camera shooting, and the reliability of food material image recognition is improved. The invention further discloses a camera visual angle regulation and control device for the inner container area, a refrigerator and a computer readable storage medium.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of refrigeration equipment, for example, to a camera view angle regulation method and device for a liner area, a refrigerator, and a computer readable storage medium. BACKGROUND

[0002] At present, with the rapid development of science and technology, refrigerators with refrigeration and freezing and food management functions are widely used in the field of intelligent household appliances. In order to better realize food management, a refrigerator is provided with a camera at the top of the refrigeration chamber to identify the food storage in the refrigeration chamber. However, the drawers in the refrigeration chamber are arranged from top to bottom, and in the process of use, there may be a situation that multiple drawers are pulled out at the same time. At this time, the camera installed at the top of the refrigeration chamber can only capture the food image of the uppermost drawer that is pulled out, and cannot capture the food images of other pulled-out drawers, so the shooting range is relatively limited, which is not conducive to food management.

[0003] In order to expand the shooting range of food in the refrigeration chamber of the refrigerator, the related technology discloses a kind of intelligent refrigerator, comprising: refrigeration chamber, layer frame is arranged in refrigeration chamber, layer frame is separated into multiple layers of storage space in refrigeration chamber, temperature sensor is arranged in each layer of storage space;Camera, set on the door body of refrigeration chamber, for shooting the food in refrigeration chamber;Refrigeration air duct, refrigeration air duct is connected with multiple storage spaces by multiple groups of air outlets arranged on air duct front cover plate, and cold air in each storage space corresponds to a group of air outlets and enters refrigeration air duct after evaporator and fan;Air door assembly, for opening or closing one or more groups of multiple air outlets;Control unit, respectively in camera and air door assembly connection, control unit can determine the set temperature of each layer of storage space according to the image information of camera shooting, and simultaneously controls the opening and closing of one or more groups of air outlets of air door assembly according to the comparison between real-time temperature value and set temperature value detected by temperature sensor. When the refrigeration door body is rotated to the plane where the refrigeration chamber opening is located, the set angle is between 0 and 90 degrees. When the set angle is between 20 and 40 degrees, the distortion of the two ends of the captured image is small.

[0004] In the process of implementing the embodiments of the present disclosure, it is found that at least the following problems exist in the related art:

[0005] When the refrigeration door body is rotated, the set angle is configured to be between 0 and 90 degrees, and the camera shooting range is relatively fixed. Therefore, when the refrigeration door body is rotated, part of the area in the refrigeration space will move out of the camera shooting range, so that the camera cannot capture a relatively complete food image, affecting the reliability of food image recognition.

[0006] It is to be understood that the information disclosed in the Background section is merely for the purpose of enhancing the understanding of the present application and thus can include information that does not constitute prior art to those skilled in the art. SUMMARY

[0007] The following presents a simplified summary of some aspects of the disclosed embodiments in order to provide a basic understanding of such embodiments. The summary is not an extensive overview of the application. It is not intended to identify key / critical elements of the embodiments or to delineate the scope of the embodiments but to present some aspects of the disclosed embodiments in a simplified form as a prelude to the more detailed description that is presented later.

[0008] Embodiments of the present disclosure provide a camera view angle adjustment method and device for a liner area, a refrigerator and a computer readable storage medium, to obtain a relatively complete food material area in the liner area through camera shooting, and improve the reliability of food material image recognition.

[0009] In some embodiments, the refrigerator comprises: a cabinet; a door body rotatably arranged on the front side of the cabinet through a door body rotation shaft, an end of the door body away from the rotation shaft being provided with a support rotation shaft; a camera support rotatably arranged on the door body through the support rotation shaft; and a camera arranged on the camera support. The method comprises: determining a current to-be-recognized area, an area offset along a horizontal direction, and a lens position of a camera lens center; determining a center point of a first side and a center point of a second side of the current to-be-recognized area as a first reference position and a second reference position respectively; the first side represents a side farthest from the door body rotation shaft along the thickness direction of the cabinet, and the second side represents a side closest to the door body rotation shaft along the thickness direction of the cabinet; determining a current view angle and a target view angle of the camera along the horizontal direction according to the lens position and the first reference position, the second reference position and the area offset; and determining a view angle offset of the camera according to a view angle difference between the current view angle and the target view angle.

[0010] In some embodiments, the determination of the target view angle of the camera along the horizontal direction according to the lens position and the first reference position, the second reference position and the area offset comprises: determining a third reference position according to the first reference position and the area offset; determining a fourth reference position according to the second reference position and the area offset; and determining the target view angle of the camera along the horizontal direction according to the lens position and the third reference position and the fourth reference position.

[0011] In some embodiments, the determination of the center point of the first side and the center point of the second side of the current to-be-recognized area as the first reference position and the second reference position respectively comprises: taking the intersection of the door body rotation shaft and a set straight line as the origin, the cabinet width direction towards the current to-be-recognized area as the X axis, the cabinet thickness direction away from the cabinet as the Y axis, and the gravity direction as the Z axis, to construct a three-dimensional coordinate system; the set straight line represents a straight line passing through the center point of the first side and the center point of the second side; and obtaining the cabinet width Wbox and the maximum width W of the region to be identified along the X axis ROI ; obtain the box thickness D box and the maximum thickness D of the region to be identified along the Y axis ROI ; obtain the proportionality coefficient k of the maximum identification region and the region to be identified; determine the first coordinate B0 of the first reference position according to (0.5W box / k+0.5W ROI / k, 0, D box -0.5D ROI ), and determine the second coordinate C0 of the second reference position according to (0.5W box / k-0.5W ROI / k, 0, D box -0.5D ROI ).

[0012] In some embodiments, the third reference position is determined according to the first reference position and the region offset, including: in the case that the current region to be identified is translated along the X axis in the positive direction by the region offset Δx, the third coordinate B1 of the third reference position is determined according to (0.5W box / k+0.5W ROI / k+Δx, 0, D box -0.5D ROI ); or in the case that the current region to be identified is translated along the X axis in the negative direction by the region offset Δx, the third coordinate B1 of the third reference position is determined according to (0.5W box / k+0.5W ROI / k-Δx, 0, D box -0.5D ROI ); the fourth reference position is determined according to the second reference position and the region offset, including: in the case that the current region to be identified is translated along the X axis in the positive direction by the region offset Δx, the fourth coordinate C1 of the fourth reference position is determined according to (0.5W box / k-0.5W ROI / k+Δx, 0, D box -0.5D ROI ); or in the case that the current region to be identified is translated along the X axis in the negative direction by the region offset Δx, the fourth coordinate C1 of the fourth reference position is determined according to (0.5W box / k-0.5W ROI / k-Δx, 0, D box -0.5D ROI ).

[0013] In some embodiments, the current horizontal view angle of the camera is determined according to the lens position and the first reference position, the second reference position, and the area offset, including: determining a first view boundary line according to the lens position and the first reference position; determining a second view boundary line according to the lens position and the second reference position; determining the included angle between the first view boundary line and the second view boundary line as the current horizontal view angle of the camera; determining the target horizontal view angle of the camera according to the lens position and the third reference position, the fourth reference position; determining a third view boundary line according to the lens position and the third reference position; determining a fourth view boundary line according to the lens position and the fourth reference position; determining the included angle between the third view boundary line and the fourth view boundary line as the target horizontal view angle of the camera.

[0014] In some embodiments, the included angle between the first view boundary line and the second view boundary line is determined as the current horizontal view angle of the camera, including: calculating wherein A, B0, and C0 represent the lens coordinate of the lens position, the first coordinate of the first reference position, and the second coordinate of the second reference position, respectively, and FOV0 represents the current horizontal view angle of the camera; and the included angle between the third view boundary line and the fourth view boundary line is determined as the target horizontal view angle of the camera, including: calculating wherein B1 and C1 represent the third coordinate of the third reference position and the fourth coordinate of the fourth reference position, respectively, and FOV1 represents the target horizontal view angle of the camera.

[0015] In some embodiments, the lens position of the camera lens center is determined, including: obtaining the maximum door opening angle a of the door body and the real-time installation inclination angle β of the camera support and the door body; obtaining the distance x between the camera lens center and the door body rotation axis and the distance y between the support rotation axis and the camera lens center; and determining the lens coordinate A of the lens position of the camera lens center according to (xcos a+ycos (β-180°+a), xsin a+ysin (β-180°+a), 0).

[0016] In some embodiments, the device includes a processor and a memory storing program instructions, and the processor is configured to execute the above-mentioned camera view angle control method for the liner area when running the program instructions.

[0017] In some embodiments, the refrigerator includes: a cabinet; a door body rotatably arranged on the front side of the cabinet through a door body rotation axis, an end of the door body away from the door body rotation axis being provided with a support rotation axis; a camera support rotatably arranged on the door body through the support rotation axis; a camera arranged on the camera support; and a camera view angle control device for the liner area as mentioned above installed on the door body.

[0018] In some embodiments, the computer readable storage medium stores program instructions which, when executed, cause a computer to perform the camera view angle adjustment method for liner area as described above.

[0019] The camera view angle adjustment method for liner area, the device, the refrigerator and the computer readable storage medium provided by the embodiments of the present disclosure can achieve the following technical effects:

[0020] The embodiments of the present disclosure first determine the current to-be-identified region, the region offset along the horizontal direction, the lens position of the camera lens center, and determine the center point of the first side and the center point of the second side of the current to-be-identified region as the first reference position and the second reference position respectively. Then, the current view angle and the target view angle of the camera along the horizontal direction are determined according to the lens position, the first reference position, the second reference position and the region offset. Finally, the view angle offset of the camera is determined according to the view angle difference between the current view angle and the target view angle. The embodiments of the present disclosure can adaptively adjust the view angle of the camera according to the region offset of the current to-be-identified region, so that the shooting range of the camera still covers the current to-be-identified region after the current to-be-identified region is offset, thereby obtaining a relatively complete food material region of the liner region, and improving the reliability of food material image recognition.

[0021] The foregoing general description and the following description are only exemplary and explanatory, and are not intended to limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0022] One or more embodiments are exemplarily illustrated by corresponding drawings, which do not constitute limitation on the embodiments, elements with the same reference numerals in the drawings are shown as similar elements, the drawings do not constitute proportional limitation, and wherein:

[0023] Figure 1 is a top view of a refrigerator body and a door body provided by an embodiment of the present disclosure;

[0024] Figure 2 is a front view of a refrigerator body provided by an embodiment of the present disclosure;

[0025] Figure 3 is a region schematic view of a to-be-identified region and a maximum identification region provided by an embodiment of the present disclosure;

[0026] Figure 4 is a top view of a camera horizontal direction view angle provided by an embodiment of the present disclosure;

[0027] Figure 5 is a schematic view of a camera view angle adjustment method for liner area provided by an embodiment of the present disclosure;

[0028] Figure 6is a schematic diagram of another method for adjusting the camera view angle of the inner container area provided by the embodiments of the present disclosure.

[0029] Figure 7 is a schematic diagram of a device for adjusting the camera view angle of the inner container area provided by the embodiments of the present disclosure.

[0030] Reference signs:

[0031] 10: box body; 20: door body; 30: camera support;

[0032] 101: inner container; 201: camera;

[0033] 20a: door body rotation shaft; 20b: support rotation shaft;

[0034] 70: device for determining the camera view angle for shooting the inner container area;

[0035] 700: processor; 701: memory;

[0036] 702: communication interface; 703: bus. DETAILED DESCRIPTION

[0037] In order to enable a person skilled in the art to more fully understand the features and technical contents of the embodiments of the present disclosure, the implementation of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings, which are used only for reference and are not intended to limit the embodiments of the present disclosure. In the following technical description, in order to facilitate explanation, a plurality of details are provided to provide a full understanding of the disclosed embodiments. However, one or more embodiments can still be implemented without these details. In other cases, well-known structures and devices can be simplified to facilitate the drawings.

[0038] The terms "first", "second", and the like in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion.

[0039] Unless otherwise specified, the term "a plurality of" means two or more.

[0040] In the embodiments of the present disclosure, the character " / " represents an "or" relationship between the preceding and following objects. For example, A / B represents: A or B.

[0041] The term "and / or" is a description of the relationship between the objects, which means that there can be three relationships. For example, A and / or B means: A or B, or, A and B, the three relationships.

[0042] The term "corresponding" can refer to a kind of association or binding relationship. A corresponds to B means that there is an association or binding relationship between A and B.

[0043] In combination Figure 1 As shown, the refrigerator provided by the embodiments of the present disclosure includes a cabinet 10 and a door body 20, a camera bracket 30, and a camera 201. The door body 20 is installed on the front side of the cabinet 10 through a door body rotation shaft 20a. The end of the door body 20 away from the door body rotation shaft 20a is provided with a bracket rotation shaft 20b. The camera bracket 30 is rotatably arranged on the door body 40 through the bracket rotation shaft 20b. The camera 201 is arranged on the camera bracket 30.

[0044] The cabinet 10 is configured with an inner container 101, which includes one or more shelves and a refrigeration chamber below the one or more shelves. One or more drawers are arranged in the refrigeration chamber. The one or more drawers can be pulled to the maximum position along the thickness direction of the cabinet.

[0045] Optionally, the number of door bodies 20 is two. The two door bodies 20 are symmetrically arranged on the front side of the cabinet 10 along the central axis of the thickness direction of the cabinet. The bracket rotation shaft 20b is installed on the end of any one of the door bodies 20 away from the door body rotation shaft 20a. In this way, the inner container area can be identified by the camera installed on any one of the door bodies.

[0046] In combination Figure 3 As shown, when the camera 201 is shooting, the camera 201 is associated with a to-be-identified area. The to-be-identified area represents an area that needs to be identified when the camera 201 is shooting. The to-be-identified area includes an inner container area and a periphery area of the inner container area, and the inner container area includes a shelf area and a refrigeration chamber area below the shelf area. The shelf area includes one or more shelves, and the refrigeration chamber area includes one or more drawers. The maximum thickness of the to-be-identified area along the thickness direction of the cabinet is equal to the thickness of the inner container area along the thickness direction of the cabinet. It should be noted that the periphery area of the inner container area can be an area extending outward from the inner container area along the width direction of the cabinet.

[0047] The camera 201 is also associated with a maximum identification area. The maximum identification area represents an identification area obtained by enlarging the to-be-identified area along the width direction of the cabinet, and the ratio of the area of the to-be-identified area to the area of the maximum identification area is greater than an area ratio and less than 1. The area ratio is greater than or equal to 0.7. Preferably, the area ratio is 0.7.

[0048] In one specific embodiment, in combination Figure 1 and Figure 2 As shown, the cabinet thickness of the cabinet 10 is H box and the cabinet width Wbox The maximum width of the area to be identified along the width direction of the box is W. ROI The maximum thickness of the area to be identified along the thickness direction of the box is D. ROI The maximum thickness of the region to be identified along the vertical direction is H. ROI .

[0049] Combination Figure 1 As shown, the maximum opening angle of the door 20 is α, and the angle between the camera bracket 30 and the door 20 is the real-time installation tilt angle β. The distance between the door rotation axis 20a and the bracket rotation axis 20b is x. The distance between the bracket rotation axis and the center of the camera lens is y.

[0050] In another specific embodiment, to ensure the area recognition function of the camera, the viewing angle of the camera in the horizontal or vertical direction should not be too large. If the viewing angle of the camera in the horizontal or vertical direction is too large, the target to be recognized in the area to be recognized will be too small, resulting in a low pixel ratio of the target and a decrease in the recognition rate of the camera. To this end, this embodiment configures a ratio coefficient k between the maximum recognition area and the area to be recognized, and calculates the maximum viewing angle of the camera in the horizontal or vertical direction based on the ratio coefficient k, thereby ensuring the recognition rate of the camera.

[0051] It should be noted that the scaling factor k represents the maximum width W of the area to be identified along the width direction of the box. ROI The ratio of the maximum width of the maximum recognition area along the width direction of the box, or the maximum thickness H of the area to be recognized along the vertical direction. ROI The ratio of the maximum thickness of the maximum recognition area in the vertical direction to the area ratio. The scaling factor k is determined by the area ratio. As an example, when the area ratio is 0.7, k is greater than or equal to 0.7 and less than or equal to 1.

[0052] Based on the above refrigerator structural configuration, combined with Figure 5 As shown, this disclosure provides a method for adjusting the camera viewing angle in the inner lining area, including:

[0053] S01, the refrigerator determines the current area to be identified, the area offset along the horizontal direction, and the lens position of the center of the camera lens.

[0054] In this step, the current area to be identified includes at least the area to be identified, and the current area to be identified is located within the maximum identification area and the maximum coverage area coincides with the maximum identification area.

[0055] S03, the refrigerator determines the center point of the first side and the center point of the second side of the current to-be-identified region as a first reference position and a second reference position respectively. The first side represents a side farthest from the door body rotation axis in the thickness direction of the box body, and the second side represents a side closest to the door body rotation axis in the thickness direction of the box body.

[0056] S03, the refrigerator determines the current view angle and the target view angle of the camera along the horizontal direction according to the lens position and the first reference position, the second reference position and the region offset.

[0057] S04, the refrigerator determines the view angle offset of the camera according to the view angle difference between the current view angle and the target view angle. The view angle offset = current view angle-target view angle.

[0058] By adopting the camera view angle control method for the inner container region provided in the embodiments of the present disclosure, the embodiments of the present disclosure first determine the current to-be-identified region, the region offset along the horizontal direction, the lens position of the camera lens center, and the center point of the first side and the center point of the second side of the current to-be-identified region as the first reference position and the second reference position respectively. Then, the current view angle and the target view angle of the camera along the horizontal direction are determined according to the lens position and the first reference position, the second reference position and the region offset. Finally, the view angle offset of the camera is determined according to the view angle difference between the current view angle and the target view angle. The embodiments of the present disclosure can adaptively adjust the view angle of the camera according to the region offset of the current to-be-identified region, so that the shooting range of the camera still covers the current to-be-identified region after the current to-be-identified region shifts, thereby obtaining a relatively complete food material region in the inner container region, and improving the reliability of food material image recognition.

[0059] Optionally, the refrigerator determines the target view angle of the camera along the horizontal direction according to the lens position and the first reference position, the second reference position and the region offset, including:

[0060] The refrigerator determines a third reference position according to the first reference position and the region offset.

[0061] The refrigerator determines a fourth reference position according to the second reference position and the region offset.

[0062] The refrigerator determines the target view angle of the camera along the horizontal direction according to the lens position and the third reference position and the fourth reference position.

[0063] Thus, since the reference positions (including the first reference position and the second reference position) of the target view angle of the camera in the horizontal direction are also offset in the horizontal direction after the current to-be-identified region is offset in the horizontal direction, the third reference position is determined according to the first reference position and the region offset, and the fourth reference position is determined according to the second reference position and the region offset, and then the target view angle of the camera in the horizontal direction is calculated according to the lens position and the third reference position and the fourth reference position. In this way, the third reference position and the fourth reference position after the offset can be accurately determined based on the first reference position and the second reference position, and the accuracy of the calculation of the target view angle of the camera in the horizontal direction is ensured.

[0064] Optionally, in combination with Figure 1 and Figure 4 、 Figure 6 As shown in the figure, the refrigerator determines the center point of the first side of the current to-be-identified region and the center point of the second side as the first reference position and the second reference position, including:

[0065] S11, the refrigerator constructs a three-dimensional coordinate system with the intersection of the door body rotation axis and the set straight line as the origin, the direction of the width of the cabinet of the current to-be-identified region as the X axis, the direction of the thickness of the cabinet away from the cabinet as the Y axis, and the direction of gravity as the Z axis. The set straight line represents a straight line passing through the center point of the first side and the center point of the second side.

[0066] S12, the refrigerator obtains the width W box of the cabinet and the maximum width W ROI of the to-be-identified region along the X axis.

[0067] S13, the refrigerator obtains the thickness D box of the cabinet and the maximum thickness D ROI of the to-be-identified region along the Y axis.

[0068] S14, the refrigerator obtains the proportionality coefficient k of the maximum identified region and the to-be-identified region.

[0069] S15, the refrigerator determines the first coordinate B0 of the first reference position according to (0.5W box / k+0.5W ROI / k, 0, D box -0.5D ROI ), and determines the second coordinate C0 of the second reference position according to (0.5W box / k-0.5W ROI / k, 0, D box -0.5D ROI ).

[0070] In this way, the embodiment of the disclosure can reduce the calculation difficulty of the current horizontal view angle of the camera and improve the accuracy of the current view angle calculation by constructing a three-dimensional coordinate system in the above manner and determining the first coordinate B0 and the second coordinate C0 according to the three-dimensional coordinate system.

[0071] Optionally, in combination with Figure 4 As shown in the figure, the refrigerator determines a third reference position according to the first reference position and the area offset, including:

[0072] In the case that the current to-be-identified area is translated along the positive axis of the X axis according to the area offset Δx, the refrigerator determines the third coordinate B1 of the third reference position according to (0.5W box / k+0.5W ROI / k+Δx,0,D box -0.5D ROI ).

[0073] In the case that the current to-be-identified area is translated along the negative axis of the X axis according to the area offset Δx, the refrigerator determines the third coordinate B1 of the third reference position according to (0.5W box / k+0.5W ROI / k-Δx,0,D box -0.5D ROI ).

[0074] The refrigerator determines a fourth reference position according to the second reference position and the area offset, including:

[0075] In the case that the current to-be-identified area is translated along the positive axis of the X axis according to the area offset Δx, the refrigerator determines the fourth coordinate C1 of the fourth reference position according to (0.5W box / k-0.5W ROI / k+Δx,0,D box -0.5D ROI ).

[0076] In the case that the current to-be-identified area is translated along the negative axis of the X axis according to the area offset Δx, the refrigerator determines the fourth coordinate C1 of the fourth reference position according to (0.5W box / k-0.5W ROI / k-Δx,0,D box -0.5D ROI ).

[0077] In this way, when the current to-be-identified area is translated along the positive axis of the X axis according to the area offset Δx, the horizontal coordinate of the third reference position is increased by Δx compared with the first reference position, and the vertical coordinate and the vertical coordinate are unchanged, so the embodiment of the disclosure can determine the third coordinate B1 of the third reference position according to (0.5W box / k+0.5W ROI / k+Δx,0,D box-0.5D ROI ) the third coordinate B1 of the third reference position. When the current to-be-identified region is translated along the X-axis negative axis by the region offset amount Δx, the horizontal coordinate of the third reference position is reduced by Δx compared with the first reference position, and the vertical coordinate and the vertical coordinate are unchanged, therefore, the embodiment of the present disclosure can determine the third coordinate B1 of the third reference position according to (0.5W box / k+0.5W ROI / k-Δx,0,D box -0.5D ROI ) the third coordinate B1 of the third reference position. Based on the same translation manner, the present disclosure can determine the fourth coordinate C1 of the fourth reference position after the current to-be-identified region is translated along the X-axis positive axis by Δx according to (0.5W box / k-0.5W ROI / k+Δx,0,D box -0.5D ROI ) the third coordinate B1 of the third reference position. Based on the same translation manner, the present disclosure can determine the fourth coordinate C1 of the fourth reference position after the current to-be-identified region is translated along the X-axis positive axis by Δx according to (0.5W box / k-0.5W ROI / k-Δx,0,D box -0.5D ROI ) the third coordinate B1 of the third reference position. Based on the same translation manner, the present disclosure can determine the fourth coordinate C1 of the fourth reference position after the current to-be-identified region is translated along the X-axis positive axis by Δx according to (0.5W

[0078] Optionally, as shown in Figure 4 , the refrigerator determines a current view angle of the camera along the horizontal direction according to the lens position and the first reference position, the second reference position, and the region offset amount, including:

[0079] The refrigerator determines a first view angle boundary line according to the lens position and the first reference position.

[0080] The refrigerator determines a second view angle boundary line according to the lens position and the second reference position.

[0081] The refrigerator determines an included angle between the first view angle boundary line and the second view angle boundary line as the current view angle of the camera along the horizontal direction.

[0082] Optionally, the refrigerator determines a target view angle of the camera along the horizontal direction according to the lens position and the third reference position and the fourth reference position.

[0083] The refrigerator determines a third view angle boundary line according to the lens position and the third reference position.

[0084] The refrigerator determines a fourth view angle boundary line according to the lens position and the fourth reference position.

[0085] The refrigerator determines an included angle between the third view angle boundary line and the fourth view angle boundary line as the target view angle of the camera along the horizontal direction.

[0086] In this way, the first view angle boundary line is determined according to the lens position and the first reference position, and the second view angle boundary line is determined according to the lens position and the second reference position. The included angle formed by the first view angle boundary line and the second view angle boundary line is the current view angle of the camera along the horizontal direction, and the field of view range corresponding to the current view angle is the current field of view range. In this way, the current field of view range can cover at least the to-be-identified region, achieving full coverage of the inner container region and the food material region. Accordingly, after the current to-be-identified region is offset, the field of view range corresponding to the target view angle is the target field of view range, and the target field of view range can cover at least the offset to-be-identified region, and full coverage of the inner container region and the food material region can also be achieved, thereby achieving full coverage of the inner container region and the food material region after the current to-be-identified region is offset, and a relatively complete food material region is obtained, which helps to improve the reliability of food material image recognition.

[0087] Optionally, the refrigerator determines that the included angle of the first view angle boundary line and the second view angle boundary line is the current view angle of the camera along the horizontal direction, and the refrigerator comprises:

[0088] The refrigerator calculates

[0089] wherein A, B0 and C0 represent the lens coordinate of the lens position, the first coordinate of the first reference position and the second coordinate of the second reference position respectively, and HFOV0 represents the current view angle of the camera along the horizontal direction.

[0090] The refrigerator determines that the included angle of the third view angle boundary line and the fourth view angle boundary line is the target view angle of the camera along the horizontal direction, and the refrigerator comprises:

[0091] The refrigerator calculates

[0092] wherein B1 and C1 represent the third coordinate of the third reference position and the fourth coordinate of the fourth reference position respectively, and HFOV1 represents the target view angle of the camera along the horizontal direction.

[0093] In this way, after the lens coordinate of the lens position, the first coordinate of the first reference position and the second coordinate of the second reference position are determined, the first view angle boundary line and the second view angle boundary line can be determined and respectively, and the included angle of the first view angle boundary line and the second view angle boundary line can be determined based on the cosine theorem, that is, the current view angle of the camera along the horizontal direction is calculated, and after the third coordinate of the third reference position and the fourth coordinate of the fourth reference position are determined, the third view angle boundary line and the fourth view angle boundary line can be determined and The angle between the third view boundary line and the fourth view boundary line can be determined based on the cosine theorem, that is, the target view angle in the horizontal direction of the camera is calculated. In this way, the current view angle and the target view angle of the camera in the horizontal direction can be accurately calculated based on the cosine theorem, the current to-be-identified region is offset, the inner container region and the food material region are fully covered, a relatively complete food material region is obtained, and the reliability of food material image recognition is further improved.

[0094] Optionally, in combination with Figure 1 As shown, the refrigerator determines the lens position of the camera lens center, including:

[0095] The refrigerator obtains the maximum door opening angle a of the door body and the real-time installation inclination angle β of the camera support and the door body.

[0096] The refrigerator obtains the distance x between the camera lens center and the door body rotation axis and the distance y between the support rotation axis and the camera lens center.

[0097] The refrigerator determines the lens coordinates A of the lens position of the camera lens center according to (xcos a+ycos (β-180°+a), xsina+ysin (β-180°+a), 0).

[0098] In this way, after determining the maximum door opening angle a of the door body and the distance x between the camera lens center and the door body rotation axis and the real-time installation inclination angle β of the camera support and the door body, the projection of x to the X axis can be obtained through xcos a, and the projection of y to the X axis can be obtained through ycos (β-180°+a). The horizontal coordinate of the lens coordinates A can be obtained by summing the two X axis projections. At the same time, the projection of x to the Y axis can be obtained through xsina, and the projection of y to the Y axis can be obtained through ysin (β-180°+a). The vertical coordinate of the lens coordinates A can be obtained by summing the two Y axis projections, and the vertical coordinate of the lens coordinates A is zero. Therefore, the lens coordinates A can be determined according to the maximum door opening angle a of the door body and the real-time installation inclination angle β of the camera support and the door body and the distance x between the camera lens center and the door body rotation axis, ensuring the accuracy of the lens position calculation, improving the accuracy of the current view angle and the target view angle calculation, and ensuring the completeness of the food material region obtained by shooting.

[0099] In combination with Figure 7As shown, the camera view angle control device 70 for the liner area provided by the embodiments of the present disclosure includes a processor 700 and a memory 701. Optionally, the device 70 can further include a communication interface 702 and a bus 703. The processor 700, the communication interface 702, and the memory 701 can complete communication with each other through the bus 703. The communication interface 702 can be used for information transmission. The processor 700 can invoke the logic instructions in the memory 701 to execute the camera view angle control method for the liner area of the above-mentioned embodiments.

[0100] In addition, the logic instructions in the memory 701 described above can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer readable storage medium.

[0101] The memory 701 as a computer readable storage medium can be used to store software programs, computer executable programs, such as program instructions / modules corresponding to the method in the embodiments of the present disclosure. The processor 700 executes the function application and data processing by running the program instructions / modules stored in the memory 701, that is, implements the camera view angle control method for the liner area in the above-mentioned embodiments.

[0102] The memory 701 can include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 701 can include a high-speed random access memory, and can also include a non-volatile memory.

[0103] The embodiments of the present disclosure provide a refrigerator, which includes a cabinet, a door body, a camera support, a camera, and the above-mentioned camera view angle control device 70 for the liner area. The door body is rotatably arranged on the front side of the cabinet through a door body rotation shaft, and a support rotation shaft is arranged at the end of the door body away from the door body rotation shaft. The camera support is rotatably arranged on the door body through the support rotation shaft. The camera is arranged on the camera support. The camera view angle control device 70 for the liner area is installed on the door body. The installation relationship described herein is not limited to being placed in the interior of the door body, but also includes installation connection with other components of the refrigerator, including but not limited to physical connection, electrical connection, or signal transmission connection, etc. Those skilled in the art can understand that the camera view angle control device 70 for the liner area can be adapted to a feasible product body, and thus other feasible embodiments can be realized.

[0104] The embodiment of the present disclosure provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are configured to execute the camera view angle regulation method for the liner area.

[0105] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes one or more instructions to make a computer device (which can be a personal computer, a server, or a network device) execute all or part of the steps of the method disclosed in the embodiment of the present disclosure. The storage medium mentioned above can be a non-transitory storage medium, for example, a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0106] The above description and drawings sufficiently show the embodiments of the present disclosure to enable a person skilled in the art to practice them. Other embodiments can include structural, logical, electrical, process, and other changes. The embodiments only represent possible changes. Unless explicitly required, individual components and functions are optional, and the order of operations can be changed. Parts and features of some embodiments can be included in or replace parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and not to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly requires otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms as well. Similarly, as used in this application, the term "and / or" refers to any and all possible combinations of one or more associated listed items. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or comprises" and the like mean the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, or device that includes the stated element. In this document, each embodiment focuses on the differences from other embodiments, and the same or similar parts between various embodiments can be referred to each other. For the method, product, and the like disclosed in the embodiments, if it corresponds to the method part disclosed in the embodiments, the relevant part can be referred to the description of the method part.

[0107] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods for each specific application to realize the described functions, but such implementation should not be considered beyond the scope of the embodiments of the present disclosure. The skilled person can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0108] In the embodiments disclosed herein, the disclosed methods, products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units can be only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms. The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to implement the embodiments. In addition, each functional unit in the embodiments of the present disclosure can be integrated in one processing unit, or each unit can be a physically independent unit, or two or more units can be integrated in one unit.

[0109] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other processing device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other processing device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

Claims

1. A camera view angle regulation method for a liner area, characterized in that a refrigerator The method comprises the following steps: The box body; The door body is rotatably arranged on the front side of the box body through a door body rotation shaft, and a support rotation shaft is arranged at the end of the door body away from the rotation shaft; The camera support is rotatably arranged on the door body through the support rotation shaft; The camera is arranged on the camera support; The method comprises the following steps: Determine the current to be identified area and the area offset along the horizontal direction, the lens position of the camera lens center; Determine the center points of the first side and the second side of the current to be identified area as the first reference position and the second reference position respectively; The first side represents the side farthest from the door body rotation shaft along the thickness direction of the box body, and the second side represents the side closest to the door body rotation shaft along the thickness direction of the box body; According to the lens position and the first reference position, the second reference position and the area offset, the current view angle and the target view angle of the camera along the horizontal direction are determined; According to the view angle difference value of the current view angle and the target view angle, the view angle offset of the camera is determined.

2. The method of claim 1, wherein, According to the lens position and the first reference position, the second reference position and the area offset, the target view angle of the camera along the horizontal direction is determined, which comprises: According to the first reference position and the area offset, the third reference position is determined; According to the second reference position and the area offset, the fourth reference position is determined; According to the lens position and the third reference position, the fourth reference position, the target view angle of the camera along the horizontal direction is determined.

3. The method of claim 2, wherein, Determine the center points of the first side and the second side of the current to be identified area as the first reference position and the second reference position respectively, which comprises: The intersection of the door body rotation shaft and the set straight line is taken as the origin, the box width direction of the current to be identified area is taken as the X axis, the box thickness direction away from the box is taken as the Y axis, and the gravity direction is taken as the Z axis, so as to construct a three-dimensional coordinate system; The set straight line represents the straight line passing through the center points of the first side and the second side; Obtaining the width W of the box box and the maximum width W of the region to be recognized along the X axis ROI ; Obtaining the thickness D of the box box and the maximum thickness D of the region to be identified along the Y axis ROI ; Obtain the proportional coefficient k of the maximum identification area and the to-be-identified area; According to (0.5W box / k+0.5W ROI / k,0,D box -0.5D ROI ), a first coordinate B0 of a first reference position is determined, and according to (0.5W box / k-0.5W ROI / k,0,D box -0.5D ROI ), a second coordinate C0 of a second reference position is determined.

4. The method of claim 3, wherein, According to the first reference position and the area offset, the third reference position is determined, which comprises: In the case that the current to-be-identified region is translated along the X-axis positive axis by the region offset amount Δx, the third coordinate B1 of the third reference position is determined according to (0.5W box / k+0.5W ROI / k+Δx,0,D box -0.5D ROI ), or In the case that the current to-be-identified region is translated along the X axis negative axis by the region offset amount Δx, the third coordinate B1 of the third reference position is determined according to (0.5W box / k+0.5W ROI / k-Δx,0,D box -0.5D ROI ). According to the second reference position and the area offset, the fourth reference position is determined, which comprises: In the case that the current to-be-identified region is translated along the X-axis positive axis by the region offset Δx, the fourth coordinate C1 of the fourth reference position is determined according to (0.5W box / k-0.5W ROI / k+Δx,0,D box -0.5D ROI ), or In the case that the current to-be-identified region is translated along the X axis negative axis by the region offset Δx, the fourth coordinate C1 of the fourth reference position is determined according to (0.5W box / k-0.5W ROI / k-Δx,0,D box -0.5D ROI ).

5. The method according to any one of claims 1 to 4, characterized in that, According to the lens position and the first reference position, the first view angle boundary line is determined; According to the lens position and the second reference position, the second view angle boundary line is determined; The included angle between the first view angle boundary line and the second view angle boundary line is determined as the current view angle of the camera along the horizontal direction; According to the lens position and the third reference position, the third view angle boundary line is determined; According to the lens position and the fourth reference position, the fourth view angle boundary line is determined; The included angle between the third view angle boundary line and the fourth view angle boundary line is determined as the target view angle of the camera along the horizontal direction. Determine the included angle between the first view angle boundary line and the second view angle boundary line as the current view angle of the camera along the horizontal direction, which comprises: ​ 6. The method of claim 5, wherein, ​ Wherein, A, B0 and C0 represent the lens coordinate of the lens position, the first coordinate of the first reference position and the second coordinate of the second reference position respectively, and FOV0 represents the current view angle of the camera along the horizontal direction; The angle between the third view angle boundary line and the fourth view angle boundary line is determined as the target view angle of the camera along the horizontal direction, comprising: Computing Wherein, B1 and C1 represent the third coordinate of the third reference position and the fourth coordinate of the fourth reference position respectively, and FOV1 represents the target view angle of the camera along the horizontal direction.

7. The method of claim 6, wherein, The lens position of the camera lens center is determined, comprising: The maximum door opening angle α of the door body and the real-time installation angle β of the camera support and the door body are obtained; The distance x between the camera lens center and the door body rotation axis and the distance y between the support rotation axis and the camera lens center are obtained; According to (xcosα+ycos(β-180°+α), xsinα+ysin(β-180°+α), 0), the lens coordinate A of the lens position of the camera lens center is determined.

8. A camera view angle regulating device for liner area, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to execute the program instructions when running, the camera view angle control method for the liner area as claimed in any one of claims 1 to 7.

9. A refrigerator characterized by comprising: Comprising: A box body; A door body rotatably arranged on the front side of the box body through a door body rotation axis, and a support rotation axis arranged at the end of the door body away from the door body rotation axis; A camera support rotatably arranged on the door body through the support rotation axis; A camera arranged on the camera support; The camera view angle control device for the liner area as claimed in claim 8 is installed on the door body.

10. A computer readable storage medium storing program instructions, wherein the program instructions comprise instructions for causing a computer to perform the method of any one of claims 1-9. The program instructions are used to make the computer execute the camera view angle control method for the liner area as claimed in any one of claims 1 to 7 when running.

Images