Opening degree detection method and storage

Abstract

To provide an opening degree detection method, etc., for highly accurately detecting the opening degree of a storage drawer.SOLUTION: An opening degree detection method includes: edge extraction processing extracting, on the basis of a captured image obtained by capturing an image of the drawer of a storage from above, the edge line of a front plate at which a drawer knob is provided, as an edge in the captured image; and opening degree calculation processing calculating the opening degree of the drawer on the basis of the edge position in the captured image. The edge extraction processing includes: scan processing reading in the luminance value of a pixel included in a first pixel line 71 and reading in the luminance value of a pixel included in a second pixel line 72 while moving mutually adjacent first pixel line 71 and second pixel line 72 that are a row of pixels appearing in the captured image and are parallel to the extending direction of the front plate, in a direction perpendicular to the extending direction; and edge position identification processing for identifying the position of an edge in the captured image on the basis of the difference in luminance value between the first pixel line 71 and the second pixel line 72.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present disclosure relates to an opening degree detection method and the like. [Background technology]

[0002] Known technology for detecting the open / closed state of a cabinet door or doors is, for example, the open / close detection device described in Patent Document 1. Patent Document 1 states that "the reader module analyzes the positional information data of the open / close body sent from the sensor module, and if it determines that the sensor module is in a predetermined position, it determines that the condition is normal, that is, the door (or doors) is closed, and if it is not in the predetermined position, it determines that the condition is abnormal, that is, the door (or doors) is open, and issues an alarm." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-165298 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 can determine whether a sliding door serving as an opening / closing body is in an open state or a closed state, but it is difficult to accurately detect how open the sliding door is.

[0005] Therefore, an object of the present disclosure is to provide an opening degree detection method and the like that can detect the opening degree of a drawer of a storage cabinet with high accuracy. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, the opening degree detection method of the present disclosure includes an edge extraction process that extracts the ridge line of the front panel on which the handle portion of the drawer is provided as an edge on the captured image based on the captured image obtained by capturing an image of a drawer of a storage cabinet from above, and an opening degree calculation process that calculates the opening degree of the drawer based on the position of the edge on the captured image, wherein the edge extraction process includes a scanning process that reads the luminance values ​​of the pixels included in the first pixel line and the luminance values ​​of the pixels included in the second pixel line while moving adjacent first pixel lines and second pixel lines that are arranged parallel to the extension direction of the front panel that appear in the captured image in a direction perpendicular to the extension direction, and an edge position identification process that identifies the position of the edge on the captured image based on the difference in luminance values ​​of the first pixel line and the second pixel line. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to provide an opening degree detection method and the like that detects the opening degree of a drawer of a storage cabinet with high accuracy. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a schematic side view of a storage cabinet in an opening degree detection method according to an embodiment. FIG. [Figure 2] FIG. 2 is a functional block diagram of a storage cabinet in an opening degree detection method according to an embodiment. [Figure 3A] 10 is a side view of the storage cabinet when the drawer is in a fully closed state in the opening degree detection method according to the embodiment. FIG. [Figure 3B] 10 is a side view of the storage cabinet when the drawer is opened to a predetermined opening degree in the opening degree detection method according to the embodiment. FIG. [Figure 3C] 10 is a side view of the storage cabinet when the drawer is fully open in the opening degree detection method according to the embodiment. FIG. [Figure 4] 1A to 1C are plan views showing the fully closed state, the intermediate open state, and the fully open state of a drawer of a storage cabinet in an opening degree detection method according to an embodiment. [Figure 5]10 is a flowchart of a process executed by a processing unit of a storage cabinet in an opening degree detection method according to the embodiment. [Figure 6] 10 is a flowchart illustrating a virtual plane setting process in the opening degree detection method according to the embodiment. [Figure 7] 10A and 10B are explanatory diagrams relating to setting of a virtual plane on a captured image in the opening degree detection method according to the embodiment. [Figure 8A] 10 is an example of a captured image before gradation processing is performed in the opening degree detection method according to the embodiment. [Figure 8B] 10 is an example of a captured image after gradation processing is performed in the opening degree detection method according to the embodiment. [Figure 9] 10A and 10B are explanatory diagrams relating to gradation processing and two-pixel line setting processing in the opening degree detection method according to the embodiment. [Figure 10] 4 is a flowchart illustrating an edge extraction process in the opening degree detection method according to the embodiment. [Figure 11A] 10A and 10B are explanatory diagrams illustrating a case where the edge of the front panel is detected at the fully open position in the opening degree detection method according to the embodiment. [Figure 11B] 10A and 10B are explanatory diagrams illustrating a case where the edge of the front panel is detected at an intermediate opening position in the opening degree detection method according to the embodiment. [Figure 11C] 10 is an explanatory diagram illustrating a case where the edge of the front plate is detected at the fully closed position in the opening degree detection method according to the embodiment. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0009] <Embodiment> FIG. 1 is a schematic side view of a storage 10 in an opening degree detection method according to an embodiment. 1 indicates the imaging range (field of view) of the imaging device 3. Also, the outline arrow in Fig. 1 indicates the movement direction when the drawer 2 is opened or closed. The storage cabinet 10 shown in Fig. 1 is for storing predetermined items. Examples of such storage cabinets 10 include, but are not limited to, refrigerators with drawer doors, as well as safes and security boxes. As shown in Fig. 1, the storage cabinet 10 includes a housing 1, a drawer 2, an imaging device 3, and a support 4.

[0010] The housing 1 is a box body in which the drawer 2 is provided. The housing 1 may be made of either metal or resin. Alternatively, the housing 1 may be a metal outer box (not shown) with a resin inner box (not shown) provided inside. The housing 1 is provided with a storage section (not shown) for storing the drawer 2. This storage section has a shape corresponding to the outer shape of the drawer 2, and is open at the front side of the housing 1. A pair of rails (not shown) for guiding the movement of the drawer 2 in the front-to-rear direction may be provided on the inner surface of the storage section.

[0011] The drawer 2 is a box-shaped drawer with an open top and is provided in the housing 1. The drawer 2 can be inserted and removed from the housing 1 in the front-rear direction. As shown in FIG. 1, the drawer 2 includes a container 2a and a front panel 2b. The container 2a is configured to hold a predetermined item and is box-shaped with an open top. When viewed from above, the opening edge of the container 2a has a rectangular frame shape. The front panel 2b is a plate-shaped member provided with a handle portion 21b and is installed on the front surface of the container 2a. As shown in FIG. 1, the upper end of the front panel 2b is positioned higher than the upper end of the container 2a. When the drawer 2 is fully closed, the portion of the front panel 2b that protrudes outward from the container 2a (for example, near the upper end) is in contact with or close to the opening edge of the housing 1. In the drawer 2, the container 2a and the front panel 2b may be integrally formed.

[0012] A handle portion 21b is provided on the upper part of the front panel 2b. The handle portion 21b is a portion into which a user's fingers are inserted when opening or closing the drawer 2. In the example of FIG. 1, the handle portion 21b is formed by partially cutting out a corner between the upper and front surfaces of the front panel 2b and further by recessing a portion of the upper surface of the front panel 2b downward by a predetermined amount so that a user can insert their fingers from above. Note that the handle portion 21b may be formed on only a portion of the front panel 2b in the horizontal direction (the direction perpendicular to the drawer direction), or may be formed across the entire horizontal area.

[0013] In the example of FIG. 1, one drawer 2 is provided at the bottom of the housing 1, but the position and number of drawers 2 can be changed as appropriate. For example, the housing 1 may be configured with multiple drawers arranged vertically or horizontally. If the storage cabinet 10 is a refrigerator, the drawers 2 may include a vegetable compartment door, a freezer compartment door, and an ice compartment door. If the storage cabinet 10 is a safe or a security box, predetermined valuables, important documents, etc. are placed in the drawer 2. If the storage cabinet 10 is a safe or a security box, the front panel 2b of the drawer 2 may be exposed by opening a hinged door (not shown).

[0014] The imaging device 3 is a camera for capturing images of the drawer 2 of the storage cabinet 10 from above. In the example of FIG. 1, the imaging device 3 is disposed forward of the front end of the housing 1. The imaging device 3 is supported by a support body 4. The imaging device 3 is disposed in a position that allows it to look down on the drawer 2 in a fully open state. In other words, the position of the imaging device 3 is set in advance so that the drawer 2 in a fully open state (see FIG. 3C) is included in the field of view of the imaging device 3.

[0015] The support body 4 is a member for supporting the imaging device 3, and extends in the front-to-rear direction. In the example of FIG. 1, one end of the support body 4 is fixed near the upper end of the housing 1, and the other end is fixed to the imaging device 3.

[0016] FIG. 2 is a functional block diagram of the repository 10. As shown in FIG. 2, the storage cabinet 10 includes an imaging device 3, a processing unit 5, and a buzzer 6. As described above, the imaging device 3 is a camera that captures an image of the drawer 2 (see FIG. 1) from above, and includes a lens 3a and an imaging element 3b. The lens 3a is an optical element that refracts and focuses light. When the imaging device 3 is installed in the housing 1 (see FIG. 1) via the support 4 (see FIG. 1), the optical axis of the lens 3a is approximately parallel to the vertical direction. The lens 3a faces downward so that the imaging device 3 can capture an image of the drawer 2 (see FIG. 1).

[0017] The image sensor 3b photoelectrically converts light incident through the lens 3a to generate captured image data. For example, a CCD sensor (Charge Coupled Device) or a CMOS sensor (Complementary Metal Oxide Semiconductor) is used as the image sensor 3b. The captured image data generated by the image sensor 3b is output to the processing unit 5.

[0018] The processing unit 5 is, for example, an MCU (Micro Controller Unit), and although not shown, is configured to include electronic circuits such as a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and various interfaces. The CPU reads out a program stored in the ROM and loads it into the RAM, and the CPU executes various processes. The processing unit 5 has a function of calculating the opening degree of the drawer 2 based on the image captured by the imaging device 3. The processes executed by the processing unit 5 will be described later.

[0019] The buzzer 6 emits a predetermined alarm sound when the drawer 2 of the storage cabinet 10 remains open for a predetermined time. While Fig. 2 shows an example in which the buzzer 6 is provided inside the storage cabinet 10, this is not limiting. For example, a buzzer that emits a predetermined sound based on a command signal from the processing unit 5 may be provided outside the storage cabinet 10. Alternatively, an alarm sound may be sounded or a predetermined alarm screen may be displayed on the user's mobile terminal (not shown).

[0020] FIG. 3A is a side view of the storage cabinet 10 when the drawer 2 is in a fully closed state. Note that in Figure 3A, the handle portion 21b of the drawer 2 (see Figure 1) is omitted (the same applies to Figures 3B and 3C). Furthermore, the "fully closed state" refers to a state in which there is no particular obstacle (not shown) at the back (rear) of the drawer 2, and further movement of the drawer 2 toward the back is restricted. In the "fully closed state," the front plate 2b of the drawer 2 is in contact with (or close to) the periphery of the opening of the housing 1. The position of the front plate 2b of the drawer 2 in the fully closed state is also included in the imaging range (field of view) of the imaging device 3.

[0021] FIG. 3B is a side view of the storage cabinet 10 when the drawer 2 is opened to a predetermined opening angle. When the drawer 2 is opened to a predetermined degree (for example, 50%), with the fully open state being defined as 100% open, the portion of the drawer 2 that protrudes from the housing 1 (the portion forward of the front end of the housing 1) is imaged by the imaging device 3.

[0022] FIG. 3C is a side view of the storage cabinet 10 when the drawer 2 is fully open. 3C, the position of the front panel 2b of the drawer 2 in the fully open state is also included in the imaging range (field of view) of the imaging device 3. Note that the "fully open state" refers to a state in which further movement of the drawer 2 toward the front is restricted.

[0023] FIG. 4 is a plan view showing the drawer 2 of the storage cabinet 10 in a fully closed state, an intermediate open state, and a fully open state. Note that Fig. 4 shows the case where the drawer 2 is fully open at 100% opening, as well as the case where it is intermediately open at 50% opening and fully closed at 0% opening. The white arrow in the front-to-rear direction shown in Fig. 4 indicates the movement direction of the drawer 2 when it is opened or closed. Furthermore, the "imaging range" shown in Fig. 4 indicates the imaging range of the imaging device 3 (see Fig. 1) in the front-to-rear direction.

[0024] As shown in Fig. 4, the imaging range of the imaging device 3 (see Fig. 1) is set so that when the drawer 2 is fully open, a predetermined area further forward than the front end of the housing 1 in a plan view is captured. Furthermore, the greater the opening angle of the drawer 2, the closer the front panel 2b appears to the front (to the right of the page in Fig. 4) in the captured image. For example, when the drawer 2 is opened to an intermediate degree, 50%, the front panel 2b appears in a position intermediate in the front-to-back direction between the position of the front panel 2b in the fully closed state and the position of the front panel 2b in the fully open state. In this way, the position of the front panel 2b in the captured image changes as the opening angle of the drawer 2 changes.

[0025] Therefore, in this embodiment, the ridge line of the front panel 2b of the drawer 2 is extracted as an edge on the captured image, and the processing unit 5 (see FIG. 2) calculates the opening degree of the drawer 2 based on the position of the edge on the captured image. Note that the imaging range is the same when imaging the fully closed state, the intermediate open state, and the fully open state of the drawer 2. Furthermore, an even wider range may be imaged, including the imaging range in the front-to-rear direction shown in FIG. 4.

[0026] FIG. 5 is a flowchart of the processing executed by the processing unit (also see FIGS. 1 and 2 as appropriate). In step S101, the processing unit 5 executes a virtual plane setting process (setting process). Details of this virtual plane setting process will be described with reference to the flowchart in FIG.

[0027] FIG. 6 is a flowchart relating to the virtual plane setting process (also see FIGS. 1 and 2 as appropriate). The series of processes shown in FIG. 6 may be performed by an operator during the setup stage of the storage 10 before product shipment, or may be performed as appropriate by a user based on the instruction manual. In step S101a, the processing unit 5 acquires captured images of the drawer 2 in both the fully closed and fully open states by the imaging device 3 (image acquisition process). That is, the processing unit 5 uses the imaging device 3 placed at a predetermined imaging position above the drawer 2 to acquire a captured image when the drawer 2 is in the fully open state and a captured image when the drawer 2 is in the fully closed state.

[0028] Specifically, a person places the drawer 2 in a fully closed state and then operates the image capture button on the storage cabinet 10 or a mobile terminal (not shown), thereby acquiring an image of the drawer 2 in the fully closed state. Similarly, an image of the drawer 2 in the fully open state is acquired. Such processing in step S101a (image acquisition processing) is included in the processing in step S101 in FIG. 5 (setting processing).

[0029] Next, in step S101b, the processing unit 5 converts the captured image (color image) to grayscale and then performs gradation processing. For example, the processing unit 5 represents the brightness of each pixel of the captured image with one of 256 values. Specifically, the brightness value of black pixels is set to '0', and the brightness value of white pixels is set to '255'.

[0030] In step S101c, the processing unit 5 extracts edges on the captured image. For example, the processing unit 5 extracts edges on the captured image by using a Sobel filter. Note that the edge extraction method is not limited to the Sobel filter, and other well-known methods such as a Laplacian filter or the Canny algorithm may also be used. As a result, ridgelines when the front panel 2b of the drawer 2 is viewed in plan are extracted as edges on the captured image.

[0031] Although it is possible to extract ridgelines in the front-to-back direction (i.e., the opening / closing direction of the drawer 2) of the container 2a (see FIG. 1) when the drawer 2 is viewed in plan, in this embodiment, lateral ridgelines parallel to the extension direction of the front plate 2b are extracted as edges. Furthermore, since the front plate 2b has a predetermined thickness in the front-to-back direction, in addition to the ridgeline at the front end of the upper surface of the front plate 2b, the ridgeline at the rear end of the upper surface of the front plate 2b is also extracted as an edge. In this embodiment, the processing unit 5 extracts the ridgeline at the front end of the upper surface of the front plate 2b (edge ​​on the captured image).

[0032] Next, in step S101d of Fig. 6, the processing unit 5 sets a virtual plane for the captured image (setting process). That is, the processing unit 5 sets a predetermined rectangular area included in the captured image as the virtual plane. Setting of this virtual plane will be described with reference to Fig. 7.

[0033] FIG. 7 is an explanatory diagram regarding the setting of the virtual plane P1 on the captured image. It is assumed that a captured image is generated in the area of ​​a rectangular virtual plane P1 based on electrical signals from a large number of image sensors 3b (see FIG. 2) arranged vertically and horizontally in a matrix. It is also assumed that the ridge lines of the front panel 2b (see FIG. 4) of the drawer 2 in the fully open and fully closed states have already been extracted as edges on the captured image (S101c in FIG. 6).

[0034] As a preliminary step to the edge extraction process (S104 in FIG. 5) described below, the processing unit 5 sets a rectangular area specified by the fully open and fully closed positions of the drawer 2 as a virtual plane P1. The "fully open position" mentioned above is the position of the edge of the front panel 2b (see FIG. 1) on the captured image when the drawer 2 is in the fully open state. The "fully closed position" is the position of the edge of the front panel 2b on the captured image when the drawer 2 is in the fully closed state.

[0035] When setting the virtual plane P1, the processing unit 5 sets the origin (0, 0) and the first coordinate position (0, Y Max ) and the second coordinate position (X Max ,0) and the third coordinate position (X Max ,Y Max) on the captured image. Specifically, in the virtual plane setting process (i.e., setting process: S101 in FIG. 5), the processing unit 5 sets the pixel at one end of the edge of the front panel 2b in the fully open position as the origin (0,0) on the captured image, and sets the pixel at the other end as the second coordinate position (X Max , 0). The processing unit 5 also sets the pixel at one end of the edge of the front panel 2b in the fully closed position as the first coordinate position (0, Y Max ), and the pixel at the other end is set as the third coordinate position (X Max ,Y Max )

[0036] Then, the processing unit 5 calculates the coordinates of the origin (0,0) and the first coordinate position (0,Y Max ) and the second coordinate position (X Max ,0) and the third coordinate position (X Max ,Y Max ) and sets a rectangular area specified by the above as a virtual plane P1. In addition, in the virtual plane setting process (i.e., setting process: S101 in FIG. 5), the processing unit 5 sets the origin (0,0) and the second coordinate position (X Max The line passing through the origin (0,0) and the first coordinate position (0,Y Max ) is set as the Y-axis.

[0037] After setting the virtual plane P1 in this way (S101d in Fig. 6), the processing of the processing unit 5 proceeds to step S102 in Fig. 5. In step S102, the processing unit 5 uses the imaging device 3 to obtain a current captured image of the drawer 2.

[0038] That is, after setting the virtual plane P1 (setting process), the processing unit 5 acquires a current captured image by again capturing an image of the drawer 2 at a predetermined capturing position (the same capturing position as in S101a of FIG. 6) using the imaging device 3. Then, the processing unit 5 sequentially performs the next step S103, as well as edge extraction processing (S104) and opening degree calculation processing (S105) on this captured image.

[0039] As a result, compared to when the virtual plane P1 is not set, the opening degree of the drawer 2 can be detected with high accuracy based on the captured image even if the imaging device 3 is shifted or tilted from its original position. For example, if the storage cabinet 10 is a refrigerator, and the foaming pressure of the foam insulation causes the top surface of the housing 1 to bulge slightly, causing the imaging device 3 to tilt from its original position, the actual fully open and fully closed positions of the drawer 2 will be reflected on the virtual plane P1 on the captured image. Therefore, as will be described later, the processing unit 5 can appropriately set the first pixel line 71 (see FIG. 9) and the second pixel line 72 (see FIG. 9) based on the virtual plane P1, and the opening degree of the drawer 2 can be calculated with high accuracy.

[0040] The processing of step S102 (acquisition of captured images) may be performed periodically regardless of the state of drawer 2, or may be performed when an open signal for drawer 2 is input from a door switch (not shown).

[0041] Next, in step S103, the processing unit 5 performs gradation processing and two-pixel line setting processing. In the gradation processing, the processing unit 5 represents the luminance value of each pixel of the captured image as one of 256 gradation values, for example.

[0042] FIG. 8A is an example of a captured image before gradation processing is performed. 8A shows an example in which drawer 2 is a vegetable compartment door of a refrigerator. As shown in FIG. 8A, a captured image of an area including drawer 2 that is opened in a predetermined manner is obtained.

[0043] FIG. 8B is an example of a captured image after gradation processing has been performed. As described above, one of 256 brightness values ​​is assigned to each pixel included in the captured image. In the example of Fig. 8B, the smaller the brightness value, the closer each pixel is to being displayed in a color that is closer to black.

[0044] Next, in the two-pixel line setting process included in step S103 in FIG. 5, the processing unit 5 sets a first pixel line 71 (see FIG. 9) and a second pixel line 72 (see FIG. 9) as two pixel lines.

[0045] FIG. 9 is an explanatory diagram relating to the gradation processing and the two-pixel line setting processing (see FIG. 1 and FIG. 2 as appropriate). The processing unit 5 performs a predetermined gradation process on the captured image of the current state of the storage 10, and sets the area of ​​the virtual plane P1 (see FIG. 7) on the captured image that has been subjected to the gradation process as the gradation area T1. Here, the gradation area T1 means the area of ​​the virtual plane P1 (see FIG. 7) in the state that the gradation process has been performed.

[0046] In the two-pixel line setting process (S103 in FIG. 5), the processing unit 5 sets a first pixel line 71 and a second pixel line 72 to identify the position of the front panel 2b of the drawer 2 in the captured image. These first pixel line 71 and second pixel line 72 have one pixel in the Y-axis direction and are formed by a series of multiple pixels in the X-axis direction. The first pixel line 71 and second pixel line 72 are parallel to the X-axis direction and are adjacent to each other in the Y-axis direction.

[0047] Although details will be described later, the first pixel line 71 and the second pixel line 72 are used when scanning the luminance values ​​of multiple pixels aligned in the X-axis direction (i.e., reading the luminance value of each pixel). At the start of such scanning processing, the first pixel line 71 and the second pixel line 72 are set at the positions shown in FIG. 9. That is, as a preliminary step to the edge extraction processing (S104 in FIG. 5), the processing unit 5 sets the initial position of the first pixel line 71 outside the virtual plane P1 (see FIG. 7) and sets the initial position of the second pixel line 72 inside the virtual plane P1 (setting processing). As described above, the multiple pixels included in the gradation area T1 in FIG. 9 are the same as the multiple pixels included in the virtual plane P1 (see FIG. 7).

[0048] Next, in step S104 of Fig. 5, the processing unit 5 performs edge extraction processing. That is, based on a captured image obtained by capturing an image of the drawer 2 of the storage cabinet 10 from above, the processing unit 5 extracts, as an edge in the captured image, the ridge line of the front panel 2b on which the handle portion 21b of the drawer 2 is provided. This edge extraction processing will be described with reference to Fig. 10.

[0049] FIG. 10 is a flowchart relating to the edge extraction process (also see FIG. 9 as appropriate). In the edge extraction process (S104 in FIG. 5), a scanning process in step S104a and an edge position identification process in step S104b are performed sequentially. First, in the scanning process in step S104a, the processing unit 5 scans the first pixel line 71 and the second pixel line 72 in the Y-axis direction. That is, the processing unit 5 moves the first pixel line 71 and the second pixel line 72, which are arranged in a row of pixels parallel to the extension direction (X-axis direction) of the front plate 2b (see FIG. 4) reflected in the captured image, and which are adjacent to each other, by one pixel in the direction perpendicular to the extension direction (Y-axis direction).

[0050] In this "scanning process," the processing unit 5 moves the first pixel line 71 and the second pixel line 72 in the Y-axis direction, while reading the luminance values ​​of the pixels included in the first pixel line 71, and also reading the luminance values ​​of the pixels included in the second pixel line 72. In other words, the processing unit 5 reads the luminance values ​​of the pixels in the first pixel line 71 and the second pixel line 72, while shifting the positions of the first pixel line 71 and the second pixel line 72 by one pixel at a time in the positive direction in the Y-axis direction from the initial positions shown in Fig. 9.

[0051] In the next step S104b, which is edge position identification processing, the processing unit 5 identifies the position of the edge of the front plate 2b on the captured image based on the difference in brightness values ​​between the first pixel line 71 and the second pixel line 72. That is, the processing unit 5 identifies the position of the edge of the front plate 2b (position in the Y-axis direction) based on the difference in brightness values ​​between pixels adjacent to each other in the Y-axis direction in the first pixel line 71 and the second pixel line 72.

[0052] For example, if neither the first pixel line 71 nor the second pixel line 72 includes an edge of the front plate 2b in the captured image, the luminance value of the first pixel line 71 and the luminance value of the second pixel line 72 will be approximately the same. On the other hand, if the second pixel line 72 is located at a position where an edge exists in the captured image, the difference between the luminance values ​​of the first pixel line 71 and the second pixel line 72 will be large. Specifically, the luminance value of the second pixel line 72 that includes an edge will be lower than the luminance value of the first pixel line 71 that does not include an edge. In this way, the processing unit 5 performs scanning (reading pixel luminance values) based on the first pixel line 71 and the second pixel line 72 over the entire gradation area T1.

[0053] In most cases, edges on a captured image after gradation processing have a thickness of about one pixel. Therefore, it is unlikely that a pixel on a given edge will be included in both the first pixel line 71 and the second pixel line 72 at the same time.

[0054] 5 (S104), the following processing may be performed. That is, in the first pixel line 71 and the second pixel line 72, if the number of pairs of pixels whose difference in brightness value is equal to or greater than a predetermined threshold value among a plurality of pairs of pixels adjacent to each other in the direction (Y-axis direction) perpendicular to the extension direction (X-axis direction) of the front plate 2b on the captured image is equal to or greater than a predetermined value, the processing unit 5 may extract the position of the first pixel line 71 or the second pixel line 72 as an edge candidate. In this case, the processing unit 5 may extract the position of the first pixel line 71 or the second pixel line 72 as an edge candidate by dividing the number of pairs of pixels whose difference in brightness value is equal to or greater than a predetermined threshold value by the number of pairs of pixels whose difference in brightness value is equal to or greater than a predetermined threshold value. Max The scan is performed up to the position (position ), and the edge candidate with the largest number of pixel pairs whose brightness difference is equal to or greater than the predetermined threshold is extracted as the edge of the front panel 2b. This makes it possible to reduce false edge detection when noise occurs in the captured image or when other components or people are captured.

[0055] In the predetermined setting process, the processor 5 may acquire a test captured image with the drawer 2 set to a predetermined intermediate opening state between the fully open state and the fully closed state, and the "predetermined threshold" (brightness threshold) may be set based on this test captured image so that the edge of the front panel 2b in the captured image is extracted. This setting process is performed, for example, during the virtual plane setting process in step S101. The intermediate opening state may be a state in which the drawer 2 is opened 50%, or may also be a state in which the drawer 2 is opened 40% or 60%. In an experiment, it was confirmed that the front panel 2b in the captured image could be detected with high accuracy by setting the "predetermined threshold" (brightness threshold) to 15 in 256 gradations.

[0056] Next, in step S105 of FIG. 5, the processing unit 5 performs an opening degree calculation process. That is, the processing unit 5 calculates the opening degree of the drawer 2 based on the position of the edge of the front panel 2b on the captured image. For example, the processing unit 5 calculates the opening degree of the drawer 2 based on the origin (0,0) and the first coordinate position (0,Y Max ) and the first coordinate position (0,Y ) are divided into 100 equal parts to set the scale (setting process). Furthermore, the processing unit 5 associates the value of '0' with the origin (0,0) and Max ) to the value '100' to set the scale from 0 to 100.

[0057] Then, in the opening degree calculation process (S105 in FIG. 5), the processing unit 5 calculates the opening degree of the drawer 2 based on the value of the scale where the edge on the captured image is located. Specifically, the processing unit 5 identifies which of the above-mentioned scales the position of the edge extracted in the edge extraction process (S104 in FIG. 5) is closest to, and determines the position of the scale closest to the edge as the approximate position of the edge. The opening degree of the drawer 2 is calculated by subtracting the value of the edge's scale from the value '100'. This opening degree is a ratio that indicates the degree to which the drawer 2 is opened, with the fully open state being 100% open.

[0058] If the drawer 2 is not closed within a predetermined time after the door sensor (not shown) detects an opening operation of the drawer 2, the processing unit 5 may sound the buzzer 6 and display the opening degree value on the user's mobile terminal (not shown). In such control, a predetermined command signal for sounding the buzzer 6 is transmitted from the processing unit 5 to a server (not shown) via a communication device (not shown) and a network (not shown) in this order, and the command signal is transmitted from the server to the user's mobile terminal (not shown). The mobile terminal may be, for example, a smartphone, a tablet, a wearable terminal, or a mobile phone.

[0059] Furthermore, by including information about the opening degree of drawer 2 in the message displayed on the user's mobile terminal (not shown), the user can know, for example, how far the vegetable compartment door of the refrigerator is open. Specifically, even when the user is out, the user can know that the refrigerator drawer is left slightly open (for example, 5% open).

[0060] FIG. 11A is an explanatory diagram illustrating a case where the edge of the front panel is detected in the fully open position. 11A, the area in which the drawer 2 is captured is indicated by dots. For example, at the initial position described in FIG. 9, if the number of pixel pairs (pairs of pixels adjacent in the Y-axis direction) in which the difference in luminance value between the first pixel line 71 and the second pixel line 72 is equal to or greater than a predetermined threshold is the largest, the processing unit 5 determines that the drawer 2 is fully open.

[0061] FIG. 11B is an explanatory diagram illustrating a case where the edge of the front panel is detected at the intermediate opening position. In Fig. 11B, the area in which the drawer 2 is shown is indicated by dots. Also, in Fig. 11B, the position of the drawer 2 when it is fully open is indicated by a dashed line. For example, the middle position in the Y-axis direction (the Y-axis coordinate value is Y Max If the number of pixel pairs where the difference between the luminance value of the first pixel line 71 and the luminance value of the second pixel line 72 is equal to or greater than the predetermined threshold value at the position (position / 2) is the largest, the processing unit 5 determines that the drawer 2 is open to 50%.

[0062] FIG. 11C is an explanatory diagram of a case where the edge of the front plate is detected in the fully closed position. 11C, the position of the drawer 2 when it is fully open is indicated by a dashed line. For example, if the number of pixel pairs where the difference between the luminance value of the first pixel line 71 and the luminance value of the second pixel line 72 is equal to or greater than the predetermined threshold is the largest at the end position of the scan in the Y-axis direction, the processing unit 5 determines that the drawer 2 is fully closed.

[0063] <Effects> According to this embodiment, it is possible to accurately detect how far the drawer 2 of the storage cabinet 10 is opened. As a result, if, for example, the vegetable compartment door of a refrigerator or a safe for storing valuables is opened more than necessary, the buzzer 6 can be sounded to notify the user.

[0064] <<Variations>> Although the opening degree detection method and the storage shed 10 according to the present disclosure have been described above in the embodiments, they are not limited to these descriptions and various modifications can be made. For example, in the embodiment, the case where the number of drawers 2 in the storage cabinet 10 (see FIG. 1) is one has been described, but the number of drawers may be multiple. That is, the housing 1 of the storage cabinet 10 may be provided with at least one drawer.

[0065] In the embodiment, the imaging device 3 is installed in the storage cabinet 10 (see FIG. 1), but this is not limiting. For example, the imaging device 3 may be installed on a wall or ceiling of a room (not shown) in which the storage cabinet 10 is installed, and the imaging device 3 may capture an image of the drawer 2 from above.

[0066] In addition, in the embodiment, the processing unit 5 (see FIG. 2) that calculates the opening degree of the drawer 2 (see FIG. 1) is described as being included in the storage cabinet 10, but this is not limiting. For example, data of an image captured by the imaging device 3 may be transmitted to a server (not shown) via a communication device (not shown), and the opening degree of the drawer 2 may be calculated by this server.

[0067] In the embodiment, the initial position of the scan process (S104a in FIG. 10) is set to the fully open position on the captured image (the position where the coordinate value of the Y axis is 0), but this is not limiting. That is, the initial position of the scan process may be set on the negative side of the fully open position on the captured image in the Y axis direction. Also, the end position of the scan process may be set to the fully closed position on the captured image (the position where the coordinate value of the Y axis is 0). Max For example, the end position of the scanning process may be set on the positive side of the Y-axis direction from the fully closed position on the captured image.

[0068] In the embodiment, the position of the edge of the front plate 2b is identified based on the difference in luminance values ​​between pixels adjacent to each other in the Y-axis direction in the first pixel line 71 and the second pixel line 72. However, the present invention is not limited to this. For example, the edge of the front plate 2b may be identified based on the difference between the average luminance of the pixels included in the first pixel line 71 and the average luminance of the pixels included in the second pixel line 72.

[0069] In the embodiment, the handle portion 21b (see FIG. 1) is provided on the upper part of the front panel 2b (see FIG. 1), but this is not limiting. For example, the handle portion 21b may be provided on the lower part of the front panel 2b.

[0070] In the embodiment, the storage cabinet 10 (see FIG. 1) is a refrigerator, but this is not limiting. For example, in addition to a safe or a security box, a desk with drawers, a warming cabinet, a dry cabinet, or a moisture-proof cabinet are also included in the "storage cabinet."

[0071] Furthermore, all or part of the program for realizing the method for detecting the opening degree of the drawer 2 (opening degree detection method) described in the embodiment may be executed by one or more computers. The program can be provided via a communication line, or can be written to a recording medium such as a CD-ROM and distributed.

[0072] Furthermore, the present disclosure is not limited to the embodiments and includes various modifications. For example, the embodiments have been described in detail to clearly explain the present disclosure, and the present disclosure is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to add, delete, or replace part of the configuration of the embodiments with other configurations.

[0073] Furthermore, the above-mentioned configurations, functions, processing units, processing means, etc. may be partly or entirely implemented in hardware, for example, by designing them as integrated circuits. Furthermore, the above-mentioned configurations, functions, etc. may be implemented in software, with a processor interpreting and executing a program that implements each function. Information such as the programs, tables, and files that implement each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0074] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0075] 1 chassis 2 drawers 2a container 2b Front panel 3. Imaging device 3a lens 3b Image sensor 4 Support 5 Processing section 6 Buzzer 10 Storage 21b Handle 71 1st pixel line 72 Second pixel line P1 Virtual plane S101 step (setting process) S101a Step (Image acquisition process) S104 step (edge ​​extraction processing) S104a Step (Scanning process) Step S104b (edge ​​position identification process) S105 step (opening calculation process) T1 Tone Area

Claims

1. an edge extraction process for extracting, based on a captured image obtained by capturing an image of a drawer of a storage cabinet from above, a ridge line of a front panel on which a handle portion of the drawer is provided as an edge on the captured image; an opening degree calculation process for calculating an opening degree of the drawer based on the position of the edge on the captured image, The edge extraction process includes: a scanning process of reading luminance values ​​of pixels included in a first pixel line and a second pixel line, the first pixel line and the second pixel line being arranged parallel to the extension direction of the front plate and appearing in the captured image, while moving adjacent first pixel lines and second pixel lines in a direction perpendicular to the extension direction; an edge position specifying process for specifying a position of the edge on the captured image based on a difference in luminance values ​​between the first pixel line and the second pixel line.

2. As a preliminary step of the edge extraction process, a setting process is performed in which a rectangular area specified by a fully open position, which is the position of the edge on the captured image when the drawer is fully open, and a fully closed position, which is the position of the edge on the captured image when the drawer is fully closed, is set as a virtual plane, and an initial position of the first pixel line is set outside the virtual plane, and an initial position of the second pixel line is set inside the virtual plane.

2. The opening detection method according to claim 1,

3. In the setting process, a pixel at one end of the edge at the fully open position is set as an origin on the captured image, and a pixel at the other end is set as a second coordinate position on the captured image; a pixel at one end of the edge at the fully closed position is set as a first coordinate position on the captured image, and a pixel at the other end of the edge is set as a third coordinate position on the captured image; setting a rectangular area specified by the origin, the first coordinate position, the second coordinate position, and the third coordinate position as the virtual plane; 3. The opening detection method according to claim 2, wherein:

4. In the setting process, a straight line passing through the origin and the second coordinate position on the captured image is set as an X-axis, and a straight line passing through the origin and the first coordinate position on the captured image is set as a Y-axis; A predetermined gradation process is performed on the captured image of the current state of the storage, and the area of ​​the virtual plane on the captured image that has been subjected to the gradation process is set as a gradation area.

4. The opening detection method according to claim 3, wherein:

5. In the scanning process, the first pixel line and the second pixel line are scanned in the Y-axis direction; In the edge position specifying process, the position of the edge is specified based on a difference in luminance values ​​between pixels adjacent to each other in the Y-axis direction in the first pixel line and the second pixel line.

5. The opening detection method according to claim 4, wherein:

6. In the setting process, a scale is set by dividing a line segment connecting the origin and the first coordinate position into 100 equal parts; In the opening degree calculation process, the opening degree of the drawer is calculated based on the value of the scale where the edge is located.

6. The opening detection method according to claim 5,

7. the setting process includes an image acquisition process for acquiring an image when the drawer is fully open and an image when the drawer is fully closed using an imaging device arranged at a predetermined imaging position above the drawer; After the setting process, the imaging device is used to capture an image of the drawer again at the imaging position to obtain a current captured image, and the edge extraction process and the opening degree calculation process are performed sequentially on the captured image.

3. The opening detection method according to claim 2, wherein:

8. In the edge extraction process, when the number of pixel pairs whose difference in luminance value is equal to or greater than a predetermined threshold value is equal to or greater than a predetermined value among a plurality of pixel pairs adjacent to each other in a direction perpendicular to the extending direction in the first pixel line and the second pixel line, the position of the first pixel line or the second pixel line is extracted as an edge candidate, and the edge candidate having the largest number of pixel pairs whose difference in luminance value is equal to or greater than the predetermined threshold value is extracted as the edge.

3. The opening detection method according to claim 2, wherein:

9. In the setting process, a test captured image is acquired by setting the drawer to a predetermined intermediate open state between the fully open state and the fully closed state, and the predetermined threshold value is set based on the test captured image so that the edge is extracted.

9. The opening detection method according to claim 8,

10. The housing and At least one drawer provided in the housing; an imaging device for imaging the drawer from above; a processing unit that calculates the opening degree of the drawer based on the image captured by the imaging device, The processing unit an edge extraction process for extracting, based on the captured image obtained by capturing an image of the drawer from above, a ridge line of a front panel on which a handle portion of the drawer is provided as an edge on the captured image; an opening degree calculation process for calculating an opening degree of the drawer based on the position of the edge on the captured image; The edge extraction process includes: a scanning process of reading luminance values ​​of pixels included in a first pixel line and a second pixel line, the first pixel line and the second pixel line being arranged parallel to the extension direction of the front plate and appearing in the captured image, while moving adjacent first pixel lines and second pixel lines in a direction perpendicular to the extension direction; and an edge position identification process for identifying the position of the edge on the captured image based on the difference in brightness values ​​between the first pixel line and the second pixel line.

Images